Profile Picture

Lip Sync Improvement...Any Tips or Suggestions

Posted By TopOneTone 6 Years Ago
You don't have permission to rate!
Author
Message
animagic
animagic
Posted 6 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)

Group: Forum Members
Last Active: 1 hour ago
Posts: 15.7K, Visits: 30.5K
ckalan1 (7/14/2018)
The real challenge is to make the lips move in an asymmetrical manner. Being able to add in snarls, lip curls, deformations and various mouth shapes to correspond to the emotions is what will make iclone avatars come alive. The eyes and forehead deformation plays a big part in making the lip sync aesthetically pleasing. 
My next little project is to figure out the best way to bring facial morphs from DAZ to iclone. 
Craig

There are now a lot more facial morphs in iClone, so if you use CC characters that would be easier. The Facial Key option makes these available, for example.



https://forum.reallusion.com/uploads/images/436b0ffd-1242-44d6-a876-d631.jpg

doubblesixx
doubblesixx
Posted 4 Years Ago
View Quick Profile
Veteran Member

Veteran Member (577 reputation)Veteran Member (577 reputation)Veteran Member (577 reputation)Veteran Member (577 reputation)Veteran Member (577 reputation)Veteran Member (577 reputation)Veteran Member (577 reputation)Veteran Member (577 reputation)Veteran Member (577 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 86, Visits: 470
Read through this post and i hoped to find a better answer.. I got curious how this all works from another post and started to investigate but when loading in an audiofile and finding it's off by about 8 - 10 frames( counting the initial transient and where the bulk starts maybe another 2 frames)  so between 8 - 12 frames early.. it gets tedious to compensate for this and get a working result when you see is not what you get.. i find this useless

One (crappy) workaround could be using a 3'rdparty app like Pure Ref - take screenshot of the audiofile zoomed out and lay it ontop on the current visual representation in to get a reference to place the pnonemes against.. but.. how hard would it be to add a timeshift function in iclone so the visual and audio always would be in sync.. this would help tremendously.

As i see it now knowing the limitations other tools like what Mike's using  (papagayo) seems to be the only option.


This is a silent signature

Edited
4 Years Ago by doubblesixx
Kelleytoons
Kelleytoons
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)

Group: Forum Members
Last Active: 9 hours ago
Posts: 9.2K, Visits: 21.8K
The good news is this will be improved -- we know they are working on a much better system and IIRC it *should* be before the end of the year (but I'm old and don't necessarily remember -- search here for the timeline and you might find more details).

So -- if you have patience you will be rewarded.



Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley
toystorylab
toystorylab
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)

Group: Forum Members
Last Active: 28 minutes ago
Posts: 4.4K, Visits: 28.8K
New Lipsync is planned for Q3...

Enhanced Facial Lip-sync  (iClone 7.9)
  • A highly accurate viseme generation based on voice detection will be added (supports English, French, German in the first release) 
  • Facial animation can be driven by audio strength for different animation styles
  • Enhanced smooth lip algorithm for co-articulation.

https://forum.reallusion.com/uploads/images/d417e18a-e3f4-475a-970b-3529.png


Toystorylab on Vimeo : https://forum.reallusion.com/uploads/images/ce44ea78-6984-47d8-9bf4-b783.png    Crassitudes (my "Alter Ego") on Youtube: https://forum.reallusion.com/uploads/images/fcc4df30-b3a8-40a5-a427-0735.png




Reading This Topic