Profile Picture

Setting Facial (Viseme) Keyframe in iClone 7

Posted By mattypattyfp 3 Months Ago
You don't have permission to rate!
Author
Message
mattypattyfp
mattypattyfp
Posted 3 Months Ago
View Quick Profile
Senior Member

Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 14, Visits: 72
Hi,
I'm trying to setup a model (CC3) for use with FaceFx.  To do so, I have to set keyframes for the visemes at certain times.  Here's the instructions:
https://facefx.github.io/documentation/doc/default-character-setup

I have two related questions:
1) Is there a way to set key frames using the Lips Editor in iClone (since the lips editor already contains the visemes I need)?  It adds the visemes to the Lips track, but I'm assuming I need them to actually modify the Expression >> Muscle track.
2) How can I set key frames from the tongue?
animagic
animagic
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)

Group: Forum Members
Last Active: 10 minutes ago
Posts: 11.4K, Visits: 19.6K
It surely is an interesting concept.

Recently we had a discussion about how to improve iClone lip-syncing using Papagayo (which also can use a text file to improve the results), but this may be even more convenient.

Tentatively, I believe you can accomplish what you want to do. (Someone smarter may give you an already more definitive response).

I haven't done much with FBX export of facial animation, but as far as I know you should be able to export the viseme track as part of a MotionPlus clip, which includes facial animation.

Animation for the tongue needs to be set up I think (used to be at least). I have to see if that is still possible. There may already be an existing animatable morph for it.


https://forum.reallusion.com/uploads/images/1a09220f-ab50-42ac-ad1a-33ec.pnghttps://forum.reallusion.com/Uploads/Images/d14339d0-cd32-4b35-88f9-40a0.png


animagic
animagic
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)

Group: Forum Members
Last Active: 10 minutes ago
Posts: 11.4K, Visits: 19.6K
I'm reading that the shapes need to be keyed, but that should still be possible with visemes (perhaps adding None before and after the target viseme). I just don't know how it ends up in the FBX file.


https://forum.reallusion.com/uploads/images/1a09220f-ab50-42ac-ad1a-33ec.pnghttps://forum.reallusion.com/Uploads/Images/d14339d0-cd32-4b35-88f9-40a0.png


Kelleytoons
Kelleytoons
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)

Group: Forum Members
Last Active: 3 hours ago
Posts: 6.0K, Visits: 12.2K
I just woke up so I may not quite get all this -- first of all, what the hell is FaceFX?  Is that for export to Unity or some other animation program?

In any case, this seems real straightforward to create a Python program to do this in iClone.  But, again, I may just be asleep and not getting the whole point here (because just a cursory glance at the FaceFX docs indicates there's a Python Import pipeline so I'm not sure where that even is -- import to where?  And if there is, why didn't they write the Python to do the correct setup in iClone?

I may need to wake up more -- or perhaps someone can tell me the bigger picture here.



Alienware Aurora R7, Win 10, i7-8700k, 4.7GHz CPU, 32GB RAM, GTX Titan XP (12GB), Samsung 960 Pro 2TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley
mattypattyfp
mattypattyfp
Posted 3 Months Ago
View Quick Profile
Senior Member

Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 14, Visits: 72
Thanks, animagic.

I can setup the base visemes using the Modify tab of the Face Key Editor.  Interestingly, the FV key isn't recognized in FaceFX (I use the Bite Lip slider to set that.)  Then I just do an FBX export directly from iClone.  It'd just be more convenient from the Lips Editor.  
I would definitely be interested in a feature that leverages a text file to improve the lip syncing.  I get a better quality output letting FaceFX automatically lip sync , than I do in iClone or CrazyTalk.  Otherwise, I would just stay inside iClone/Crazytalk.  Although FaceFX has some nice features - the big one is that you can place "marks" inside the text file to trigger gestures and animations.

Regarding, "getting the visemes into an FBX" -- I'll try setting them with the Lips Editor exporting the RLTalk file into 3DXchange and exporting that way.  Also I'm wondering if I can set the key frames directly inside 3DXchange -- doesn't it have a lips editor as well?
mattypattyfp
mattypattyfp
Posted 3 Months Ago
View Quick Profile
Senior Member

Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 14, Visits: 72
Hi Kelleytoons,
I think FaceFX uses the keyframes at specific locations to then animate the lips. So, I'm not sure the python script approach would work.  I don't know how you'd set the morphs in the python code. But I'm very new at this, so I could be missing something here.

The bigger picture here (for me) is to get better quality lipsyncing in less time.  I currently spend a lot of time in iClone - largely because I end up discarding most of the automated visemes and re-doing them.
Kelleytoons
Kelleytoons
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)Distinguished Member (19.4K reputation)

Group: Forum Members
Last Active: 3 hours ago
Posts: 6.0K, Visits: 12.2K
Okay, I'm more awake now and am starting to get the picture.  You use FaceFX outside of iClone somewhere to generate your sync, and then import.  If you do want to look at my Papagayo route (it might help -- got us lip sync good enough for Fox television) here's the video:






Alienware Aurora R7, Win 10, i7-8700k, 4.7GHz CPU, 32GB RAM, GTX Titan XP (12GB), Samsung 960 Pro 2TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley
mattypattyfp
mattypattyfp
Posted 3 Months Ago
View Quick Profile
Senior Member

Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)Senior Member (263 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 14, Visits: 72
Thanks guys. This gave me some things to try.  Kelleytoons, I got the script to work for a couple of lip-sync
Dr. Nemesis
Dr. Nemesis
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.9K reputation)Distinguished Member (1.9K reputation)Distinguished Member (1.9K reputation)Distinguished Member (1.9K reputation)Distinguished Member (1.9K reputation)Distinguished Member (1.9K reputation)Distinguished Member (1.9K reputation)Distinguished Member (1.9K reputation)Distinguished Member (1.9K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 226, Visits: 4.5K
Hey matty, I used FaceFx for games back in the day and am now starting to look at using it for Iclone. Please keep us posted with your progress. I won’t activately start work on this yet since I’ll (hopefully) be using it on my next project but I’ll be interested to see your results!

From what I can see FaceFX is the best choice for high quality automatic lip sync so this is fairly important.



Reading This Topic