English


Profile Picture

What do I need to use facial animations (record, save, retarget) in Unity3d, using Character Creator...

Posted By lollosone 5 Years Ago
You don't have permission to rate!

What do I need to use facial animations (record, save, retarget) in...

Author
Message
lollosone
lollosone
Posted 5 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)

Group: Forum Members
Last Active: Last Year
Posts: 95, Visits: 340
I have CC3, and found an application on the iphone store called live face; which seems to be able to stream data to a RL figure. If I get this right; I can technically do facial animations on any CC3 figure, and as such, anything that can be imported as CC3 character, so any DAZ figure.
A quick look at the page show that I can't go directly to CC3, but I need to buy iClone7, plus a plugin that is called "motion live"; although the RL site is so confusing that it is quite hard to figure out exactly what I need. The final result should be a FBX I save from this iclone + motion live combo, that can be technically imported into Unity3d and be re-targeted.
Do I have this right? If not, can someone please clarify?

My plan is the following:
- record facial animations, so I can have a digitized version of the performance recorded from an actor, including audio.
- save those recordings in a format that can be used to be retargeted easily
- Use CC3 to either create characters or import characters from DAZ, and apply those animations to those characters, to then export them into Unity3d
- Use the recorded facial animations and retarget them in Unity3d; especially for characters not made via CC3 (an example is animated figures, like the MMD models that are so popular in anime community), so I can have in Unity a talking character that can play those animations and sound files.

Does anyone use such workflow or part of it, and could help me understand what do I need? I find extreme to have to buy 2 software just to get the FBX with the facial recording done; since I won't use iclone for anything else beside mocap of the facial performances; so I hope there is an easier way to get facial animations to both CC3 and Unity.
Kaen_SG
Kaen_SG
Posted 5 Years Ago
View Quick Profile
New Member

New Member (87 reputation)New Member (87 reputation)New Member (87 reputation)New Member (87 reputation)New Member (87 reputation)New Member (87 reputation)New Member (87 reputation)New Member (87 reputation)New Member (87 reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 5, Visits: 86
I'm exploring using CC3 characters with a iphone X facial mocap workflow as well.
Like you, I'll be exporting to unity and have no use for iclone except as a medium for recording the animations so I looked for cheaper alternatives.

I'll share some of my findings here:

I'm not sure if the CC3 standard facial blendshapes would be compatible with other tools so you might need to use "Face It" $60 from the blender addon store to help setup the required blendshapes.
https://blendermarket.com/products/faceit

Then you can use a tool from the app store called "Face Cap" ($59) https://www.bannaflak.com/face-cap/index.html
Faceit was designed to work with Face Cap so it will definitely work.

If you wish to do livestreaming of the Face Mocap session in unity you can use "Face Cap's" live mode https://www.bannaflak.com/face-cap/livemode.html#1.2.3
There are also alternatives to livestream into Unreal or Blender (http://www.jpfep.net/pages/addroutes/)

If you want to try a Free Open Source alternative for Facial mocap you can try out BMC (note it can't livestream into unity directly)
https://apps.apple.com/app/id1494579713
https://github.com/pjardin/Blender-Motion-Capture

Comes with a Free Blender Addon to playback the recorded mocaps.
https://github.com/pjardin/iphoneCaptureAddon

However, the tutorial videos and documentation for this free solution are very poor so you will have to figure out a lot of it yourself.
https://www.youtube.com/watch?v=9p_NSk-Vd4c

Fun to explore but IMHO not worth the extra effort if you can afford the licences for  "Face Cap $59" and "Face It $60"

Hope that helps.

Cheers,
Kaen




Reading This Topic

0 active, 0 guests, 0 members, 0 anonymous.
No members currently viewing this topic!