Profile Picture

How to Get Facial Mocap into Unity Export?

Posted By XanderDavis 3 Years Ago
You don't have permission to rate!
Author
Message
XanderDavis
XanderDavis
Posted 3 Years Ago
View Quick Profile
Senior Member

Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)

Group: Forum Members
Last Active: 3 Years Ago
Posts: 5, Visits: 29
Hi,

I am able to export an iClone Motion track with Facial Mocap as MotionPlus to Unity FBX from Collect Clip, but it doesn't actually seem to include any of the Facial Mocap I have from LIVE Face that looks fine in iClone.  When I try to send this to 3DXchange, I have the same issue.

Can someone run me through how to get Facial Mocap into Unity?  This is super critical to our goals of using Reallusion software for our character creation and animation solution for Unity.

Thanks


www.xanderdavis.studio
www.astrogun.com

XanderDavis
XanderDavis
Posted 3 Years Ago
View Quick Profile
Senior Member

Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)

Group: Forum Members
Last Active: 3 Years Ago
Posts: 5, Visits: 29
It looks like the mods moved this post from where I originally made it in the iClone forum to Character Creator... which isn't accurate.  In fact, it seems CC alone decidedly does not work at all for this purpose, as far as I can tell.  Exporting directly from CC does not get me Facial Mocap, which has to be done in iClone, nor does it get me ARKit Expressions used for that.  Using a character exported directly from CC before Mocap is made, importing that into Unity, and then later mapping on a separate motion-only export from iClone / CC into Unity also doesn't work perfectly.  Only base non-expression facial animation remains in tact with auto blinking.  It even fails to automatically rig up the eyes.

The issue is how to export the Facial Mocap performance from iClone so I can get it into Unity, just as I wrote in the OP.  This is an iClone topic.

After some more experimentation since this post was made (and the gap between approval), I got it to work.

Part of why I asked this is because original characters created with Headshot in CC and then brought into iClone would fail to export to FBX.  So I was stuck trying to set ranges for the Collected Clip layer and exported MotionPlus files directly to then import into CC and export as motion-only from there or export MotionPlus to 3DXchange.  Then I would apply that in Unity into a CC-only exported character.  All elements brought in via the Unity CC_Assets pipeline.  Both methods apparently fail to translate facial expressions, in part.  Head movement seemed to be the only thing in tact.  No blinking.  No facial muscle capture.  What made this confusing is on top of everything else the characters would look as-intended with facial mocap in iClone, but not upon MotionPlus export and Unity import.  So what was missing?

Today, I started a new document directly in iClone with stock Kevin and did a facial mocap test.  I was able to export him out as an FBX for Unity.  I figured there was something unique about the avatar I wasn't aware of with my Headshot-CC created character.  Some Facial Expressions / morphs mapping I'd need to apply?  That nothing is telling me about?  I have to scour the Internet and YouTube to get a clue on this?

I opened up the same original Headshot made character and sent him to iClone.  Facial Mocap could connect to LIVE Face but the character wouldn't move during Preview or Record.  Back in CC, I applied CC3 Motion and Expressions > _Default.iTalk to the character based on some other random forum post I found from like 5 years ago.  I'm not sure if this did anything but when I sent him back to iClone, I had to again apply ARKit Expressions, but then it worked.  I've gotten the character to work with facial mocap before but then ran into the 'Failed to Export to FBX' error.

Well during this attempt, for whatever reason, I was able to export the character with facial mocap and audio recorded to FBX for Unity via iClone directly.  No 'Failed to Export FBX' error.  Brought it into Unity and it worked perfectly.  I also have no idea how Unity is reading facial morphs (like muscles) / expressions and applying them because all I can see in Unity are Humanoid rigging and bones.

I'm not exactly sure what's going on but it was a bit frustrating to run into this when, honestly, it should just work.  It's also not comforting to discover a method that apparently sometimes works?  We chose Reallusion specifically as a solution for Unity that could be our Unity alternative for Unreal's exclusive Metahuman Creator.  We would love it if Reallusion doubles, triples down on direct integration with Unity, like the Unreal Live Link plug-in.  CC3 characters should be ready to go for Mocap without iClone having to apply anything to it and especially to get it to successfully Export to FBX for Unity or better yet to just automagically send to Unity live.  And there should absolutely be up to date tutorials for doing exactly this... which to us is the entire point of using any of these tools at all.  I mean, why else would we be using CC if not to make characters that work with iClone in a few clicks?  And we want iClone for its Mocap tools.

All of that being said, we are generally very happy with the Reallusion software, but, the point is, this could be made a lot less tedious and confusing yet.

Anyway, relived to have got it working.  If anyone can shed some light on this, that'd not only be helpful to me but others I'm sure.


www.xanderdavis.studio
www.astrogun.com

Edited
3 Years Ago by XanderDavis
Peter (RL)
Peter (RL)
Posted 3 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)

Group: Administrators
Last Active: 15 hours ago
Posts: 23.1K, Visits: 36.5K
Hi Xander

I'm pleased to hear you have been able to resolve the issues you were having. Smile

To clarify the reason for moving your post, if your question or issue relates to exporting to Unity or Unreal then this is the forum it should be in. Of course if the issue is an iClone usage or technical issue then that's where it should be posted but questions related to exporting to 3D engines go in this forum.

And just to correct one thing you said, Character Creator 3 Pipeline can certainly export facial and body animation and is in fact the recommended method to use. You would obviously need to create and save your animation data in iClone, but then you can export the animations from CC3 Pipeline along with your character.

                                                                

Peter
Forum Administrator

www.reallusion.com


XanderDavis
XanderDavis
Posted 3 Years Ago
View Quick Profile
Senior Member

Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)

Group: Forum Members
Last Active: 3 Years Ago
Posts: 5, Visits: 29
Yeah pardon the frustration hehe Tongue

Okay-- can you explain why CC3 is the recommended method to use for exporting facial and body animation and not... y'know... iClone that does facial and body animation?  Perfect example of how there's UX that's counter-intuitive here and requires inside-baseball knowledge to get working...

Also when I did it by that method, the full facial Mocap didn't translate out and into Unity.  It seemed to only get head movement but not muscle (including mocapped blinking).  The only way I've gotten full facial mocap out and into Unity is by exporting as FBX out of iClone.

Especially worth noting, for any CC3 generated characters I've made and sent to iClone and given ARKit Expressions for mocap, they will give me an FBX export fail error unless, as I've discovered, I go to Animation > Face Puppet > and click on 01_Universal Closed.  Then FBX export will succeed.  Which seems really random to know to do that.  Why is that?


www.xanderdavis.studio
www.astrogun.com

Edited
3 Years Ago by XanderDavis
Peter (RL)
Peter (RL)
Posted 3 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)Distinguished Member (123.3K reputation)

Group: Administrators
Last Active: 15 hours ago
Posts: 23.1K, Visits: 36.5K
XanderDavis (6/20/2021)

Okay-- can you explain why CC3 is the recommended method to use for exporting facial and body animation and not... y'know... iClone that does facial and body animation?  Perfect example of how there's UX that's counter-intuitive here and requires inside-baseball knowledge to get working...


iClone on its own does not have any export features but instead it utilises the export features of 3DXchange 7 Pipeline which must be owned to be able to export from iClone. 3DXchange itself is now quite an old product (still 32bit) and lacks some of the advanced export features found in Character Creator 3 Pipeline (LOD for example).

So generally for characters and animations we recommend Character Creator 3 Pipeline but of course if you prefer to export from iClone and it works for you then there is no reason to change your workflow. Smile



                                                                

Peter
Forum Administrator

www.reallusion.com


XanderDavis
XanderDavis
Posted 3 Years Ago
View Quick Profile
Senior Member

Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)Senior Member (251 reputation)

Group: Forum Members
Last Active: 3 Years Ago
Posts: 5, Visits: 29
So the problem with bringing Collected Clip MotionPlus into Character Creator and exporting it out from there, I'm not getting a lot of the facial Expressions data (namely, the data that was mocapped) once brought into Unity.  It's only retaining head movement, but doing nothing for muscle (including blinking).  Eye movement is coming through, but only after I rig up the eyes in Unity, which import doesn't automatically do unless we're importing a total FBX exported from iClone instead.

Conversely, if I export via iClone (not using Collect Clip but instead selecting the character and going File > Export > FBX > Unity), then it does successfully export the facial expressions that were mocapped.

What do you think is going on with that?  Can you describe how to export from Character Creator to make sure it gets all Expressions, namely facial mocap / muscle?

I appreciate you taking the time to help us sort this out.


www.xanderdavis.studio
www.astrogun.com

Edited
3 Years Ago by XanderDavis
tsterk
tsterk
Posted 3 Years Ago
View Quick Profile
New Member

New Member (1 reputation)New Member (1 reputation)New Member (1 reputation)New Member (1 reputation)New Member (1 reputation)New Member (1 reputation)New Member (1 reputation)New Member (1 reputation)New Member (1 reputation)

Group: Forum Members
Last Active: 3 Years Ago
Posts: 1, Visits: 7
Hi XanderDavis,
Did you find a workflow that allows you to use the facial mocap "seemlessly" in Unity?
I was planning on buying the digital human + animation bundle, but after seeing this post I am a bit hesitant as I would primarily be using it for facial capture in Unity.

@Peter, maybe you can clarify this issue a bit for a prospective customer Smile



Reading This Topic