Export expressions from CC3 or DAZ figures to Unity does not work


https://forum.reallusion.com/Topic395716.aspx
Print Topic | Close Window

By lollosone - 6 Years Ago
Hi,
Still in a quest to figure out if I can use CC3 for Unity; and I did get another roadblock.

In DAZ studio I can simply export the morphs I have for the various expressions (happy, sad, scared, angry and so on); and in Unity they are available at runtime, so I can code them in the application logic and make changes at runtime.
I can't find a way to do the same with the CC exported figures. The exported figure has blendshapes but there is nothing that set multiple parameters to make an expression; and this is confusing because I can set an expression in CC3; but the blendshapes exported do not have those.
Is this expected or a bug?

Also for DAZ figures imported in CC3, there is no way to export actual morphs from DAZ figures. From what I can tell, since baking or exporting has no effect when I export the figure to Unity; I assume that the only morphs exported are the ones from CC3? Which as mentioned above, lack the expressions morphs sadly; so to use them in Unity I have to write down ALL the single morphs for "happy"; and modify the CC3 exported morphs in Unity one by one. Which is borderline insane if you ask me Sad Thanks
By OlanderDA - 6 Years Ago
From all the testing in Beta and up to current only the Phonemes are exported and the names are changed a bit so you have to figure out which one does what. With some scripting you can get the facial emotion/expressions from those. Lip Sync Pro makes audio voice to mouth movement (including the Jaw bone) work well.

And you are correct...this is borderline...it IS insane. I cannot understand why RL does not have blendshapes other than Phonemes exportable...even a list of 10-25 to select from. I have a suspicion why but I will leave it that.

We are going through this right now....almost have this mess solved. Here is my bitter rant on why....read after what I say below to tie it together.
https://forum.reallusion.com/394058/iClone7-and-3DXChange7-I-have-Had-Enough-and-WhyGame-Development-Related-Pipeline

So CC3 or DAZ right? Since we are talking CC3 it can work very well in Unity. It will take A LOT of extra work to make it work....you will have to make a ton different body/head types since there are no 'Body Blends' in Unity. So your character systems will have to switch out entire bodies and heads in character creation (NPCs or PCs). So if you would have 4 body types and 30 heads you would have 4x30=120 body prefabs (each with LOD0/1/2). So instead of 2 bodies (1 Female and 1 Male) with clothing and blends recipes that get saved you would basically set a body and clothing switcher.

Advantage? The 'Switcher Method' is actually better performance. With the Blends setting the Vertex of the mesh....every frame...there is a bit of overhead to set them then set up the lighting data so the shaders will look right. Unity is still doing this really fast but having a lot of blends in a Skinned Mesh Renderer does affect performance....especially if one uses the same tech on NPCs. Very convenient for design but the slug on performance needs a much closer look. So not having to go through a large list of blends each frame helps out enough to make it worth a look.

CC3 is easier to animate (much better re-targetting via 3DXChange7 and iClone7) than DAZ models...even G1 & G2 which is fairly close to a standard mocap rig but with some strange bone positions in a couple places. CC3 also has better licensing. Lastly, since the Remesher in CC3 is 'half baked' and worthless for Real-Time Game Engine shaders....you will need to do the LODs by hand in CC3. This works and the end result is what is needed.

So.....if you are thinking of sticking with CC3 then you will need to make your own 'Switcher' system and also figure out which exported CC3 blends do what (also you will need to figure out the actual scale needed too).

Now you can go read my bitter rant/complaint. BigGrin  Or this guy has some great videos that came up with the same results...with lots of other testing as well. Great vids!
*also in my post above
VirtualRealities.school  https://www.youtube.com/channel/UCVnaxNPvGCOlj4WW6aj1MNA/videos
Character Creator 3 has finally arrived: First Impressions: https://www.youtube.com/watch?v=CZp0a1kbheA
Character Creator 3: Using LODs and Merge Materials in Unity: https://www.youtube.com/watch?v=sCypSg3Kz0I

Hope that helps.
By lollosone - 6 Years Ago
Thanks for the reply Smile I am still learning the Reallusion way to deal with Unity; but from what I can tell, it is clear to me that most of the focus is on iClone and whatever is left, is for Unreal.

Maybe Unity is considered second class citizen for them; due to the fact that things never look good in Unity, out of the box; or maybe Unreal is more user friendly...can't say; I only use Unity so cannot make a real educated comparison with my knowledge of both engines.
What I would like is a list of phonemes or expressions; that you can export. They did a good job in setting up some expressions; which works fine for standard animations; but would be nice to export like you do in DAZ, only the blendshapes you want; so I can save efforts when doing facial animation (I do not use any other Reallusion product, so the face animation is all done "by hand" old way, in a programmatical way).

Regarding LOD; I did give up the idea to use LOD feature because it mess up with hair and transparent materials...until they fix CC3 to export in LOD using different materials; that way is shut. The workaround is to reduce the polygon number of the clothes and everything else by doing the accessory LOD; but the main figure retain the full poly number, so once exported, I need to do a pass in Blender to decimate that. So much extra work that could be avoided with a simple change in how the LOD feature works. For any character that does not have transparency in the hair or clothes; it works out of the box; but this is fundamentally a rare case, and limit severely your creative freedom. I can say that they will change the LOD at 30%; which means most likely it will never happen.
For now  I am forced to use DAZ for swapping morphs and clothes; so a character creation in Unity; using CC3 meshes is still something that won't ever happen. I will have to try your approach; I usually did build a large set of meshes and enable/disable the mesh renderer in Unity; to make it visible or invisible. From what I can tell, the draw calls are related to the visibility of the mesh renderer component, so as long as clothes or other props remain deselected; they won't be slowing down the application.

I do see value in CC; although not 200 dollars value; considering the limitations on blendshapes, unity exported figures and the LOD that does not work as one would expect; but if the devs make the changes, it will be really nice to use CC3 for Unity content.
By Rampa - 6 Years Ago
Try converting to Game bone in the modify panel. It'll half the character poly-count. Then make sure to hide any mes that is not visible and tick the "delete hidden mesh" box when you export.
By lollosone - 6 Years Ago
Rampa (11/26/2018)
Try converting to Game bone in the modify panel. It'll half the character poly-count. Then make sure to hide any mes that is not visible and tick the "delete hidden mesh" box when you export.



You mean this?

https://forum.reallusion.com/uploads/images/6b077be3-786b-46a6-8afc-48bd.jpg

I can't hide the mesh parts because I do swap clothes; so I always need to have most of the body available for when some clothes are swapped Sad

By Rampa - 6 Years Ago
That's it!
It'll drop the character to a lower mesh, yet retain all morphs and blendshapes.
By OlanderDA - 6 Years Ago
Rampa (11/26/2018)
That's it!
It'll drop the character to a lower mesh, yet retain all morphs and blendshapes.

Come on Rampa...this is only partly true. You should have done the test yourself before giving this advice. This feature does not work properly and has mesh and bone issues.
Why this issue still exists is beyond me.

.....
So here is the REAL truth about the Game UV and exporting Body Morphs....Expression Blendshapes also tested here. Just another half-baked feature in CC3.
Also note...ALL the bone names and structure are different in the Game UV which has an affect on the animations coming out of iClone7...but we are talking about CC3 here....not iClone7....but...getting this CC3 Game UV body to animate without fixing the animations is a hard swallow to customers. I tested it...I know. Test it if curious and see.
And of course here is the proof in pics. =)

Game UV does not work properly and is a Non-Path Workflow to Game Development/Engine Real-Time Rendering. It just isn't until RL fixes it.
So the best Option is LOD Manually using  LOD0 = 100%   LOD1 = 70%   LOD2  = 30%  LOD3  =  15%  LOD4 is not necessary....simply cull at 5-10%
**Always LOD from the LOD0 to the lower LOD# or things break. All LOD systems have this issue.
With this we get matching mesh reduction/decimation with clothing items as well in a consistent method....as should be in the InstaLOD....which is also half-baked.

Test (about 40min includng screen pics)....CC3 Default Exported (FBX) with Bra and Panty. Then Convert to Game UV the Default CC3 and Exported (FBX)
*Remember to UNCHECK the Embed Textures when going to a Game Engine or they will not work properly.
- All textures set to Maximum 2048 (some export smaller...which is fine)
- Default UV = 29 textures 17.9mb
- Default UV = 14 Materials
- Default UV = 35334 (just the Body), 35654 Total
- Game UV = 30 textures 41.8mb  <= Huh? I did not compare which is the extra texture but the file space got larger during the baking.
- Game UV = 15 Materials <= Ummm....Okay?
- Game UV = 19294 (just the Body), 19614 Total

You can see in the 2nd pic the mesh reduction for everything in the scene....but it is ONLY the character mesh....not the Bra and Panty.
https://forum.reallusion.com/uploads/images/97959548-a5ce-4574-a3ba-8b72.png
https://forum.reallusion.com/uploads/images/ecac8419-1f05-45c6-bb4c-3532.png
https://forum.reallusion.com/uploads/images/11301b73-6b97-4f5f-aecf-c610.png

Default Textures on Export (Unity 3D Template with no adjustments except Unchecking Embed Textures) JPG format....should be PNG by default
https://forum.reallusion.com/uploads/images/b7bb3f40-af03-4822-8827-d570.png
Game UV Textures on Export PNG format on Bake...so Unity 3D Template is JPG but a Convert to Game UV is PNG on baking? Should be PNG/PNG
https://forum.reallusion.com/uploads/images/03c3be74-961a-4670-9cbd-9072.png
Imported in Unity. Materials have to be extracted to fix the Transparent setting in the materials (most of them not all). Normals are also messed up and need to be fixed (never has been fixed).
https://forum.reallusion.com/uploads/images/6deae455-2fff-4c73-87ef-c148.png

After fixing the Normals. Upper body and Bra and Panty are all fine now....but the Game UV legs have serious issues.
https://forum.reallusion.com/uploads/images/488fc49b-3e68-4e99-af4e-bef2.png
https://forum.reallusion.com/uploads/images/f8e7b067-9cac-4889-9ad9-9ab6.png

Here you can see the mesh reduction is actually well done on the Body. The Bra and Panty are untouched.
https://forum.reallusion.com/uploads/images/1e21d2f5-055f-41a1-b375-35f6.png
https://forum.reallusion.com/uploads/images/f4c7e1e5-3107-41ca-8bda-607d.png

And here you can see the Skinned Mesh Renderer has identical Phoneme and Facial Blends are present but no Body or Expression blends. To make expressions one would need to mix and match in C# to set a 'Recipe' and LERP in each blend based upon the Recipe 0-100.0% scaling....example....adding Wide to an expression you only need 12.5% so the Scaling is 0.0 to 12.5% = 100.0% over  0.000 to 0.250s. If you have 10 blends in an expression then all 10 would need to be done this way.
Another method is using Morph Mixer and/or PolyMorpher from the Asset Store and make new Morphs from these combinations and simply setting 0.0 to 1.0 (100.0%)
https://forum.reallusion.com/uploads/images/ae2795bc-d3e8-48b1-b792-bb92.png

And the bones. WTF and Why?
https://forum.reallusion.com/uploads/images/c172886b-b8ac-43c5-aee7-760c.png

By e1_scheer - 5 Years Ago
Hi,

We've decided to use CC3 for our game character pipeline and this thread now has me pretty worried.
Have any of the issues raised by Oleander in this thread been adressed or have temporary workarounds?

Thanks,

Oddbody
By Rampa - 5 Years Ago
Converting to Game Bone effects the avatar body, not the clothing. The clothing can be reduced separately. Or you can reduce and combine both body and clothing together on export using InstaLOD.
By Alexxander - 5 Years Ago
Vote this
https://www.reallusion.com/FeedBackTracker/Issue/Export-expressions-into-FBX-as-additional-blendshapes