Hello guys, today Reallusion released this scanned pack for CC3 https://www.reallusion.com/ContentStore/ccdproduct.html?contentid=ACC300DIENU012420181115009 This package provides a new impulse for the motivation to add facial animations, with voice to a project. The teaser video https://youtu.be/KaCOYXtXhxk

As a Unreal Engine developer I wonder what would be the best way to facilitate imported facial morph targets, make them work via blueprints, and how exactly is the best way to sync them up with voice.

Approaches
I could imagine to use the UE Cinematics feature for cut scenes, with facial animations, syncing voice on a manual adjusted per keyframe level, but this appears to be very tedious, unless this is only very brief?

 

 

Pre-made
For pre-made facial sync there is an existing tool called FaceFX, however it costs $900, includes a UE plugin (not tested) https://www.facefx.com I am aware Reallusion has a motion capture solution, but I do not know about export/pipeline, will look into this later, but very likely also a pre.-made approach (rendering animations outside of UE) 

Real time
For manual setting up face emotions (and eventually triggering them via Anim Notifies), there is this example https://youtu.be/5wWGaJL_Q_8?t=38m
There are two UE plugins for realtime approach but do not include real face morphings / lip sync https://forums.unrealengine.com/unreal-engine/marketplace/1553622-ultimate-dialogue-plugin - https://www.unrealengine.com/marketplace/text-to-lip-sync