Unreal Live Link Face FREE / AI Creates Realistic and Precise Facial Animation From Audio

Print Topic | Close Window

By ZenMuse - Last Year
We are all big fans of iClone but for Facial MoCap and animation solutions we all want to make things better and easier to integrate without expensive upgrades and plugins that still struggle with realistic Facial MoCap.   
Generally, it takes a very long time to perfect Facial MoCap and the results are not completely realistic even with hours and hours of tweaking and adjustments and all the plugins Kai was struggling in his latest webinar. 

So inspired by this latest Webinar with Kai on Facial Mocap Tips & Techniques 
I started looking around for further developments with Facial MoCap. There has to be an easier solution to Facial MoCap that gets more realistic results.

Then I came across this - WOW..!!!!

AI Creates Facial Animation From Audio | Two Minute Papers #185 
It is a research paper created in 2017 so I can imagine things have moved on since then. He was working with Nvidia so perhaps it is the basis of Unreal Live Link Face?

Miranda (RL) 

Tip - Maybe it would be an idea for the Dev Team at Reallusion to create a free App closer with Unreal Live Link Face (free App) and revamp the Facial Mocap solutions for iClone so the Audio to Visme results are more realistic and accurate. 

Please Make it free like Unreal Live Link Face iOS App and watch the users flood into iClone and the world of animation. Keeping them in iClone for a complete animation solution.


Here is Dave Workman from Cinematographer Database testing the free iOS app in Unreal Engine - Live. 

Thanks RL - big changes ahead for iClone and in animation.

By ZenMuse - Last Year
Thanks for the reply animagic . Yes of course I understand the Unreal/Epic Games and RL business models are different. Most users don´t look at that, they just want to get animating quickly with great results. So they go to Unreal because the software is free and so is the Unreal Face App. You still need an iPhone X though.  As the market changes business models have to adapt.

So there is no basic Webcam facial mocap feature included in iClone for free. At the moment you require an investment of a Mocap Live plugin and a Live Face Profile, that´s $500* just to do facial mocap and then you need a $800* iPhone X.  *more or less depending on offers/deals.

iClone could be great with a simple basic 2D webcam addition, similar to the 2D Mocap for Cartoon Animator ( a RL product) or Adobe Character Animator. No iPhone depth camera is needed for either. 
Adobe Character Animator works with a webcam very accurately by the way.

With a free webcam facial mocap solution we can all get on with enjoying the benefits of iClone and be animating and creating great sequences and animating rather than than pushing Visme keyframes around to get mediocre results. It makes the output from users look bad.
This simple addition would make the output of work from the community stronger and more users would use want to dive into CC3 and iClone. Both excellent character software. Giant leaps forward with every release. so h/t to the Dev Team for that.
By Bassline303 - Last Year
You can have the best and most expensive facial capture system , but most important are the morph targets / blend shapes of your Character , you will need dynamic wrinkles , animated muscles a.s.o  to get believable facial expressions. 
By ZenMuse - Last Year
More great reference for Facial MoCap. Always good to see where the competition is heading. AI and machine learning seem to be a focus. Check out Cubic Motion and Dynamixyz for realism.


Specifically Films & Games Service (good benchmark) https://cubicmotion.com/ and https://www.youtube.com/watch?v=mIFiftCLQsc

https://www.dynamixyz.com/ and https://www.dynamixyz.com/support  great demo reel https://www.dynamixyz.com/demoreel


Unreal Live Face (FREE) App https://apps.apple.com/app/id1495370836

Sean Bean Deepfake https://www.fxguide.com/fxfeatured/pinscreens-advanced-face-ai-neural-rendering/

Glassbox (
connected to Faceware) 


AI Creates Facial Animation From Audio 

Please post more links to this post if you know of any more inspiring reference.

By Unshattered - Last Year
Unreal recently acquired Cubit Motion. As with other technology they have acquired they will undoubtly be making it accessible for free to UE4 users. The iOS face app is just the start. https://www.google.com/amp/s/venturebeat.com/2020/03/12/epic-games-acquires-facial-animation-technology-maker-cubic-motion/amp/ 

By ZenMuse - Last Year
Absolutely agree on that Bassline303  

Looking forward to the release of the SkinGen and CC3. 
Besides wrinkles and other features, realistic Eyes for Real-time Digital Human is the detail that will set CC3 apart. 

The Dev Team always deliver great advances on each release and that is not easy. I am all in with CC3.
By animagic - Last Year
There is big difference between Unreal's and RL's financial position, so demanding free this and that may not be feasible. Also, RL has wisely chosen to stay out of Mocap development as this has not been too successful in the past.