Profile Picture

Unreal Live Link Face FREE / AI Creates Realistic and Precise Facial Animation From Audio

Posted By ZenMuse Last Year
Rated 5 stars based on 1 vote.

Unreal Live Link Face FREE / AI Creates Realistic and Precise Facial...

Author
Message
ZenMuse
Praise.png
ZenMuse
Posted Last Year
View Quick Profile
Senior Member

Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 15, Visits: 45
We are all big fans of iClone but for Facial MoCap and animation solutions we all want to make things better and easier to integrate without expensive upgrades and plugins that still struggle with realistic Facial MoCap.   
Generally, it takes a very long time to perfect Facial MoCap and the results are not completely realistic even with hours and hours of tweaking and adjustments and all the plugins Kai was struggling in his latest webinar. 

So inspired by this latest Webinar with Kai on Facial Mocap Tips & Techniques 
https://www.youtube.com/watch?v=RkpUQ0f0-Yg 
I started looking around for further developments with Facial MoCap. There has to be an easier solution to Facial MoCap that gets more realistic results.

Then I came across this - WOW..!!!!

AI Creates Facial Animation From Audio | Two Minute Papers #185 
https://www.youtube.com/watch?v=ZtP3gl_2kBM
It is a research paper created in 2017 so I can imagine things have moved on since then. He was working with Nvidia so perhaps it is the basis of Unreal Live Link Face?

Miranda (RL) 

Tip - Maybe it would be an idea for the Dev Team at Reallusion to create a free App closer with Unreal Live Link Face (free App) and revamp the Facial Mocap solutions for iClone so the Audio to Visme results are more realistic and accurate. 

Please Make it free like Unreal Live Link Face iOS App and watch the users flood into iClone and the world of animation. Keeping them in iClone for a complete animation solution.

https://www.unrealengine.com/en-US/blog/new-live-link-face-ios-app-now-available-for-real-time-facial-capture-with-unreal-engine

Here is Dave Workman from Cinematographer Database testing the free iOS app in Unreal Engine - Live. 
https://www.youtube.com/watch?v=-2ZDN1in4ss

Thanks RL - big changes ahead for iClone and in animation.




ZenMuse
ZenMuse
Posted Last Year
View Quick Profile
Senior Member

Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 15, Visits: 45
More great reference for Facial MoCap. Always good to see where the competition is heading. AI and machine learning seem to be a focus. Check out Cubic Motion and Dynamixyz for realism.

https://la.disneyresearch.com/publication/realtimeperformancecapture/ 

Specifically Films & Games Service (good benchmark) https://cubicmotion.com/ and https://www.youtube.com/watch?v=mIFiftCLQsc

https://www.dynamixyz.com/ and https://www.dynamixyz.com/support  great demo reel https://www.dynamixyz.com/demoreel

https://www.unrealengine.com/en-US/blog/new-live-link-face-ios-app-now-available-for-real-time-facial-capture-with-unreal-engine

Unreal Live Face (FREE) App https://apps.apple.com/app/id1495370836

Sean Bean Deepfake https://www.fxguide.com/fxfeatured/pinscreens-advanced-face-ai-neural-rendering/


Glassbox (
connected to Faceware) 

https://glassboxtech.com/products/live-client 

AI Creates Facial Animation From Audio 
https://www.youtube.com/watch?v=ZtP3gl_2kBM

Please post more links to this post if you know of any more inspiring reference.

Unshattered
Unshattered
Posted Last Year
View Quick Profile
New Member

New Member (49 reputation)New Member (49 reputation)New Member (49 reputation)New Member (49 reputation)New Member (49 reputation)New Member (49 reputation)New Member (49 reputation)New Member (49 reputation)New Member (49 reputation)

Group: Forum Members
Last Active: Last Month
Posts: 5, Visits: 118
Unreal recently acquired Cubit Motion. As with other technology they have acquired they will undoubtly be making it accessible for free to UE4 users. The iOS face app is just the start. https://www.google.com/amp/s/venturebeat.com/2020/03/12/epic-games-acquires-facial-animation-technology-maker-cubic-motion/amp/ 

Edited
Last Year by Unshattered
animagic
animagic
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (24.3K reputation)Distinguished Member (24.3K reputation)Distinguished Member (24.3K reputation)Distinguished Member (24.3K reputation)Distinguished Member (24.3K reputation)Distinguished Member (24.3K reputation)Distinguished Member (24.3K reputation)Distinguished Member (24.3K reputation)Distinguished Member (24.3K reputation)

Group: Forum Members
Last Active: 44 minutes ago
Posts: 13.1K, Visits: 23.2K
There is big difference between Unreal's and RL's financial position, so demanding free this and that may not be feasible. Also, RL has wisely chosen to stay out of Mocap development as this has not been too successful in the past.


https://forum.reallusion.com/uploads/images/1f3bed13-1788-45e1-9ace-4fd0.jpg


ZenMuse
ZenMuse
Posted Last Year
View Quick Profile
Senior Member

Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 15, Visits: 45
Thanks for the reply animagic . Yes of course I understand the Unreal/Epic Games and RL business models are different. Most users don´t look at that, they just want to get animating quickly with great results. So they go to Unreal because the software is free and so is the Unreal Face App. You still need an iPhone X though.  As the market changes business models have to adapt.

So there is no basic Webcam facial mocap feature included in iClone for free. At the moment you require an investment of a Mocap Live plugin and a Live Face Profile, that´s $500* just to do facial mocap and then you need a $800* iPhone X.  *more or less depending on offers/deals.

iClone could be great with a simple basic 2D webcam addition, similar to the 2D Mocap for Cartoon Animator ( a RL product) or Adobe Character Animator. No iPhone depth camera is needed for either. 
Adobe Character Animator works with a webcam very accurately by the way.

With a free webcam facial mocap solution we can all get on with enjoying the benefits of iClone and be animating and creating great sequences and animating rather than than pushing Visme keyframes around to get mediocre results. It makes the output from users look bad.
This simple addition would make the output of work from the community stronger and more users would use want to dive into CC3 and iClone. Both excellent character software. Giant leaps forward with every release. so h/t to the Dev Team for that.
Edited
Last Year by ZenMuse
Bassline303
Bassline303
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (4.9K reputation)Distinguished Member (4.9K reputation)Distinguished Member (4.9K reputation)Distinguished Member (4.9K reputation)Distinguished Member (4.9K reputation)Distinguished Member (4.9K reputation)Distinguished Member (4.9K reputation)Distinguished Member (4.9K reputation)Distinguished Member (4.9K reputation)

Group: Forum Members
Last Active: 7 hours ago
Posts: 1.9K, Visits: 8.7K
You can have the best and most expensive facial capture system , but most important are the morph targets / blend shapes of your Character , you will need dynamic wrinkles , animated muscles a.s.o  to get believable facial expressions. 
ZenMuse
ZenMuse
Posted Last Year
View Quick Profile
Senior Member

Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)Senior Member (252 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 15, Visits: 45
Absolutely agree on that Bassline303  

Looking forward to the release of the SkinGen and CC3. 
Besides wrinkles and other features, realistic Eyes for Real-time Digital Human is the detail that will set CC3 apart. 
https://www.youtube.com/watch?v=XDGpgo2ONvo

The Dev Team always deliver great advances on each release and that is not easy. I am all in with CC3.



Reading This Topic