|
Author
|
Message
|
|
Miranda (RL)
|
Miranda (RL)
Posted 3 Years Ago
|
|
Group: Administrators
Last Active: 2 Years Ago
Posts: 1.8K,
Visits: 11.2K
|
Hello everyone, When we first introduced Audio2Face in late July, we mentioned that we are working on the solution to bring Audio2Face animation back to iClone. Now it's time for everyone to experience it! What is Audio2Face?Omniverse Audio2Face is an AI-powered application that generates expressive facial animation from just an audio source. Audio2Face 2021.2.6 supports Character Creator CC3 Base+ and Game Base presets, which largely simplifies the wrap process for facial and lip-sync animation creation.
In ver. 2021.3.0, Audio2Face supports exporting animation as blendshapes (.json). This guide will also offer a solution for loading back the json data onto CC characters in iClone. - With the ability to bake Audio2Face blendshapes and export it back to iClone, and in combination with iClone's native facial animation tools, users now have another option for lip-syncing besides AccuLips. On top of this, comes the ability to export the character animations to other mainstream 3D engines such as Blender, Unreal Engine, Unity, etc.
- For Omniverse users, here is a total solution for full facial and body animation in combination with Audio2Face animation. Users can send the result to Omniverse via the upcoming iClone Omniverse Connector.
Please note that this is a Beta solution for sneak preview. The official solution will be released with iClone 8.Required Software & Files - Character Creator 3.44 or above
- iClone 7.93 or above
- iClone Python script for loading Audio2Face blendshape JSON
*Script is updated on Nov 4th for UI optimization. - Omniverse Audio2Face 2021.3.0
- Omniverse USD Sample Model (CC3+ Neutral Base) for generating animation and blendshape baking
*This file is no longer needed while using CC version 3.44.
*In the previous CC version 3.43, the blendshapes of the exported A2F head mesh are not named clearly, which makes it difficult to be identified in A2F. Before CC v3.44 releases, you can use the USD file that we provide as the sample model used in A2F, for generating animation and blendshape baking.
Workflow
Current version - iClone 7.93 has native USD export function: (3DXChange 7 Pipeline required)
 Previous version:
 Video Demo (no dubbing)*Recorded by using of CC3.43 and iClone 7.92: Part 1: Audio2Face Character Transfer Part 2: Audio2Face Blendshape Conversion (Viking)
Part 3: Using Sample Model for Blendshape Export (JSON) Part 4: Importing JSON, and Adjusting Strength Smooth in iClone Step-by-Step Guide
Please check THIS DOCUMENT for details. Reallusion Team
Edited
3 Years Ago by
Miranda (RL)
|
|
|
|
|
Jeasy
|
|
|
Group: Forum Members
Last Active: 3 days ago
Posts: 174,
Visits: 2.4K
|
This is amazing news Miranda (RL), can't wait to check it out  Thanks!
Edited
3 Years Ago by
Miranda (RL)
|
|
|
|
|
GARCIAGA
|
GARCIAGA
Posted 3 Years Ago
|
|
Group: Forum Members
Last Active: Last Month
Posts: 10,
Visits: 206
|
It looks great, wich one will be better Accu-Lips or Audio2Face??
|
|
|
|
|
mtakerkart
|
mtakerkart
Posted 3 Years Ago
|
|
Group: Forum Members
Last Active: 4 hours ago
Posts: 3.1K,
Visits: 28.1K
|
I would like some help to test Audio2face with the french language. Reallusion's example offers 6 languages but not French ...
I don't have an rtx graphics card which is mandatory to use it. If it is accurate I would buy the smaller rtx card because it does not require resources. I put below an audio file of 20 seconds for the test. can you take a screenshot in video?
|
|
|
|
|
ttamd
|
|
|
Group: Forum Members
Last Active: 2 days ago
Posts: 6,
Visits: 214
|
Amazing, thanks, will digg into this.
|
|
|
|
|
Miranda (RL)
|
Miranda (RL)
Posted 3 Years Ago
|
|
Group: Administrators
Last Active: 2 Years Ago
Posts: 1.8K,
Visits: 11.2K
|
Hi GARCIAGA, They are developed by using different algorithms. In my personal opinion, AccuLips provides flexibility in generating and fine tuning (e.g. word dictionary), while Audio2Face supports multi-languages, and more automatic functions will be introduced in the future. When Audio2Face is getting mature, it's definitely a great tool to work with. GARCIAGA (9/20/2021)
It looks great, wich one will be better Accu-Lips or Audio2Face??
|
|
|
|
|
Miranda (RL)
|
Miranda (RL)
Posted 3 Years Ago
|
|
Group: Administrators
Last Active: 2 Years Ago
Posts: 1.8K,
Visits: 11.2K
|
@Jeasy, ttamd, Welcome to show what you come up with. @Mark, This is French. Doesn't sound like?  https://youtu.be/Q64C4CWOV8s?t=17
|
|
|
|
|
rosuckmedia
|
rosuckmedia
Posted 3 Years Ago
|
|
Group: Forum Members
Last Active: 2 hours ago
Posts: 3.1K,
Visits: 4.7K
|
I see that the German language file is also missing.😢😢 Will it be added later???
Greetings rosuckmedia
|
|
|
|
|
Miranda (RL)
|
Miranda (RL)
Posted 3 Years Ago
|
|
Group: Administrators
Last Active: 2 Years Ago
Posts: 1.8K,
Visits: 11.2K
|
Hi rosuckmedia, Do you have a chance to try it? Audio2Face shall work with any languages, and Omniverse Team continually updating with more and more languages. rosuckmedia (9/22/2021)
I see that the German language file is also missing.😢😢 Will it be added later???
Greetings rosuckmedia
|
|
|
|
|
rosuckmedia
|
rosuckmedia
Posted 3 Years Ago
|
|
Group: Forum Members
Last Active: 2 hours ago
Posts: 3.1K,
Visits: 4.7K
|
Hi Miranda, Thanks for your quick reply.😊 I need to reinstall Omniverse first, then I will try it. I have a RTX 2080 should actually work.
Greetings rosuckmedia
|
|
|
|