Profile Picture

Replica Studios - AI Voice Actor Plugin for Lip-sync Animation

Posted By Charlene (RL) Last Year
You don't have permission to rate!

Replica Studios - AI Voice Actor Plugin for Lip-sync Animation

Author
Message
Charlene (RL)
Charlene (RL)
Posted Last Year
View Quick Profile
Senior Member

Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)

Group: Administrators
Last Active: 3 Months Ago
Posts: 11, Visits: 331
Dear Users,

AI Voice Actor is a collaborative project between Reallusion and Replica Studios. The goal is to create a fully automated text to animation pipeline with minimal user intervention. This trailblazing solution enables fast talking animation production without the need to hire voice talent, or even animators. In addition, designers can still fine-tune the AI generated results with iClone's Talking Styles, Viseme Editing, Face Mocap, Face Puppet, or Facial Key Editor.

*While iClone AccuLips provides text-to-speech animation for the lips, it does not support facial animations. Using Replica’s AI Voice Actor plugin, one can easily and conveniently apply different facial expressions on top of the lip motions to give a mood to the performance, thereby, offering more emotional impact to the audience.

Visit the product page at https://www.reallusion.com/iclone/ai-voice-replica/.

Required Tools:
  • iClone 7.92 or above.
  • Replica Plugin (Marketplace > iClone > Plug-in).
  • Replica Studios desktop application.

Installation Guide:


Getting Started:



Step 1: Go to Replicastudios.com, create an account and download Replica Studios Desktop Application.
https://forum.reallusion.com/uploads/images/1d3006b2-7a16-436c-8a64-7eb8.jpg

Step 2: Go to the Reallusion Marketplace and download the Replica Plugin.


Step 3: Uncompress the ReplicaStudios zip folder  https://forum.reallusion.com/uploads/images/598228dc-ea14-4d44-8492-f97e.jpg
Move the unzipped ReplicaStudios folder to C:\Program Files\Reallusion\iClone 7\Bin64\OpenPlugin
https://forum.reallusion.com/uploads/images/bab2c0b5-ac70-4b62-819c-ca1d.png

Step 4: Go to iClone and create a character in the scene.
https://forum.reallusion.com/uploads/images/c03cb8cf-5418-45f6-8aea-8937.jpg

Step 5: Click on Plugins > Replica Studios > Replica Link
https://forum.reallusion.com/uploads/images/8ae59b8b-116f-40bc-9616-833c.jpg

Step6: Sign in to Replica Studios desktop application and choose Sandbox.
https://forum.reallusion.com/uploads/images/d40e5042-d716-45d4-9639-b8de.jpg

Step7: Choose a speaker, enter the text for TTS conversion, and click on Save Take
https://forum.reallusion.com/uploads/images/86215cde-b61a-4130-b154-e841.jpg

Step 8: Choose the character in the scene and click on Replica Link > Select Avatar and notice the status becomes Connected.
Note: The character is named RD4, so it becomes evident in the dropdown list. 
https://forum.reallusion.com/uploads/images/e8d8b032-4108-46f4-9ed4-9a1d.jpg

Step 9: Export to iClone.
https://forum.reallusion.com/uploads/images/10b58770-132b-43a0-a0fa-e320.jpg

Step 10: Set the Style and adjust the Strength in the Select Animation window and click on Apply.
https://forum.reallusion.com/uploads/images/86903c38-6d91-43b8-8207-36b6.jpg



Edited
Last Year by Charlene (RL)
Charlene (RL)
Charlene (RL)
Posted Last Year
View Quick Profile
Senior Member

Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)Senior Member (498 reputation)

Group: Administrators
Last Active: 3 Months Ago
Posts: 11, Visits: 331
Troubleshooting:


Scenes with multiple characters
Replica Link works on one character at a time. Use the drop-down box in the Replica Link window to select a target character for importing audio.
https://forum.reallusion.com/uploads/images/a4a98602-5177-4b0b-8823-a357.jpg


Audio is longer than total frames of project
This message appears when your audio is longer than the total frames for the project. Simply click on “Edit > Project Settings > change Total Frames > Export to iClone from the Replica Studios desktop application once more.
https://forum.reallusion.com/uploads/images/b80418fb-4051-45ea-9620-7931.jpg
https://forum.reallusion.com/uploads/images/fa23c808-e284-4c74-8166-8ab8.jpg
https://forum.reallusion.com/uploads/images/7a6b9964-8ba1-4cb8-925c-af3b.jpg


No character in the scene
This message appears when there is no character in the scene.
https://forum.reallusion.com/uploads/images/4518f504-ec8e-46fe-8d89-c9ae.jpg


Connection error
This message appears when there are issues with the connection. Check your internet connection or try restarting the Replica Studios desktop application.
https://forum.reallusion.com/uploads/images/cf824b49-6814-4e7b-b3c7-97f0.jpg


Replica Studios desktop application is not open
This message appears when either Replica Studios desktop application is closed or when a character has not been selected, in which case you should click on the Select Avatar button.
https://forum.reallusion.com/uploads/images/4cb699a0-71eb-40ed-a8be-a04a.jpg


Update iClone to 7.92 or above
This message appears when you haven’t updated to iClone 7.92 or above, so auto-animation and auto-expressions are not available.
https://forum.reallusion.com/uploads/images/f5a7ade1-768f-45a3-9435-cd24.jpg

More guidance on Replica
If you need more guidance or help to use Replica Studios desktop application, click on Replica Link > Getting Started Guide to access Replica Studio’s Help Center.
https://forum.reallusion.com/uploads/images/acf66ca5-0379-4647-b8af-749a.jpg

Or you can join Replica Studios’s Discord 
https://forum.reallusion.com/uploads/images/55ad7439-b0cc-4502-90ed-aa39.jpg


Changing expressions
The new embedded Expression_Loop asset in iClone 7.92 lets you change expressions. Just go to Content > Animation > Template > Animation Template > Face > Expression Loop > All.
https://forum.reallusion.com/uploads/images/0964b564-6f6a-4147-850e-afaf.jpg





Edited
Last Year by Charlene (RL)
wires
wires
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 5.6K, Visits: 26.2K
Thank you all at Reallusion for this truly amazing addition for iClone, and one that many have been requesting for a long time. CoolSmoooothSmile

I sincerely hope that it doesn't get cancelled in the same way as the last TTS offer. Unsure

Gerry



System: Win 10 Pro (21H2), Asus X99-E WS, CPU i7-5930K -3,5 GHz, 32 GB DDR4 2666-16 RAM, NVidia GTX 1080 Ti GPU - 11 GB VRAM(Driver Studio-516.94), Samsung 850 Pro 512 GB SSD, 6 TB HD storage.


toystorylab
toystorylab
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)

Group: Forum Members
Last Active: 38 minutes ago
Posts: 3.8K, Visits: 24.4K
Cool... Works fine!




Toystorylab on Vimeo : https://forum.reallusion.com/uploads/images/ce44ea78-6984-47d8-9bf4-b783.png    Crassitudes (my "Alter Ego") on Youtube: https://forum.reallusion.com/uploads/images/fcc4df30-b3a8-40a5-a427-0735.png

argus1000
argus1000
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (3.0K reputation)Distinguished Member (3.0K reputation)Distinguished Member (3.0K reputation)Distinguished Member (3.0K reputation)Distinguished Member (3.0K reputation)Distinguished Member (3.0K reputation)Distinguished Member (3.0K reputation)Distinguished Member (3.0K reputation)Distinguished Member (3.0K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 496, Visits: 10.0K
Wow! This is fantastic!

My latest movie THE GOLDEN MAN, and a few othershttps://www.youtube.com/channel/UCIl1EqVCKitZzLqaNnLK0BA
rosuckmedia
rosuckmedia
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (7.8K reputation)Distinguished Member (7.8K reputation)Distinguished Member (7.8K reputation)Distinguished Member (7.8K reputation)Distinguished Member (7.8K reputation)Distinguished Member (7.8K reputation)Distinguished Member (7.8K reputation)Distinguished Member (7.8K reputation)Distinguished Member (7.8K reputation)

Group: Forum Members
Last Active: 4 hours ago
Posts: 2.6K, Visits: 4.0K
Cool Many Thank 👍👍 you, I've already tried it,
works very well.
Greetings rosuckmedia
4u2ges
4u2ges
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)

Group: Forum Members
Last Active: 10 hours ago
Posts: 4.4K, Visits: 14.8K
It would have been great if important announcements threads would also stay pinned at the very top for a few days in the "Latest" threads list.
A lot of people (including myself) are using "Latest" exclusively.




michaelrbarton
michaelrbarton
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)

Group: Forum Members
Last Active: 7 hours ago
Posts: 785, Visits: 8.2K
I got it working now.
Edited
Last Year by michaelrbarton
Leowald
Leowald
Posted Last Year
View Quick Profile
Veteran Member

Veteran Member (714 reputation)Veteran Member (714 reputation)Veteran Member (714 reputation)Veteran Member (714 reputation)Veteran Member (714 reputation)Veteran Member (714 reputation)Veteran Member (714 reputation)Veteran Member (714 reputation)Veteran Member (714 reputation)

Group: Forum Members
Last Active: 3 hours ago
Posts: 53, Visits: 1.6K
This plugin looks very intersting, but it looks like it's english only. But maybe I'm wrong?
toystorylab
toystorylab
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)Distinguished Member (13.3K reputation)

Group: Forum Members
Last Active: 38 minutes ago
Posts: 3.8K, Visits: 24.4K
You get 4 hours free as iClone User + regular free 30 min as starter.
So plenty enough to test and see/hear whether this is something you want to add to pipeline.
Regarding price, I would say 6,- $ per hour is not expensive.
And if you are sure you want to use this for several projects,
you even get package for 3,- $ per hour (100 hours for 300,- $)
For comparison, prices Acapela Box (which I used quite a lot):
https://acapela-box.com/AcaBox/acabox-prices.php
So way cheaper than Acapela Box...
Plus, your text automatically transforms into an AccuLips animation within seconds!
You can also create your own TTS voice, I did the first steps 6 months ago,
but did not complete as it takes several recordings. Might take that up again.
So, I would say, give it a try as it is for free. Wink

Here is a quick test:


Toystorylab on Vimeo : https://forum.reallusion.com/uploads/images/ce44ea78-6984-47d8-9bf4-b783.png    Crassitudes (my "Alter Ego") on Youtube: https://forum.reallusion.com/uploads/images/fcc4df30-b3a8-40a5-a427-0735.png




Reading This Topic