Replica Studios - AI Voice Actor Plugin for Lip-sync Animation


https://forum.reallusion.com/Topic486959.aspx
Print Topic | Close Window

By Charlene (RL) - 4 Years Ago
Dear Users,

AI Voice Actor is a collaborative project between Reallusion and Replica Studios. The goal is to create a fully automated text to animation pipeline with minimal user intervention. This trailblazing solution enables fast talking animation production without the need to hire voice talent, or even animators. In addition, designers can still fine-tune the AI generated results with iClone's Talking Styles, Viseme Editing, Face Mocap, Face Puppet, or Facial Key Editor.

*While iClone AccuLips provides text-to-speech animation for the lips, it does not support facial animations. Using Replica’s AI Voice Actor plugin, one can easily and conveniently apply different facial expressions on top of the lip motions to give a mood to the performance, thereby, offering more emotional impact to the audience.

Visit the product page at https://www.reallusion.com/iclone/ai-voice-replica/.

Required Tools:
  • iClone 7.92 or above.
  • Replica Plugin (Marketplace > iClone > Plug-in).
  • Replica Studios desktop application.

Installation Guide:


Getting Started:



Step 1: Go to Replicastudios.com, create an account and download Replica Studios Desktop Application.
https://forum.reallusion.com/uploads/images/1d3006b2-7a16-436c-8a64-7eb8.jpg

Step 2: Go to the Reallusion Marketplace and download the Replica Plugin.


Step 3: Uncompress the ReplicaStudios zip folder  https://forum.reallusion.com/uploads/images/598228dc-ea14-4d44-8492-f97e.jpg
Move the unzipped ReplicaStudios folder to C:\Program Files\Reallusion\iClone 7\Bin64\OpenPlugin
https://forum.reallusion.com/uploads/images/bab2c0b5-ac70-4b62-819c-ca1d.png

Step 4: Go to iClone and create a character in the scene.
https://forum.reallusion.com/uploads/images/c03cb8cf-5418-45f6-8aea-8937.jpg

Step 5: Click on Plugins > Replica Studios > Replica Link
https://forum.reallusion.com/uploads/images/8ae59b8b-116f-40bc-9616-833c.jpg

Step6: Sign in to Replica Studios desktop application and choose Sandbox.
https://forum.reallusion.com/uploads/images/d40e5042-d716-45d4-9639-b8de.jpg

Step7: Choose a speaker, enter the text for TTS conversion, and click on Save Take
https://forum.reallusion.com/uploads/images/86215cde-b61a-4130-b154-e841.jpg

Step 8: Choose the character in the scene and click on Replica Link > Select Avatar and notice the status becomes Connected.
Note: The character is named RD4, so it becomes evident in the dropdown list. 
https://forum.reallusion.com/uploads/images/e8d8b032-4108-46f4-9ed4-9a1d.jpg

Step 9: Export to iClone.
https://forum.reallusion.com/uploads/images/10b58770-132b-43a0-a0fa-e320.jpg

Step 10: Set the Style and adjust the Strength in the Select Animation window and click on Apply.
https://forum.reallusion.com/uploads/images/86903c38-6d91-43b8-8207-36b6.jpg



By Charlene (RL) - 4 Years Ago
Troubleshooting:


Scenes with multiple characters
Replica Link works on one character at a time. Use the drop-down box in the Replica Link window to select a target character for importing audio.
https://forum.reallusion.com/uploads/images/a4a98602-5177-4b0b-8823-a357.jpg


Audio is longer than total frames of project
This message appears when your audio is longer than the total frames for the project. Simply click on “Edit > Project Settings > change Total Frames > Export to iClone from the Replica Studios desktop application once more.
https://forum.reallusion.com/uploads/images/b80418fb-4051-45ea-9620-7931.jpg
https://forum.reallusion.com/uploads/images/fa23c808-e284-4c74-8166-8ab8.jpg
https://forum.reallusion.com/uploads/images/7a6b9964-8ba1-4cb8-925c-af3b.jpg


No character in the scene
This message appears when there is no character in the scene.
https://forum.reallusion.com/uploads/images/4518f504-ec8e-46fe-8d89-c9ae.jpg


Connection error
This message appears when there are issues with the connection. Check your internet connection or try restarting the Replica Studios desktop application.
https://forum.reallusion.com/uploads/images/cf824b49-6814-4e7b-b3c7-97f0.jpg


Replica Studios desktop application is not open
This message appears when either Replica Studios desktop application is closed or when a character has not been selected, in which case you should click on the Select Avatar button.
https://forum.reallusion.com/uploads/images/4cb699a0-71eb-40ed-a8be-a04a.jpg


Update iClone to 7.92 or above
This message appears when you haven’t updated to iClone 7.92 or above, so auto-animation and auto-expressions are not available.
https://forum.reallusion.com/uploads/images/f5a7ade1-768f-45a3-9435-cd24.jpg

More guidance on Replica
If you need more guidance or help to use Replica Studios desktop application, click on Replica Link > Getting Started Guide to access Replica Studio’s Help Center.
https://forum.reallusion.com/uploads/images/acf66ca5-0379-4647-b8af-749a.jpg

Or you can join Replica Studios’s Discord 
https://forum.reallusion.com/uploads/images/55ad7439-b0cc-4502-90ed-aa39.jpg


Changing expressions
The new embedded Expression_Loop asset in iClone 7.92 lets you change expressions. Just go to Content > Animation > Template > Animation Template > Face > Expression Loop > All.
https://forum.reallusion.com/uploads/images/0964b564-6f6a-4147-850e-afaf.jpg





By wires - 4 Years Ago
Thank you all at Reallusion for this truly amazing addition for iClone, and one that many have been requesting for a long time. :cool::smooooth::)

I sincerely hope that it doesn't get cancelled in the same way as the last TTS offer. :unsure:
By toystorylab - 4 Years Ago
Cool... Works fine!


By argus1000 - 4 Years Ago
Wow! This is fantastic!
By rosuckmedia - 4 Years Ago
Cool Many Thank 👍👍 you, I've already tried it,
works very well.
Greetings rosuckmedia
By 4u2ges - 4 Years Ago
It would have been great if important announcements threads would also stay pinned at the very top for a few days in the "Latest" threads list.
A lot of people (including myself) are using "Latest" exclusively.
By michaelrbarton - 4 Years Ago
I got it working now.
By Leowald - 4 Years Ago
This plugin looks very intersting, but it looks like it's english only. But maybe I'm wrong?
By toystorylab - 4 Years Ago
You get 4 hours free as iClone User + regular free 30 min as starter.
So plenty enough to test and see/hear whether this is something you want to add to pipeline.
Regarding price, I would say 6,- $ per hour is not expensive.
And if you are sure you want to use this for several projects,
you even get package for 3,- $ per hour (100 hours for 300,- $)
For comparison, prices Acapela Box (which I used quite a lot):
https://acapela-box.com/AcaBox/acabox-prices.php
So way cheaper than Acapela Box...
Plus, your text automatically transforms into an AccuLips animation within seconds!
You can also create your own TTS voice, I did the first steps 6 months ago,
but did not complete as it takes several recordings. Might take that up again.
So, I would say, give it a try as it is for free. ;)

Here is a quick test:
By lianchiu - 4 Years Ago
toystorylab (7/1/2021)
Cool... Works fine!




Nice toystorylab !!!!
add more Expression go go go~~

Haven't talk to you for a while~

Lian
By lianchiu - 4 Years Ago
4u2ges (7/2/2021)
It would have been great if important announcements threads would also stay pinned at the very top for a few days in the "Latest" threads list.
A lot of people (including myself) are using "Latest" exclusively.

Great i will pass to marketing team
Thanks~!

Lian RL
By lianchiu - 4 Years Ago
Leowald (7/2/2021)
This plugin looks very intersting, but it looks like it's english only. But maybe I'm wrong?

For now iClone and Replica Studios both side only support English!
Lian RL
By mark - 4 Years Ago
Does this plugin work with Windows 7?   I'm getting the "Connection" Error and I have a good internet connection.   I'm still on 7 I'm afraid :ermm:
By wires - 4 Years Ago
mark (7/2/2021)
Does this plugin work with Windows 7?   I'm getting the "Connection" Error and I have a good internet connection.   I'm still on 7 I'm afraid :ermm:


Sorry Mark, but on the Replica download page it says Windows 10 or later. :crying:
By gordryd - 4 Years Ago
I'm also using Win 7 with limited success -- it tries to work for awhile, but after auditioning a few different voices it stops working.  Make sure that the Replica desktop app is running (choose "Sandbox" option after startup) before you try to run the plug-in inside iClone.  Also make sure you select an avatar once the plugin loads before doing anything else.
By mark - 4 Years Ago
Thanks guys! Yep missed that mention of W10...getting closer to moving up I guess!!:crazy:
By theschemer - 4 Years Ago
mark (7/2/2021)
Thanks guys! Yep missed that mention of W10...getting closer to moving up I guess!!:crazy:


May as well as Windows 11 is around the corner with special OS requirements (Gen 8 or newer CPU) for security. But don't worry as Win10 will be supported until 2025.

By mark - 4 Years Ago
theschemer (7/2/2021)
[quote]
May as well as Windows 11 is around the corner with special OS requirements (Gen 8 or newer CPU) for security. But don't worry as Win10 will be supported until 2025.


Good point,  hadn't heard about W11...I'm sooo behind the times! :blink:

By argus1000 - 4 Years Ago
I see the usefulness of this plugin, but only for characters with a line of dialogue or two. For the more fleshed out characters, nothing replaces a real voice-actor for the range of facial and voice emotions.
By animagic - 4 Years Ago
argus1000 (7/5/2021)
I see the usefulness of this plugin, but only for characters with a line of dialogue or two. For the more fleshed out characters, nothing replaces a real voice-actor for the range of facial and voice emotions.

Not all of us have easy access to that, so we have to make do. It is all fake anyway...:P

And, voice actors who offer their services online should at least make sure that their demos are not distorted.

But we have had this discussion here time and again, so I think everyone should do as they see fit.
By yoshizumir - 4 Years Ago
After 2 hours of trying to resolve connection issue, I decided to uninstall this plugin as it simply doesn't work on my computer. I made sure all the specs are met. Turn off firewall protection so nothing is blocking with the connection. My character is CC3 base Plus. Window 10 and downloaded window version of plugin. I still get the message connection error: try in 10 seconds.
I guess until the forum have other useful information to resolve this, I will continue to use voice actors and face mocap device. otherwise, this would be my go to plugin and I don't even mind paying for the voice as it will be cheaper for my film. What a disappointment.
By animagic - 4 Years Ago
yoshizumir (7/9/2021)
After 2 hours of trying to resolve connection issue, I decided to uninstall this plugin as it simply doesn't work on my computer. I made sure all the specs are met. Turn off firewall protection so nothing is blocking with the connection. My character is CC3 base Plus. Window 10 and downloaded window version of plugin. I still get the message connection error: try in 10 seconds.
I guess until the forum have other useful information to resolve this, I will continue to use voice actors and face mocap device. otherwise, this would be my go to plugin and I don't even mind paying for the voice as it will be cheaper for my film. What a disappointment.

I had that problem as well, and what I did was run Replica Studios as administrator (i.e., right-click and select "Run as Administrator"). 

That give me an initial connection and after that I could run Replica Studios normally and still get a connection. My guess is that with running as administrator the necessary firewall rules were established.
By wujackjack - 4 Years Ago
same condition as yoshizumir and still get connection error after run replica studio as administrator. Hope to get a solution.
By wujackjack - 4 Years Ago
animagic (7/11/2021)
yoshizumir (7/9/2021)
After 2 hours of trying to resolve connection issue, I decided to uninstall this plugin as it simply doesn't work on my computer. I made sure all the specs are met. Turn off firewall protection so nothing is blocking with the connection. My character is CC3 base Plus. Window 10 and downloaded window version of plugin. I still get the message connection error: try in 10 seconds.
I guess until the forum have other useful information to resolve this, I will continue to use voice actors and face mocap device. otherwise, this would be my go to plugin and I don't even mind paying for the voice as it will be cheaper for my film. What a disappointment.

I had that problem as well, and what I did was run Replica Studios as administrator (i.e., right-click and select "Run as Administrator"). 

That give me an initial connection and after that I could run Replica Studios normally and still get a connection. My guess is that with running as administrator the necessary firewall rules were established.


Same condition as yoshizumir and stil get connection error after run replica studio as administrator. Hope to get a solution.
By david sharp - 4 Years Ago
I've just tried this out now 27th July 2021 and it worked well in iClone 7. But I can only see just under 30 minutes credit on the Replica account. How do do you claim the 4 hours Replica voice bonus or has this offer expired ? I bought iClone 7 on the 12th July 2021.

Edit: actually I'm not 100% sure but I think you may get the 4 hours free Replica sound if you submit an entry to the Lipsynch competition. Dunno where I read that you get it if you buy iClone 7.
By toystorylab - 4 Years Ago
david.boris.fernandez (7/27/2021)
I've just tried this out now 27th July 2021 and it worked well in iClone 7. But I can only see just under 30 minutes credit on the Replica account. How do do you claim the 4 hours Replica voice bonus or has this offer expired ? I bought iClone 7 on the 12th July 2021.

Edit: actually I'm not 100% sure but I think you may get the 4 hours free Replica sound if you submit an entry to the Lipsynch competition. Dunno where I read that you get it if you buy iClone 7.

Visit the product page at https://www.reallusion.com/iclone/ai-voice-replica/.
Scroll down to nearly bottom, green field iclone 7 users, click "redeem now"...
By animagic - 4 Years Ago
david.boris.fernandez (7/27/2021)
Edit: actually I'm not 100% sure but I think you may get the 4 hours free Replica sound if you submit an entry to the Lipsynch competition. Dunno where I read that you get it if you buy iClone 7.

It is independent of your participation in the contest. You only need to buy or already have iClone. 

Make sure you are logged into your Reallusion user account and go to this page: https://www.reallusion.com/iclone/ai-voice-replica/

Near the bottom you should see a Redeem Now button. Click that and you should see a page with a voucher code and further instructions.
By david sharp - 4 Years Ago
cheers