Hyper Realistic Text To Speech


https://forum.reallusion.com/Topic470320.aspx 		Print Topic | Close Window

By pwnz7321 - 5 Years Ago
I know Reallusion often looks into ways to enhance their products by forming relationships with other companies so I figured I'd link this here. https://www.sonantic.io/
They offer AI Text to Speech That looks very promising from the demos I've seen on their page.
Someone with Iclone7 already got access to it while it was in beta and made this test video: https://www.youtube.com/watch?v=DHhNwsC_XG4
I was thinking maybe Reallusion could look into this farther and potentially open up relations that granted iclone users access to this technology.
By Zeronimo - 5 Years Ago
if you listened to the example with Dina's voice, you can feel that the voice is still a bit robotic.
Apparently this system only deals with English voices. I haven't seen anything about other languages.
On the other hand it doesn't solve the problem of lips movement.
By Data Juggler - 5 Years Ago
Just to respond since this topic is here. The best company I have seen for speech synthesis is called CereProc: https://cereproc.com 

One reason I like this company, is I watched a video and people that know they will be losing their voice soon due to a disease or other causes, can record their voice before their conditions gets to where they cannot speak at all. So instead of a robotic voice, a Text To Speech interface can be built for them using their own voice. I looked into this top as I started to write a tool called Narrator that takes the AI generated text and lets you add pauses and other natural sounds but the project was a bit involved.

The one thing I noticed is generated female speech sounds better than males, and probably because I am an American, Scottish voices sound the best.

I think the lip syncing has gotten better than when I first bought IClone, or maybe I learned how to pronounce words better.
By 0oseven - 5 Years Ago
I'm quite impressed with this as most text to speech fall down on the lack of expression / emotion sounding mechanical or just without the emphasis needed on certain words. .
its one thing having a realistic 3D models and motions both of which  have come a very long way but we need much more for a speaking character.
Dialogue and lipsinc rely on an audio file  being provided  and that really 'is ther hard part ' if you are making any sort of movie  If you are wanting voices for a script you need different voices speaking with expression and emotion just like real.
Unless you are a professional studio who might afford to employ people to do voice over and  maybe use facial recognition software etc etc then  its uphill fo we lesser "animators" without those resources.
So this is why I think Sonantic's software could be a game changer.  At last you could write script, designate voices to it and have reasonable outcome. Integrated with iclone would be the icing on the cake.
By 0oseven - 5 Years Ago
pwnz7321 (1/20/2021)
I know Reallusion often looks into ways to enhance their products by forming relationships with other companies so I figured I'd link this here. https://www.sonantic.io/
They offer AI Text to Speech That looks very promising from the demos I've seen on their page.
Someone with Iclone7 already got access to it while it was in beta and made this test video: https://www.youtube.com/watch?v=DHhNwsC_XG4
I was thinking maybe Reallusion could look into this farther and potentially open up relations that granted iclone users access to this technology.


I know in their roadmap they are promising exciting things for lipsinc for 7.9  release  I think.  Maybe our dreams will come true!