Profile Picture

Mix and match workflow or multiple layers blended?

Posted By daniel.eriksson Last Month
You don't have permission to rate!
Author
Message
daniel.eriksson
daniel.eriksson
Posted Last Month
View Quick Profile
New Member

New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)

Group: Forum Members
Last Active: 3 Weeks Ago
Posts: 2, Visits: 45
Is there any way to either mix and match layers of the same animation type?
For example, I think the TTS lipsync is of very high quality out of the box, but I don't want to use the robotic voices, I want to use voice actors.
I have not found a way to get that to work. Is there any way to unlink the audio tracks from the viseme keys?
If I could do that  I could start from the strong foundation that is the TTS lipsync and simply retime it by hand.
Peter (RL)
Peter (RL)
Posted Last Month
View Quick Profile
Distinguished Member

Distinguished Member (52.2K reputation)Distinguished Member (52.2K reputation)Distinguished Member (52.2K reputation)Distinguished Member (52.2K reputation)Distinguished Member (52.2K reputation)Distinguished Member (52.2K reputation)Distinguished Member (52.2K reputation)Distinguished Member (52.2K reputation)Distinguished Member (52.2K reputation)

Group: Administrators
Last Active: 2 hours ago
Posts: 16.0K, Visits: 24.1K
Which product is the question about?

                                                                

Peter
Forum Administrator

www.reallusion.com


animagic
animagic
Posted Last Month
View Quick Profile
Distinguished Member

Distinguished Member (19.3K reputation)Distinguished Member (19.3K reputation)Distinguished Member (19.3K reputation)Distinguished Member (19.3K reputation)Distinguished Member (19.3K reputation)Distinguished Member (19.3K reputation)Distinguished Member (19.3K reputation)Distinguished Member (19.3K reputation)Distinguished Member (19.3K reputation)

Group: Forum Members
Last Active: Today @ 5:04 AM
Posts: 11.3K, Visits: 19.6K
@daniel: You cannot delete the dialog file without deleting the visemes, but you can do sort of a work around, which I have done in the post (there's also a tutorial by use 3DTest about that on YouTube, I'll have to find it).

As you observed, viseme allocation is much better when using TTS voices. The reason seems to be that iClone (if that's what you are using) uses the text for the TTS is a guideline to determine the visemes. We have asked a million time to do the same for regular lip-syncing but it has fallen on deaf ears thus far.

After you have done your TTS lip-syncing, you can then mute the out put for the voice (there is sliders on the time line for that). Than you can import a sound file with your voice actor. Unfortunately, you won't see the waveform, but you can scrub the audio.

I think the way I have done it is to split things up by sentences or phrases to make the lining up easier.

Here is the tutorial. The only difference is that you don't need to use CT8 to do the TTS lip-syncing; you can now do that in iClone directly.



I'm in a bit of a hurry right now, but recently Mike Kelley introduced a way to use Papagayo to generate the visemes and then use a Python script to "inject" those in iClone. I have worked on some improvements of that script.

Of course, it would be much easier if this was actually integrated in iClone. The technology is there and has been for a long time, but there seems to be some reluctance on RL's part to make use of that. Even though we have facial mocap, that is not always a solution.


https://forum.reallusion.com/uploads/images/1a09220f-ab50-42ac-ad1a-33ec.pnghttps://forum.reallusion.com/Uploads/Images/d14339d0-cd32-4b35-88f9-40a0.png


daniel.eriksson
daniel.eriksson
Posted Last Month
View Quick Profile
New Member

New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)New Member (38 reputation)

Group: Forum Members
Last Active: 3 Weeks Ago
Posts: 2, Visits: 45
Peter (RL) (5/3/2019)
Which product is the question about?

iClone 7, sorry about that.


@animagic
Yeah this is mighty frustrating. I feel like there is so much untapped potential here.
I have found a way to get the waveform and audio in the timeline by importing a second dummy character while I animate.
So the dummy carries the actual audio that I want and I clean its timeline display to only display the waveform.
So now everything works, except for one little detail.
animagic (5/3/2019)
After you have done your TTS lip-syncing, you can then mute the out put for the voice (there is sliders on the time line for that).

I might be blind or stupid or something but I can not find this option.
Where is the button to mute a character or lower the volume?
All I have found that even resembles a volume button/slider is this button in the image below.
IIt pops up a slider but the slider does not do anything.
https://forum.reallusion.com/uploads/images/672ffa68-171f-45ba-a4ba-d77c.jpg






Reading This Topic