Profile Picture

Synchronizing speech

Posted By TimV 9 Years Ago
You don't have permission to rate!
Author
Message
TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
Hi

As a learning project, I am at the beginning of creating a music video. The castle I posted about previously will be involved in this once I get it built in Hexagon. At any rate, here is what I have done.

There is a song that I really like by Libby McGrath called Princess. I give fair warning that it might stick in your head for a while if you search it out, but I love the concept and the lyrics.

I have a text file of the words which I have run through a text to speech program (Balabolka). I have that file on the sound line of my project in animation, which nicely drives the facial movements so that my avatar (Heidi) will act as my singer.

I have dropped the mp3 of the song into the main window and it is working fine as the background music. So, I have the audio of the text file and the mp3 both playing at the same time.

The next step is to synchronize the spoken audio with the voice of the mp3. The timing is worlds apart right now, so I am wondering if there is a better approach than just trying to speed up the spoken word delivery.

Is there a way to perhaps use the text of the song from the full mp3 (music and voice) to drive the lip-syncing instead of a separate text file?

High level suggestions would be great. I can figure out the process with a little prodding on the overall direction.

Thanks,

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

prabhatM
prabhatM
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 1.8K, Visits: 8.1K
If you think the Text to Speech has given you good lip syncing, then render that to Image Sequence / video without audio.

In a video editing programme, mix with the original song.
Edited
9 Years Ago by prabhatM
Cricky
Cricky
Posted 9 Years Ago
View Quick Profile
Skype ID= learn.iclone

Skype ID= learn.iclone (13.4K reputation)Skype ID= learn.iclone (13.4K reputation)Skype ID= learn.iclone (13.4K reputation)Skype ID= learn.iclone (13.4K reputation)Skype ID= learn.iclone (13.4K reputation)Skype ID= learn.iclone (13.4K reputation)Skype ID= learn.iclone (13.4K reputation)Skype ID= learn.iclone (13.4K reputation)Skype ID= learn.iclone (13.4K reputation)

Group: Forum Members
Last Active: 2 Years Ago
Posts: 2.8K, Visits: 13.8K
TimV (5/3/2015)
Hi

As a learning project, I am at the beginning of creating a music video. The castle I posted about previously will be involved in this once I get it built in Hexagon. At any rate, here is what I have done.

There is a song that I really like by Libby McGrath called Princess. I give fair warning that it might stick in your head for a while if you search it out, but I love the concept and the lyrics.

I have a text file of the words which I have run through a text to speech program (Balabolka). I have that file on the sound line of my project in animation, which nicely drives the facial movements so that my avatar (Heidi) will act as my singer.

I have dropped the mp3 of the song into the main window and it is working fine as the background music. So, I have the audio of the text file and the mp3 both playing at the same time.

The next step is to synchronize the spoken audio with the voice of the mp3. The timing is worlds apart right now, so I am wondering if there is a better approach than just trying to speed up the spoken word delivery.

Is there a way to perhaps use the text of the song from the full mp3 (music and voice) to drive the lip-syncing instead of a separate text file?

High level suggestions would be great. I can figure out the process with a little prodding on the overall direction.

Thanks,

Tim



Consider this:




SKYPE ID: rc.650

http://city.reallusion.com/store/ModuleExpand.aspx?id=GS634545170497417354&mdid=18866

http://city.reallusion.com/ContentTag.aspx?tagname=Cricky&AuthorID=20081020688550191201#]Reallusion Store Content

" If I had Alzheimer's (Old Timers), would I know it? "
Rampa
Rampa
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)

Group: Forum Members
Last Active: 3 hours ago
Posts: 8.1K, Visits: 60.5K
Might I suggest:

Getting the timing right may prove to be very difficult. If you have access to a microphone, I would suggest you record it as a "sing-along". It doesn't need to sound good or anything, it's just to get the timing right. Then drop out the voice as your doing now.

prabhatM
prabhatM
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 1.8K, Visits: 8.1K
rampa (5/3/2015)
Might I suggest:

Getting the timing right may prove to be very difficult. If you have access to a microphone, I would suggest you record it as a "sing-along". It doesn't need to sound good or anything, it's just to get the timing right. Then drop out the voice as your doing now.



This is definitely a better method.

I never needed it because I always get an independent voice track for all my original recording.
TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
Thank you all very much. I really appreciate the input!!!!

I have downloaded audacity and added a track for my spoken word text file. Once I figure out how to splice in spaces, I should be able to line the words up to the mp3 and then I think I saw something that would let me adjust the speed of the audio on the text file which would allow me to line up the length of time it takes to speak the work with it's actual length in the mp3.

I have a licenced version of 3DXChange 5 Pro if that is of any help

I don't have a mic at the moment, but it is a good suggestion to set the timing with a voice over personal lip sync of the song. I think I could manage to good that pretty close will a little practice.

I'm going to see if I can find any free software that will let me adjust the timing in the voice text file. I don't have a budget for "expensive" software, but I will look into lequendo to see what I should be searching for.

best regards,

Tim


There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
Hi All:

My search for something similar to Lequendo came back with Balabolka as the first recommendation. I downloaded Balabolka to create the speech file that I used to drive the facial animation.

I have never used it before today, but it may allow me to do the timing adjustments I need in the text file prior to bringing the file into IClone6. That just might do the trick for me.

Time to learn a new program.....Never enough time, but I never waste an opportunity to learn something new. It always pays off in the end.

Thanks for your help in heading me down this path with your suggestions.

Best,

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

Rampa
Rampa
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)

Group: Forum Members
Last Active: 3 hours ago
Posts: 8.1K, Visits: 60.5K
sw00000p (5/3/2015)
"Does iClone 6 import facial animation?"
...rampa, ...or does anybody else know? Ermm

If so, Tim would just import the animation and Drop the Sound Track!Smile
This is what I do using (3dxchange 4.0) Wow






Kind Regards,
sw00000p Cool






You can do what you used to do in 3DX4, that is import bone based animation that happens to be facial bones, but not characterize it. It's the getting of the animation that may prove difficult. You could try "frankensteining" a mocap head onto an iClone body.

Do you have a way to record bone based facial animation, sw00000p?

So.... Sort of.
prabhatM
prabhatM
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 1.8K, Visits: 8.1K
rampa (5/3/2015)
sw00000p (5/3/2015)
"Does iClone 6 import facial animation?"
...rampa, ...or does anybody else know? Ermm

If so, Tim would just import the animation and Drop the Sound Track!Smile
This is what I do using (3dxchange 4.0) Wow






Kind Regards,
sw00000p Cool






You can do what you used to do in 3DX4, that is import bone based animation that happens to be facial bones, but not characterize it. It's the getting of the animation that may prove difficult. You could try "frankensteining" a mocap head onto an iClone body.

Do you have a way to record bone based facial animation, sw00000p?

So.... Sort of.


Sw00000p has been claiming too often bringing the facial animation to ICLONE through 3DX4. I and the newbies would love to see a sample of that.
Edited
9 Years Ago by prabhatM
Rampa
Rampa
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)

Group: Forum Members
Last Active: 3 hours ago
Posts: 8.1K, Visits: 60.5K
prabhatM (5/3/2015)
rampa (5/3/2015)
sw00000p (5/3/2015)
"Does iClone 6 import facial animation?"
...rampa, ...or does anybody else know? Ermm

If so, Tim would just import the animation and Drop the Sound Track!Smile
This is what I do using (3dxchange 4.0) Wow






Kind Regards,
sw00000p Cool






You can do what you used to do in 3DX4, that is import bone based animation that happens to be facial bones, but not characterize it. It's the getting of the animation that may prove difficult. You could try "frankensteining" a mocap head onto an iClone body.

Do you have a way to record bone based facial animation, sw00000p?

So.... Sort of.


Sw00000p has been claiming too often bringing the facial animation to ICLONE through 3DX4. I and the newbies would love to see a sample of that.


Until you characterize it and define certain bones as "facial", your playing prop animation that happens to be in the shape of a human face. So yeah, it's not hard to do. The difficutly comes in making it an iClone avatar, and capturing it in the first place if you don't have the software.



Reading This Topic