Profile Picture

Synchronizing speech

Posted By TimV 9 Years Ago
You don't have permission to rate!
Author
Message
TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
@Rampa

Thanks for the location information.  Just what I needed.

@Mark

I might have to break down and buy a mic.  I will look into that option.

In the meantime, I am becoming great friends with Audacity.  I used a program feature to split my audio track automatically based on "detach at silence".  That gave me individual blocks of spoken text that I could then line up with the vocals on the mp3 track.  Prior to that I went back to Balabolka and re-read my text file at a slower rate.  Some of the blocks are now an almost exact match.  With a little time and effort I can output a line at a time, or even a word for areas that aren't close enough after the first pass.  I'm thinking this is going to get me the results I want as long as my patience holds up.

Thanks for mentioning that you attached to a prop.  That is a great idea.  I am attaching my speech to Heidi and the actual mp3 is in as background audio.  Attaching the mp3 should give me a sound line for the music which will help me sync things up.

Thanks,

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

prabhatM
prabhatM
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 1.8K, Visits: 8.1K
sw00000p (5/6/2015)
Is this the song?



Couldn't this be done inside ICLONE without any external help ?

TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
Yes, that is the song and the lip-synching is pretty much spot on.  If, as I expect, I am about to find out that this was done entirely in IClone6, I have obviously spent time learning things External to IClone6 that I would much rather have spent learning things Inside IClone6.

That being said. I would point to the large friendly "Newbie" that indicates my experience with IClone6 and very politely ask how this was done.

If it was with a mic, I obviously should have bought one of those instead of MediaMonkey which did accept my lyrics but as a whole block of text at the beginning of the track that I have been unable to "sync" with the music. I was going to work on the spoken text file with SYLTEditor today as it allows time stamps that I was imagining would help in Audacity.  I had hoped to find a tidy digital solution that I could be sure I could line up accurately at a price that I could afford.

Please advise on how this was done.

Thanks and best regards,

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
Hi sw00000p

Thank you so much for taking the time to create this tutorial for me.  I really do appreciate it.  I can see that I have a great deal of learning to do.  It's clear from the results that you have shown, that what I wish to do requires an investment in time and patience.  As is usually the case, there clearly are no shortcuts if one wants to do it properly and convincingly.  I am a big fan of realism and so I will take my time and learn what I have to learn.

Thanks for sharing.  I now know how I want to proceed rather than trying to rush my way through.

best,

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

prabhatM
prabhatM
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 1.8K, Visits: 8.1K
sw00000p (5/6/2015)
TimV (5/6/2015)

Please advise on how this was done.

Thanks and best regards,

Tim

Tim, this is One Way!





I missed the context.
Was this tutorial about "how to do better LipSync in ICLONE?"
Or,  doing the LipSync outside ICLONE and then importing the facial animation into ICLONE ?

TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
Hi sw00000p

Again, my sincere thanks for your efforts in laying out the process and providing specific, useful information.

It would obviously seem that starting / continuing with the vertex animation would be a good choice since I know little about any animation.  I am pleased with the animation that was achieved using a Daz model, exporting the .fbx, and then loading the model into 3DXchange5 (Pro).  That provides a decent group of visemes for the facial movements.  I see that 3DXchange5 provides the ability to tweak those visemes to enhance how they appear.  Not sure about the realism aspect, but it would be a good learning environment I think.

I upgraded my version of IClone6 from Standard to Pro a couple of days ago and haven't done anything with it, but I am hoping the enhanced facial puppeteering will help on the realism side and it does support bone level expression for when I am ready to take that step.  I don't know what "detail solo feature control" is yet, but I will find out.  It also has more flexible timeline features which I expect will be useful.

At the moment, I do have a spoken word text file that has no background noise.  Can I not use the timeline enhancements to adjust my spoken word track's timing to manually sync that file with the actual mp3 of the song?  Your sample in this post is perfect from a timing perspective.  I would like to have a process that synchornizes my text file with the mp3.  

I want to focus on the balance of the facial animation because that is where the reality of a live performance lives.  I went to concert and had a good seat to watch Colin Raye.  Like most professionals, he does not sing his songs.  He lives his songs from the heart.  That's what you don't see when you buy an MP3.  That's what I want in my animation.  I think the aspect of timing the lip-sync should be like putting a canvas on a stand.  Applying the paint is where the magic lives.

You mentioned a package in a previous post and I will go back and see what it was after I post this.  The implication that I got out of your note was that this software was capable of preparing the canvas. (so to speak by my analogy above.)  Is that the case? and are you aware of any other packages that might be a little more affordable.  I think you noted that it was around $1000.

Can't wait to hear your thoughts on IClone6 Pro and if I can do enough tweaking on the timing there.  Then I could focus on adding bones to handle the facial animation.

best,

Tim



There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

prabhatM
prabhatM
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)Distinguished Member (6.4K reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 1.8K, Visits: 8.1K
Tim,

With the help of 3DX and ICLONE you can create reasonable good Lipsyncing. 

1) Start with the basic lipsyncing
2) Play around and Practice to improve in ICLONE.
3) You may have problem getting the facial expressions from other softwares to ICLONE. So stay focussed on ICLONE.
4) If you keep switching between Maya, Blender, 3DX MAX, DAZ and ICLONE, then you will never be able to complete a project.


I just read your posting. I guess you would do fine with "JUST" 3DX and ICLONE. 

Please don't set yourself on a wild goose chase when you are LEARNING ANIMATION. 





TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
Thanks prabhatM

I think the first thing I want to do is investigate the features in IClone6 Pro.  

If I can control the timing on the text speech in there, to line it up with the MP3, I think that is a good place to start.  

If I can't get what I want specifically, I know I can get pretty close with lining up blocks in Audacity.  Usually, with this particular song, I can put two lines in a block and sync that block to the audio track of the song and get very close.

Since I now have a good grip on editing in Audacity, I'm comfortable.  I have also discovered the art of creative spelling in my spoken text file.  For example, matching the word "perfect" with the timing doesn't work well.  Matching "purr fect" works very well.  Some words are spread over several beats, so that will be more challenging.  Destiny overlays much better as "desk in he" 

I'm sorry I said anything about $1000.  I went back and looked at sw00000ps post and didn't find anywhere where it said that.

best,

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

Rampa
Rampa
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)Distinguished Member (35.8K reputation)

Group: Forum Members
Last Active: 2 hours ago
Posts: 8.1K, Visits: 60.6K
Good call on the upgrade, Tim! Many of us just assume everyone gets "Pro" to start with, so we would end up giving advice that wouldn't make sense.

Yes, with the full timeline, you will be able to cut-up and stretch your audio just as you need. The "Facial Puppet" and "Facial Key" windows are also very powerful, and allow for animation of facial features both in groups and separately. You will also be able to change/add/delete the visemes. For those longer beats, sometimes just re-positioning a viseme a bit later in the timeline is enough.

That fancy stuff sw00000p was showing you from Max can also be done right in iClone (Pro), with the advantage of it being on an iClone character which you can further animate.

Good luck!
TimV
TimV
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 126, Visits: 426
Hi Rampa:

Thanks for your note and the info on the pro version.  Now I am excited

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.




Reading This Topic