Profile Picture

Synchronizing speech

Posted By TimV 10 Years Ago
You don't have permission to rate!
1
2
3
4
5
6
7

Author
Message
prabhatM
prabhatM
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 1.8K, Visits: 8.1K
sw00000p (5/7/2015)

I use iClone to "Render." I import everything!







That sounds very interesting.

How do you import a Pre-Composed Multi-actor scene with body and facial animations into ICLONE?

TimV
TimV
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)

Group: Forum Members
Last Active: 6 Years Ago
Posts: 126, Visits: 426
@sw00000p

Thanks for clarifying your process.  I will certainly keep heading toward the facial bones.  In the meantime, my time is available for learning whatever I need to build a foundation that I can understand.  I would love at some point to be able to do this as a paid activity, but

First you get good, then you get fast.  It's a quote from somewhere but I'm sure of the source.  (I heard it in M.A.S.H.)
@all
I was writing this to find out how to make the "break" icon work on the sound file, but I tried something I thought I had done before which was to left click on the file in the actual viseme line and then click the icon and it worked, so I suspect that I had not actually tried that before.  I think I was trying to "break" the voice track, but it wouldn't break.

Is there a way to display the waveform in the Sound line under Project like it is displayed in the Voice line under Viseme.  That would be helpful for me be able to see where the silent spots are, to help line up the two tracks.

Thanks,

Tim




There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

TimV
TimV
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)

Group: Forum Members
Last Active: 6 Years Ago
Posts: 126, Visits: 426
For the benefit of any other newbie reading along.

I have learned that I do not need to use Audacity.  Too bad, it was such a good price.  :)

 I have loaded the file into the Modify panel in .wav format, and I have also copied the text into the TTS window.  Both ways allowed me to enter the file.  I'm not sure they both put the file in the Viseme / Voice track, but it is there now and I am happy.  The actual mp3 is in the Project / Sound track, so all is well.

It looks like I will be ok adjusting the speed of the text track section now that I have figured out how to break up the file.  

I am trying to use Flags in the project track to line up where my voice starts for the lines of the song.  Im finding it challenging to find the right spot.  Once I get that in place, I can use it as the indicator for where the spoken text track should start up for the next section / (line or verse)

If I hadn't gone through what I did to get to this point, I wouldn't have the understanding of what I'm trying to do in a framework that I can now make use of in IClone6.

All of the help and suggestions have been very useful to me and I thank you for the hours of frustration that you have saved me.

best,

Tim

The edit was to fix a missed letter, but I just realized that this learning exercise has driven my promotion from NewBie to Junior Member.  That actually seems true:)

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

Rampa
Rampa
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)

Group: Forum Members
Last Active: Last Week
Posts: 8.2K, Visits: 62.6K
I think I can help you with the alignment thing, a little anyway. As you noticed, the only wave-form display is for "voice". If you want to have both the music wave-form, and your TTS wave-forms and visemes all visible together, then assign the music track as the voice of a second avatar. Since you can have everybody's tracks open at once in "Pro", open your voice track, and the music track. You can close all the tracks and sub-tracks you don't need at the moment so that you can have both wave-form tracks together.
TimV
TimV
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)

Group: Forum Members
Last Active: 6 Years Ago
Posts: 126, Visits: 426
rampa

Ohhhhh, thank you.  That's brilliant!!!  I will try that.  I had actually gotten pretty good with the flags and broken up voice text file.  I still have more to do so I will try your suggestion.

Thanks again

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

prabhatM
prabhatM
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 1.8K, Visits: 8.1K
sw00000p (5/7/2015)
prabhatM (5/7/2015)
[quote]sw00000p (5/7/2015)

I use iClone to "Render." I import everything!
/quote]

That sounds very interesting.

How do you import a Pre-Composed Multi-actor scene with body and facial animations into ICLONE?

Look! I don't want to argue nor debate YOU!

I take what I need from iClone and various other apps. into:

1. MotionBuilder
2. 3ds Max
3. Endorphin
4. REALFLOW
5. TopoGun
6. After Effects

Pre-Composed... this or that BS doesn't bother me. I tear it apart and make it the way I WANT IT.
_________________________________________________________

Import and render what I need to iClone .... then finish in Post... with AE and Premiere Pro.










@Tutor !!!
===============


First you say you use ICLONE ONLY FOR RENDERING and IMPORT EVERYTHING.
When we ask you to clarify your IMPORT ( INTO ICLONE )  process, you start  talking about  the EXPORT ( OUT OF ICLONE )  process !
In your High-end Process, are they same ?:D

TimV
TimV
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)

Group: Forum Members
Last Active: 6 Years Ago
Posts: 126, Visits: 426
Hi:

@rampa
Thanks again.  I have added another character and exported the mp3 of the song as a .wav file.  I have that set at the Viseme track for the second character so I can now see my waveforms for both tracks.

@all

I have taken another little side trip, but I really believe it is important and worth the effort.  

I did a lot of work last night on the overall timing, and with breaking the spoken track to give me the flexibility to line up the lyrics line by line, or phrase by phrase. That was the original "final" plan.

That did bring some of the text into a decent synchronization between the words and the song.  Adjusting the text track segment lengths of the broken up text .wav speech also closed a number of out of sync areas.  

It doesn't look close enough.  I still have out of sync areas that look like watching old kung fu movies - english sound stops, characters keep talking.

In my case, since the language is the same on the two tracks that I want to sync, some of that mismatch is phrasing, while some of it is syllable stress and some of it is pronunciation.  There is also the creative aspect of singing where words are stretched or altered to fit the beat of the music.  Playing with the length of a track segment that represents a line of the song may allow for end to end synchronization but how the line is spoken becomes apparent.

To that end, TTS sytems allow control of the flow of the speech, enunciation, pronunciation, delays, voice speed and pitch variances, etc.  This is done through the use of XML tags embedded in the text.  I have experimented with adding these tags in my speech text prior to having the software create my .wav file of the lyrics.  I'm stumbling at the moment, but I will certainly get better at it.  The results are most encouraging.

I have been able to match the way the words are sung in the song by using tags to change syllable stresses and by specifying pronunciations.  The plan is to get the spoken file as close as I can, create a new .wav file and bring that into IClone6.  From there, when I break the spoken file into segments, I believe I will be able to very close to an exact match.

I expect it will take some time, but I will post the result in a small sample which I hope will demonstrate the effectiveness of the process.

best,

Tim

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

prabhatM
prabhatM
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 1.8K, Visits: 8.1K
sw00000p (5/8/2015)
prabhatM (5/8/2015)
First you say you use ICLONE ONLY FOR RENDERING and IMPORT EVERYTHING.
When we ask you to clarify your IMPORT ( INTO ICLONE )  process, you start  talking about  the EXPORT ( OUT OF ICLONE )  process !
In your High-end Process, are they same ?:D


Ummm! Clarify Import, OK!  I use 3dxchange.
"Plain Simple English!" :pinch:


===========================
You famously said you  IMPORT everything and use ICLONE only for the rendering.
===========================

What all do you IMPORT through 3dX for RENDERING in ICLONE ?

Do we presume that you set your SCENE in another software and IMPORT it to ICLONE ONLY FOR the RENDERING ?





TimV
TimV
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)

Group: Forum Members
Last Active: 6 Years Ago
Posts: 126, Visits: 426
@sw00000p and @prabhatM

I'm not sure if there are any moderators.  I'm not sure how long you two have been feuding.  I'd like to ignore it and pretend that it isn't happening.

What I do know is that the conversation that you two are having in MY thread doesn't have anything to do with MY issue and is not helping ME or ANYONE else make any progress.  

If the two of you have any further suggestions that would help ME, by all means, please share.

If you really want nothing more than to slug it out with each other, might I kindly suggest an exchange of email addresses might provide you with a direct link to each other that wouldn't be interrupted by my posts of what I am trying to do and how I am approaching it.

It has been my hope that someone would pick up on something I was saying and think of something to share with me that might be helpful.  Rampa's post is a perfect example where he suggested adding a second character in order to be able to assign the full version of the song to a voice channel where I could then see the wave form.

If there are any suggestions on embedding MAPI5 tags in my text file, or other ways of manipulating that file that can impact the timing of the words, THAT would be very much appreciated.  

Thanks,

Tim

Edited to add the I for If

There are surely times when it is good to be stubborn.  Knowing when is the gift of experience.

prabhatM
prabhatM
Posted 10 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)Distinguished Member (6.6K reputation)

Group: Forum Members
Last Active: 5 Years Ago
Posts: 1.8K, Visits: 8.1K
sw00000p (5/8/2015)
1. I work on animated characters.
2. I work on Props... some animated.
3. I use 3dxchange to import.

What part of this you don't understand?:w00t:


Then you do a whole lot in ICLONE than just rendering. You set up these animated characters and props. You set up the scene and interactions between the elements. It's not just bringing in a WHOLE PRECOMPOSED SCENE TO ICLONE AND JUST RENDERING IT.
This is not just clicking the Rendering Button in ICLONE and ..................... Poof ...........you are done with ICLONE !!!

And you are not alone. Most people do that in ICLONE. They import many individual elements to ICLONE and work.

When you say :" I just use ICLONE for Rendering and IMPORT everything" , you almost sound as if ICLONE is GOOD FOR NOTHING AS FAR AS YOU ARE CONCERNED and only attractive  feature that it has is RENDERING. You use ICLONE just for a few seconds.

You almost made the World believe that you have a SECRET WEAPON / ALEMBIC IMPORTER FOR ICLONE through which you can IMPORT the whole scene with a single click and then just click the ICLONE render button....Poof !!!

You offer out of context suggestions and blatant wrong impressions to the newbies who look for sincere guidance. 

You take them on an unnecessary wild goose chase. Do that with your clients. They might get impression that you are an EXTRAORDINARY TALENT, and their only hope, and may dole out a few extra bucks. Don't do that with the newbies here on the forum.

Resist that. DO NOT SAY WHAT YOU DO NOT INTEND TO SAY. YOU ARE A TUTOR. YOU MUST KNOW WHAT TO SAY AS A PART OF YOUR REPLIES.

@TimV

You see a few, perhaps 10-15 people being active on the forum. Does it mean ICLONE has only these 15 users all over the world ? Does it mean that only these 15 people have problems with ICLONE ?

No. 98% people do not ask questions. They search the forum for the answers to their problems, read diligently and learn.

So your question actually does not remain yours once you put on the forum. You represent a whole lot of people who have similar issues. When the problem is resolved, it is resolved not only for you but also for whole lot of people out there.
In any reply, every sentence is important, particularly the conclusion part. 

People may offer TENTATIVE SOLUTIONS. That's OK. They are just offering suggestions and may not be perfect. They are participating in a debate with their suggestions. They can make mistakes. No issues.
But.....

When somebody PRETENDS and POSES as an authoratative Industry Expert and offers a DEFINITE ANSWER, he must not BS and be careful what he says.

Hope you got the context of this debate. There is nothing personal about it.


1
2
3
4
5
6
7



Reading This Topic