I realized that when I play audio in iClone 7 in the Viseme track in the timeline, it doesn't match at all with my audio waveform. I realized this when trying to add phonemes via the Lips Editor and couldn't get anything to match. After some inspection, I see that the scrubbing of audio actually matches well to the audio waveform. However, when I simply press play, the play head begins to fall way behind the audio waveform. If I stop the playback, it will start as expected and, for a moment, will play the audio at that point. But the longer it plays, the more the audio/voice gets ahead of the audio waveform. And I don't' mean by a little; it's off by a lot. If I play three of four sentences, it falls behind by about a sentence. To say it another way, in normal playback, the audio waveform is too long in the timeline. When scrubbing, it seems fine.  

I'm exporting/bouncing from Logic Pro. I've tried bouncing as both .wav and .mp3 as well as 16bit and 24bit with a sample rate of 44.1. But no luck.        

What could be wrong?!
Thanks

This is well known:



My Python routine for using Papagayo adjusts for this:



In fairness, since I first reported this issue, they HAVE made a few improvements, but it's still off so you'll need to adjust a bit.

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley

Thanks, Mike (my last name is Kelley, too, BTW).

The issue in your first video sounds like it's the same as mine. However, mine isn't out of sync by a fixed number of frames. It falls out of sync progressively over the course of my audio file. So I can't simply correct it with an offset. If I set the play head at any point in my audio waveform, the sound will be in sync at that point and then begin to fall off.  

I will look into the Papagayo script, but it doesn't seem like that quite solves the issue either? 

I had much better luck in Crazy Talk in the past. I wonder if I should just do the phoneme work there?  

If you know of any other solutions or know of other threads relating to this issue, I would love to know. I couldn't find any others, but maybe I'm using the wrong search terms. 

This is a long shot, but on the Play bar, do you perhaps have "By Frame" selected rather than "Realtime"?

https://manual.reallusion.com/iClone_7/ENU/Content/iClone_7/7.7/04_Introducing_the_User_Interface/Play_bar_and_play_control_pane.htm.

---

In addition to what Job said you might just understand that the audio track in iClone isn't really meant to be anything other than a placeholder.  IOW, you really want to do your audio work in your video editor after you render to frames anyway.

In addition, visimes will be much improved in iClone 8 -- so at least if you're younger than me you'll see this solved in your lifetime.

(As I'm sure you know, it's VERY unusual to spell our name that way.  I'm sure you've also experienced folks not getting it right -- reporters used to get it wrong so many times I just gave up on the theory that any publicity was better than no publicity).

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley

I think Animagic's point about Realtime vs By Frame might have been the issue. I've set it to Realtime (at least, "Realtime" is what I see on the screen now) and now when I press play, the play head seems to match the audio in the waveform. Thanks! 

I don't think I quite made this clear in my original post, but, previously, even when I tried to match the visemes to the audible sounds when scrubbing (ignoring how the waveform looked altogether) I still couldn't get the mouth movement to match in playback. But, as I play it now, I think it might be working as expected. I'll report back after I do a little more work on it. 

(Re Kelley: I know, it's a very uncommon name. My first name is also an unusual spelling of a common word. It's very rare for both of my names to be spelled correctly by others!) 

Thanks a bunch, everyone! I see you're all active on here. I've been reading some insightful threads.  

The "Realtime" "By Frame" option was the issue. I just completed a viseme track without any playback issues and the waveform is the right length (the play head stays at the correct part of the waveform in playback). 

The only slightly annoying issue that remains is that the scrub audio isn't accurate. It always suggests that the voice starts before it actually does (and before the waveform for the voice begins), but I suspect this is just how iClone works.  

Thanks again! 

I rendered a video of my iClone project and I believe the video (and viseme) is slightly out of sync with the audio in the rendered .mp4 file. Has anyone had a similar issue? I'm posting here because I wonder if this might be related to my previous question above where the solution was to change the Time Mode to Realtime. 

The audio and video look out of sync when I play the exported .mp4 file on my Windows machine as well as when I play it in Adobe Premiere on my Mac. Unless I'm imagining things, it seems like the video falls very slightly behind the audio as the time goes on. It doesn't look terrible, it's just not as tight as it looks in iClone. 

I rendered my .mp4 at 30 fps 1920x1080 with the Compressor Qualities at maximum. All other settings were on default. My Premiere project was set up with the same settings: 1920x1080 at 30 fps. I'm using iClone V 7.73

I tried importing my source audio into Premiere and aligning it up with the rendered iClone video, but I couldn't get a perfect sync that way either.

Any suggestions would be appreciated. 

 

Oh my goodness thank you, I knew I clicked something, but didnt know what. Thank you 

Markom3D

Please Subscribe to the Markom3D Youtube Channel. 

You just saved my life with your long shot!!!