Profile Picture

Audio playback out of sync with waveform in the Visime track in timeline

Posted By forestkelley 4 Years Ago
You don't have permission to rate!

Audio playback out of sync with waveform in the Visime track in...

Author
Message
forestkelley
forestkelley
Posted 4 Years Ago
View Quick Profile
Veteran Member

Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 18, Visits: 121
I realized that when I play audio in iClone 7 in the Viseme track in the timeline, it doesn't match at all with my audio waveform. I realized this when trying to add phonemes via the Lips Editor and couldn't get anything to match. After some inspection, I see that the scrubbing of audio actually matches well to the audio waveform. However, when I simply press play, the play head begins to fall way behind the audio waveform. If I stop the playback, it will start as expected and, for a moment, will play the audio at that point. But the longer it plays, the more the audio/voice gets ahead of the audio waveform. And I don't' mean by a little; it's off by a lot. If I play three of four sentences, it falls behind by about a sentence. To say it another way, in normal playback, the audio waveform is too long in the timeline. When scrubbing, it seems fine.  

I'm exporting/bouncing from Logic Pro. I've tried bouncing as both .wav and .mp3 as well as 16bit and 24bit with a sample rate of 44.1. But no luck.        

What could be wrong?!
Thanks
Kelleytoons
Kelleytoons
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)

Group: Forum Members
Last Active: 1 hour ago
Posts: 9.1K, Visits: 21.8K
This is well known:



My Python routine for using Papagayo adjusts for this:



In fairness, since I first reported this issue, they HAVE made a few improvements, but it's still off so you'll need to adjust a bit.



Alienware Aurora R12, Win 10, i9-119000KF, 3.5GHz CPU, 128GB RAM, RTX 3090 (24GB), Samsung 960 Pro 4TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley
Edited
4 Years Ago by Kelleytoons
forestkelley
forestkelley
Posted 4 Years Ago
View Quick Profile
Veteran Member

Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 18, Visits: 121
Thanks, Mike (my last name is Kelley, too, BTW).

The issue in your first video sounds like it's the same as mine. However, mine isn't out of sync by a fixed number of frames. It falls out of sync progressively over the course of my audio file. So I can't simply correct it with an offset. If I set the play head at any point in my audio waveform, the sound will be in sync at that point and then begin to fall off.  

I will look into the Papagayo script, but it doesn't seem like that quite solves the issue either? 

I had much better luck in Crazy Talk in the past. I wonder if I should just do the phoneme work there?  

If you know of any other solutions or know of other threads relating to this issue, I would love to know. I couldn't find any others, but maybe I'm using the wrong search terms. 
Edited
4 Years Ago by forestkelley
animagic
animagic
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)Distinguished Member (32.5K reputation)

Group: Forum Members
Last Active: 9 hours ago
Posts: 15.7K, Visits: 30.5K
This is a long shot, but on the Play bar, do you perhaps have "By Frame" selected rather than "Realtime"?

https://manual.reallusion.com/iClone_7/ENU/Content/iClone_7/7.7/04_Introducing_the_User_Interface/Play_bar_and_play_control_pane.htm.


https://forum.reallusion.com/uploads/images/436b0ffd-1242-44d6-a876-d631.jpg

Kelleytoons
Kelleytoons
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)Distinguished Member (35.6K reputation)

Group: Forum Members
Last Active: 1 hour ago
Posts: 9.1K, Visits: 21.8K
In addition to what Job said you might just understand that the audio track in iClone isn't really meant to be anything other than a placeholder.  IOW, you really want to do your audio work in your video editor after you render to frames anyway.

In addition, visimes will be much improved in iClone 8 -- so at least if you're younger than me you'll see this solved in your lifetime.

(As I'm sure you know, it's VERY unusual to spell our name that way.  I'm sure you've also experienced folks not getting it right -- reporters used to get it wrong so many times I just gave up on the theory that any publicity was better than no publicity).



Alienware Aurora R12, Win 10, i9-119000KF, 3.5GHz CPU, 128GB RAM, RTX 3090 (24GB), Samsung 960 Pro 4TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley
forestkelley
forestkelley
Posted 4 Years Ago
View Quick Profile
Veteran Member

Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 18, Visits: 121
I think Animagic's point about Realtime vs By Frame might have been the issue. I've set it to Realtime (at least, "Realtime" is what I see on the screen now) and now when I press play, the play head seems to match the audio in the waveform. Thanks! 

I don't think I quite made this clear in my original post, but, previously, even when I tried to match the visemes to the audible sounds when scrubbing (ignoring how the waveform looked altogether) I still couldn't get the mouth movement to match in playback. But, as I play it now, I think it might be working as expected. I'll report back after I do a little more work on it. 

(Re Kelley: I know, it's a very uncommon name. My first name is also an unusual spelling of a common word. It's very rare for both of my names to be spelled correctly by others!) 

Thanks a bunch, everyone! I see you're all active on here. I've been reading some insightful threads.  
forestkelley
forestkelley
Posted 4 Years Ago
View Quick Profile
Veteran Member

Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 18, Visits: 121
The "Realtime" "By Frame" option was the issue. I just completed a viseme track without any playback issues and the waveform is the right length (the play head stays at the correct part of the waveform in playback). 

The only slightly annoying issue that remains is that the scrub audio isn't accurate. It always suggests that the voice starts before it actually does (and before the waveform for the voice begins), but I suspect this is just how iClone works.  

Thanks again! 
forestkelley
forestkelley
Posted 4 Years Ago
View Quick Profile
Veteran Member

Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)Veteran Member (956 reputation)

Group: Forum Members
Last Active: 4 Years Ago
Posts: 18, Visits: 121
I rendered a video of my iClone project and I believe the video (and viseme) is slightly out of sync with the audio in the rendered .mp4 file. Has anyone had a similar issue? I'm posting here because I wonder if this might be related to my previous question above where the solution was to change the Time Mode to Realtime. 

The audio and video look out of sync when I play the exported .mp4 file on my Windows machine as well as when I play it in Adobe Premiere on my Mac. Unless I'm imagining things, it seems like the video falls very slightly behind the audio as the time goes on. It doesn't look terrible, it's just not as tight as it looks in iClone. 

I rendered my .mp4 at 30 fps 1920x1080 with the Compressor Qualities at maximum. All other settings were on default. My Premiere project was set up with the same settings: 1920x1080 at 30 fps. I'm using iClone V 7.73

I tried importing my source audio into Premiere and aligning it up with the rendered iClone video, but I couldn't get a perfect sync that way either.

Any suggestions would be appreciated. 

 
Edited
4 Years Ago by forestkelley
Markom3D
Markom3D
Posted 3 Years Ago
View Quick Profile
Senior Member

Senior Member (262 reputation)Senior Member (262 reputation)Senior Member (262 reputation)Senior Member (262 reputation)Senior Member (262 reputation)Senior Member (262 reputation)Senior Member (262 reputation)Senior Member (262 reputation)Senior Member (262 reputation)

Group: Forum Members
Last Active: 3 Months Ago
Posts: 19, Visits: 176
Oh my goodness thank you, I knew I clicked something, but didnt know what. Thank you 

Markom3D

Please Subscribe to the Markom3D Youtube Channel


Happy Sheep
Happy Sheep
Posted 2 Years Ago
View Quick Profile
Junior Member

Junior Member (244 reputation)Junior Member (244 reputation)Junior Member (244 reputation)Junior Member (244 reputation)Junior Member (244 reputation)Junior Member (244 reputation)Junior Member (244 reputation)Junior Member (244 reputation)Junior Member (244 reputation)

Group: Forum Members
Last Active: 2 Years Ago
Posts: 13, Visits: 121
You just saved my life with your long shot!!!



Reading This Topic