Just found out something today that might help any of you with issues you are having with your lip sync.

I've been doing a lot of "lip syncing" myself, singing along to others, but today I went back and was recording my voice and face and I noticed, once again, that the lip sync seems a bit "off".  So on a hunch I slipped the audio track forward by some frames and suddenly it not only looked better, it looked perfect.

A few things (and for those of you know know, you can skip this): humans are VERY sensitive to sync issues.  At 24fps (the only thing I use) even a single frame off can be detected, and two or more looks *really* bad.  It's much worse for the audio to be sooner, and for an easy to understand reason -- you can open your mouth without producing sound, but if you hear the sound before the mouth opens (unless you're a ventriloquist) it immediately raises red flags.

In the biz (decades ago when I used to sync reel to reel with actual film footage) we would often "trick" things by slipping the sync (the audio) forward a bit.  It looks fine if someone opens their mouth without the sound coming out, and it actually tends to look better sometimes (particularly with fast dialog).  Naturally you don't want to go too far forward, or you'll have the mouth closing before the sound ends.

I was a bit surprised when I finally ended up with five frames forward looking the best -- that's a LOT (remember we are at 24fps, and that's almost a fifth of a second).  It didn't seem to matter if it was wi-fi or USB, the same issue prevailed (I'm guessing it has something to do with iClone's lag in recording things).  And you can't do it in iClone -- you'll need to do it in your video editor (as if you needed another reason to export single frames -- only way to get 24fps anyway).

You might want to experiment with this if you are also having issues.  It looks so much better to me I don't even think I'm going to do quick edit tests (to MP4) that I used to do.  And, of course, YMMV (may vary quite a bit, particularly if you don't use 24fps).  But it's another trick to have in your arsenal.

---

Alienware Aurora R12, Win 10, i9-119000KF, 3.5GHz CPU, 128GB RAM, RTX 3090 (24GB), Samsung 960 Pro 4TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley

Mike, thank you for this post. On different threads you have helped me understand some performance issues I've been having, and the sync problem has been an issue for me as well. I, too, have been able to rectify this by detaching the audio from the clip and sliding it so that it lines up with the picture. This solution works great, except my primary use for the technology is to bring characters into live programs so I'm hoping that by upgrading my system with faster graphics and more video memory will prevent the problem from happening in a live situation. Since you have a speedy system, it looks like even upgrading my computer won't fix the sync problem. 

Are you able to get convincing lip sync when performing live using your system?

Thanks, always, for your help and comments!

Jeff

It's an interesting question.  On the rare occasions I actually watch my lip sync (most of the time I'm reading text off the screen and not even seeing the character I am using to record) it seems just fine.  IOW, I don't actually notice any slippage until after the recording.  So I suspect that a live performance (if I'm understanding you correct and you are going to somehow use the iClone screen itself as your "performance output") should be okay.  I'd have to run some tests to confirm this, but you can do the same on your end.

---

Alienware Aurora R12, Win 10, i9-119000KF, 3.5GHz CPU, 128GB RAM, RTX 3090 (24GB), Samsung 960 Pro 4TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley