I've actually taken the audio recorded direct in iClone and brought it into my audio programs (much higher end than Premiere, and even using such tools as Melodyne) and gotten good results.  The audio records pretty well in iClone, as long as you use a decent microphone.  

Now, you could easily also record at the same time and then audio match (Premiere has that tool built-in, and there are also plugins that can do that) but I like the ease of being able to get the .WAV file "in-house" so to speak.  None of the examples here had any audio processing, of course, and they sound pretty bad out of the box.

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley

KelleyToons,

I apologize if this information is somewhere in the thread but can you show a recording in which you have facial motion capture and also connected the audio for lip syncing?  I am on the fence about purchasing this and I have been reading you posts which helps since as you stated Reallusion did not provide the ability to truly test the plugin.  Currently i am trying to see if I should purchase this or continue to use Face Cap.  Face Cap is a iphone x app that is $10 and you can capture your facial animation and then send it to yourself as an attachment.  You then can connect them re-target the blendshapes to your character in Maya. All my characters are created from Daz and imported into iclone strictly for animation and exported out to Maya or blender as iClone's renderer is really bad.

I'm a little confused by what you are asking.  The way Faceshift OR Faceware for iClone (they don't call the iPhone X plugin Faceshift but I find it's easier to say to differentiate it from Faceware) both work for lipsync is they capture the muscles of the face and those movements of the lips sync up with the audio.  At the same time you *can* capture the audio for the auto phonome generation.  These phonemes aren't used because they aren't nearly as accurate or as good as the facial mocap.  What IS used are the phonemes for tongue movement (which isn't captured by either camera system -- my wife was really funny because she kept sticking her tongue out and looking at the avatar and saying "why isn't my tongue showing?"  You had to be there).

So I'm not really sure what I'd show you in a video -- all my videos in this thread (starting at the beginning) have the audio captured for the tongue purpose.  I'm not really so sure it's that helpful, but perhaps it is at times.  As I said, iClone doesn't really do a good job at transferring audio into phonemes, not nearly as good as Papagayo (but PG also uses text to match up to it -- I was hoping the Python implementation in iClone would allow us to do things with this, but apparently it won't be very useful at all).

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley

Sorry for any confusion as i appreciate you taking the time to reply.  I have read that the Faceshift plugin records the audio as you stated, but I have seen where you can go in and edit the Visemes (may have mispelled that) in order to create a more accurate representation.  It briefly shows this in one of the Reallusion demos or tutorials.  Here is the video I saw the audio track.  You can briefly see it under the Viseme - Voice - Lips.  You can FFW to 2:10 and pause the video.  https://www.youtube.com/watch?v=89dW2LFR07I
I wondered if you are able to go in and fix any Visemes being used that are not correct.   As you may know face capture and lip sync are 2 different things.  I would like to know if I have the ability to modify lip sync if needed with the FaceShift plugin.

I really hope this clarifies things but if not I do appreciate the info. :) 

I may not be explaining this properly -- let me try it one more time.

The audio has NOTHING to do with lip sync.  Lip sync is wholly generated by the recording of the movements of the lips and areas around the mouth.  These muscle movements are what is captured and what produces the sync.  You actually don't need to edit this at all -- Faceshift is VERY accurate and there is no need to try and improve it (you could not).

The audio that is recorded and the visemes generated are ONLY used for the tongue.  Nothing more.  I hope this explains it (you *could* edit those for the tongue movements, but I haven't found the tongue movements to be all that bad, nor really noticeable much).

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley