Hi,
It seems the iClone lipsync engine is having hard time to detect or at least place correctly the detected visemes.

I did a little test to verify that, and to compare I also did a manual detection of the visemes (up until "its like a jungle../.."):



(iClone script done on the acapella file)
As you can see the results are far from optimal but perhaps some of you got better results ?

I would be interested to hear from you guys, if you have done lipsync tests and experiments on your side.

Cheers,
Guy.


--
guy rabiller | GOETZIWOOD STUDIOS
"N.O.E." (Nations Of Earth) Sci-Fi TV Show, Showrunner.

This looks really cool. Great song too.
I have been trying a few things out with Crazy talk 8 and getting worse results. I have considered using the text to speech tool to create the correct lip / mouth movements then doing a voice over in another program but then the timing wouldn't flow as well.

I will also try in iclone and let you know how I get on.


----
Designer, Illustrator, Artist, Developer, Modeller - Augmented Reality : Instagram and Facebook Camera Effects
www.realityaugmented.co.uk

I think iC6 is improved over iC6, but if you want the best-possible results, plan on doing some manual viseme editing.  For clean-up, usually removing some helps.  That's step #1 for me.   

iC5 had way too many, especially too many "None" visemes, making the mouth look spastic.  A good "live" recording has always given me better results that Text-to-Speech.  And it should be "voice-only."  The engine cannot separate out vocals from music in a song.  I record my own voice-over for a song to get the visemes, and replace the audio in my video editing software.

If you spend too much time editing visemes, real live people talking to you will start to look funny, and after a while you're not sure what looks real and natural anymore. 

---

iClone 7... Character Creator... Substance Designer/Painter... Blender... Audacity...
Desktop (homebuilt) - Windows 10, Ryzen 9 3900x CPU, GTX 1080 GPU (8GB), 32GB RAM, Asus X570 Pro motherboard, 2TB SSD, terabytes of disk space, dual  monitors.
Laptop - Windows 10, MSI GS63VR STEALTH-252, 16GB RAM, GTX 1060 (6GB), 256GB SSD and 1TB HDD

One of my favorite song ever Grabiller! Motions on the beat is very rare in the Iclone world. I never had better results and my problem is that the majority of my voices are in french language and as you know there're sounds that does'nt exist in english language that the Iclone detection algorythm is not enable to detect. because lot of  network cartoons productions are so so with the match between viseme and mouth shape, the audience is very tolerant about lipsync. I focus only on the vowel.

I know Swooop. I can have perfect result but the time consuming nedeed for it is not in my philosophy why I bought Iclone.It will be the same time consiming as  blender or Maya.
But I notice that Iclone is optimize for english viseme. In the expression editor , if I build a perfect "th" viseme with visible tongue on up-teeth, it's useless
because this sound doesn't exist in french. When english people said "e" it's "i" for french language. May be  with the new render release I'll built an
efficient CC viseme build for French language. ;-)

@grabiller: I usually edit the visemes, especially for close-ups. iClone viseme detection has not substantially improved since it was introduced. It got a little bit better in iClone 6 as far as blending goes, but there are still many spurious visemes being generated.

I have suggested for a couple of years now an approach where the viseme detection gets help from the text of the dialog. This approach is not new, I came across in a product called LipSync (now Mimic) that allowed you to do lip-synching in Poser. It worked quite will. Maybe I should make this a wishful feature, because things could be much better. Or, we get your SDK/scripting approved and we can build our own.

---

You did a helluva job on this!

justaviking and sw00000p had some very helpful comments, I generally delete all the suggested visemes straight away as it takes longer to hunt, peck, move and delete than it is to put what I want when and where I want it. One thing I would add is the mouth always makes the shape a few frames before the sound, and to paraphrase JV what you don't set is as important as what you do, just like playing music. Don't forget holds, avoid twinning (Two similar sounds next to each other) tweak one to show movement of the mouth while retaining a slightly different but similar shape. Its a skill that develops over time and eventually becomes second nature, you eventually won't need charts for reference.

One thing I hope that gets addressed soon is the IC  facial animation limitations, one of the first rules of creating believable character facial animation and lipsync is to avoid twinning as in (The mouth repeats the same static shapes over and over - watch anyone talk and notice their mouth, eyes, cheeks, lips.. seldom move symmetrically, the left side seldom mimics the right side) hence the robotic wooden appearance. Trying to make asymmetrical facial expressions and visemes in IC is difficult when you only have a couple sections of the face to move which often makes a characters face look disfigured.

Rant aside, you did a great job on this, you obviously know what your doing.


Virtual Media® / VTV® / The Virtual Entertainment Network...
http://VTVLive.com

Thanks for the comments sw00000p, I've been at it for awhile and always learning.  I can't understand why RL keeps adding all the random bells and whistles and doesn't address the animation limitations, it's really a shame considering IC at it's core is an animation program.


Virtual Media® / VTV® / The Virtual Entertainment Network...
http://VTVLive.com

Yep keys alignment is an issue, though not as big as the excess redundant keys issue
Your actors look like CC... (and simple but effective use of HDR btw)
I trust that you've already cleaned up the auto-generated mess with the 2 extra right click options activated only on CC



CC certainly have more refined viseme shapes than G5
(even then I always redo and custom all lipsync morphs and face profiles)
But no face profile customization experties can overcome antique lip sync process
iCloners who use G5 G6 Genesis1/2/3 Poser/ Max/ Maya/ Blender/ Mixamo rig
still suffer unnecessary time loss: no reduce redundant keys mess, nor smoothing fix
either manual and eyeball it, carefully click on cute little overlapped TLD TLD TLD keys 
or do the "faster" way: erase auto-generated keys and start all over from scratch!
because...
Two crucial LipSync fix: Reduce Lip Keys + Lip Sync Options
are STILL not implemented globally for Non-Standards or G5/G6 characters!

I have just added a new suggestion
Enable Lip Sync Options & Reduce Lip Keys for ALL iClone iAvatars
Do vote or (if you've run out of votes) do add comment



  >>> Indie Generalist modeler-rigger-animator <<< Hardware: AlienWare i7-HK 32G GTX1080 2T-SSDs BenqSW320 Wacom-Intuous 3Dconnexion ||| Software: IC6 Pipeline - CTA3 Pipeline - ATK2 - Indigo ||| Zbrush4R8. Blender 2.79. DazStudio 4.9. Carrara. Poser. Octane.

I'm slightly more optimistic, and I think...
Adding "standard basic" animation tools
will not lead to anim-novice suddenly dumping presets for DIY custom sets
Just like
Adding "standard basic" dynamic vertex morph tools
will not lead to modeling-novice suddenly dumping pre-made assets for sculpting
Case in point:
RL CC dev team has made it SUPER EASY to mouse-manipulate CC body parts
as in gamer-easy to transform neck/ trunk/ chest/ hip...No need to touch sliders even!
Yet?
There remained many perma-novice who'd scoff at "mastering" EZmouse-CC-interface...
but would spend crazy amount of mone/ time/ energy to have?
One single hyper-specific PRESET something!
So no worries, RL can add as many new functionalities
as many parameters, splines, controllers, skeleton-hierarchy exposure as they have been requested...
the novice preset content consumers market, a done deal.

In STARK contrast...
The pro-ish scripts & functionalities consumers front
97% under-exploited!
IMO of course

Regardless of what we think RL is thinking
"Pose to pose" possibility is, at least, being considered
at best, "Assigned and WIP"
which incidently - if the face bones are incorporated as detailed in that request
will take care of most face expression adjustment issues mentioned in this thread.

So it's not if RL will improve iClone for character animators
They will
It is, when?
Yesterday, I hope



  >>> Indie Generalist modeler-rigger-animator <<< Hardware: AlienWare i7-HK 32G GTX1080 2T-SSDs BenqSW320 Wacom-Intuous 3Dconnexion ||| Software: IC6 Pipeline - CTA3 Pipeline - ATK2 - Indigo ||| Zbrush4R8. Blender 2.79. DazStudio 4.9. Carrara. Poser. Octane.