Hi AlbinosGambit,

SALSA is designed to provide a fast and easy implementation of lip-sync, emotion control, head, eye, and eyelid tracking, and blink control. It only works in the Unity Game Engine.

Hi Michael.

Good advice, so with SALSA we can use it for lip sync our Characters, and also back to iClone again as well or just for Unity?

---

WIN 10, COOLERMASTER MASTERCASE 500H, MSI GAMING PRO CARBON X470 RGB, AMD Ryzen 7 2700X CPU, MSI GeForce GTX 1650 GAMING X - 4GB GDDR5 RAM GPU, Corsair Vengeance RGB PRO DDR4 3200MHz 16GB RAM, SSD M.2 Corsair Force Series MP510 480 GB, SEAGATE BARRACUDA 3.5 HDD 4 TB.

Best regards
Albin
(AlbinosGambit)

Thanks a lot for the reply, I'll check out the resources you linked to.

Hi,

This is Michael from Crazy Minnow Studio (SALSA LipSync Suite). Our contact at Reallusion dropped us a line to let us know there was some SALSA discussion going on here. I'll just clarify a few things and if anyone has additional questions you can inquire on our Unity forum. SALSA is a real-time system, meaning it can analyze audio in real-time to perform lip-sync. It doesn't matter if that audio is a prerecorded clip, a live microphone stream, real-time text-to-speech, or VoIP data in a multiplayer game chat. There are many parameters available to adjust and tune your lip-sync results, we've got thorough documentation on the topic at the link below.

https://crazyminnowstudio.com/docs/salsa-lip-sync/modules/salsa/using/

..our Unity forum.
https://forum.unity.com/threads/salsa-lipsync-suite-lip-sync-emote-head-eye-and-eyelid-control-system.242135/page-36

Michael

Ahh, I see what you mean. I really don't know. Maybe one of the more knowledgeable guys will see this and help you out. Too bad the results are not very good, it looked like a great solution to work within Unity. 

Thank you for your replies.

I am already using the 'oneclick' scripts from SALSA, and it provides reasonable results. However it's obviously not as good as it would be if I'm doing it directly through IClone. So I was wondering if there's anything I can do to improve it before export, e.g. if I lipsync it in Iclone, whether this would make any difference to how well SALSA performs? My instinct is that it won't, and that at best Lipsyncing in Iclone would result in exported animation clips which SALSA won't work with.

The more videos I watch on this asset, I'm liking it. I don't think iClone works in this pipeline. From my understanding so far:

1. Export CC3 character for Unity (fbx).
2. Make sure auto script is used. It currently does not support anything newer than Unity 2019.1.
3. Import CC3 character by dragging into the CC_Assets folder.

This can be used right on the Timeline in Unity.

Any lip-syncing can be done in iClone, and then you will be able to lip-sync unseen audio. An update has been announced to improve the lip-syncing process.

Someone else will have to answer if that can actually be exported via FBX. I don't use that pipeline.

---

I know that if you have iclone and 3d exchange pipeline you can create all the animations and lipsyncing right inside iclone and then export the alembic file over into unity and it will have all the animations including soft cloth.

Sorry, I'm not sure what this means, this is still baking certain mouth animations into the model and then exporting them, right? 

By real-time do you mean to show the lip-sync on the character while speaking? 

It seems that Salsa uses an audio file as input, so it's not really real-time. It seems that Salsa uses an audio file as input, so it's not really real-time. 

Real real-time is possible with a facial mocap system (Faceware or Live Face)Real real-time is possible with a facial mocap system (Faceware or Live Face)

To be clear I don't mean realtime as in, the character needs to speak given live mic input. I mean it needs to be able to do it at runtime having never seen the audio input before. Basically my scenario is, when In Unity I will have audio files/text files with given phrases, and I want to be able to feed these to SALSA, which in turn will animate the model appropriately. The audio/text will be generated dynamically so I can't know what will be said ahead of time. What I'm trying to figure out is is this possible with Iclone's animation? Because as far as I can see it's more designed for prerecording and baking speech animations into the model, rather than being able to adapt to new text.

Thank you both