To tweak it ? Maybe 3 minuts.
I'm looking at the Python API to see what is possible. 
Maybe it could possible to use the Montréal aligner to generate the grid of visemes and then import them, but I'm not sure right now of the process.



--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

[quote]magnoliaPower (6/26/2022)

---

As you can see in this very quick test, the nasalisation is not a problem.
R is changed in Ah, btw.
Indeed it is possible to use Acculips with other languages (I do it with Spanish and French). It can be corrected but it takes a lot of time.

It's a good start. How many time you spend for this 10sec ? 

As you can see in this very quick test, the nasalisation is not a problem.
R is changed in Ah, btw.




--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

As said in the other thread, the labial are still the same with nasalisation.
AN = A ON = O 
Test it in front of a mirror, look at your mouth like Monsieur Jourdain.



--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

give as a reference a french dict with for each word the correspondance in this ARPABET english phonemes

The problem is that some french language sound doesn't exist in english. Exemple : "Le jambon est tentant" .The "an" and "on" have
no really concordance with english blendshape viseme. However You can create  "Hiboux" with english blendshape viseme.

76% of original size (was 658x19) - Click to enlarge

I no longer use acculips for my dialogue. I use facial mocap , being careful to have an accentuated diction and a slightly slower delivery than reality because of the latency of the steaming.

That would be great magnoliaPower, please do try 😊

Ok, but if I use the actual ML model for english and give as a reference a french dict with for each word the correspondance in this ARPABET english phonemes (it looks like that), it should work. I will make a try with few words.

--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

The dictionary is one part and you can add a custom distionary. It's specified in Preference.

The other part is the actual speech recognition, which is where the language model comes in (I didn't make that quite clear in my previous post). That's why I needed to write the Dutch so weird in AccuLips to make it as if the Dutch was really some strange kind of English.

I thought about creating a dictionary but decided it is a lot of work.

---

76% of original size (was 658x19) - Click to enlarge

I think it should be possible.
I do not see why you should have a machine learning while there is a list of english words with their correspondence in phonemes. If you do the same file for french it should load it up. The second file is another table to match phonemes to their viseme. If reallusion says it is allowed to tweak it then they should add their reply here :-)

--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse