I would like to create a new Dict file for Acculips to be more precise with french voices.
Does acculips takes into account other dict than the one specified by default?
Thanks for your reply.

--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

I'm glad you brought that up...:D

When AccuLips was introduced (actually during testing) the option of other dictionaries was mentioned, but it never came to fruition. There was a test version of a German dictionary, as I recall.

In fact, if you look under C:\Program Files\Reallusion\iClone 8\Resource\ICTextToSpeech\montreal-forced-aligner\pretrained_models, there's still something for German there...:unsure:

According to RL, the issue was that besides a dictionary you need a language model. This is a specification of how the words in the dictionary map to visemes for the specific language as well as the blend together (simply out).

There was back and forth about it and I still believe it is possible, given that Nvidia can do it with Audio2Face. In other words, the language models are obtainable, but I guess there wasn't much incentive to implement it.

It saddens of course all of us with native languages other than English or an interest in such languages, but such is the situation.

What I tried once is to write a Dutch sentence as it it were mangled English:

76% of original size (was 661x19) - Click to enlarge

The Dutch text is: "Ik wil AccuLips in het Nederlands. Waarom is dat zo moeilijk?" ("I want AccuLips in Dutch. Why is that so difficult?")

That is of course quite a lot of work, but currently it's the only way to do it, AFAIK.

---

76% of original size (was 661x19) - Click to enlarge

Tu n'as pas idee comment j'attendais le french acculips avec impatience!!!!  ;)

I started this thread last year. I live near Montreal and I was intriged why a folder was named "montreal-forced-aligner".

https://forum.reallusion.com/486876/Montrealforcedaligner

I think it should be possible.
I do not see why you should have a machine learning while there is a list of english words with their correspondence in phonemes. If you do the same file for french it should load it up. The second file is another table to match phonemes to their viseme. If reallusion says it is allowed to tweak it then they should add their reply here :-)

--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

The dictionary is one part and you can add a custom distionary. It's specified in Preference.

The other part is the actual speech recognition, which is where the language model comes in (I didn't make that quite clear in my previous post). That's why I needed to write the Dutch so weird in AccuLips to make it as if the Dutch was really some strange kind of English.

I thought about creating a dictionary but decided it is a lot of work.

---

76% of original size (was 661x19) - Click to enlarge

Ok, but if I use the actual ML model for english and give as a reference a french dict with for each word the correspondance in this ARPABET english phonemes (it looks like that), it should work. I will make a try with few words.

--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

That would be great magnoliaPower, please do try 😊

give as a reference a french dict with for each word the correspondance in this ARPABET english phonemes

The problem is that some french language sound doesn't exist in english. Exemple : "Le jambon est tentant" .The "an" and "on" have
no really concordance with english blendshape viseme. However You can create  "Hiboux" with english blendshape viseme.

76% of original size (was 661x19) - Click to enlarge

I no longer use acculips for my dialogue. I use facial mocap , being careful to have an accentuated diction and a slightly slower delivery than reality because of the latency of the steaming.

As said in the other thread, the labial are still the same with nasalisation.
AN = A ON = O 
Test it in front of a mirror, look at your mouth like Monsieur Jourdain.



--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

As you can see in this very quick test, the nasalisation is not a problem.
R is changed in Ah, btw.




--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse