|
By magnoliaPower - 3 Years Ago
|
I would like to create a new Dict file for Acculips to be more precise with french voices.
Does acculips takes into account other dict than the one specified by default?
Thanks for your reply.
|
|
By animagic - 3 Years Ago
|
I'm glad you brought that up...:D
When AccuLips was introduced (actually during testing) the option of other dictionaries was mentioned, but it never came to fruition. There was a test version of a German dictionary, as I recall.
In fact, if you look under C:\Program Files\Reallusion\iClone 8\Resource\ICTextToSpeech\montreal-forced-aligner\pretrained_models, there's still something for German there...:unsure:
According to RL, the issue was that besides a dictionary you need a language model. This is a specification of how the words in the dictionary map to visemes for the specific language as well as the blend together (simply out).
There was back and forth about it and I still believe it is possible, given that Nvidia can do it with Audio2Face. In other words, the language models are obtainable, but I guess there wasn't much incentive to implement it.
It saddens of course all of us with native languages other than English or an interest in such languages, but such is the situation.
What I tried once is to write a Dutch sentence as it it were mangled English:
The Dutch text is: "Ik wil AccuLips in het Nederlands. Waarom is dat zo moeilijk?" ("I want AccuLips in Dutch. Why is that so difficult?")
That is of course quite a lot of work, but currently it's the only way to do it, AFAIK.
|
|
By mtakerkart - 3 Years Ago
|
Tu n'as pas idee comment j'attendais le french acculips avec impatience!!!! ;)
I started this thread last year. I live near Montreal and I was intriged why a folder was named "montreal-forced-aligner".
https://forum.reallusion.com/486876/Montrealforcedaligner
|
|
By magnoliaPower - 3 Years Ago
|
I think it should be possible.
I do not see why you should have a machine learning while there is a list of english words with their correspondence in phonemes. If you do the same file for french it should load it up. The second file is another table to match phonemes to their viseme. If reallusion says it is allowed to tweak it then they should add their reply here :-)
|
|
By animagic - 3 Years Ago
|
The dictionary is one part and you can add a custom distionary. It's specified in Preference.
The other part is the actual speech recognition, which is where the language model comes in (I didn't make that quite clear in my previous post). That's why I needed to write the Dutch so weird in AccuLips to make it as if the Dutch was really some strange kind of English.
I thought about creating a dictionary but decided it is a lot of work.
|
|
By magnoliaPower - 3 Years Ago
|
|
Ok, but if I use the actual ML model for english and give as a reference a french dict with for each word the correspondance in this ARPABET english phonemes (it looks like that), it should work. I will make a try with few words.
|
|
By Pollux - 3 Years Ago
|
That would be great magnoliaPower, please do try 😊
|
|
By mtakerkart - 3 Years Ago
|
|
give as a reference a french dict with for each word the correspondance in this ARPABET english phonemes
The problem is that some french language sound doesn't exist in english. Exemple : "Le jambon est tentant" .The "an" and "on" have
no really concordance with english blendshape viseme. However You can create "Hiboux" with english blendshape viseme.
I no longer use acculips for my dialogue. I use facial mocap , being careful to have an accentuated diction and a slightly slower delivery than reality because of the latency of the steaming.
|
|
By magnoliaPower - 3 Years Ago
|
As said in the other thread, the labial are still the same with nasalisation.
AN = A ON = O
Test it in front of a mirror, look at your mouth like Monsieur Jourdain.
|
|
By magnoliaPower - 3 Years Ago
|
As you can see in this very quick test, the nasalisation is not a problem.
R is changed in Ah, btw.
|
|
By mtakerkart - 3 Years Ago
|
|
It's a good start. How many time you spend for this 10sec ?
|
|
By Pollux - 3 Years Ago
|
[quote]magnoliaPower (6/26/2022)
As you can see in this very quick test, the nasalisation is not a problem.
R is changed in Ah, btw.
Indeed it is possible to use Acculips with other languages (I do it with Spanish and French). It can be corrected but it takes a lot of time.
|
|
By magnoliaPower - 3 Years Ago
|
To tweak it ? Maybe 3 minuts.
I'm looking at the Python API to see what is possible.
Maybe it could possible to use the Montréal aligner to generate the grid of visemes and then import them, but I'm not sure right now of the process.
|