English


Profile Picture

Montreal-forced-aligner

Posted By mtakerkart 4 Years Ago
You don't have permission to rate!
1
2

Montreal-forced-aligner

Author
Message
magnoliaPower
magnoliaPower
Posted 3 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)Distinguished Member (1.8K reputation)

Group: Forum Members
Last Active: Last Year
Posts: 93, Visits: 353
The nasal phonemes do not imply change to labial. When you say "an", your lips move like "A", "ON" = O and so on...
Maybe in french canadian you still make the distinction with the words "brin" and "brun", but we do not in France (except some regions).
So the dictionnary for visemes would be usable after some minor changes :
- Except the stress symbol like that does not make sense in french
- and you may have to add NONE visemes after words, as we (french) articulate more than US people (aristocratic UK may have more articulation) and don't keep the mouth half opened between words.

There is many ways to proceed : 
(1) try to make a plugin, but not sure it is possible because the RIVisemeComponent does not seem to be clearly connected to the Acculips function.
(2) ask Reallusion to let volunteers to train a ML (or get one through university) and set the dict for new languages.
(3) ask Reallusion to let user copy/paste visemes directly to a text field instead of natural language text.






--------
PC : AMD Ryzen 9 5900X 12-Core Processor 3.70 GHz
RAM: 128 Go
OS: Windows 11 64 bits
Graphics Card : NVidia GeForce RTX 3090


Softwares: Reallusion products, Adobe suite, Blender, NVidia Omniverse

animagic
animagic
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 15.8K, Visits: 31.3K
I wonder if RL could just include a Language plugin so that we could add languages ourselves and then make available to others as well.

RL users are everywhere so I think there would certainly be interest for this and it would make iClone attractive internationally.



mtakerkart
mtakerkart
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 3.2K, Visits: 29.2K
I have already tried to replace with the French file by renaming it en.dict but unfortunately Iclone uses the English file.
Moreover the blendshape phoneme are specific to the English language, it would be necessary to add some for the sound "in", "on", "an" which is specific to the French language.
 I remain convinced that it is child's play for Reallusion to add this french dictionary ...
animagic
animagic
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 15.8K, Visits: 31.3K
I played a little bit with that using Dutch, but currently it seems that the system only accepts a dictionary called en.dict and the corresponding en.PhonemeVisemeMapping. As I don't make films in Dutch, I didn't go any further with it.

So your French files would need to be renamed as if they were English and you would have to figure out the mapping. What I don't know if the system also expects a specific naming of the phonemes or if the mapping file would take care of that.



mtakerkart
mtakerkart
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 3.2K, Visits: 29.2K
Oh my god Animagic! Thank you

It seams that the fr.dict have the same structure like the en.dict.
it would just be necessary to create a file fr.PhonemeVisemeMapping isn't it ?  :ermm:
animagic
animagic
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 15.8K, Visits: 31.3K
Here is the link for French: https://sourceforge.net/projects/cmusphinx/files/Acoustic%20and%20Language%20Models/French/.

The dictionary is the file fr.dict. It lists words followed by an encoding of the phonemes.

This is from the CMU Sphinx project. The idea is to take a list like that and then have a conversion routine that generates the visemes.




mtakerkart
mtakerkart
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 3.2K, Visits: 29.2K
hohoho! you guys are real investigators!

Indeed Yoshua Bengio (who is a professor at McGill) is one of the pioneers of deep learning, the approach that revolutionized artificial intelligence.

Animagic could you show the link where you found these online pronunciation dictionaries ?
animagic
animagic
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 15.8K, Visits: 31.3K
The aligner seems to be based on work done at McGill University, which is in Montreal. 

I don't know exactly what is holding things up because there are large online pronunciation dictionaries (I was looking for Dutch and found one), unless there are licensing issues for commercial use.



mtakerkart
mtakerkart
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 3.2K, Visits: 29.2K
Thank you Jeff

I looked in this file to see if the update announced by Miranda had a link with the French dictionary that we are waiting for.
https://forum.reallusion.com/FindPost486813.aspx
 I was curious to know why RL called this file "Montreal" because I live in Montreal which is a French Canadian city.
Let's say I was hoping for something ...  :ermm:
jlittle
jlittle
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (7.0K reputation)Distinguished Member (7.0K reputation)Distinguished Member (7.0K reputation)Distinguished Member (7.0K reputation)Distinguished Member (7.0K reputation)Distinguished Member (7.0K reputation)Distinguished Member (7.0K reputation)Distinguished Member (7.0K reputation)Distinguished Member (7.0K reputation)

Group: Forum Members
Last Active: Last Year
Posts: 1.9K, Visits: 17.3K
I did a search online for "montreal forced aligner" and came up with this:

What is forced alignment?

Forced alignment is a technique to take an orthographic transcription of an audio file and generate a time-aligned version using a pronunciationdictionary to look up phones for words.

Might be part of acculips or something to do with phonemes?

Jeff




Get my Plugin Manager and TaskNotes plugins for iClone.
Check out EZColors and Other products for CTA/CA. EZColors: the easy way to change Render Styled (RS) colors!
See my CTA Tutorials on the YouTube channel CTAStepByStep


1
2



Reading This Topic

0 active, 0 guests, 0 members, 0 anonymous.
No members currently viewing this topic!