import a custom dictionary for acculips for italian language


https://forum.reallusion.com/Topic537535.aspx 		Print Topic | Close Window

By bianchi.lo - 2 Years Ago

I need to use acculips with the Italian language, which is not natively supported. I thought that a possible solution could be to create my own it.dict dictionary containing the main Italian words of my interest and their phonetic translation. already here I met the first problem... on the documentation https://manual.reallusion.com/iClone-8/Content/ENU/8.0/50-Animation/Facial-Animation/Working-on-Dictionary.htm it is said that the dictionary must contain lines of the type WORD VISEMA VISEMA ... while if I look at the content of the native en.dict file I see that it contains WORD PHONEMA PHONEMA ....
and so I was already in doubt about what to do..... to try I took the words of my interest and converted them with epitran (a python utility for phonetic transcription). So I used these phonemes, possibly I could do the same by putting the visemes (since there is the file that contains the phoneme-viseme association in the same folder of the dictionary), but for the moment I'm only trying with the phonemes. Working with them I then found another problem, the IPA standard phonemes do not correspond as symbols exactly to the iClone ones, so I drafted a translation table (based on information found online)to map them to the iclone phonemes.
In the end I did some tests and there is some result, but it's not optimal: the audio is quite in sync with the pauses and with certain syllables but for others it seems that a decidedly wrong viseme is applied. And so now I don't know if this depends on the phonetic translation I had epitran do and which maybe is wrong, or on my conversion table, on the fact that I used phonemes and not visemes or something else :(
I don't know if anyone has ever encountered this kind of situation trying to use a language other than English in acculips, but any suggestions would be really welcome! what do you think about it?
By animagic - 2 Years Ago
There have been requests for other languages in the past, but so far nothing from Reallusion.

I tried to look at it for Dutch, which is my native language, but it seemed too much of a task. All we have are visemes for English, which may or may not correspond to what you need in other languages.

A further problem is, I believe, that besides a dictionary you need a language model, which determines how the visemes are strung together based on the input (to put it very simple).

Also, AccuLips is greatly assisted by the text of the speech, so you need to somehow change the other language into something that looks like English. I think that was the point where I gave up...:unsure: