Profile Picture

Creating AccuLips with Python

Posted By josh_177552 3 Months Ago
You don't have permission to rate!
Author
Message
josh_177552
josh_177552
Posted 3 Months Ago
View Quick Profile
Junior Member

Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 6, Visits: 56
I would like to use Python to access the AccuLips feature with a .wav and .txt source file.

I am able to create lipsync that is equivalent to choosing "Audio File" as the lipsync source with the following code:
import RLPy
avatar_list = RLPy.RScene.GetAvatars()
avatar = avatar_list[0]
viseme_component = avatar.GetVisemeComponent()
audio = RLPy.RAudio.CreateAudioObject()
audio.Load("filename.wav")
result = viseme_component.LoadVocal(audio, RLPy.RGlobal.GetTime(), "Clip Name")

However, I see no way to do this with the AccuLips feature. How can I accomplish this?
I have looked through the documentation and RLPy.py and I was not able to find anything at all.
wires
wires
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (12.9K reputation)Distinguished Member (12.9K reputation)Distinguished Member (12.9K reputation)Distinguished Member (12.9K reputation)Distinguished Member (12.9K reputation)Distinguished Member (12.9K reputation)Distinguished Member (12.9K reputation)Distinguished Member (12.9K reputation)Distinguished Member (12.9K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 5.2K, Visits: 24.6K
Why would you need to load the files in that way? AccuLips will import both an audio and a text file if both have the same name and sync them.

Gerry



System: Win 10 Pro (21H1), Asus X99-E WS, CPU i7-5930K -3,5 GHz, 32 GB DDR4 2666-16 RAM, NVidia GTX 1080 Ti GPU - 11 GB VRAM(Driver Studio-472.12), Samsung 850 Pro 512 GB SSD, 6 TB HD storage.


josh_177552
josh_177552
Posted 3 Months Ago
View Quick Profile
Junior Member

Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 6, Visits: 56
wires (7/20/2021)
Why would you need to load the files in that way? AccuLips will import both an audio and a text file if both have the same name and sync them.

I have 4000 lines of dialogue to process and would prefer not to do it manually.
animagic
animagic
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 13.8K, Visits: 24.5K
I  don't think that the Python API has been updated to include AccuLips.

4000 lines of dialog is an awful lot, so you would need to split that up anyway. That in itself would be more work than running AccuLips after that.


https://forum.reallusion.com/uploads/images/1f3bed13-1788-45e1-9ace-4fd0.jpg


Rampa
Rampa
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (31.3K reputation)Distinguished Member (31.3K reputation)Distinguished Member (31.3K reputation)Distinguished Member (31.3K reputation)Distinguished Member (31.3K reputation)Distinguished Member (31.3K reputation)Distinguished Member (31.3K reputation)Distinguished Member (31.3K reputation)Distinguished Member (31.3K reputation)

Group: Forum Members
Last Active: 5 minutes ago
Posts: 7.7K, Visits: 49.5K
You might be able to automate the import process with your previous method, and then manually convert to Acculips. See here:
https://manual.reallusion.com/iClone_7/ENU/Content/iClone_7/7.9/08-Animation/Facial-Animation/Converting-Old-Talking-Script-with-AccLips.htm
josh_177552
josh_177552
Posted 3 Months Ago
View Quick Profile
Junior Member

Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 6, Visits: 56
animagic (7/20/2021)
I  don't think that the Python API has been updated to include AccuLips.

4000 lines of dialog is an awful lot, so you would need to split that up anyway. That in itself would be more work than running AccuLips after that.

The lines are split up and I have .wav files and .txt files for each of them.
Using the user interface, even if I spent 1 minute on each line it would take 2 weeks to get through them.
animagic
animagic
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)Distinguished Member (26.0K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 13.8K, Visits: 24.5K
@josh: So I assume your desired output is a MotionPlus file for each line?

Currently, that would be the only way to record the facial animation, AFAIK. We used to have .iTalk files, but I think that's old hat.

That said, there may be other options that I'm not aware of...Unsure


https://forum.reallusion.com/uploads/images/1f3bed13-1788-45e1-9ace-4fd0.jpg


Edited
3 Months Ago by animagic
josh_177552
josh_177552
Posted 3 Months Ago
View Quick Profile
Junior Member

Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 6, Visits: 56
animagic (7/21/2021)
@josh: So I assume your desired output is a MotionPlus file for each line?

Currently, that would be the only way to record the facial animation, AFAIK. We used to have .iTalk files, but I think that's old hat.

That said, there may be other options that I'm not aware of...Unsure


I am exporting a custom file format from the blend weight, bone transform, and phoneme values I am able to access on the viseme component.
I have the whole automation process fully working using the older lipsync function that only uses the audio file for input.
So the problem is not with the export, only with making the input be used in the desired way (AccuLips as opposed to the older, previously existing automatic lipsync function).
The only thing I am missing is the AccuLips feature which produces noticeably better lipsync.
josh_177552
This post has been flagged as an answer
josh_177552
Posted 3 Months Ago
View Quick Profile
Junior Member

Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)Junior Member (155 reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 6, Visits: 56
I found that iClone is using a free, open source software called Montreal Forced Aligner to generate the phonemes, which is the main data I wanted to generate.
So, I have instead built my pipeline around this same software and skipped iClone entirely. Maybe I will find a use for iClone in the future.
mtakerkart
mtakerkart
Posted 3 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (13.0K reputation)Distinguished Member (13.0K reputation)Distinguished Member (13.0K reputation)Distinguished Member (13.0K reputation)Distinguished Member (13.0K reputation)Distinguished Member (13.0K reputation)Distinguished Member (13.0K reputation)Distinguished Member (13.0K reputation)Distinguished Member (13.0K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 2.8K, Visits: 24.2K
I found that iClone is using a free, open source software called Montreal Forced Aligner to generate the phonemes, which is the main data I wanted to generate.
So, I have instead built my pipeline around this same software and skipped iClone entirely. Maybe I will find a use for iClone in the future


I started this thread last month about the montreal forced aligner :
https://forum.reallusion.com/Topic486876.aspx

Could you share/show what pipeline you use ? I would like to use french acculips.

Thank you
Edited
3 Months Ago by mtakerkart



Reading This Topic