Profile Picture

"Enhancement" of Mike's Papagayo script

Posted By animagic 2 Months Ago
You don't have permission to rate!
Author
Message
animagic
animagic
Posted 2 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)

Group: Forum Members
Last Active: 3 hours ago
Posts: 11.3K, Visits: 19.5K
As an exercise to get into Python, I added a few enhancements to Mike's Papagayo script. Papagayo has text-supported generation of visemes, which greatly improves the accuracy. The program has a nice user interface which makes lining up visemes very easy.

The purpose of this script is to use a Papagayo-generated file with visemes and insert them into the iClone viseme track for the character.

When the script was first written, it was not possible to delete visemes, so the voice audio file could not be loaded as the created visemes would interfere with those from Papagayo.

As it is now possible to delete visemes, my version of the script loads the audio file and then deletes the generated visemes. After that the viseme track is populated with the Papagayo-generated visemes. The presence of the audio waveform helps when making further minor adjustments of the viseme track.

The original script takes the first avatar from a list of avatars. I changed that to a selection by avatar name. I really would like to take whatever avatar is selected in the scene, but I couldn't find a function for that.

As per Mike's original script, there is no UI, and avatar name, file names, and viseme strength are hard-coded . Once my learning improves, I may add a UI, so things can be adjusted more easily.

import RLPy

# Select avatar by name; should ideally be the currently selected avatar
avatar = RLPy.RScene.FindObject(RLPy.EObjectType_Avatar, "fem_55906")
viseme_animator = avatar.GetVisemeComponent()

# Specify paths to voice audio file and Papagayo *.dat file
wav_path = "H:\\Media\\Audio\\Film\\Glint\\Poem_Job_timed_from_zero.wav"
dat_path = "H:\\Media\\Audio\\Film\\Glint\\Papagayo\\Poem_Job_timed_from_zero.dat"

# Define Papagayo viseme list (XX serves as a spacer to line visemes up with iClone viseme list
vList = ["XX","E","U","XX","AI","O","WQ","etc","XX","FV","XX","L","MBP"] #Papagayo
# iClone:      EE  ER       AH   OH  W-OO S-Z        F-V  T-L-D-N  B-M-P
ST = RLPy.RGlobal.GetTime()
current_time = RLPy.RTime(0)

# Load voice audio file and clear iClone-generated viseme keys
audio_object = RLPy.RAudio.CreateAudioObject()
audio_object.Load(wav_path)
viseme_animator.LoadVocal (audio_object, ST, "Poem_Job_timed_from_zero.wav")      
keys = viseme_animator.GetVisemeKeys();
for key in keys:
    viseme_animator.RemoveVisemesKey(key);    

# Insert Papagayo-created visemes from *.dat file
f = open(dat_path,'r')
the_frame = 0
while True:
    text = f.readline()
    if text == '':
        break
    if 'MohoSwitch1' in text:
        print ("Start")
    else:
        my_Frame, my_Vise = text.split()
        the_frame = int(my_Frame)
#        print (my_Vise)
        if 'rest' in text:
            the_frame += 2
            scene_time = RLPy.RTime(ST + (42 * the_frame))            
            viseme_animator.AddVisemeKey(scene_time,0,60.0)
            text = f.readline()
            if text == '':
                break
        else:
            myVNum = vList.index(my_Vise)
            the_frame += 2
            scene_time = RLPy.RTime(ST + (42 * the_frame))
            viseme_animator.AddVisemeKey(scene_time,myVNum,60.0)
         
print("Finished")




https://forum.reallusion.com/uploads/images/1a09220f-ab50-42ac-ad1a-33ec.pnghttps://forum.reallusion.com/Uploads/Images/d14339d0-cd32-4b35-88f9-40a0.png


Kelleytoons
Kelleytoons
Posted 2 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)

Group: Forum Members
Last Active: 7 hours ago
Posts: 5.9K, Visits: 11.9K
Good for you, Ani.

This gets the currently selected avatar(s):

selected_list = RLPy.RScene.GetSelectedObjects()
for object in selected_list:
    if object.GetType() == RLPy.EObjectType_Avatar:
        #we now have an avatar -- object can be any type including avatar




Alienware Aurora R7, Win 10, i7-8700k, 4.7GHz CPU, 32GB RAM, GTX Titan XP (12GB), Samsung 960 Pro 2TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley
gordryd
gordryd
Posted 2 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)Distinguished Member (1.3K reputation)

Group: Forum Members
Last Active: 7 hours ago
Posts: 190, Visits: 2.7K
Mike, forgive my dumb question, but how do we get from here (in your snippet):
selected_list = RLPy.RScene.GetSelectedObjects()
for object in selected_list:
    if object.GetType() == RLPy.EObjectType_Avatar:
to here (in Job's code)?
viseme_animator = avatar.GetVisemeComponent()
In other words, how would "avatar" get defined?  Job's was explicit:  avatar = RLPy.RScene.FindObject(RLPy.EObjectType_Avatar, "fem_55906").  I don't know what variable holds the value that we would use for the avatar in your snippet (not trying to be argumentative, I just don't know Python - I'm mostly a BASIC programmer that muddles through other languages, so object-oriented stuff throws me for a loop)
Kelleytoons
Kelleytoons
Posted 2 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)

Group: Forum Members
Last Active: 7 hours ago
Posts: 5.9K, Visits: 11.9K
It's okay -- hey, BASIC was my first language (and I went on to teach programming at the college level.  We all have to start somewhere :>Wink.

In the snippet I provided "object" is the avatar.  Without looking too closely at what Job did this is equivalent:

viseme_animator = object.GetVisemeComponent()

For readability purposes you *could* add another line like:

avatar = object

following my snippet but in general Python programmers HATE to be verbose (where as I would much rather be clearer than not).  Luke's Python implementation actually makes few distinctions between types, so all avatars are objects (but not all objects are avatars, if you get the drift).  I actually wish we did have a GetSelectedAvatars but if you wanted to write it you could:

def GetSelectedAvatars ():
    myAvatars = []
    selected_list = RLPy.RScene.GetSelectedObjects()
    for object in selected_list:
        if object.GetType() == RLPy.EObjectType_Avatar:
            myAvatar.append(object)
    return myAvatars

but note that you'll then need to check to see if the function returns null or otherwise be careful (sorry if I'm not being clear here -- I just got up from a nap and may be a bit off).
    



Alienware Aurora R7, Win 10, i7-8700k, 4.7GHz CPU, 32GB RAM, GTX Titan XP (12GB), Samsung 960 Pro 2TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley
animagic
animagic
Posted 2 Months Ago
View Quick Profile
Distinguished Member

Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)Distinguished Member (19.2K reputation)

Group: Forum Members
Last Active: 3 hours ago
Posts: 11.3K, Visits: 19.5K
Thanks for the feedback.

I worked a bit more on the script so that it can handle the absence of an avatar in the scene, provide a default when no avatar is selected, and use the selected avatar if one is selected. The new version is attached.

It 's long time since I have done some coding, but it is kind of fun!

EDIT: Due to a change in function name on RL's part, I needed to attach an updated version.


https://forum.reallusion.com/uploads/images/1a09220f-ab50-42ac-ad1a-33ec.pnghttps://forum.reallusion.com/Uploads/Images/d14339d0-cd32-4b35-88f9-40a0.png


Attachments
pg2_enhanced_2.zip (11 views, 1.00 KB)
Edited
Last Month by animagic
hcameron90
hcameron90
Posted Last Month
View Quick Profile
Distinguished Member

Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)

Group: Forum Members
Last Active: 5 days ago
Posts: 247, Visits: 1.6K
Hello @animagic and @Kelleytoons

So I saw your Papagayo video and wanted to give it a try. 

I added the same code but I'm getting errors in my iClone7 console log.

Here Is an image of the Error message, My code, and the RLpy file.
Can you help me out so I can fix the viseme, so I can see it on the timeline

PLease and Thank You 
https://forum.reallusion.com/uploads/images/506eeb21-578e-4d57-8d59-bcb2.png


https://forum.reallusion.com/uploads/images/37fe57bb-ac7a-44db-9157-755c.png


https://forum.reallusion.com/uploads/images/2f6c5815-4cf6-4484-9bff-8349.png
The-any-Key
The-any-Key
Posted Last Month
View Quick Profile
Distinguished Member

Distinguished Member (1.4K reputation)Distinguished Member (1.4K reputation)Distinguished Member (1.4K reputation)Distinguished Member (1.4K reputation)Distinguished Member (1.4K reputation)Distinguished Member (1.4K reputation)Distinguished Member (1.4K reputation)Distinguished Member (1.4K reputation)Distinguished Member (1.4K reputation)

Group: Forum Members
Last Active: 50 minutes ago
Posts: 260, Visits: 1.3K
The error suggest that you have an empty line in your JayText.dat



iClone 7, 3DXchange 7 Pipeline, CC3 Pipeline, CT8 Pipeline, Iray, Live Face, iPhone Xr, Win 10, i9 9900K ~5GHz, Nvidia RTX 2080 8GB, 16GB Ram, M.2 SSD
hcameron90
hcameron90
Posted Last Month
View Quick Profile
Distinguished Member

Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)

Group: Forum Members
Last Active: 5 days ago
Posts: 247, Visits: 1.6K
OK so do you know how to fix that. 

I didn't see any empty lines in my code.
Kelleytoons
Kelleytoons
Posted Last Month
View Quick Profile
Distinguished Member

Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)Distinguished Member (18.9K reputation)

Group: Forum Members
Last Active: 7 hours ago
Posts: 5.9K, Visits: 11.9K
Can you provide the .DAT file for us here?  (And when I say "we" I mean for perhaps someone else, as I'm shortly about to get offline until tomorrow).



Alienware Aurora R7, Win 10, i7-8700k, 4.7GHz CPU, 32GB RAM, GTX Titan XP (12GB), Samsung 960 Pro 2TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley
Edited
Last Month by Kelleytoons
hcameron90
hcameron90
Posted Last Month
View Quick Profile
Distinguished Member

Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)Distinguished Member (2.9K reputation)

Group: Forum Members
Last Active: 5 days ago
Posts: 247, Visits: 1.6K
OK here is two different .dat files both are the same I just if I redid the code and ran it, that I would get a different result in iclone
Attachments
lipSyncCode.zip (3 views, 3.00 KB)



Reading This Topic