|
By Data Juggler - 6 Years Ago
|
Today I made a quick video, and the lip syncing was probably not perfect to anyone who really pays attention, but it was good enough I could publish a video and focus on content without wasting time on mouth poses (vsemes is the word I think).
My microphone (Shure SM7B) is probably too good for my noisy apartment, so I have to record with my microphones interface 'amp' option turned down, then amplify using audio software (Audacity and / or AVS Audio Editor) and it comes out to sounding near studio quality now that I moved my PC away from my desk and put a 2 inch thick gymnast mat in front of it.
When I upload an audio file, the mouth poses just don't seem to sync as well.
From my past experiences, audio files were so bad I used Text to Speech and adjusted my audio file to match, but the recorded voice option is close enough I could eliminate the audio editing.
The only problem is this means I have to record between 3:00 AM and 4:45 AM, and I usually go to bed about midnight.
Just curious if this is unique to my environment or if anyone else has experienced this?
Thanks
|
|
By Kelleytoons - 6 Years Ago
|
|
|
|
By Data Juggler - 6 Years Ago
|
Thank you very much for the video. And I agree, I wish Reallusion would add something like PapaGayo, what a great editor for vsemes compared to a timeline.
Python is on my to do list, I am in the process of releasing several open source projects so I can go back to playing animator and wasting (investing) a bunch of money.
Open source doesn't pay that well, but animation is an expensive hobby when you count the new graphics card I had to buy, the new case to hold the new graphics card, paying someone to move my mother board to the new case because of the if something goes wrong I would lose money at my day job.
Then you take into account all the characters and scenes and props and software and microphones and it adds up.
I will play around with this on the weekend when I have more time.
Thanks again.
|
|
By Kelleytoons - 6 Years Ago
|
Yeah, the purpose of that video is to show you can use Papagayo with iClone. Indeed, since I made that the new Python release for iClone makes it easier (because now we can delete and move visemes on the track -- so just add the command to delete them all first. If this turns out something you want to use, let me know and I'll alter my script for that purpose).
I also have a number of tutorials on just using Papagayo (for the other animation program, but they are then applicable for iClone). It's really easy, though, and I think the results would be great in iClone (as I said in the video, I use Liveface so I don't really need this, but it was kind of a thing on my "to-do" list long before we got facial mocap available. I still might use it for some of my pre-recorded audio tracks I have).
|
|
By TonyDPrime - 6 Years Ago
|
Just a theory - The difference you notice may be due to the sound quality of the audio.
But in this case, less is more perhaps....
If the recorded audio has higher treble/volume peaks that are less compressed, it may be the case that such peaks more easily trigger phonemes mapped on the timeline by the conversion process.
|
|
By Data Juggler - 6 Years Ago
|
|
TonyDPrime (3/11/2019)
Just a theory - The difference you notice may be due to the sound quality of the audio.
But in this case, less is more perhaps....
If the recorded audio has higher treble/volume peaks that are less compressed, it may be the case that such peaks more easily trigger phonemes mapped on the timeline by the conversion process.
Actually more is more probably. When I record directly from IClone, I have to crank up my AMP just like I do for Skype conversations.
For Skype it doesn't matter if people hear a little background noise, but for a video it just ruins it.
But now that I think about what you said, by using the raw video amplified IClone picks up the syllable changes better probably.
I only did 1 test, I was planning on doing more testing if I ever get caught up.
I am going to try using an amplified version to do the vsemes, and then I can use a edited version for produced animation.
Kellytoons option looks great if I can get it working, but any steps that can be saved is always best.
I will try some more tests after work.
|
|
By AutoDidact - 6 Years Ago
|
I wish Reallusion would add something like PapaGayo, what a great editor for vsemes compared to a timeline.what a great editor for vsemes compared to a timeline. PapaGayo!! it has been a while since I heard that name. It is a great utility, I still have it installed over on my older Mac where I used to use it quit a bit it during my brief foray into 2D animation with the old anime studio pro software years ago.
|
|
By Data Juggler - 6 Years Ago
|
@Kellytoons, is that code file published on GitHub or anywhere?
If not, could you post the source code here, or email me please?
I am ready to Python tonight (is Python also a verb, or would be slither like a lawyer)?
Thanks
|
|
By Kelleytoons - 6 Years Ago
|
In that video on YouTube there's a link to the code (it's Dropbox so it lives there permanently). Hmmm, tried to copy it and it didn't work. Let me know if you can't find it there (it's in the comments section where someone else asks for it).
|
|
By 3dtester - 6 Years Ago
|
Pretty awesome stuff, although currently beyond my scope ;)
I will surely check this out at a later time.
Thanks guys!
|
|
By Data Juggler - 6 Years Ago
|
Thanks, I didn't read the description.
I will dive in as soon as I get the chance.
|
|
By Data Juggler - 6 Years Ago
|
I finally had the time to try Python tonight, and I got an error on my first attempt at slithering, so kind of disappointing.
It took me a while to figure out Papagayo, but once I did I would pay IClone $100 to upgrade the audio and add an interface like this.
I tried to run the python script, but I get an error:
<class 'AttributeError'>, File: C:\Program Files\Reallusion\iClone 7\Bin64\RLPy.py, Line: 80
I opened the file above, and on line 80 just raises some previous error, but that is about all I can tell by looking:
raise AttributeError("'%s' object has no attribute '%s'" % (class_type.__name__, name))
All I can tell by looking at Python from my first 5 minutes, is Visual Studio thinks Python is a third class citizen as when you open a Python File with VS with Python installed, you still have to go to File Open and reopen the file because attempting that from Windows and nothing happens.
This could be due to VS 2019's new start menu is horrible.
Anyway, I don't know what to do to resolve the error, so I am going to give up on animating and stick to wrestling sharks, aerial cart wheels and other easier tasks that I know how to do.
I was all excited after conquering Papagayo.
|
|
By Kelleytoons - 6 Years Ago
|
My gut tells me you aren't using the latest iClone (the last Python version was about two weeks ago). You will need to install the latest to have any of my scripts work (well, it's ALWAYS better to install the latest).
7.41.2529.1 is the earliest version you should be using.
|
|
By Data Juggler - 6 Years Ago
|
|
Kelleytoons (3/18/2019)
My gut tells me you aren't using the latest iClone (the last Python version was about two weeks ago). You will need to install the latest to have any of my scripts work (well, it's ALWAYS better to install the latest).
7.41.2529.1 is the earliest version you should be using.
You must be part of an IClone insider program, because I have the latest according to this, and it is 7.41.2525.
I guess I will just have to wait.
|
|
By Kelleytoons - 6 Years Ago
|
No, it's no insider version, DJ. You just need to download the latest from the Python thread here (they show the last candidate as 2/27, which you don't have, but note there is another one coming in April as well):
https://forum.reallusion.com/399188/Basic-Samples-Request-Thread-iClone-Developer-Edition
|
|
By Data Juggler - 6 Years Ago
|
Thank you for the update. I installed the update, ran the script, quickly crashed IClone.
I modified your script to match my path(s) there was one other place I mdified for the wave file.
From your code, it looks like you have to use double back slashes as an escape character to indicate an actual backslash, is that correct?
C# is like that, unless you put an @ symbol before the string you can use a literal backslash.
This path exists: C:\Video Projects\Codey\Audio\CodeIntro.dat on my hard drive. The error shown was on line 23, which is the last line I pasted where it parses the text into two variables, and it blows up. Is there a way in Python to check if the file exists before it opens? Just trying to verify it isn't a path issue.
And in C# you can say something like While Read Line, instead of While True, so if it doesn't Read the first line you don't go into the loop at all. Sorry I don't know Python, or I could debug it.
I just wanted to write and say thank you for your help, but alas my pesky day job is calling so I can afford to play with animation tools.
After work I will resume again if I have time, else this weekend I will investigate further.
f = open("C:\\Video Projects\\Codey\\Audio\\CodeIntro.dat",'r') the_frame = 0 while True: text = f.readline() if text == '': break if 'MohoSwitch1' in text: print ("Start") else: my_Frame, my_Vise = text.split()
|
|
By Kelleytoons - 6 Years Ago
|
Did you watch my video about using this? You will need to create a viseme track first (and then delete all the visemes in it -- if I get the time I'll mod this to do that first). It won't work without that because as it stands Python won't allow us to create a blank track. That may be where it's blowing up (it's not a path issue -- it should just throw an error if it can't find the file).
And if you watch my video you'll note I just say this is a proof of concept -- it will work, and *should* work for you, but it's really not intended for anyone unless they know Python or at least understand a *bit* about how it works in iClone. Knowing any other language will help, of course, but iClone's Python needs to be studied so you can see the various objects it creates and handles (so you would see that "avatar.GetVisemeComponent" needs both the avatar AND that viseme track first).
If you do decide to work on this tonight (or this weekend) holler at me and I can walk you through some things (if we could get enough folks interested we could do an online webinar, but I suspect that's coming from RL soon enough).
|
|
By Data Juggler - 6 Years Ago
|
Yes I watched the video, and I created a 3 second blank recording that showed up as None None where the only two vsemes.
I set the cursor at frame 10 and these two were at frames less than 10. I will try deleting them after work and try again.
Thanks
|
|
By Kelleytoons - 6 Years Ago
|
|
Hmmm. Well, holler at me, then (my email is easy to find if you go to my YouTube channel, and from there I can give you my phone or you can give me yours). We'll figure it out.
|