Profile Picture

What would be a general workflow for working with mocap in clone?

Posted By Ibis Fernandez 9 Years Ago
You don't have permission to rate!
1
2

What would be a general workflow for working with mocap in clone?

Author
Message
Ibis Fernandez
Ibis Fernandez
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)

Group: Forum Members
Last Active: 11 Months Ago
Posts: 1.2K, Visits: 4.3K
James in terms of making the shot lists and capturing just what you need in short bursts is provable a very sound approach to dealing with it for the most part.
Kinda frustrating for me cause im used to recording my scenes in master scene format which lends itself to more natural acting from the talent. But it is what it is hahaha.

Im also thinking to maybe experimenting with different sync markers somewhere in the middle and at the end of each capture in case of long sequences to allow for quicker resynching. i don't know ill give it a shot next chance i get. see how that works.


https://forum.reallusion.com/uploads/images/ad51c90b-00a8-459d-8ca4-2136.gif
Ibis Fernandez   |   (available for hire)
----------------------------------------------------------------------------------------
Professional Animator, Filmmaker | Creator of the highest quality (modular) G2 rigs for cartoon animator and developer of Toon Titan and Puppet Producer 
Author of Flash Animation and Cartooning: A Creative Guide
>>> be sure to check out http://toontitan.com for professional grade assets, templates, and custom tools for Cartoon Animator and more.
animagic
animagic
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 15.8K, Visits: 31.4K
rampa (2/21/2016)
Probably just a latency issue. For speech, make sure your recording mono. A lower sampling rate is fine too, as you don't need as wide a frequency response for speaking. Try 32 Khz, or even 22 Khz. Keep the bit depth at 16, as that effects your dynamic range (soft to loud).

It certainly is not really critical for most body mocap. It will be an issue when we get facial mocap though.


From what I understand the latency is in the motion capture. I would think though that when played back it would be OK. Would it be possible to adjust the playback frame rate?



https://forum.reallusion.com/uploads/images/436b0ffd-1242-44d6-a876-d631.jpg

Rampa
Rampa
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)

Group: Forum Members
Last Active: Last Week
Posts: 8.2K, Visits: 62.6K
Probably just a latency issue. For speech, make sure your recording mono. A lower sampling rate is fine too, as you don't need as wide a frequency response for speaking. Try 32 Khz, or even 22 Khz. Keep the bit depth at 16, as that effects your dynamic range (soft to loud).

It certainly is not really critical for most body mocap. It will be an issue when we get facial mocap though.
Ibis Fernandez
Ibis Fernandez
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)

Group: Forum Members
Last Active: 11 Months Ago
Posts: 1.2K, Visits: 4.3K
No. This happens even when capturing directly to the axis neuron software. 

https://forum.reallusion.com/uploads/images/ad51c90b-00a8-459d-8ca4-2136.gif
Ibis Fernandez   |   (available for hire)
----------------------------------------------------------------------------------------
Professional Animator, Filmmaker | Creator of the highest quality (modular) G2 rigs for cartoon animator and developer of Toon Titan and Puppet Producer 
Author of Flash Animation and Cartooning: A Creative Guide
>>> be sure to check out http://toontitan.com for professional grade assets, templates, and custom tools for Cartoon Animator and more.
Rampa
Rampa
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)Distinguished Member (37.5K reputation)

Group: Forum Members
Last Active: Last Week
Posts: 8.2K, Visits: 62.6K
Maybe don't stream it to iClone? Just capture with the Noitom software and an audio capture running concurrently?

I would suspect that the BVH broadcast to iClone might be where the issue is.
Ibis Fernandez
Ibis Fernandez
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)

Group: Forum Members
Last Active: 11 Months Ago
Posts: 1.2K, Visits: 4.3K
I've been experimenting. Been noticing that the mocap data somehow becomes out of sync with the audio even when the performances are captured at the same time. It seems the mocap is not recorded in real time. instead the motions are added in as they come in. So capturing 1minute of audio and 1 minute of mocap, at the same time, seems to yeild 1 min of audio and about 1.25 mins of mocap. 

So while the movements are fairly accurate, there is some kind of time shifting involved. 




https://forum.reallusion.com/uploads/images/ad51c90b-00a8-459d-8ca4-2136.gif
Ibis Fernandez   |   (available for hire)
----------------------------------------------------------------------------------------
Professional Animator, Filmmaker | Creator of the highest quality (modular) G2 rigs for cartoon animator and developer of Toon Titan and Puppet Producer 
Author of Flash Animation and Cartooning: A Creative Guide
>>> be sure to check out http://toontitan.com for professional grade assets, templates, and custom tools for Cartoon Animator and more.
animagic
animagic
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)Distinguished Member (33.4K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 15.8K, Visits: 31.4K
I'm following this with interest, because even though I don't have PN right now, I may get it if the iClone plugin becomes a reality. As Rampa points out, using "canned" gestures with dialog is problematic. It's not impossible but takes a lot of tweaking.

I would think that if you have an audiovisual reference point, such as a clapper, to mark the beginning of your capture, you would be fine. I would also think that synching gestures and dialog would be less critical than say mouth movements and dialog, where you need frame accuracy.

The problem I see is that the performance actor and the voice actor may not be the same, in which case the gesturing would have to follow what is being said. I've tried that (with Kinect) and it's not so easy.



https://forum.reallusion.com/uploads/images/436b0ffd-1242-44d6-a876-d631.jpg

Ibis Fernandez
Ibis Fernandez
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)

Group: Forum Members
Last Active: 11 Months Ago
Posts: 1.2K, Visits: 4.3K
I think you keep missing the point. I'm already perfectly capable of "animating" from scratch. What I'm experimenting with atvthus time is motion captures performances. 

At this time I'm only really interested in finding or establishing a conventional and comprehensive workflow. 

It's only natural that any type of capture will need to be tweaked and fine tuned manually to achieve the best results, but as a filmmaker I really don't care about how it's done, as long as it gets done. 

Bones, morph targets, curve editors ... It doest matter. What matters is that the objective is achieved. If I wanted to make life more difficult for myself I wouldn't be using iclone or neurons lol.


https://forum.reallusion.com/uploads/images/ad51c90b-00a8-459d-8ca4-2136.gif
Ibis Fernandez   |   (available for hire)
----------------------------------------------------------------------------------------
Professional Animator, Filmmaker | Creator of the highest quality (modular) G2 rigs for cartoon animator and developer of Toon Titan and Puppet Producer 
Author of Flash Animation and Cartooning: A Creative Guide
>>> be sure to check out http://toontitan.com for professional grade assets, templates, and custom tools for Cartoon Animator and more.
Ibis Fernandez
Ibis Fernandez
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)

Group: Forum Members
Last Active: 11 Months Ago
Posts: 1.2K, Visits: 4.3K
still feel like its more efficient to just clap. you see the character clap and you sync the visual cue of the clap to the sound of the clap. once synced, you just trim off the unwanted parts.

Ill have to try your method though, see how that goes.


https://forum.reallusion.com/uploads/images/ad51c90b-00a8-459d-8ca4-2136.gif
Ibis Fernandez   |   (available for hire)
----------------------------------------------------------------------------------------
Professional Animator, Filmmaker | Creator of the highest quality (modular) G2 rigs for cartoon animator and developer of Toon Titan and Puppet Producer 
Author of Flash Animation and Cartooning: A Creative Guide
>>> be sure to check out http://toontitan.com for professional grade assets, templates, and custom tools for Cartoon Animator and more.
Ibis Fernandez
Ibis Fernandez
Posted 9 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)Distinguished Member (16.6K reputation)

Group: Forum Members
Last Active: 11 Months Ago
Posts: 1.2K, Visits: 4.3K
sw00000p (2/18/2016)
Ibis Fernandez (2/18/2016)
What is the technique?

... you have awesome raw animation data that's PRECISE! :w00t::Wow:

Yes, i realize this. I have compared the results of the data in terms of what iClone sees and what other programs see and there is a huge difference. The hands don't even work in the iClone captures most of the time. The thumbs, never do.

Ive actually switched to recording the raw data using the Axis Neuron software. I am thinking at sometime if it comes down to it i can use that data on other apps that can better handle it. To get the data from axis neuron into iclone or anything else all you have to do is hit play, and it gets broadcasted over to whatever plugin is set up to listen for it. Stick it doesn't address what the sync issues. This is something that has always been an issue even in regular live action film making. that why they came up with the clapper board. You see video of the object clapping, and you listen for the clap (or look at the waveform in the audio file) you match the visual with the audio and boom its synced. (by sliding the track).


...Slide the track with precision to Sync.:Wow:...

Yes, this is basically what im doing now. Except I give myself a visual cue in the performance track to match my audio cue so as not to be aimlessly sliding around indefinitely.

It feels like there would have been some standardized technique by now. How the people in planet of the apes or avatar handle it? Seems like they just brute force synchronizing their stuff. I guess with  millions of dollars in budget you can afford to just brute force your way through anything hahaha



https://forum.reallusion.com/uploads/images/ad51c90b-00a8-459d-8ca4-2136.gif
Ibis Fernandez   |   (available for hire)
----------------------------------------------------------------------------------------
Professional Animator, Filmmaker | Creator of the highest quality (modular) G2 rigs for cartoon animator and developer of Toon Titan and Puppet Producer 
Author of Flash Animation and Cartooning: A Creative Guide
>>> be sure to check out http://toontitan.com for professional grade assets, templates, and custom tools for Cartoon Animator and more.

1
2



Reading This Topic