Profile Picture

Motion capture with a video only

Posted By mtakerkart Last Year
You don't have permission to rate!
1
2
3
4

Author
Message
mtakerkart
mtakerkart
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: 3 Months Ago
Posts: 3.2K, Visits: 29.2K
I will give a try next week with the dual camera Rokoko vision. There's a 14 days trial. Let's see what it gives...

https://www.rokoko.com/products/vision
AutoDidact
AutoDidact
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 2.1K, Visits: 13.6K
The import process into UDK showed by JSFILMZ required NASA engineer skills which required PC and wifi to stream data.



LOL!!:D
I really do not get why people insist that unreal engine is a good choice for animated filmmaking
with Characters 
What the hell good is “realtime” when you have to do 15 complicated steps before your Character can even start moving. 



RAG DOLL COLLISION ANIMATIONS FOR ICLONE 8 & 7
---------------------------------------------------------------------------------------------------------------------
Ghost Origins
My latest Feature length film created with Iclone.
https://forum.reallusion.com/uploads/images/adf9b210-df59-4cb6-aa1b-9de5.jpg
My Sci- Fi Graphic Novel on Amazon: https://a.co/d/9k3cwoY


mtakerkart
mtakerkart
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: 3 Months Ago
Posts: 3.2K, Visits: 29.2K
4u2ges (9/5/2024)
Yes, I am afraid there would be a need to do some sort of manual surgical procedures with mocaps motions until AI figures it all out.


I think it's just a root positioning issue that their AI is unable to handle.
If we look at the rest of the body it's very faithful, and just 1 camera!!!
Maybe it should use the iPhone's zbuffer sensor data....


mtakerkart
mtakerkart
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: 3 Months Ago
Posts: 3.2K, Visits: 29.2K
AutoDidact (9/5/2024)
JSFILMZ made a new video with the mocopi kit wich can record in bvh format. It seems quite good . May it could be 
better the the motion correction...

better the the motion correction...


JSFILMZ recommends export from UE5 to FBX for Iclone with his Mocapi setu
BVH is typically a horrible format unless it is bespoke from your specific character that was retargeted in Iclone or 
Motionbuilder.
I suspect that a raw generic BVH from Mocapi will have the same foot sliding (or worse), and still require the type of tedious manual fixing that  4u2ges had to perform.  


Never had a horrible problem with bhv file into Iclone.
The import process into UDK showed by JSFILMZ required NASA ingeneer skills wich required PC and wifi to stream data  which adds a source of recording error...
Mocopi made an update one year ago. What interests me in mocopi is that it uses bluetooth  to record bvh file on a cell phone.
So you can record anywhere with no distance limit.

4u2ges
4u2ges
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 5.3K, Visits: 16.8K
Yes, I am afraid there would be a need to do some sort of manual surgical procedures with mocaps motions until AI figures it all out.

Just one more note about the routine I posted.
In some cases you might not even need to animate toe bones at all (only dummy objects for feet).
Alternative pivoting method to toes I posted later on there, allows to use Foot Contact feature to drive toe bones automatically.
Foot Contact has it's own quirks and bugs, but still workable.

The best part is, that I learned recently, a deformation with Foot Contact feature will be baked along with all other constraints.
Meaning there would not be any dependency on this iClone feature when you export motion.




AutoDidact
AutoDidact
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)Distinguished Member (6.1K reputation)

Group: Forum Members
Last Active: 2 Months Ago
Posts: 2.1K, Visits: 13.6K
JSFILMZ made a new video with the mocopi kit wich can record in bvh format. It seems quite good . May it could be 
better the the motion correction...

better the the motion correction...


JSFILMZ recommends export from UE5 to FBX for Iclone with his Mocapi setu
BVH is typically a horrible format unless it is bespoke from your specific character that was retargeted in Iclone or 
Motionbuilder.
I suspect that a raw generic BVH from Mocapi will have the same foot sliding (or worse), and still require the type of tedious manual fixing that  4u2ges had to perform.  



RAG DOLL COLLISION ANIMATIONS FOR ICLONE 8 & 7
---------------------------------------------------------------------------------------------------------------------
Ghost Origins
My latest Feature length film created with Iclone.
https://forum.reallusion.com/uploads/images/adf9b210-df59-4cb6-aa1b-9de5.jpg
My Sci- Fi Graphic Novel on Amazon: https://a.co/d/9k3cwoY


mtakerkart
mtakerkart
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: 3 Months Ago
Posts: 3.2K, Visits: 29.2K
Thank you 4u2ges , very instructive. But to much time consuming for me ;)
JSFILMZ made a new video with the mocopi kit wich can record in bvh format. It seems quite good . May it could be 
better the the motion correction...


4u2ges
4u2ges
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 5.3K, Visits: 16.8K
Just a few details on fixing feet sliding/sinking for motions which cannot be effectively fixed with Motion Correction.
I hope someone find it interesting and/or introduce their own alternative methods or tips to improve the workflow.
Mid to advanced experience level with animating on the timeline required.

It's not actually "fixing", but rather recreating motions for 4 bones - L/R Foot and L/R ToeBase.

The idea is to have 2 characters with the same motion at the same location in the project.
You then animate 4 mentioned bones of one character without sliding while having second character as a reference for planting feet.
During the process you would want to hide/un-hide reference character to have a clear view of the main character you are trying to fix the motion for.

Animating L/R Foot

You do not actually animate bones. You do it by animating Reach Targets attached to  L/R Foot nodes.
So you have to create 2 reach targets, place them on the timeline and start animating to have feet of the target character follow the feet of the reference character.
You select an L/R foot node and click Create Dummy in Edit Reach Target:
https://forum.reallusion.com/uploads/images/b67d26ae-e8be-4b7b-8397-255b.jpg

The biggest issue during animation is target rotation. Vast majority of human feet motions pivoted to toes.
But foot bones and consequently reach targets pivoted to heels.
So a slight turn with rotation would require repositioning reach target with move to arbitrary match initial toe position.
And mismatch with positioning even for a tiny fraction of an inch would result in sliding.

I came up with orthodox foot rotation by setting a Pivot point for Reach targets to the location of toes.
It would feel a bit unusual at first, that while rotating reach target by X the Calf and Thigh bones are pushed up. But it's a matter of getting used to.
The benefits are obvious. Once foot is rotated with reach target, the toe would firmly stay planted at the intended location with zero sliding
and further tweaking and guessing of the foot position would NOT be required. Making turns and twists would be a snap.
Here is how to make an alternative pivoting of the feet Reach Targets:



Alternative method for setting a Pivot to toes



So animating Reach targets for the feet (specially with alternative pivots) is nothing really special.
You make pose-to-pose animation moving foot from one pose on the ground to another
and fill the blanks in between (such as raising foot, rotating up/down for steps) while trying to mimic the foot movement of the reference character.

Animating L/R ToeBase

This has to be done with character bones. As the matter of fact you only need to animate ToeBase\Rotation X.
But first, you need to remove ALL ToeBase bone animation from a given clip.
They are jittery most of the time (specially having Y and Z rotation while there has to be none by those axises).

Sample Selected parts
https://forum.reallusion.com/uploads/images/c3bd27b0-663e-4ea0-8ac6-d30a.jpg

Then open Curve Editor, select ALL keys for the ToeBase bones and delete them.
https://forum.reallusion.com/uploads/images/9a8e58d4-7357-4255-8d0f-282e.jpg

After that, you may Flatten the clip back.

You animate ToeBase bones by X rotation where appropriate while making Reach targets pose-to-pose animation for feet.
You may animate toes afterwards when you complete Reach Targets animation for feet. In which case slight tweaking of Reach targets would most likely require in a process.
You have to use Edit Motion Layer with Edit Body Parts mode.
Try to mimic the reference character, or if toe bending is completely messed up (like in a case with boxer animation in my previous post) try to make them perform like in real life.

Keep Motion Layer clean. Do not animate anything else other than ToeBase bones.
It's very easy to lose track of that's happening at the motion layer if you start fixing other parts. You can do that later.
The only possible exception is Hip bone (Pelvis). For out of touch mocaps you may get inadequate Hip movements where it would prevent planting feet properly.
So you might have to tweak Hip bone in a way.

I found it's a good practice to cleanup unnecessary keys from timeline.
Every once in a while I would go to Curve Editor, select and delete all keys leaving only:
CC_Base_R_ToeBase \Rotation X
CC_Base_L_ToeBase \Rotation X

This is how my timeline looked at the end from Curve Editor perspective (only 2 curves):

https://forum.reallusion.com/uploads/images/5cb18ec3-8241-4d31-9e08-be0b.jpg

At the end when all is done and you are happy with the result, go ahead and bake with Flatten All Motion with Constraint option. Delete reach dummies once motion is baked.

It might seem overly complicated, but it's actually not. Once you get done a few poses, you'd come up with steady routine and your own shortcuts for animating Reach dummies and Toe bones at the timeline.
It's also a good practice to keep your skills for manual animations sharp.




4u2ges
4u2ges
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)Distinguished Member (22.3K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 5.3K, Visits: 16.8K
I am surprised no one came out with that yet. The issue of sliding/sinking feet is part of any mocap out there.
Meantime, I would mostly rely on manual correction with reach targets. Yes, it takes time though.

This case was as bad as it can be and could not be fixed with motion correction.
I will explain a bit in details later on how it was done with some tips.






mtakerkart
mtakerkart
Posted Last Year
View Quick Profile
Distinguished Member

Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)Distinguished Member (16.8K reputation)

Group: Forum Members
Last Active: 3 Months Ago
Posts: 3.2K, Visits: 29.2K


Thank 4u2ges. It's more clear. In their exemple one click makes the result nearly perfect. In my case I have to rework all footsteps and the result is so so.
As I said this AI tech is ready for very marked steps. May in few months ;)

1
2
3
4



Reading This Topic