Profile Picture

How to write a facial mocap plug-in

Posted By jeff.lin_712595 3 Years Ago
You don't have permission to rate!
Author
Message
jeff.lin_712595
jeff.lin_712595
Posted 3 Years Ago
View Quick Profile
Junior Member

Junior Member (208 reputation)Junior Member (208 reputation)Junior Member (208 reputation)Junior Member (208 reputation)Junior Member (208 reputation)Junior Member (208 reputation)Junior Member (208 reputation)Junior Member (208 reputation)Junior Member (208 reputation)

Group: Forum Members
Last Active: 3 Years Ago
Posts: 1, Visits: 24
Hello, Support,
We developed our own facial mocap algorithm and want to integrate it to iClone8,
I followed the RIFacialDevice document example and I expect that code will record keys to morph data channels but it doesn't work.
Is there any sample or tutorial of writing custom mocap device plugin for iClone8?
thanks
Johnny (RL)
Johnny (RL)
Posted 3 Years Ago
View Quick Profile
Junior Member

Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 17, Visits: 646
Hi Jeff,

The attached zip file is an example of the facial mocap, you can follow the steps of the video to execute it.
Please feel free to ask me if you encounter any problems, thanks.



Mocap Document
Attachments
FacailMocapSample.zip (78 views, 6.00 KB)
Jean-Sé
Jean-Sé
Posted 3 Years Ago
View Quick Profile
Senior Member

Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 13, Visits: 4.1K
Hi Johnny,

Thanks for the code FacialMocap sample.
Could you please also upload the BodyMocap sample?

Regarding the 
FacialMocap sample, I would like to create my own version of the LIVE Face App for my personal need.
I already own the Motion Live plug-in but I would like to be able to record the sound and facial mocap information 'offline' on a iPad and send the data (sound + mocap data) 
later to iClone 8.
Is it possible to send this information directly to MotionLive, instead of the FacialMocap code sample? It would really help.

Thanks,
Jean-Sebastien

Johnny (RL)
Johnny (RL)
Posted 3 Years Ago
View Quick Profile
Junior Member

Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 17, Visits: 646
Hi Jean,

Please see the attached zip file for the full Mocap sample.

Sorry, as the iPhone Live Face Profile is paid, there are currently no plans to expose the data format.
So it is suggested that you can change the FacialMocap sample to the version you need.

For the "offline" function you mentioned.
If you have a demand, please provide a more detailed description. We are happy to include the Live Face feature in the next version.

Thanks,
Johnny Yang
Attachments
MocapSample.zip (74 views, 13.00 KB)
Jean-Sé
Jean-Sé
Posted 3 Years Ago
View Quick Profile
Senior Member

Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)Senior Member (260 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 13, Visits: 4.1K
Thank you for the code sample, that's exactly what I needed.
sunnyviewtech
sunnyviewtech
Posted 2 Years Ago
View Quick Profile
Senior Member

Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 27, Visits: 457
Johnny (RL) (9/27/2022)
Hi Jeff,
The attached zip file is an example of the facial mocap, you can follow the steps of the video to execute it.
Please feel free to ask me if you encounter any problems, thanks.
Mocap Document


Hi, Johnny, the FacialMocapSample fails in 8.33,
facial_device.ProcessData(mocap_avatar, exp_list) still returns success, but the face morphs don't change.

Wish there will be an update, thank you!

Johnny (RL)
Johnny (RL)
Posted Last Year
View Quick Profile
Junior Member

Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)Junior Member (180 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 17, Visits: 646
Hi sunnyviewtech,
The sample code didn't account for different numbers of expressions for characters, which caused some errors.
We've updated the Sample Code, so give it another shot!
Thanks!
Attachments
MocapSample.zip (50 views, 465.00 KB)
sunnyviewtech
sunnyviewtech
Posted Last Year
View Quick Profile
Senior Member

Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 27, Visits: 457
Thank you Johnny, it's of great help!
sunnyviewtech
sunnyviewtech
Posted Last Year
View Quick Profile
Senior Member

Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)Senior Member (336 reputation)

Group: Forum Members
Last Active: Last Year
Posts: 27, Visits: 457
Hi Johnny, thank you for the code and documentation. We have learned a lot from them. But we have two follow-up questions,

1. When sending the same data used in SetTPoseData, the character will be in a T-pose, but the overall position will be elevated. Is this normal? Please refer to the uploaded image, and we have also uploaded the modified MocapSample for your reference

Origin position
https://forum.reallusion.com/uploads/images/82892fe9-a189-4cba-9774-7a00.png

After sending the TPose data, the feet position is higher

https://forum.reallusion.com/uploads/images/0655c9b9-6651-49ca-9271-72dc.png

2. How can we configure the settings so that the motion capture data uses the initial transform of the character as the origin, instead of the system's origin? We have tried various combinations of SetMotionApplyMode, SetCoordinateOffset, SetCoordinateSpace, etc., but have not achieved the desired results. Is this possible to implement? It would be greatly appreciated if you could shed some light on this for us. Thank you very much.
Attachments
MocapSample.zip (38 views, 6.00 KB)



Reading This Topic