Plugin: Move with Unity (iClone and Unity motion sync)


https://forum.reallusion.com/Topic401810.aspx
Print Topic | Close Window

By The-any-Key - 5 Years Ago
Tried to make my first network plugin. The idea is to use Unity3D movement and animations and sync these to iClone to a prop or avatar. You can then ex use Unity locomotion plugins or other plugins that move or animate object and directly sync these to iClone avatars and props.
It is still a work in progress and the current sync need to be fine tuned, better speed timing, ...:

Simple AI move in Unity where we are connected to iClone:
https://forum.reallusion.com/uploads/images/226e6851-4491-4792-91d2-37c6.gif

In iClone:
https://forum.reallusion.com/uploads/images/a2b7ba68-7540-47f9-b215-eee9.gif
By Rampa - 5 Years Ago
This is really pretty amazing!
Opens up all kinds of possibilities. There are a lot of pretty interesting character motion addons for Unity. Not to mention Mecanim itself!
By SeanMac - 5 Years Ago
Re: Unity.

I am puzzled at the appeal of Unity until they resolve their odd behaviour as regards SpatialOS and the Improbable company.

Unity changed their terms of service without notice shutting down all of this firm's Steam offerings.

Why wouldn't they do the same to any iCloners who had a successful offering?

Home Built in Coolermaster ATX case with GigabyteGA-Z170X-GAMING 7 MoBo, i7-6700 @ 3.4 Ghz, Asus GeForce GTX 1080 Ti GraphicsCard, 32Gb DDR4 RAM, an Acer CB241HQK  & a Samsung S24D300 monitor, Huion H610Pro Graphics Tablet, Kinect v2, Logitech C920 Webcam, Win 10 64 Bit OS

By RobertoColombo - 5 Years Ago
SeanMac (1/20/2019)
Re: Unity.

I am puzzled at the appeal of Unity until they resolve their odd behaviour as regards SpatialOS and the Improbable company.

Unity changed their terms of service without notice shutting down all of this firm's Steam offerings.

Why wouldn't they do the same to any iCloners who had a successful offering?

Home Built in Coolermaster ATX case with GigabyteGA-Z170X-GAMING 7 MoBo, i7-6700 @ 3.4 Ghz, Asus GeForce GTX 1080 Ti GraphicsCard, 32Gb DDR4 RAM, an Acer CB241HQK  & a Samsung S24D300 monitor, Huion H610Pro Graphics Tablet, Kinect v2, Logitech C920 Webcam, Win 10 64 Bit OS



I think the demonstrator here goes beyond Unity: here it is shown how the object movement can be controlled via a socket, which at the other endpoint can eventually be anything.
I would say, GREAT and INSPIRING example done by TheAnyKey!!

  Roberto

By SeanMac - 5 Years Ago
"I would say, GREAT and INSPIRING example done by TheAnyKey!!"
Roberto

I agree, Roberto. 100%.

I was just under the impression that it only applied to Unity.

Thanks for clearing that up.
By RobertoColombo - 5 Years Ago
Next step is to connect a joystick... ;-)
I would not be surprised to discover soon that "The Master" (TheAnyKey) is already working on it...
By The-any-Key - 5 Years Ago
Why I use Unity is because the sample used Unity. I have never used Unity before, so I can't comment on how their manners as humans are (it's a cooperation, what do you expect :-). But it seem Unity have a bunch of plugins that can create interesting animations.
Ex I want to try Puppetmaster and sync it to iClone: https://assetstore.unity.com/packages/tools/physics/puppetmaster-48977

But the goal is to make it generic so I can adapt it to connect to mostly anything that support sockets. 

About joysticks: I actually have a plan to use WiiMote controllers for prop movements. Ex you can duct tape a wiimote to a box or something and move it around. And sync the movement back to iClone. But I have no idea about the quality of the data from the wiimote. So we will see what happens.
By SeanMac - 5 Years Ago
Hi Tak

Thanks for that.

"But the goal is to make it generic so I canadapt it to connect to mostly anything that support sockets. "

Very much approve - proper middleware programming.

More power to your elbow!
SeanMac
By The-any-Key - 5 Years Ago
Hmm. Trying to link bones from Unity to iClone. It seem the orientation of the axis is different when export to Unity. So I need a way to translate it. But my math skills is not that good when it comes to matrix and quaternions math. We will see if the people at Unity forum can help.
https://forum.unity.com/threads/calc-local-rotation-with-local-axes-based-on-first-parent-local-rotation.619480/

Do anyone have an idea what I can send instead of the euler or 
quaternions to make the same rotation in Unity as iClone. I can use the world angles. But the problem with tthat is that the world angles is affected by other bones and I would need to "local" world angles.
By The-any-Key - 5 Years Ago
Was able to figure out the maths. But seem the API only allow to get the local quaternion rotation. The world transform get return wrong values. So I need to wait until they update the API.
By RobertoColombo - 5 Years Ago
Hi Jeff,

I can't help on this because I have no experience on quaternion and Euler angles (even though I suppose soon or later i will have to dig a bit into them...).
Sorry.

  Roberto
By The-any-Key - 5 Years Ago
Jeff? (maybe this is a meme or language thing like Jack?)

I am learning quaternions via internet. A good way to experiment with them is to use Unity as they are heavily integrated.
Never mind the latest question about world space. It seem you can get the world space quaternion by multiply all local quaternions from root to the local object. Ex: root quaternion*hips rotation*spine quaternion...
So no update is needed it seems :-)
By RobertoColombo - 5 Years Ago
Yeah... i confused you with another member Tongue
I will call you TAK (The Any Key) ...

Cheers

  Roberto
By RobertoColombo - 5 Years Ago
Yeah... i confused you with another member Tongue
I will call you TAK (The Any Key) ...

Cheers

  Roberto
By AllenLee (RL) - 5 Years Ago
Hi The-any-Key,

You can use the "RINode.WorldTransform()" API to get bone's world quaternions.
This API is already in the latest development version(2/1/2019).

-------------------------------------------------------------------------------------------------------------

You can try this code:

import RLPy
avatar_list = RLPy.RScene.GetAvatars()
avatar = avatar_list[0]
bone_compoment = avatar.GetSkeletonComponent()
skin_bones = bone_compoment.GetSkinBones()
for bone in skin_bones:
    world_transform = bone.WorldTransform()
    world_q = world_transform.R()
    print( world_q.X() )
    print( world_q.Y() )
    print( world_q.Z() )
    print( world_q.W() )


Allen
By The-any-Key - 5 Years Ago
Thanks. But I already tested that. And it seem the values returned from the WorldTransform are wrong (filed a bug ticket)
Ex if I have a prop or bone with Euler rotation [90,0,0] (x,y,z) with no parent I should get the quaternion [0.707,0,0,0.707] (x,y,z,w). But I get [0.85,0,0,0.53] aka [116,0,0] (x,y,z)