Profile Picture

How do you create a script file that uses manual mocap data to move the hand or a finger?

Posted By delebash Last Year
Rated 5 stars based on 1 vote.

How do you create a script file that uses manual mocap data to move...

Author
Message
delebash
delebash
Posted Last Year
View Quick Profile
Veteran Member

Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)

Group: Forum Members
Last Active: Last Week
Posts: 32, Visits: 400
I have tried this script https://wiki.reallusion.com/IC_Python_API:RLPy_RIHandDevice
I also looked at https://github.com/reallusion/iClone/tree/master/HandGesturesPuppeteering for further information but I still cannot seem to get manual mocap data to move a finger on the hand or any part of the hand so I can see how to use mocap data.

Here is my current code that is not working.  Any help or working example would be greatly appreciated!

import ptvsd
ptvsd.enable_attach(address=("localhost"5678), redirect_output=True)
ptvsd.wait_for_attach()

import RLPy

hips = ["hips""""Hips"]
right_up_leg = ["rightupleg""hips""RightUpLeg"]
right_leg = ["rightleg""rightupleg""RightLeg"]
right_foot = ["rightfoot""rightleg""RightFoot"]
left_up_leg = ["leftupleg""hips""LeftUpLeg"]
left_leg = ["leftleg""leftupleg""LeftLeg"]
left_foot = ["leftfoot""leftleg""LeftFoot"]
spine = ["spine""hips""Spine"]
spine_1 = ["spine1""spine""Spine3"]
spine_2 = ["spine2""spine1""Spine6"]
spine_3 = ["spine3""spine2""Spine9"]
neck = ["neck""spine3""Neck"]
head = ["head""neck""Head"]
right_shoulder = ["rightshoulder""spine3""RightShoulder"]
right_arm = ["rightarm""rightshoulder""RightArm"]
right_forearm = ["rightforearm""rightarm""RightForeArm"]
right_hand = ["righthand""rightforearm""RightHand"]
right_hand_thumb_1 = ["righthandthumb1""righthand""RightHandThumb1"]
right_hand_thumb_2 = ["righthandthumb2""righthandthumb1""RightHandThumb2"]
right_hand_thumb_3 = ["righthandthumb3""righthandthumb2""RightHandThumb3"]
right_in_hand_index = ["rightinhandindex""righthand""RightInHandIndex"]
right_hand_index_1 = ["righthandindex1""rightinhandindex""RightHandIndex1"]
right_hand_index_2 = ["righthandindex2""righthandindex1""RightHandIndex2"]
right_hand_index_3 = ["righthandindex3""righthandindex2""RightHandIndex3"]
right_in_hand_middle = ["rightinhandmiddle""righthand""RightInHandMiddle"]
right_hand_middle_1 = ["righthandmiddle1""rightinhandmiddle""RightHandMiddle1"]
right_hand_middle_2 = ["righthandmiddle2""righthandmiddle1""RightHandMiddle2"]
right_hand_middle_3 = ["righthandmiddle3""righthandmiddle2""RightHandMiddle3"]
right_in_hand_ring = ["rightinhandring""righthand""RightInHandRing"]
right_hand_ring_1 = ["righthandring1""rightinhandring""RightHandRing1"]
right_hand_ring_2 = ["righthandring2""righthandring1""RightHandRing2"]
right_hand_ring_3 = ["righthandring3""righthandring2""RightHandRing3"]
right_in_hand_pinky = ["rightinhandpinky""righthand""RightInHandPinky"]
right_hand_pinky_1 = ["righthandpinky1""rightinhandpinky""RightHandPinky1"]
right_hand_pinky_2 = ["righthandpinky2""righthandpinky1""RightHandPinky2"]
right_hand_pinky_3 = ["righthandpinky3""righthandpinky2""RightHandPinky3"]
left_shoulder = ["leftshoulder""spine3""LeftShoulder"]
left_arm = ["leftarm""leftshoulder""LeftArm"]
left_forearm = ["leftforearm""leftarm""LeftForeArm"]
left_hand = ["lefthand""leftforearm""LeftHand"]
left_hand_thumb_1 = ["lefthandthumb1""lefthand""LeftHandThumb1"]
left_hand_thumb_2 = ["lefthandthumb2""lefthandthumb1""LeftHandThumb2"]
left_hand_thumb_3 = ["lefthandthumb3""lefthandthumb2""LeftHandThumb3"]
left_in_hand_index = ["leftinhandindex""lefthand""LeftInHandIndex"]
left_hand_index_1 = ["lefthandindex1""leftinhandindex""LeftHandIndex1"]
left_hand_index_2 = ["lefthandindex2""lefthandindex1""LeftHandIndex2"]
left_hand_index_3 = ["lefthandindex3""lefthandindex2""LeftHandIndex3"]
left_in_hand_middle = ["leftinhandmiddle""lefthand""LeftInHandMiddle"]
left_hand_middle_1 = ["lefthandmiddle1""leftinhandmiddle""LeftHandMiddle1"]
left_hand_middle_2 = ["lefthandmiddle2""lefthandmiddle1""LeftHandMiddle2"]
left_hand_middle_3 = ["lefthandmiddle3""lefthandmiddle2""LeftHandMiddle3"]
left_in_hand_ring = ["leftinhandring""lefthand""LeftInHandRing"]
left_hand_ring_1 = ["lefthandring1""leftinhandring""LeftHandRing1"]
left_hand_ring_2 = ["lefthandring2""lefthandring1""LeftHandRing2"]
left_hand_ring_3 = ["lefthandring3""lefthandring2""LeftHandRing3"]
left_hand_pinky = ["leftinhandpinky""lefthand""LeftInHandPinky"]
left_hand_pinky_1 = ["lefthandpinky1""leftinhandpinky""LeftHandPinky1"]
left_hand_pinky_2 = ["lefthandpinky2""lefthandpinky1""LeftHandPinky2"]
left_hand_pinky_3 = ["lefthandpinky3""lefthandpinky2""LeftHandPinky3"]

hik_bone_list = [hips, right_up_leg, right_leg, right_foot, left_up_leg, left_leg, left_foot,
                 spine, spine_1, spine_2, spine_3, neck, head,
                 right_shoulder, right_arm, right_forearm, right_hand,
                 right_hand_thumb_1, right_hand_thumb_2, right_hand_thumb_3,
                 right_in_hand_index, right_hand_index_1, right_hand_index_2, right_hand_index_3,
                 right_in_hand_middle, right_hand_middle_1, right_hand_middle_2, right_hand_middle_3,
                 right_in_hand_ring, right_hand_ring_1, right_hand_ring_2, right_hand_ring_3,
                 right_in_hand_pinky, right_hand_pinky_1, right_hand_pinky_2, right_hand_pinky_3,
                 left_shoulder, left_arm, left_forearm, left_hand,
                 left_hand_thumb_1, left_hand_thumb_2, left_hand_thumb_3,
                 left_in_hand_index, left_hand_index_1, left_hand_index_2, left_hand_index_3,
                 left_in_hand_middle, left_hand_middle_1, left_hand_middle_2, left_hand_middle_3,
                 left_in_hand_ring, left_hand_ring_1, left_hand_ring_2, left_hand_ring_3,
                 left_hand_pinky, left_hand_pinky_1, left_hand_pinky_2, left_hand_pinky_3]

tpose = [
          01050000,       #hips                   [0]
          -111050000,
          -11560000,
          -1180000,
          111050000,
          11560000,
          1180000.0,
          01180000,       #spine                  [7]
          01290000,
          01410000,
          01520000,
          01640000,
          01730000,
          -31600000,      #right_shoulder         [13]
          -171600000,
          -461600000,
          -741600000,     #right_hand             [16]

          -7716130300,    #right_hand_thumb_1     [17]
          -811613000,
          -831613000,
          -781612000,     #right_in_hand_index    [20]
          -831613000,
          -871613000,
          -891613000,
          -781610000,     #right_in_hand_middle   [24]
          -831611000,
          -881611000,
          -901611000,
          -78161, -0000,    #right_in_hand_ring     [28]
          -83161, -0000,
          -86161, -0000,
          -89161, -0000,
          -77161, -1000,    #right_in_hand_pinky    [32]
          -82161, -2000,
          -85161, -2000,
          -87161, -2000,

          31600000,       #left_shoulder          [36]
          171600000,
          461600000,
          741600000,      #left_hand              [39]

          7716130, -300,    #left_hand_thumb_1      [40]
          811613000,
          831613000,
          781612000,
          831613000,
          871613000,
          891613000,
          781610000,
          831611000,
          881611000,
          901611000,
          781610000,
          831610000,
          861610000,
          891610000,
          77161, -1000,
          82161, -2000,
          85161, -2000,
          87161, -2000]     #   [58]

fram1_data=[-0.05106.89, -3.65, -6.91173.25, -1.78, -11.44, -1.88, -0.06, -6.1412.7, -9.170.19, -48.070.0227.03, -11.940.710.52, -48.92, -0.88, -7.01, -2.0316.6311.54, -1.81, -0.156.9520.594.210.11, -48.010.076.32, -4.320.420.84, -47.31, -0.06, -7.021.87, -1.47, -0.116.66, -0.053.420.62, -2.97, -0.0411.32, -0.060.860.14, -0.75, -0.0211.780.011.430.21, -1.24, -0.0511.31, -0.030.33, -1.320.27, -0.1512.08, -0.00.34, -1.330.27, -0.149.010.018.37, -24.143.38, -3.528.02, -0.02, -2.341.0210.14, -14.04, -0.15, -0.07, -27.4830.5918.74, -28.99, -0.1, -0.127.7836.336.57, -27.99, -0.02, -0.0218.56, -1.48, -48.22, -2.70.213.39, -0.056.6524.69, -4.00.00.0, -0.013.017.06, -2.780.00.0, -0.0, -3.830.0, -3.50.552.15, -0.00.00.0, -5.66, -0.11.08, -0.015.012.78, -3.930.00.0, -0.00.012.78, -2.230.00.0, -0.00.012.78, -3.670.560.82, -0.00.00.0, -5.62, -0.090.34, -0.00.014.26, -4.290.00.0, -0.00.014.26, -2.690.00.0, -0.00.014.26, -3.650.58, -0.14, -0.00.00.0, -5.03, -0.02, -0.52, -0.0, -15.013.89, -3.740.00.0, -0.00.013.89, -2.590.00.0, -0.00.013.89, -3.430.51, -1.3, -0.00.00.0, -4.5, -0.02, -1.18, -0.0, -25.010.03, -2.990.00.0, -0.00.010.03, -1.890.00.0, -0.00.010.033.488.090.045.9219.77, -1.9314.080.060.031.13, -12.31, -70.2928.980.110.073.24, -2.941.2228.0, -0.020.264.0810.776.192.70.213.39, -0.0, -52.26, -28.524.00.00.0, -0.0, -2.75, -5.082.780.00.0, -0.0, -5.530.03.50.552.15, -0.00.00.05.66, -0.11.08, -0.0, -15.0, -16.843.930.00.0, -0.00.0, -16.842.230.00.0, -0.00.0, -16.843.670.560.82, -0.00.00.05.62, -0.090.34, -0.00.0, -18.324.290.00.0, -0.00.0, -18.322.690.00.0, -0.00.0, -18.323.650.58, -0.14, -0.00.00.05.03, -0.02, -0.52, -0.015.0, -21.53.740.00.0, -0.00.0, -21.52.590.00.0, -0.00.0, -21.53.430.51, -1.3, -0.00.00.04.5, -0.02, -1.18, -0.025.0, -24.82.990.00.0, -0.00.0, -24.81.890.00.0, -0.00.0, -24.8]

fram2_data=[6.44106.862.11, -6.0144.52, -2.36, -11.52, -1.83, -0.057.039.661.24, -0.07, -48.0, -0.043.28, -11.53, -0.020.81, -47.99, -0.85, -4.792.472.4811.55, -1.860.025.8610.688.650.17, -47.990.056.93, -0.614.33, -0.25, -47.970.22, -3.0822.35, -10.460.0416.650.03.11, -0.772.420.0111.31, -0.030.78, -0.180.61, -0.0211.78, -0.01.31, -0.281.01, -0.0211.31, -0.010.04, -0.330.20.0212.09, -0.040.04, -0.330.2, -0.039.0, -0.081.2, -5.813.62, -3.518.060.02, -3.8, -0.277.17, -14.01, -0.01, -0.01, -0.677.8171.31, -29.00.01, -0.15, -4.460.726.46, -28.00.05, -0.0713.32, -6.34, -14.73, -2.70.213.39, -0.051.6120.16, -4.00.00.0, -0.02.942.53, -2.780.00.0, -0.05.460.0, -3.50.552.15, -0.00.00.0, -5.66, -0.11.08, -0.015.019.35, -3.930.00.0, -0.00.019.35, -2.230.00.0, -0.00.019.35, -3.670.560.82, -0.00.00.0, -5.62, -0.090.34, -0.00.023.71, -4.290.00.0, -0.00.023.71, -2.690.00.0, -0.00.023.71, -3.650.58, -0.14, -0.00.00.0, -5.03, -0.02, -0.52, -0.0, -15.022.43, -3.740.00.0, -0.00.022.43, -2.590.00.0, -0.00.022.43, -3.430.51, -1.3, -0.00.00.0, -4.5, -0.02, -1.18, -0.0, -25.018.58, -2.990.00.0, -0.00.018.58, -1.890.00.0, -0.00.018.583.498.080.032.2610.43, -4.9314.020.020.074.27, -8.99, -74.9329.00.10.04, -2.67, -3.664.1928.010.050.034.623.087.232.70.213.39, -0.0, -52.71, -22.764.00.00.0, -0.0, -3.650.682.780.00.0, -0.0, -2.840.03.50.552.15, -0.00.00.05.66, -0.11.08, -0.0, -15.0, -14.13.930.00.0, -0.00.0, -14.12.230.00.0, -0.00.0, -14.13.670.560.82, -0.00.00.05.62, -0.090.34, -0.00.0, -17.74.290.00.0, -0.00.0, -17.72.690.00.0, -0.00.0, -17.73.650.58, -0.14, -0.00.00.05.03, -0.02, -0.52, -0.015.0, -18.983.740.00.0, -0.00.0, -18.982.590.00.0, -0.00.0, -18.983.430.51, -1.3, -0.00.00.04.5, -0.02, -1.18, -0.025.0, -24.322.990.00.0, -0.00.0, -24.321.890.00.0, -0.00.0, -24.32]

mocap_manager = RLPy.RGlobal.GetMocapManager()
hand_device_ID = "HandDevice"
hand_device = mocap_manager.AddHandDevice(hand_device_ID)

# avatar_list = RLPy.RScene.GetAvatars()
# avatar = avatar_list[0]


selection_list = RLPy.RScene.GetSelectedObjects()
if len(selection_list) > 0:
            for object in selection_list:  # find first avatar
                object_type = object.GetType()
                if object_type == RLPy.EObjectType_Avatar:
                    avatar = object
                    if hand_device is not None:
                        hand_device.AddAvatar(avatar)
                        hand_device.SetProcessDataIndex(avatar, 0)

# hand_device.AddAvatar(avatar)
hand_device.SetEnable(avatar, True)

hand_setting = hand_device.GetHandSetting(avatar)

hand_setting.SetRightHandJoin(RLPy.EHandJoin_Wrist)
hand_setting.SetLeftHandJoin(RLPy.EHandJoin_Wrist)
hand_setting.SetHandJoinType(RLPy.EHandJoinType_UseParentBone)
hand_setting.SetRightHandDataSource(RLPy.EHandDataSource_RightHand)
hand_setting.SetLeftHandDataSource(RLPy.EHandDataSource_RightHand)
hand_setting.SetActivePart(RLPy.EBodyActivePart_Hand_R | RLPy.EBodyActivePart_Finger_R |
RLPy.EBodyActivePart_Hand_L | RLPy.EBodyActivePart_Finger_L)

device_setting = hand_device.GetDeviceSetting()
device_setting.SetMocapCoordinate(RLPy.ECoordinateAxis_Y, RLPy.ECoordinateAxes_Z, RLPy.ECoordinateSystem_RightHand)
device_setting.SetCoordinateOffset(0, [000])

position_setting = device_setting.GetPositionSetting()
rotation_setting = device_setting.GetRotationSetting()
rotation_setting.SetType(RLPy.ERotationType_Euler)
rotation_setting.SetUnit(RLPy.ERotationUnit_Degrees)
rotation_setting.SetEulerOrder(RLPy.EEulerOrder_ZXY)
rotation_setting.SetCoordinateSpace(RLPy.ECoordinateSpace_Local)
position_setting.SetUnit(RLPy.EPositionUnit_Centimeters)
position_setting.SetCoordinateSpace(RLPy.ECoordinateSpace_Local)

hand_device.SetTPoseData(avatar, tpose)

hand_device.Initialize(hik_bone_list)

if hand_device.IsTPoseReady(avatar) == True:
    mocap_manager.Start(RLPy.EMocapState_Record)

fram1_time = 1000
fram2_time = 5000
hand_device.ProcessData(0, fram1_data, fram1_time)
hand_device.ProcessData(0, fram2_data, fram2_time)

mocap_manager.Stop()
Yukari.
Yukari.
Posted Last Year
View Quick Profile
Senior Member

Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)

Group: Forum Members
Last Active: 2 days ago
Posts: 7, Visits: 498
Hi delebash,
Have you solved this problem?
I tried to run your sample code, and I found that SetTPoseData must be called after Initialize.
You can try it. Smile

    hand_device.Initialize(hik_bone_list) // First
    hand_device.SetTPoseData(avatar, tpose) // Second

BR
Edited
Last Year by Zong Ye
delebash
delebash
Posted Last Year
View Quick Profile
Veteran Member

Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)

Group: Forum Members
Last Active: Last Week
Posts: 32, Visits: 400
Zong Ye, 

Thank you that fixed the problem.  Just a note that your sample documentation needs to be corrected as https://wiki.reallusion.com/IC_Python_API:RLPy_RIHandDevice shows 

hand_device.SetTPoseData(t_pose_data) //First
hand_device.Initialize(bone_list) //Second

I have a couple of questions

1) Can you explain the tpose data?
I believe that each row corresponds to a bone but what do the numbers mean 0, 105, 0, 0, 0, 0,  ?  Does this partially correspond to x,y,z coordinates ?

2) Can you explain the what the numbers correspond to in the frame data?
I think for example frame_data = [-0.05, 106.89, -3.65, -6.91, 173.25, -1.78, -11.44, -1.88, -0.06, -6.14, 12.7, -9.17] that
-0.05, 106.89, -3.65, -6.91, 173.25, -1.78 =  hips and
-11.44, -1.88, -0.06, -6.14, 12.7, -9.17 = rightupleg

Is this correct?

Also can you explain what the coordinates for each body part mean?
The data that I have as input will be time_frame, body_part, xyz coordinates  how do I map this to your frame data coordinates?

Thank you very very very much for your help!!!

Dan
Edited
Last Year by delebash
Yukari.
Yukari.
Posted Last Year
View Quick Profile
Senior Member

Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)Senior Member (308 reputation)

Group: Forum Members
Last Active: 2 days ago
Posts: 7, Visits: 498
Hi delebash,

I really appreciate you asking me, but I also can not find the more description of these data on the Python API wiki.

But I searched for information about t-pose on iCone manual, and found that there are some documents.
I don't know if this information can help you.

BR
Edited
Last Year by Zong Ye
delebash
delebash
Posted Last Year
View Quick Profile
Veteran Member

Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)Veteran Member (748 reputation)

Group: Forum Members
Last Active: Last Week
Posts: 32, Visits: 400
Thank you for the help!!



Reading This Topic