Profile Picture

How do you create a script file that uses manual mocap data to move the hand or a finger?

Posted By delebash 4 Years Ago
Rated 5 stars based on 1 vote.

How do you create a script file that uses manual mocap data to move...

Author
Message
delebash
delebash
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 41, Visits: 448
I have tried this script https://wiki.reallusion.com/IC_Python_API:RLPy_RIHandDevice
I also looked at https://github.com/reallusion/iClone/tree/master/HandGesturesPuppeteering for further information but I still cannot seem to get manual mocap data to move a finger on the hand or any part of the hand so I can see how to use mocap data.

Here is my current code that is not working.  Any help or working example would be greatly appreciated!

import ptvsd
ptvsd.enable_attach(address=("localhost"5678), redirect_output=True)
ptvsd.wait_for_attach()

import RLPy

hips = ["hips""""Hips"]
right_up_leg = ["rightupleg""hips""RightUpLeg"]
right_leg = ["rightleg""rightupleg""RightLeg"]
right_foot = ["rightfoot""rightleg""RightFoot"]
left_up_leg = ["leftupleg""hips""LeftUpLeg"]
left_leg = ["leftleg""leftupleg""LeftLeg"]
left_foot = ["leftfoot""leftleg""LeftFoot"]
spine = ["spine""hips""Spine"]
spine_1 = ["spine1""spine""Spine3"]
spine_2 = ["spine2""spine1""Spine6"]
spine_3 = ["spine3""spine2""Spine9"]
neck = ["neck""spine3""Neck"]
head = ["head""neck""Head"]
right_shoulder = ["rightshoulder""spine3""RightShoulder"]
right_arm = ["rightarm""rightshoulder""RightArm"]
right_forearm = ["rightforearm""rightarm""RightForeArm"]
right_hand = ["righthand""rightforearm""RightHand"]
right_hand_thumb_1 = ["righthandthumb1""righthand""RightHandThumb1"]
right_hand_thumb_2 = ["righthandthumb2""righthandthumb1""RightHandThumb2"]
right_hand_thumb_3 = ["righthandthumb3""righthandthumb2""RightHandThumb3"]
right_in_hand_index = ["rightinhandindex""righthand""RightInHandIndex"]
right_hand_index_1 = ["righthandindex1""rightinhandindex""RightHandIndex1"]
right_hand_index_2 = ["righthandindex2""righthandindex1""RightHandIndex2"]
right_hand_index_3 = ["righthandindex3""righthandindex2""RightHandIndex3"]
right_in_hand_middle = ["rightinhandmiddle""righthand""RightInHandMiddle"]
right_hand_middle_1 = ["righthandmiddle1""rightinhandmiddle""RightHandMiddle1"]
right_hand_middle_2 = ["righthandmiddle2""righthandmiddle1""RightHandMiddle2"]
right_hand_middle_3 = ["righthandmiddle3""righthandmiddle2""RightHandMiddle3"]
right_in_hand_ring = ["rightinhandring""righthand""RightInHandRing"]
right_hand_ring_1 = ["righthandring1""rightinhandring""RightHandRing1"]
right_hand_ring_2 = ["righthandring2""righthandring1""RightHandRing2"]
right_hand_ring_3 = ["righthandring3""righthandring2""RightHandRing3"]
right_in_hand_pinky = ["rightinhandpinky""righthand""RightInHandPinky"]
right_hand_pinky_1 = ["righthandpinky1""rightinhandpinky""RightHandPinky1"]
right_hand_pinky_2 = ["righthandpinky2""righthandpinky1""RightHandPinky2"]
right_hand_pinky_3 = ["righthandpinky3""righthandpinky2""RightHandPinky3"]
left_shoulder = ["leftshoulder""spine3""LeftShoulder"]
left_arm = ["leftarm""leftshoulder""LeftArm"]
left_forearm = ["leftforearm""leftarm""LeftForeArm"]
left_hand = ["lefthand""leftforearm""LeftHand"]
left_hand_thumb_1 = ["lefthandthumb1""lefthand""LeftHandThumb1"]
left_hand_thumb_2 = ["lefthandthumb2""lefthandthumb1""LeftHandThumb2"]
left_hand_thumb_3 = ["lefthandthumb3""lefthandthumb2""LeftHandThumb3"]
left_in_hand_index = ["leftinhandindex""lefthand""LeftInHandIndex"]
left_hand_index_1 = ["lefthandindex1""leftinhandindex""LeftHandIndex1"]
left_hand_index_2 = ["lefthandindex2""lefthandindex1""LeftHandIndex2"]
left_hand_index_3 = ["lefthandindex3""lefthandindex2""LeftHandIndex3"]
left_in_hand_middle = ["leftinhandmiddle""lefthand""LeftInHandMiddle"]
left_hand_middle_1 = ["lefthandmiddle1""leftinhandmiddle""LeftHandMiddle1"]
left_hand_middle_2 = ["lefthandmiddle2""lefthandmiddle1""LeftHandMiddle2"]
left_hand_middle_3 = ["lefthandmiddle3""lefthandmiddle2""LeftHandMiddle3"]
left_in_hand_ring = ["leftinhandring""lefthand""LeftInHandRing"]
left_hand_ring_1 = ["lefthandring1""leftinhandring""LeftHandRing1"]
left_hand_ring_2 = ["lefthandring2""lefthandring1""LeftHandRing2"]
left_hand_ring_3 = ["lefthandring3""lefthandring2""LeftHandRing3"]
left_hand_pinky = ["leftinhandpinky""lefthand""LeftInHandPinky"]
left_hand_pinky_1 = ["lefthandpinky1""leftinhandpinky""LeftHandPinky1"]
left_hand_pinky_2 = ["lefthandpinky2""lefthandpinky1""LeftHandPinky2"]
left_hand_pinky_3 = ["lefthandpinky3""lefthandpinky2""LeftHandPinky3"]

hik_bone_list = [hips, right_up_leg, right_leg, right_foot, left_up_leg, left_leg, left_foot,
                 spine, spine_1, spine_2, spine_3, neck, head,
                 right_shoulder, right_arm, right_forearm, right_hand,
                 right_hand_thumb_1, right_hand_thumb_2, right_hand_thumb_3,
                 right_in_hand_index, right_hand_index_1, right_hand_index_2, right_hand_index_3,
                 right_in_hand_middle, right_hand_middle_1, right_hand_middle_2, right_hand_middle_3,
                 right_in_hand_ring, right_hand_ring_1, right_hand_ring_2, right_hand_ring_3,
                 right_in_hand_pinky, right_hand_pinky_1, right_hand_pinky_2, right_hand_pinky_3,
                 left_shoulder, left_arm, left_forearm, left_hand,
                 left_hand_thumb_1, left_hand_thumb_2, left_hand_thumb_3,
                 left_in_hand_index, left_hand_index_1, left_hand_index_2, left_hand_index_3,
                 left_in_hand_middle, left_hand_middle_1, left_hand_middle_2, left_hand_middle_3,
                 left_in_hand_ring, left_hand_ring_1, left_hand_ring_2, left_hand_ring_3,
                 left_hand_pinky, left_hand_pinky_1, left_hand_pinky_2, left_hand_pinky_3]

tpose = [
          01050000,       #hips                   [0]
          -111050000,
          -11560000,
          -1180000,
          111050000,
          11560000,
          1180000.0,
          01180000,       #spine                  [7]
          01290000,
          01410000,
          01520000,
          01640000,
          01730000,
          -31600000,      #right_shoulder         [13]
          -171600000,
          -461600000,
          -741600000,     #right_hand             [16]

          -7716130300,    #right_hand_thumb_1     [17]
          -811613000,
          -831613000,
          -781612000,     #right_in_hand_index    [20]
          -831613000,
          -871613000,
          -891613000,
          -781610000,     #right_in_hand_middle   [24]
          -831611000,
          -881611000,
          -901611000,
          -78161, -0000,    #right_in_hand_ring     [28]
          -83161, -0000,
          -86161, -0000,
          -89161, -0000,
          -77161, -1000,    #right_in_hand_pinky    [32]
          -82161, -2000,
          -85161, -2000,
          -87161, -2000,

          31600000,       #left_shoulder          [36]
          171600000,
          461600000,
          741600000,      #left_hand              [39]

          7716130, -300,    #left_hand_thumb_1      [40]
          811613000,
          831613000,
          781612000,
          831613000,
          871613000,
          891613000,
          781610000,
          831611000,
          881611000,
          901611000,
          781610000,
          831610000,
          861610000,
          891610000,
          77161, -1000,
          82161, -2000,
          85161, -2000,
          87161, -2000]     #   [58]

fram1_data=[-0.05106.89, -3.65, -6.91173.25, -1.78, -11.44, -1.88, -0.06, -6.1412.7, -9.170.19, -48.070.0227.03, -11.940.710.52, -48.92, -0.88, -7.01, -2.0316.6311.54, -1.81, -0.156.9520.594.210.11, -48.010.076.32, -4.320.420.84, -47.31, -0.06, -7.021.87, -1.47, -0.116.66, -0.053.420.62, -2.97, -0.0411.32, -0.060.860.14, -0.75, -0.0211.780.011.430.21, -1.24, -0.0511.31, -0.030.33, -1.320.27, -0.1512.08, -0.00.34, -1.330.27, -0.149.010.018.37, -24.143.38, -3.528.02, -0.02, -2.341.0210.14, -14.04, -0.15, -0.07, -27.4830.5918.74, -28.99, -0.1, -0.127.7836.336.57, -27.99, -0.02, -0.0218.56, -1.48, -48.22, -2.70.213.39, -0.056.6524.69, -4.00.00.0, -0.013.017.06, -2.780.00.0, -0.0, -3.830.0, -3.50.552.15, -0.00.00.0, -5.66, -0.11.08, -0.015.012.78, -3.930.00.0, -0.00.012.78, -2.230.00.0, -0.00.012.78, -3.670.560.82, -0.00.00.0, -5.62, -0.090.34, -0.00.014.26, -4.290.00.0, -0.00.014.26, -2.690.00.0, -0.00.014.26, -3.650.58, -0.14, -0.00.00.0, -5.03, -0.02, -0.52, -0.0, -15.013.89, -3.740.00.0, -0.00.013.89, -2.590.00.0, -0.00.013.89, -3.430.51, -1.3, -0.00.00.0, -4.5, -0.02, -1.18, -0.0, -25.010.03, -2.990.00.0, -0.00.010.03, -1.890.00.0, -0.00.010.033.488.090.045.9219.77, -1.9314.080.060.031.13, -12.31, -70.2928.980.110.073.24, -2.941.2228.0, -0.020.264.0810.776.192.70.213.39, -0.0, -52.26, -28.524.00.00.0, -0.0, -2.75, -5.082.780.00.0, -0.0, -5.530.03.50.552.15, -0.00.00.05.66, -0.11.08, -0.0, -15.0, -16.843.930.00.0, -0.00.0, -16.842.230.00.0, -0.00.0, -16.843.670.560.82, -0.00.00.05.62, -0.090.34, -0.00.0, -18.324.290.00.0, -0.00.0, -18.322.690.00.0, -0.00.0, -18.323.650.58, -0.14, -0.00.00.05.03, -0.02, -0.52, -0.015.0, -21.53.740.00.0, -0.00.0, -21.52.590.00.0, -0.00.0, -21.53.430.51, -1.3, -0.00.00.04.5, -0.02, -1.18, -0.025.0, -24.82.990.00.0, -0.00.0, -24.81.890.00.0, -0.00.0, -24.8]

fram2_data=[6.44106.862.11, -6.0144.52, -2.36, -11.52, -1.83, -0.057.039.661.24, -0.07, -48.0, -0.043.28, -11.53, -0.020.81, -47.99, -0.85, -4.792.472.4811.55, -1.860.025.8610.688.650.17, -47.990.056.93, -0.614.33, -0.25, -47.970.22, -3.0822.35, -10.460.0416.650.03.11, -0.772.420.0111.31, -0.030.78, -0.180.61, -0.0211.78, -0.01.31, -0.281.01, -0.0211.31, -0.010.04, -0.330.20.0212.09, -0.040.04, -0.330.2, -0.039.0, -0.081.2, -5.813.62, -3.518.060.02, -3.8, -0.277.17, -14.01, -0.01, -0.01, -0.677.8171.31, -29.00.01, -0.15, -4.460.726.46, -28.00.05, -0.0713.32, -6.34, -14.73, -2.70.213.39, -0.051.6120.16, -4.00.00.0, -0.02.942.53, -2.780.00.0, -0.05.460.0, -3.50.552.15, -0.00.00.0, -5.66, -0.11.08, -0.015.019.35, -3.930.00.0, -0.00.019.35, -2.230.00.0, -0.00.019.35, -3.670.560.82, -0.00.00.0, -5.62, -0.090.34, -0.00.023.71, -4.290.00.0, -0.00.023.71, -2.690.00.0, -0.00.023.71, -3.650.58, -0.14, -0.00.00.0, -5.03, -0.02, -0.52, -0.0, -15.022.43, -3.740.00.0, -0.00.022.43, -2.590.00.0, -0.00.022.43, -3.430.51, -1.3, -0.00.00.0, -4.5, -0.02, -1.18, -0.0, -25.018.58, -2.990.00.0, -0.00.018.58, -1.890.00.0, -0.00.018.583.498.080.032.2610.43, -4.9314.020.020.074.27, -8.99, -74.9329.00.10.04, -2.67, -3.664.1928.010.050.034.623.087.232.70.213.39, -0.0, -52.71, -22.764.00.00.0, -0.0, -3.650.682.780.00.0, -0.0, -2.840.03.50.552.15, -0.00.00.05.66, -0.11.08, -0.0, -15.0, -14.13.930.00.0, -0.00.0, -14.12.230.00.0, -0.00.0, -14.13.670.560.82, -0.00.00.05.62, -0.090.34, -0.00.0, -17.74.290.00.0, -0.00.0, -17.72.690.00.0, -0.00.0, -17.73.650.58, -0.14, -0.00.00.05.03, -0.02, -0.52, -0.015.0, -18.983.740.00.0, -0.00.0, -18.982.590.00.0, -0.00.0, -18.983.430.51, -1.3, -0.00.00.04.5, -0.02, -1.18, -0.025.0, -24.322.990.00.0, -0.00.0, -24.321.890.00.0, -0.00.0, -24.32]

mocap_manager = RLPy.RGlobal.GetMocapManager()
hand_device_ID = "HandDevice"
hand_device = mocap_manager.AddHandDevice(hand_device_ID)

# avatar_list = RLPy.RScene.GetAvatars()
# avatar = avatar_list[0]


selection_list = RLPy.RScene.GetSelectedObjects()
if len(selection_list) > 0:
            for object in selection_list:  # find first avatar
                object_type = object.GetType()
                if object_type == RLPy.EObjectType_Avatar:
                    avatar = object
                    if hand_device is not None:
                        hand_device.AddAvatar(avatar)
                        hand_device.SetProcessDataIndex(avatar, 0)

# hand_device.AddAvatar(avatar)
hand_device.SetEnable(avatar, True)

hand_setting = hand_device.GetHandSetting(avatar)

hand_setting.SetRightHandJoin(RLPy.EHandJoin_Wrist)
hand_setting.SetLeftHandJoin(RLPy.EHandJoin_Wrist)
hand_setting.SetHandJoinType(RLPy.EHandJoinType_UseParentBone)
hand_setting.SetRightHandDataSource(RLPy.EHandDataSource_RightHand)
hand_setting.SetLeftHandDataSource(RLPy.EHandDataSource_RightHand)
hand_setting.SetActivePart(RLPy.EBodyActivePart_Hand_R | RLPy.EBodyActivePart_Finger_R |
RLPy.EBodyActivePart_Hand_L | RLPy.EBodyActivePart_Finger_L)

device_setting = hand_device.GetDeviceSetting()
device_setting.SetMocapCoordinate(RLPy.ECoordinateAxis_Y, RLPy.ECoordinateAxes_Z, RLPy.ECoordinateSystem_RightHand)
device_setting.SetCoordinateOffset(0, [000])

position_setting = device_setting.GetPositionSetting()
rotation_setting = device_setting.GetRotationSetting()
rotation_setting.SetType(RLPy.ERotationType_Euler)
rotation_setting.SetUnit(RLPy.ERotationUnit_Degrees)
rotation_setting.SetEulerOrder(RLPy.EEulerOrder_ZXY)
rotation_setting.SetCoordinateSpace(RLPy.ECoordinateSpace_Local)
position_setting.SetUnit(RLPy.EPositionUnit_Centimeters)
position_setting.SetCoordinateSpace(RLPy.ECoordinateSpace_Local)

hand_device.SetTPoseData(avatar, tpose)

hand_device.Initialize(hik_bone_list)

if hand_device.IsTPoseReady(avatar) == True:
    mocap_manager.Start(RLPy.EMocapState_Record)

fram1_time = 1000
fram2_time = 5000
hand_device.ProcessData(0, fram1_data, fram1_time)
hand_device.ProcessData(0, fram2_data, fram2_time)

mocap_manager.Stop()
Yukari.
Yukari.
Posted 4 Years Ago
View Quick Profile
Senior Member

Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)

Group: Forum Members
Last Active: 2 Years Ago
Posts: 7, Visits: 506
Hi delebash,
Have you solved this problem?
I tried to run your sample code, and I found that SetTPoseData must be called after Initialize.
You can try it. Smile

    hand_device.Initialize(hik_bone_list) // First
    hand_device.SetTPoseData(avatar, tpose) // Second

BR
Edited
4 Years Ago by Zong Ye
delebash
delebash
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 41, Visits: 448
Zong Ye, 

Thank you that fixed the problem.  Just a note that your sample documentation needs to be corrected as https://wiki.reallusion.com/IC_Python_API:RLPy_RIHandDevice shows 

hand_device.SetTPoseData(t_pose_data) //First
hand_device.Initialize(bone_list) //Second

I have a couple of questions

1) Can you explain the tpose data?
I believe that each row corresponds to a bone but what do the numbers mean 0, 105, 0, 0, 0, 0,  ?  Does this partially correspond to x,y,z coordinates ?

2) Can you explain the what the numbers correspond to in the frame data?
I think for example frame_data = [-0.05, 106.89, -3.65, -6.91, 173.25, -1.78, -11.44, -1.88, -0.06, -6.14, 12.7, -9.17] that
-0.05, 106.89, -3.65, -6.91, 173.25, -1.78 =  hips and
-11.44, -1.88, -0.06, -6.14, 12.7, -9.17 = rightupleg

Is this correct?

Also can you explain what the coordinates for each body part mean?
The data that I have as input will be time_frame, body_part, xyz coordinates  how do I map this to your frame data coordinates?

Thank you very very very much for your help!!!

Dan
Edited
4 Years Ago by delebash
Yukari.
Yukari.
Posted 4 Years Ago
View Quick Profile
Senior Member

Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)Senior Member (382 reputation)

Group: Forum Members
Last Active: 2 Years Ago
Posts: 7, Visits: 506
Hi delebash,

I really appreciate you asking me, but I also can not find the more description of these data on the Python API wiki.

But I searched for information about t-pose on iCone manual, and found that there are some documents.
I don't know if this information can help you.

BR
Edited
4 Years Ago by Zong Ye
delebash
delebash
Posted 4 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)Distinguished Member (1.2K reputation)

Group: Forum Members
Last Active: Last Month
Posts: 41, Visits: 448
Thank you for the help!!



Reading This Topic