Profile Picture

Animate MetaHuman in Unreal with iClone Unreal Live Link

Posted By Charlene (RL) 3 Years Ago
Rated 5 stars based on 3 votes.
1
2
3
4
5
6
7
...
19

Animate MetaHuman in Unreal with iClone Unreal Live Link

Author
Message
Charlene (RL)
Charlene (RL)
Posted 3 Years Ago
View Quick Profile
Veteran Member

Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)

Group: Banned Members
Last Active: Last Year
Posts: 11, Visits: 341

Dear Users,

The exquisitely detailed MetaHuman, capable of natural realistic motion, has had a big impact on the media and entertainment industry. On the other hand, bringing Metahumans to life while crossing the uncanny valley remains an immediate challenge to real-world production. iClone MetaHuman Live Link is the answer you’ve been looking for, with the ability to let Metahumans talk, emote, and act according to your wish. It provides designers a highly efficient way to apply facial and body animations, along with lip-syncing—all in one application!



Three guides are provided in this thread for face, body, and face+body Live Link connection procedures. Feel free to reply to this thread with comments, suggestions, and inquiries.

Required Tools & Software:
  • iClone 7.9 or above
  • iClone Unreal Live Link
  • Unreal Engine 4.26
  • MetaHuman Creator
  • Quixel Bridge (for import MetaHuman)
  • 3DXChange 7 Pipeline (for creating unique Dummy characters)
  • MetaHuman Kit: iClone MetaHuman Live Link Kit_UE4.26
    1. Assets:
      • iClone
        • mh_rl_std_anim
        • mh_rl_std_pose
        • rl_function_lib
      • MetaHumans
    2. MetaHuman Dummies x18:
      • 9 Females (each including 3 kinds of height and 3 types of weight)
      • 9 Males (each including 3 kinds of height and 3 types of weight)
    3. MH_UE_profile.3dxProfile (Only when you create own Dummy)

Tutorial Video:



Steps to get started:

Step 1: Download the iClone MetaHuman Live Link Kit_UE4.26 zip file.https://forum.reallusion.com/uploads/images/193cc2f8-8d6e-45ed-aeef-5948.jpg

Note: iClone MetaHuman Live Link Kit_UE4.26 is a free bonus pack for iClone Unreal Live Link users.
*If you are a registered iClone Unreal Live Link user, then click THIS LINK to receive the kit.
*If you have newly purchased iClone Unreal Live Link after July 1st, 2021, then browse for the kit in your Order History alongside the Unreal Live Link Plugin.
*If you are eligible for the free indie license and request the iClone Unreal Live Link after July 1st, 2021, you will get the kit along with Live Link.
*If you are iClone Unreal Live Link trial users since July 1st, 2021, you can download the kit via the follow-up email.

Step 2: Create a new UE project:
https://forum.reallusion.com/uploads/images/83cc10cf-2f3e-47d0-8b79-b120.jpg

Step 3: Import a MetaHuman with Quixel Bridge to the UE project:
https://forum.reallusion.com/uploads/images/e4aa0239-fdd2-481c-8860-d576.jpg

Step 4: Make sure RigLogic Plugin and iClone Unreal Live Link are Enabled:
https://forum.reallusion.com/uploads/images/e98fe58c-c4e6-435a-b433-4fee.jpg

Step 5: Copy and paste the iClone and MetaHumans folder from iClone MetaHuman Live Link Kit_UE4.26\Content to the "Content" folder in UE directly.(See the below picture to know how to find the folder in UE, in this case the path is: Unreal Projects\Tutorial\Content)
https://forum.reallusion.com/uploads/images/a2d02a53-b302-4e23-80cc-37ed.jpg


Limitations and workarounds:

  • Using CC3+ characters with ExPlus as the dummy characters is recommended.
  • Limitations: 
  • UE does not support negative values for facial expression sliders:
  • Tongue motions will not be exact: 
https://forum.reallusion.com/uploads/images/70a1c8c8-438b-4460-a955-2b33.gif
  • “Mouth Close” works only when applying Jaw Open (Default & ExPlus profiles).
https://forum.reallusion.com/uploads/images/fcf67767-8d31-4bde-b584-791f.gif
  • The Look at function won’t work for face-only Live Link connection because there is no control over the head bone - the entire body will need to be Live Linked instead.
  • Use the Face Key > Muscle Panel to properly drive the eyeball motions. 



          Charlene (RL)
          Charlene (RL)
          Posted 3 Years Ago
          View Quick Profile
          Veteran Member

          Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)

          Group: Banned Members
          Last Active: Last Year
          Posts: 11, Visits: 341
          Facial Setup

          iClone provides the best of breed multi-pass iPhone facial motion capture from ARKit expressions, to easily adjusting raw mocap data as well as retargeting, to multipass facial recording and essential mocap cleanup.

          Tutorial Video:


          Step 1: Double-click on a BP_MetaHuman:
          https://forum.reallusion.com/uploads/images/1bf4a688-d1a4-4ede-a24c-2b18.jpg

          Step 2: 
          1. Click +Add Component, then Live Link Skeletal Animation.
          2. Click LiveLink.
          3. Click on the + button under Event > On Live Link Updated.
          4. Drag ARKit_Setup under Graphs-Functions to Blueprint to the blueprint window.
          5. Click on Compile.
          https://forum.reallusion.com/uploads/images/6915cbdc-f2cb-4d07-b541-26cd.jpg

          Step 3: Activate iClone Live Link:
          https://forum.reallusion.com/uploads/images/e7110dbe-ddcb-4874-943a-e8ad.jpg

          Step 4: Go to iClone and click on Link Activated:
          https://forum.reallusion.com/uploads/images/d04ef273-efbe-4f41-a8d4-2b16.jpg

          Step 5: Go back to UE > Details and select the BP_metahuman.
          1. Set LLink Face Subj to the iClone character’s name (in this case, “Debra”).
          2. Activate LLink Face Head.
          https://forum.reallusion.com/uploads/images/37cf60cf-ac4a-4363-92ae-bacb.jpg


          Charlene (RL)
          Charlene (RL)
          Posted 3 Years Ago
          View Quick Profile
          Veteran Member

          Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)

          Group: Banned Members
          Last Active: Last Year
          Posts: 11, Visits: 341
          Body Setup

          iClone provides an intuitive way of creating body animation with puppeteering tools, Human-IK motion layer editing, as well as thousands of mocap animations. The same resources are now available for MetaHuman!

          To animate MetaHuman in UE, you will need to set a dummy character in iClone, link that dummy character with MetaHuman, and perform live animation via iClone Unreal Live Link. In this post, we will show you how to use the ready-made dummy characters to complete these steps.

          Tutorial Video:


          Step 1: Double-click on a BP_MetaHuman:
          https://forum.reallusion.com/uploads/images/aaec10e5-f7d0-447a-8750-80dc.jpg

          Step 2: 
          1. Click +Add Component, then Live Link Skeletal Animation.
          2. Click LiveLink.
          3. Click on the + button under Event > On Live Link Updated.
          4. Click on Compile.
          https://forum.reallusion.com/uploads/images/d5a142d1-a80a-46d5-8bdf-f456.jpg

          Step 3: Click on Body and observe the Animation section to the right-hand side. 
          1. Select Use Animation Blueprint for Animation Mode.
          2. Select a name that matches the skeletal mesh (_animbp_C suffix) under the Mesh section.
          https://forum.reallusion.com/uploads/images/39627494-649d-40f6-901a-0210.jpg

          Step 4:
          1. Go to AnimGraph > animbp.
          2. String together the Live Link Pose and Live Link Subject Name nodes before the control rig.
          3. Click the Compile button.
          https://forum.reallusion.com/uploads/images/5dec7c9e-a44d-48a0-b48b-8890.jpg

          Step 5: Double-click the BP_MetaHuman’s name.

          Step 6: 
          1. Drag Body to Blueprint.
          2. Connect it to a Get Anim Instance node.
          3. Connect Get Anim Instance to a Cast To node. Set the Anim Class for the Cast To node to match the Anim Class Name.
          4. Connect the Cast To node to a SET node. 
          5. Create a LLink Body Subj node and set it to Promote to Variable. Connect this node to the Live Link Subject Name of the SET node.
          6. One the left-hand side, make the LLink Body Subj visible (open eye icon) to make it public (expose it).
          7. Connect SET node to a Live Link Setup node.
          8. Click Compile.
          For more details, see: Using Live Link Data.
          https://forum.reallusion.com/uploads/images/43c12d12-8da0-4f64-9c9c-c8af.jpg

          Step 7: 
          Because Metahuman has 18 different body types, the correct body type will need to be applied in order to link motion data in a way that does not cause mesh penetration and inconsistencies with different body proportions.
          1. Drag an iAvatar, that matches the MetaHuman Skeletal mesh name, from the iClone MetaHuman Live Link Kit_UE4.26\MetaHuman Dummies folder into iClone.
          2. Click on the Link Activated button.
          3. Select MetaHuman Dummy for LLink Body Subj.
          https://forum.reallusion.com/uploads/images/c0abed41-aed9-4a81-91fa-81c1.jpg

          Step 8: Apply the desired motions:
          https://forum.reallusion.com/uploads/images/c1e58d6d-faf5-4e9a-8c56-4276.jpg


          How to create a MetaHuman Dummy with 3DXchange

          At the moment, MetaHuman supports 18 different body types. The Reallusion MetaHuman Kit provides the corresponding 18 iAvatar dummies to quickly match those body types. The Dummy creation process is explained below. You can apply the same process for new body types that will become available in the future.


          Tutorial video:


          Step 1: You can find 18 different kinds of MetaHuman body shapes in UE under:
          \Content\MetaHumans\Common\Common.

          https://forum.reallusion.com/uploads/images/d6a7e3a8-c0fa-475c-9250-d719.png

          Step 2: Right-click on the figure you want to create and perform Dummy > Asset Actions > Export.
          https://forum.reallusion.com/uploads/images/6629f89d-70b7-4fd6-acc7-1c6a.png

          Step 3: Adjust the FBX Export Options as the following:
          https://forum.reallusion.com/uploads/images/48108fb3-0da3-47f2-8758-7cf6.png

           |Updated| Step 4: Import the Dummy (FBX) into 3DXchange:
          https://forum.reallusion.com/uploads/images/595ebb65-f19e-4b5e-995d-0187.png
          https://forum.reallusion.com/uploads/images/9db0539d-8972-4d68-9c41-56ab.jpg


          Step 5: In the subsequent window, press OK, press the Apply to iClone button, then click Done.
          https://forum.reallusion.com/uploads/images/58e30515-47d7-4138-8caf-1ea9.png
          https://forum.reallusion.com/uploads/images/5c831f10-e099-4ac5-9b9e-2479.png
          https://forum.reallusion.com/uploads/images/066a33eb-23ac-456a-9f22-510c.png

          Charlene (RL)
          Charlene (RL)
          Posted 3 Years Ago
          View Quick Profile
          Veteran Member

          Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)Veteran Member (766 reputation)

          Group: Banned Members
          Last Active: Last Year
          Posts: 11, Visits: 341
          Face & Body Setup

          Live Link will need to know to connect to the face and body of a character. In order to do this, you’ll have to create a Sequence node in Blueprint for the face and body, and make sure they run independently, or the animation will be wrong.

          |Updated| Tutorial video:


          Step 1: Double-click on a BP_MetaHuman:
          https://forum.reallusion.com/uploads/images/5114180d-dc41-4578-8943-0015.jpg

          Step 2: 
          1. Click +Add Component, then Live Link Skeletal Animation.
          2. Click LiveLink.
          3. Click on the + button under Event > On Live Link Updated.
          4. Click on Compile.
          https://forum.reallusion.com/uploads/images/c687ff74-b1f7-4ea9-998d-501c.jpg

          Step 3: 
          1. Click on Body and observe the Animation section to the right-hand side. 
          2. Select Use Animation Blueprint for Animation Mode.
          3. Select a name that matches the skeletal mesh (_animbp_C suffix) under the Mesh section.
          https://forum.reallusion.com/uploads/images/1a9040a6-bd9e-422d-a186-bace.jpg

          Step 4:
          1. Go to AnimGraph > animbp.
          2. String together the Live Link Pose and Live Link Subject Name nodes before the control rig.
          3. Click the Compile button.
          https://forum.reallusion.com/uploads/images/e9012471-8fa9-404c-a904-e257.jpg

          Step 5:
          1. Add a Sequence node to Blueprint.
          2. Connect Sequence > Face to a ARKit_Setup node available under Graphs-Functions.
          3. Set up Body:
            • Drag Body to Blueprint.
            • Connect it to a Get Anim Instance node.
            • Connect Get Anim Instance to a Cast To node. Set the Anim Class for the Cast To node to match the Anim Class Name.
            • Connect the Cast To node to a SET node. 
            • Create a LLink Body Subj node and set it to Promote to Variable. Connect this node to the Live Link Subject Name of the SET node.
            • One the left-hand side, make the LLink Body Subj visible (open eye icon) to make it public (expose it).
            • Connect SET node to a Live Link Setup node.
            • Click Compile.
          4. Click “Compile”
          https://forum.reallusion.com/uploads/images/b5bdbbc1-75a9-4a3e-88d2-273e.jpg

          Step 6: Activate Live Link:
          1. Set LLink Body Subj to the MetaHuman Dummy.
          2. Set LLink Face Subj to the iClone character’s name.
          3. Activate LLink Face Head
          https://forum.reallusion.com/uploads/images/c37a73c3-f93f-41e5-bdc4-750f.jpg


          StyleMarshal
          StyleMarshal
          Posted 3 Years Ago
          View Quick Profile
          Distinguished Member

          Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)

          Group: Forum Members
          Last Active: 2 days ago
          Posts: 5.8K, Visits: 14.3K
          That's so great !
          Now you can show the whole potential IClone animation tools have to offer.
          Way better than the UE4/5 control rig , in my opinion.



          :)
          toystorylab
          toystorylab
          Posted 3 Years Ago
          View Quick Profile
          Distinguished Member

          Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)

          Group: Forum Members
          Last Active: 10 days ago
          Posts: 4.6K, Visits: 29.9K
          At first glance, quite a lot to do (blueprint) to make that work but will test in the next days...
          Thanx! ;)

          EDIT
          As I already have LL 1.2, I assume I only have add the MH LL Kit?

          Übrigens, Bassline303 hatte bestimmt seine Finger im Spiel, Danke!



          Toystorylab on Vimeo : https://forum.reallusion.com/uploads/images/ce44ea78-6984-47d8-9bf4-b783.png    Crassitudes (my "Alter Ego") on Youtube: https://forum.reallusion.com/uploads/images/fcc4df30-b3a8-40a5-a427-0735.png

          tarnum888
          tarnum888
          Posted 3 Years Ago
          View Quick Profile
          New Member

          New Member (9 reputation)New Member (9 reputation)New Member (9 reputation)New Member (9 reputation)New Member (9 reputation)New Member (9 reputation)New Member (9 reputation)New Member (9 reputation)New Member (9 reputation)

          Group: Forum Members
          Last Active: 3 Years Ago
          Posts: 4, Visits: 27
          Where to find explus character?
          StyleMarshal
          StyleMarshal
          Posted 3 Years Ago
          View Quick Profile
          Distinguished Member

          Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)Distinguished Member (14.4K reputation)

          Group: Forum Members
          Last Active: 2 days ago
          Posts: 5.8K, Visits: 14.3K
          @Toy , better use Live Link 1.3 in UE4 !

          @ tarnum888 , every CC+ with Arkit Character will work.

          You can find "Debra" under IClone Project / 1. Characters / 4.Explus Facial Demo
          wires
          wires
          Posted 3 Years Ago
          View Quick Profile
          Distinguished Member

          Distinguished Member (15.4K reputation)Distinguished Member (15.4K reputation)Distinguished Member (15.4K reputation)Distinguished Member (15.4K reputation)Distinguished Member (15.4K reputation)Distinguished Member (15.4K reputation)Distinguished Member (15.4K reputation)Distinguished Member (15.4K reputation)Distinguished Member (15.4K reputation)

          Group: Forum Members
          Last Active: 3 days ago
          Posts: 5.7K, Visits: 26.5K
          Does this offer any enhanced benefits for iClone animators, or can it safely be ignored and relegated to the trashcan like all the other Gaming stuff?

          Gerry



          System: Win 10 Pro (21H2), Asus X99-E WS, CPU i7-5930K -3,5 GHz, 32 GB DDR4 2666-16 RAM, NVidia GTX 1080 Ti GPU - 11 GB VRAM(Driver Studio-536.99), Samsung 850 Pro 512 GB SSD, 6 TB HD storage.


          toystorylab
          toystorylab
          Posted 3 Years Ago
          View Quick Profile
          Distinguished Member

          Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)Distinguished Member (17.3K reputation)

          Group: Forum Members
          Last Active: 10 days ago
          Posts: 4.6K, Visits: 29.9K
          Hm, when running the new exe, I get report to remove previous one...

          https://forum.reallusion.com/uploads/images/565d4f25-285c-4e5a-8b6b-8fbc.png

          Do I simply delete ICLiveLink folder (OpenPlugin)?

          https://forum.reallusion.com/uploads/images/1e50e893-51ab-43d6-9b6b-f04c.png

          EDIT
          Uninstalled with HUB... ;)



          Toystorylab on Vimeo : https://forum.reallusion.com/uploads/images/ce44ea78-6984-47d8-9bf4-b783.png    Crassitudes (my "Alter Ego") on Youtube: https://forum.reallusion.com/uploads/images/fcc4df30-b3a8-40a5-a427-0735.png


          1
          2
          3
          4
          5
          6
          7
          ...
          19



          Reading This Topic