I am creating a game that will need a lot of face animations. It will mostly be a monologue from a NPC to the player in a scene in 3D and 2D.
As a programmer I got limited skill on the animation parts, so I figure, why not have my own face recorded and use it for the face animation in the game.
I bought Crazytalk to just get a feel about the face animation process. The auto lip sync was not that good, when you try read lips. With some manual work it got better. And I just continued to go over the lips in the timeline and I could get the quality I was after. I notice I missed some sounds like y, ue... (o and u is not the same) Can I add those myself somehow? So they appear in the lip editor?

The next step is to test the Faceware plugin for iClone. I spend some time on the forum and notice problems with licence, camera not record at 60fps and lag.
So here comes some questions:
1# Can you use Faceware plugin for iClone and record face motion in real-time but without a real-time 3D figure, ex using a wireframe figure instead (to save CPU and GPU for the camera to use instead)?
2# Can I export this motion as motion clips to Crazytalk8 pipeline or CTA?
3# Can I manually edit the face motion and lips in iClone?
4# Can I export a motion clip from Crazytalk8 to iClone or CTA?
5# I plan to make some face animations and export it to 2D. Is Crazytalk Animator able to use motions from iClone Faceware?

The biggest concern is that the Faceware will not be worth it. I watched some tutorials and it seem to work fine on most of the face. But the lip sync is not good. Not shocking, because the lips moves very fast. Can I record myself in slowmotion and make the playback faster, maybe? (When I say record in slowmotion I mean in Ace venture style). Else the plan is to use Faceware to capture the head animations. And then use the motion in CT or CTA and improve the lips. And that is possible to do?

Sorry for the many questions post. But when you plan to buy stuff it's often better to ask before and not after.

---

iClone 7, 3DXchange 7 Pipeline, CC3 Pipeline, CT8 Pipeline, Iray, Live Face, iPhone Xr, Win 10, i9 9900K ~5GHz, Nvidia RTX 2080 8GB, 16GB Ram, M.2 SSD

I hoped that someone else would respond... I agree that Faceshift is your best bet and even better if you already have an iPhone.

Anyway, I don't have an iPhone, but I do have FaceWare. First off, 30 fps is sufficient. Contrary to what you might think, mouth movements for lip-synching are not that fast. If they are, you get sort of a "flapping" effect.

My initial results with FaceWare weren't that great, but after placing a LED light bar at the top of my monitor, things improved greatly.

Lip-synching based on a sound file has always been a weak point in iClone and many discussion have been devoted to it. The mechanism in iClone is the same as in CrazyTalk, so there is no reason to use CT and then import in iClone. I can not really advise on what the best approach is for 2D as my experience is strictly 3D.

---

I checked out the Faceshift and it does a better job with the lips. For this you need iPhone X (any special model?) and a Faceshift licence ($1500/year)?
It seems a bit hard to get up to date info about this. My understanding is that iClone got a kind of plugin for the iPhone X, but that is not Faceshift.
Faceshift was bought up by Apple and Faceshift studio dont exists anymore. So basically there seem to be very little info about how to actually get the motion data from the iPhone X to a 3D or 2D program on the PC. Or have I just missed that site?

How do you use Faceshift might be a better question?

---

iClone 7, 3DXchange 7 Pipeline, CC3 Pipeline, CT8 Pipeline, Iray, Live Face, iPhone Xr, Win 10, i9 9900K ~5GHz, Nvidia RTX 2080 8GB, 16GB Ram, M.2 SSD

Thanks for the info. Ok, more an more to use Faceshift. When you say Faceshift you mean an iPhone X (any particular model?). But you would still use iClone and CC? I have a hard time separate the iPhone Faceshift from Faceware from iClone Faceware.

---

iClone 7, 3DXchange 7 Pipeline, CC3 Pipeline, CT8 Pipeline, Iray, Live Face, iPhone Xr, Win 10, i9 9900K ~5GHz, Nvidia RTX 2080 8GB, 16GB Ram, M.2 SSD

I call it Faceshift because that's what it was called before Apple bought it (and maybe even after -- dunno).  RL doesn't refer to it that way, I think they just call it "Live for iPhone X" (which, if so, is much more cumbersome).  In any case, it works with iClone (and, of course, any avatars you bring in via CC2 or CC3).  It and the iPhone is all you need.

---

Alienware Aurora R12, Win 10, i9-119000KF, 3.5GHz CPU, 128GB RAM, RTX 3090 (24GB), Samsung 960 Pro 4TB M-2 SSD, TB+ Disk space
Mike "ex-genius" Kelley

The actual facial mocap functionality is built into the phone (iPhone X and related models), which makes it attractive and fast. All RL provides is an interface to get the mocap information into iClone and CrazyTalk Animator.

Motion LIVE for iClone is a universal interface that allows you to combine different Mocap "streams" (body, gesture, and facial). The Mocap solutions themselves are provided by other vendors (such as Apple) and need to be purchased separately. You also need to purchase a so-called "profile" for the specific Mocap device, which is basically the interface between the device and Motion LIVE.

More here: https://mocap.reallusion.com/iclone-motion-live-mocap/default.html.

---

Thanks. That clear things up. I think I have a good picture of what I need now. My shopping list will be something like this:

---

iClone 7, 3DXchange 7 Pipeline, CC3 Pipeline, CT8 Pipeline, Iray, Live Face, iPhone Xr, Win 10, i9 9900K ~5GHz, Nvidia RTX 2080 8GB, 16GB Ram, M.2 SSD

Thanks, this clear things up what I need for the 3D part.

---

iClone 7, 3DXchange 7 Pipeline, CC3 Pipeline, CT8 Pipeline, Iray, Live Face, iPhone Xr, Win 10, i9 9900K ~5GHz, Nvidia RTX 2080 8GB, 16GB Ram, M.2 SSD