Question about iClone Facial Mocap

Question about iClone Facial Mocap

Home
»
Archive
»
iClone 7
»
General

https://forum.reallusion.com/Topic511138.aspx

Print Topic | Close Window

By nick.davison - 4 Years Ago

I am considering buying iClone for Facial Mocap.

My current understanding is that these programs are the basic requirements for supporting Facial MoCap on iClone:
• iClone 7 IClone 7
• Motion LIVE Plugin for iClone
• A Profile plugin for Motion LIVE

Am I correct in assuming that because I want to use a webcam for facial MoCap rather than an iPhone, I would need to buy
the Facewear Profile for Motion LIVE Plugin, which costs RRP $1590, Presale Price $990?

By Kelleytoons - 4 Years Ago

Yep, that's about it.

(Note that you don't need an iPhone with service, so you can get by with some used iPhone costing only around $500).

By animagic - 4 Years Ago

Kelleytoons (4/24/2022)

Yep, that's about it.

(Note that you don't need an iPhone with service, so you can get by with some used iPhone costing only around $500).

I bought an unlocked refurbished iPhone XR directly from Apple, which is indeed around $500.

By R Ham - 4 Years Ago

I've been evaluating the Faceware front end this week. It works, after a bit of fine tuning. It's not much more advanced than CrazyTalk c2005, but if you've got plenty of money, it's one way to be rid of that iPhone.
IMO, iClone ought to offer more than *one* webcam option, $1K is a bit rich for me.

By sungod - 4 Years Ago

White Sands (4/25/2022)

That price is a third-party pricing.

By animagic - 4 Years Ago

White Sands (4/25/2022)

I started with FaceWare and one problem is the convoluted way to get a license from them. In addition, the license is only good for one system. But it is not bad. I don't understand the CrazyTalk reference as that is not facial mocap.

I am by no means an Apple fan, but I'm also not on a mission; that's why I did go for the Live Face option and purchased a refurbished unlocked XR. The total cost is less, you have a better facial mocap system, and you have a useable phone.

By R Ham - 4 Years Ago

animagic (4/26/2022)

White Sands (4/25/2022)

-----------------------------------------------------------------------------------

No, you're right. It's lipsync, not mocap. Like Acculips. I was fascinated with it. Still am.

By R Ham - 4 Years Ago

sungod (4/26/2022)

White Sands (4/25/2022)

That price is a third-party pricing.

I'm seeing $900+ on this site. I rounded.

By linus10111 - 4 Years Ago

nick.davison (4/24/2022)

Yes, this is correct.

I was unfortunate to have spent a large amount of money on purchasing both liveface and faceware realtime. Faceware (webcam version) failed to give me the mocap quality I needed, so i purchased liveface (iPhone version) and the difference was quite noticeable even before making any adjustments.
IMO skip buying the webcam profile and rather go for the iPhone version. They both cost more or less the same, but live face has a few more features you can fiddle with to enhance mocap quality. Also, I think liveface's mocap quality is largely due to the fact that iPhones X and newer, have depth perception (which is good for reading your face). At the time i purchased Faceware, good quality webcams with depth perception were unavailable.

By linus10111 - 4 Years Ago

animagic (4/26/2022)

I started with FaceWare and one problem is the convoluted way to get a license from them. In addition, the license is only good for one system.

This in itself makes it bad. Took me almost a week to get it to work, and that was after a lot of headache with back and forth messages with support, and reinstallations of 'wibu-keys' or whatever that security software is called :crying:

By R Ham - 4 Years Ago

linus10111 (4/27/2022)

nick.davison (4/24/2022)

Yes, this is correct.

If you ever find that, after some time has passed, your login credentials cannot be located, and if you cannot produce a proof of purchase document, Apple will brick your device. This was done to me. If you're slow to grasp their procedure, like I was, and you let it happen more than once, this can run into money. There is no other hardware mfgr that would even consider this kind of behavior. Note that Apple gives NO WARNING that this is their policy. If you doubt what I say, ask them. If you're OK with this, the iPhone's for you. Recently they denied my login when my credentials were PROVEN to be valid 8 months earlier, and offered no explanation for this at all. If you have thought about actually making money with Apple tools, be advised.

IR cameras are not more accurate than webcams, they're just less fiddly.

By R Ham - 4 Years Ago

linus10111 (4/27/2022)

animagic (4/26/2022)

I started with FaceWare and one problem is the convoluted way to get a license from them. In addition, the license is only good for one system.

I had no problem getting Faceware installed and operating, but this is the first I've heard about licensing issues. I assumed you just pay RL for the software, they enable it, and that's all there is to it. Is this not so?

By linus10111 - 4 Years Ago

White Sands (4/27/2022)

[quote][b]
If you ever find that, after some time has passed, your login credentials cannot be located, and if you cannot produce a proof of purchase document, Apple will brick your device. This was done to me. If you're slow to grasp their procedure, like I was, and you let it happen more than once, this can run into money. There is no other hardware mfgr that would even consider this kind of behavior. Note that Apple gives NO WARNING that this is their policy. If you doubt what I say, ask them. If you're OK with this, the iPhone's for you. Recently they denied my login when my credentials were PROVEN to be valid 8 months earlier, and offered no explanation for this at all. If you have thought about actually making money with Apple tools, be advised.

IR cameras are not more accurate than webcams, they're just less fiddly.

Sorry about your predicament. Where I live, there are phone repair shops that can reset those errors. It's supposed to be illegal, but authorities aren't acting on it, so...:Whistling:

Any affordable IR cameras that come to mind? I could give one a try.

By linus10111 - 4 Years Ago

White Sands (4/27/2022)

Unfortunately RL told me they could not offer any support, and i had to talk to faceware support. I would enter the key provided and it would just not activate. Turns out i needed to install 'wibu-key' (i forgot the exact name of that software), then requested a replacement license. The process was tiring, time-consuming and not something I'd like to do again.

By R Ham - 4 Years Ago

linus10111 (4/27/2022)

White Sands (4/27/2022)

This is the first time I've heard of RL selling and installing something they would not stand behind. I'd like to see one of the Mods comment on this.

By R Ham - 4 Years Ago

linus10111 (4/27/2022)

White Sands (4/27/2022)

Although I have seen other IR cameras used on a job site, I know of none other that can be used for mocap. I'm not clear on why this is.
EDIT:
https://www.optitrack.com/software/

By linus10111 - 4 Years Ago

White Sands (4/27/2022)

[quote]This is the first time I've heard of RL selling and installing something they would not stand behind. I'd like to see one of the Mods comment on this.

I guess there's a first time for everything! :D It's even written on their site "For purchased users, if you have the following issues or feedback for Faceware Realtime for iClone, please contact Faceware Support" Site link: http://support.facewaretech.com/iclone_support

By R Ham - 4 Years Ago

linus10111 (4/27/2022)

White Sands (4/27/2022)

[quote]This is the first time I've heard of RL selling and installing something they would not stand behind. I'd like to see one of the Mods comment on this.

I've been to their site too and I missed this. I'm sure the background to this would be entertaining. Thank you.

By animagic - 4 Years Ago

White Sands (4/27/2022)

linus10111 (4/27/2022)

White Sands (4/27/2022)

This is the first time I've heard of RL selling and installing something they would not stand behind. I'd like to see one of the Mods comment on this.

It would be similar with any third-party Mocap product for which RL provide an interface. RL has simply no control over the vendor. If they were a huge company with a lot of clout it might be different.

By animagic - 4 Years Ago

I would like to point out that one advantage of the iPhone is not just the IR camera but the facial recognition software behind it. In other words, all recognition processing is done in the phone.

With Faceware, the camera provides the image, but the Faceware application needs to do the recognition.

Another advantage of an IR camera is that lighting is less of an issue, which I always found pain with Faceware.

By R Ham - 4 Years Ago

animagic (4/28/2022)

The concept of the IR camera is a good one. It is Apple's policies that are the problem. There are a number of high quality professional IR cameras available, but our face mocap software limits us to Apple. For the money spent for products Apple has disabled on me, I could have bought an Optitrack. A Windows or Linux device would only need the OS reinstalled. Not Apple though. They're special. Not another penny for Apple. At this moment I'm waiting 2 WEEKS for Apple to reset the PW on my iPhone MX. That's their standard disciplinary period. The password was valid and working properly 8 months ago when I put the phone in the drawer. Apple declined to explain how a perfectly functioning login could disable itself. If I had to depend on Apple products for a living, I would go out of business. The iPhone is a child's toy.

Lighting is the least concerning aspect of using a webcam, etc. Lighting is affordable, easily installed, and *never* requires a login ID. If you work with videophone, you'll need the lighting anyway.

By Kelleytoons - 4 Years Ago

It's more than lighting - I can't wear my glasses with Faceware. No issue with the iPhone. In face, the iPhone can even "read" video from the monitor (which I've demoed).

In ever single way Live Face is better (in some ways MUCH better). Although I have both I have not even installed Faceware on my new machine (just not worth it).

By R Ham - 4 Years Ago

Kelleytoons (4/28/2022)

Apple makes plenty of money in spite of their policies, and I'm not here on a crusade.
I too remove my glasses for a mocap.

Tell me please. Is your iPhone your daily-carry cell phone? I'm wondering if Apple has a policy that limits the amount of time you can go between logins. You know, "for security." That might explain my most recent experience with them.

By Peter (RL) - 4 Years Ago

White Sands (4/27/2022)

This is the first time I've heard of RL selling and installing something they would not stand behind. I'd like to see one of the Mods comment on this.

Faceware is a 3rd party product and the activation system is controlled by Faceware themselves. Unlike our own software where we can reset serials and activate/deactivate the software, for Faceware any support related to installing on new computers can only be handled by Faceware themselves unfortunately.

By R Ham - 4 Years Ago

Peter (RL) (4/28/2022)

White Sands (4/27/2022)

This is the first time I've heard of RL selling and installing something they would not stand behind. I'd like to see one of the Mods comment on this.

Thank you Peter.
I notice a lot of activity in the mocap software arena, I have a hunch both Faceware and Apple may have some competition on the way.
We'll see, eh?

By Kelleytoons - 4 Years Ago

White Sands (4/28/2022)

Kelleytoons (4/28/2022)

Yeah, the phone I'm using is my daily carry, although I've also used my wife's phone (but hers is also her daily carry). I've never heard of any restriction on time between logins, though - when I first retired I would only use my phone once a week or so (so logging in about that often). I also have an old iPhone that I only use for one particular app (it doesn't have cell service) and months go by without logging in on it but it always works (then again, I stopped upgrading the iOS a long time ago - one of the tricks about your iPhone is to NOT automatically apply any updates. As a rule if it's not broken don't fix it).

But I can't speak to your login issue without knowing all the details. And even though I'm a level 6 "expert" on the Apple forums (which takes approximately 50K worth of postings that help others) I'm no apologist for Apple. I kind of hate Macs, for example (although my Airbook is still the best piece of hardware I've ever owned). The only thing I can say for sure is if you do want an answer to any of your problems the Apple forums will do it. There are many tens of thousands of folks like myself that will help, and none of them are employed by Apple (so they have no skin in the game). In two decades of forum use I've NEVER seen a problem that couldn't be solved there. So that is always the first place to go (well, after you try the official Apple route).

By R Ham - 4 Years Ago

Kelleytoons (4/28/2022)

White Sands (4/28/2022)

Kelleytoons (4/28/2022)

I never knew about the requirement to save your purchase receipt, until they demanded it. The purchase was made 5 years earlier, in another state, through a bank account I closed. Even the Apple store where I bought it had no record of it. It was an iPad Pro in near-new condition. Apple put it to death. I have 4 Apple devices. One may work in 2 weeks if I'm lucky. Apple's idea of security is like saying "It's for the virus."

I've spotted an interesting Blender plugin, in its beginning stages. It's a facial mocap plugin. What attracts me to it is its simplicity. More simple means less to go wrong. I'm thinking that iC8's much-improved import/export ability may allow more flexibility in mocap sources. I would welcome this.
This is it:
https://www.youtube.com/watch?v=oG4qG4Tt9Y8&ab_channel=cgtinker

By Kelleytoons - 4 Years Ago

I would not count on any external porting of facial mocap - iClone's facial system is non-standard and will not be changing for at least the immediate future (think two-three years).

Now if you intend to port your stuff FROM the RL universe to somewhere else (like Blender) then you could indeed use other systems. Those of us who want to remain in the RL universe will be using Live Face.

By R Ham - 4 Years Ago

Kelleytoons (4/28/2022)

I say no to Apple.
-----------------
EDIT
Maybe, for ~$1K, I'll just kiss up to Apple for a little while.

By genao87 - 4 Years Ago

which is the best IPHONE to get for the facial capture/live face??? I have so far seen posted the IPHONE XR that was bought from Apple themselves. I went to the apple store the Iphone 12 or 13 cost $1,100 just for the phone.

By genao87 - 4 Years Ago

By R Ham - 4 Years Ago

The MX works fine for $500.
EDIT
The MX!! Ha ha ha! I can't remember what I bought! I bought the cheapest one with the mocap camera.

By animagic - 4 Years Ago

I got a refurbished unlocked XR from Apple for $500, and it works fine as well. No need to go all out and get a 12, 13, 14, or whatever...:unsure:

By Kelleytoons - 4 Years Ago

Yes, and remember you do NOT need cell service on it. A used XR is just fine.

By Kidder - 4 Years Ago

I originally purchased faceware and although it worked it seemed clunky, so never really used it, I avoided apple products and thought they were over hyped and restrictive, but when i got my new pc i didn't even try installing faceware due to all the stories i read about re-activation and picked up a iphone 10 in the uk for £199 and I just started to use it following Kellytoons advise, I did it with no sim card and a Lan wire and it worked a treat,
I still use android as my main phone and just use the iphone for facial mocap. It's now about as simple as it gets and will be doing more facial mocap now as it is so easy, Faceware is now consigned to the dustbin.A very expensive Dustbin.

By R Ham - 4 Years Ago

Kidder (4/29/2022)

If Faceware was competitively priced I would have installed it already. If Apple had professional, user-friendly login policies and practices I wouldn't have questioned the iPhone as a tool. Starting now I will treat this XR like a Royal Princess, sweeping the gravel from its path.

Have you figured out a way to get a single eyebrow to move? rather than both at once, using the iPhone? If yes, please tell how.

By genao87 - 4 Years Ago

excellent. the XR it is. going to follow Kellytoons and Animagic advised. so far they have not let me down! been talking to a couple of friends and they read a few horror stories with the Faceware. trying to avoid as much mishaps as possible.

By 4413Media - 4 Years Ago

Have you figured out a way to get a single eyebrow to move? rather than both at once, using the iPhone? If yes, please tell how.

That's a good question. I would probably go with a plan B and make small adjustments and use the face puppet tool for the brow settings. I'll have to play around with it if there is a way though. There's a mapping tool that could be useful.

By animagic - 4 Years Ago

4413Media (5/1/2022)

Have you figured out a way to get a single eyebrow to move? rather than both at once, using the iPhone? If yes, please tell how.

If I recall correctly...:unsure: this is a limitation of the facial recognition program in the iPhone. This question was asked before and I think that was the conclusion.

By genao87 - 4 Years Ago

where did you buy your IPHONE XR? which website? currently Apple themselves doesn't sell them and not too sure which are reliable vendors for buying discontinue phones.

By animagic - 4 Years Ago

genao87 (5/3/2022)

where did you buy your IPHONE XR? which website? currently Apple themselves doesn't sell them and not too sure which are reliable vendors for buying discontinue phones.

So, they're gone. I looked one or two months ago and they were still available.

Unfortunately, there are a lot of scams, so it's hard to know where to go instead.

You could check this out: https://eshop.macsales.com/configure-my-iphone/UICH4H14XXXXXXA. They are a real business at least.

By LarryPlane - 4 Years Ago

What has plenty around the 250€ price point, for the XR.

Question is???? Is it worth it? Can you get good reaults?

By genao87 - 4 Years Ago

I see, If I can't get my hands on it, I am assuming the Iphone SE is the closest thing to the XR? Apple just came out with it and the price range is very close to it. Also I dont need to worry about storage size such as 64GB and 128GB?

By genao87 - 4 Years Ago

yes. the phone is roughly only 3 years old.

By zimtower009 - 4 Years Ago

The iPhone 12 and 13 work well with LiveFace. An iPhone 13 mini also weighs 27% less than the XR.

By LarryPlane - 4 Years Ago

What about the iPhone xr? How did that model do?

The xr on eBay is cheaper than either the 12 or 13.

By 4u2ges - 4 Years Ago

Pardon my ignorance (I am not an Apple person), but what is a crucial difference between X and XR in relation to Live Face performance/stability?
I picked up X a year ago on Ebay for $300 (today it's about $250). Pretty much happy with it (I do Live Face via WiFi). Whole lot better than the Faceware with camera, which I also have.
Has anyone compared both models with iClone Live Face and could report back?

By Kelleytoons - 4 Years Ago

Works EXACTLY the same, Mr. G (I've used the X, the 12 and the 13, which I currently have).

Also - the SE phone will NOT work with Live Face, IIRC (it doesn't have a facial ID camera - the only phones that work with Live Face will NOT have a button on the bottom).

By 4u2ges - 4 Years Ago

Kelleytoons (5/4/2022)

That is what I though...

By genao87 - 4 Years Ago

I was able to find the IPHONE XR. Best Buy has it from their website or you can go to their stores and they can order it for you.

https://www.bestbuy.com/site/searchpage.jsp?st=iphone+xr&_dyncharset=UTF-8&_dynSessConf=&id=pcat17071&type=page&sc=Global&cp=1&nrp=&sp=&qp=&list=n&af=true&iht=y&usc=All+Categories&ks=960&keys=keys

By allnexus - 4 Years Ago

Hi there! I'm new to the forum and Reallusion Software and start by going necro on this thread ... I do beg your pardon for that! But I am a bit confused: I've read several times (in this forum), that an iPhone SE wont work with the Live Face App. I'm more of an android person, but I took the phone of my son to give it a try. And ... the app starts okay, the mesh is showing up correctly and data in the app reads as it should (as far as I can tell). So surely there is no problem with the camera.

But for the love of god I can't connect the app and iClone 8. I've tried connecting it by USB and by WiFi. Nothing seem to work.

So just to be clear: *This* (connection) problem can't be caused by using the "wrong" iphone or can it?

Cheers,
Wernä

By Peter (RL) - 4 Years Ago

Hi Wernä

To be able to use Live Face with iClone 8 requires an iPhone with a TrueDepth camera (eg supports Face ID). Unfortunately the iPhone SE doesn't have a TrueDepth camera so can't be used for facial mocap.

Probably the cheapest option would be to look for an iPhone XR which is the lowest cost iPhone that supports the TrueDepth camera system.

By allnexus - 4 Years Ago

Hi Peter,

thanks a bunch for your answer! Not to come across as nagging, just out of interest: The camera lacks TrueDepth but the app clearly shows a mesh, that conforms to my every muscle movement in realtime (and one can see the captured data on the bottom of the screen). How is this possible with an insufficient camera? And what is the technical hurdle that prevents iClone from using at least this captured data ?

Cheers,
W

By Kelleytoons - 4 Years Ago

Excuse me for jumping in here but - did you try connecting to it?

I know the requirements and I know LOGICALLY what RL is saying about this, but there is something else going on with the process that is very... odd. For example - you can point your phone (WITH a TrueDepth camera running the app) at a flat screen like a monitor and it WILL work correctly. That is, it will see faces and correctly animate from them. I demonstrate that here:

And now you report that a camera WITHOUT a TrueDepth camera is working with the app, seemingly correct. SO... what I wonder is - does the process even NEED such a camera? IOW, regardless of logic or what RL says, if it looks like a duck and... well, you know the rest.

So if I were you, at the very least I'd try and see if your phone works with the app (I don't know if you have the paid version or just a trial, but with the trial you can still test it out). If it does - by all mean tell us! Tell RL in particular! I don't have an iPhone I can test this theory out with, but it sure seems to me like it would work.

By allnexus - 4 Years Ago

The getting to connect part is the reason I came here in the first place. :)

It didn't work (not via USB, not using WiFi). iClone doesn't find the phone, even if they are on the same network. But everything (minus the bald guy in the picture :D) else looked so promising:

https://forum.reallusion.com/uploads/images/387b1634-25f0-4277-b58e-6c2c.jpg

So ... if somebody could point me in the right direction (but: I'm not too fond of the idea of buying an ethernet adapter for a phone I don't own and a software trial version :D), I would be very happy to test this further.

Cheers!

By animagic - 4 Years Ago

Did you enter the IP address you see for the phone in the Motion Live interface? This is not filled in automatically. You'll also need to add the port number (I believe its 999).

So in your case it would be: "192.168.1.210:999"

By allnexus - 4 Years Ago

Scratch all of this. I feel so stupid ... I didn't realize that I need the Live Face PROFILE as well. Stupid, stupid, stupid....

From what I read in the plugin manual (and saw in the video tutorial I watched) the process should be relatively straight forward ... If I am not mistaken, it should work like this:

- Connect the iPhone to the computer (using USB, WiFi or an ethernet adapter)
- Start Live Face on the iPhone
- Select a character in iClone
- Go to the modify panel and press the "Motion Live" plugin
- In the "gear list" of the now open plugin window under "gear" press the + sign next to "facial" and select the now appearing gear "LIVE FACE" <---- at this point my problems start
- Enter the IP adress and the port number
- ...

At the marked step above I can't go on, because when I press "+" there is nothing to be selected. The dropdown list which should appear is empty. Consequently, when I try to enter the IP adress of my phone anyway (into the empty row), iClone registers the pressed key not as the first digit of an IP adress but as a hotkey. For example: pressing "1" results in iClone opening the motion puppet window, "2" in opening the direct puppet window and so on. This part should be expected, I guess, if no gear was selected prior (which - as mentioned - I can't :)).

Am I missing something? Success seems to be SO close ... :D

Cheers!

By allnexus - 4 Years Ago

Sorry, to post again. But this is too cool to be true. It works! (using an iPhone SE (MXD02J/A) with iOS 15.6).

And for everbody who is as slow on the uptake as I am, all of the steps (assuming, you've installed iClone8 (trial)):

- Install LIVE FACE app on the iPhone
- Install Live Face Profile for iClone 8 (that was the step I was missing)
- Connect your iPhone to the same network as your computer
- Start Live Face app
- Start iClone 8
- Select a character in iClone
- Go to the modify panel and press the "Motion Live" plugin
- In the "gear list" of the now open plugin window under "gear" press the + sign next to "facial" and select the now appearing gear "LIVE FACE"
- Enter the IP adress and the port number
- Click on the little green circle
- Tick the "face" box of a character in the character list (I had to wait a while/click until it worked at first - now it connects right away)
- Press "Preview" or "Record" and fool around. :)

And here is "me" realising this actually works.

By Kelleytoons - 4 Years Ago

allnexus (8/14/2022)

Sorry, to post again. But this is too cool to be true. It works! (using an iPhone SE (MXD02J/A) with iOS 15.6).

AHA!

So, yes, no need for a TrueDepth camera. I really hope Pete is still following this thread - this needs to get back to RL devs and product managers as well (and the advertising department).

And, of course, all begs the question - if a phone without a TrueDepth camera can work, why not an Android phone? Clearly the app would need to be recompiled, but if it isn't "looking" for a TrueDepth camera it ought to work just fine, I would think. That would DOUBLE the number of folks who could buy this (at the very least). I would think RL is leaving money on the table if they don't follow this up.

It kind of makes sense - if the camera itself can somehow "see" a proper face even from a flat screen, then it clearly doesn't need any sort of 3D image.

By allnexus - 4 Years Ago

Do you think it's possible for the developers really not to *know* what the required hardware is? o_O
I would have thought, that maybe it isn't as exact/robust/reliable without TrueDepth so that they didn't advertise it? Or some similar reason...?

On the other hand... if I really expanded the potential customer base for the plugin, I could very well do with a further discount on CC4/iclone 8 (as it's quiet pricey for someone who isn't going to generate any kind of profit from it) . *rotfl*

By Kelleytoons - 4 Years Ago

Let me just say - as nicely as possible - that as someone who has worked VERY closely with the devs over the last couple of years... oh, yes, it's more than possible. They don't know quite what you think they know. But - iClone is a VERY complicated 3D animation program, with hundreds of thousands of lines of code. It's understandable that there are areas in it that even the best dev could not anticipate. Programs with similar complexity (like Max) have teams that are 10x as large as the ones that work on RL's products and they STILL need help as well (I know - I helped the Autodesk teams).

And as my tests indicate, it works just fine without having a 3D head in front of the camera. It's no more or less robust, trust me. That I didn't make the connection between this and it not needing a TrueDepth camera is on me, for sure, but it's also on the devs (who often don't even test this stuff out very well).

By allnexus - 4 Years Ago

I find this strangely charming. :D
*gets out his popcorn and watches how things unfold*

By wetcircuit - 4 Years Ago

allnexus (8/14/2022)

Sorry, to post again. But this is too cool to be true. It works! (using an iPhone SE (MXD02J/A) with iOS 15.6).

allnexus, thanks!

The specs for AR-Kit 4 face tracking (announced in 2020 with iOS 14) requires a truedepth sensor OR an 'A12 Bionic' processor. Other iPhone apps should also 'just work' with a newer iPhone SE, but I think it's up to the dev to list which models are compatible in the app store.

Without Truedepth, the head depth position tracking is supposedly limited (tracking the head in 3D space)

You are the first confirmation I've seen. This is great news!

Kelleytoons (8/14/2022)

And, of course, all begs the question - if a phone without a TrueDepth camera can work, why not an Android phone? Clearly the app would need to be recompiled, but if it isn't "looking" for a TrueDepth camera it ought to work just fine, I would think.

Not to beat a dead horse, but this is more than just recompiling an app.

The blend shape stream is a feature of the iPhone's os (AR-Kit). There is no equivalent expression tracking on Android – or any other OS. An iPhone developer only needs to turn on the front camera and read from AR-Kit. iOS is supplying a data stream that is handled by the hardware and doesn't require any extra processing, it's information that is 'on tap' for free.

Reallusion would need to literally re-invent the wheel (and the tire, and the lugnuts, and the axel) to get that function from some other os –

they would have to create, from scratch, the entire process of tracking the head, stabilizing the image, detecting facial land marks… etc. We are starting to see 3rd-party Android apps that do exactly this (some using opensource kits, but many are proprietary and subscription) . They are significantly slower, and significantly more work for the developer.

RL's faceLive app is super-basic (at least the app that's on the phone) they are literally just turning on the camera and sending the stream (it's fine, it works. RL could try harder, like adding a record button on the app to start iClone). AR-Kit/Faceware is doing the actual tracking.... knowing RL products, I think they would be happy to support ANY mocap software that sends a data stream and has a userbase large enough to justify another plugin. That's not a hardware Apple/Android issue, that's more a 900lb gorilla issue.

Apple has well-leaked world domination plans for AR (a Hololens-like headset, then consumer glasses, AR maps, and eventually a car)… they are playing a long-game to control the AR future –– it's the equivalent of Zuckerberg 'homesteading' VR by under-pricing Quest headsets, but with an actual plan. Apple owns Pixar (hence FACS-based animation blend shapes), and the new 3D standard (USD) to replace FBX, they have an 'iMovie' for AR... LiDAR sensors on iPhones feed to AR maps feed to self-driving cars… it's Apple patents all the way down. They will manufacture another 90 million more iPhones this year….

There is better facial mocap but iPhone is a popular consumer product with broad support and a big head start. No Android manufacturer has any reason to support a speculative AR standard that Apple will control, so no one but Apple is going to supply expression tracking 'on tap' over built-in hardware for free.

I was considering buying an iPhone mini, but glad to hear the cheaper SE will work.

By allnexus - 4 Years Ago

I love how much interesting, helpful and insightful information you guys give! If I had been able to install everything first try I would never had come here and would have missed out. Seems sometimes ignorance *is* bliss - even if the original meaning of the quote couldn' t be far more stretched. :D

By Kelleytoons - 4 Years Ago

Sorry, forgot about the Apple proprietary thingee.

I'm an old man <g>. But the fact that almost any newer iPhone will work is something RL should make known.