Realtime *Clean* Facial Mocap Via Webcam - open source


https://forum.reallusion.com/Topic163648.aspx
Print Topic | Close Window

By Ascensi - 12 Years Ago
This is the BEST realtime facial mocap I've seen (no jitters) quick easy setup (seemingly no calibration needed) and oddly someone made it free/open source. Now wouldn't this be a big attraction/incentive for users to migrate to iClone if it were added hopefully -as a bonus feature..

Here is a youtube video of the demonstration and a link to the open source code

http://www.youtube.com/watch?v=-cB8vS6rUbY

Now after this if iClone Dev somehow added the "leap motion" sensor for hands we may have a complete realtime mocap system!

 

Please rate 5 stars if you would like iClone to develop this into iClone, I don't know how often Reallusion team checks the forums for development requests - a high thread rating may get some action.

By Pollux - 12 Years Ago
Interesting software kurtin, thanks for sharing :)
By prabhatM - 12 Years Ago
Very Interesting Software. ICLONE should grab it, develop and release ASAP.
By animagic - 12 Years Ago
Mixamo just announced a markerless webcam-based Mocap solution for use in Unity, called Face Plus.

Of course this is not free or open source, but I would rather have something that works. 

By Pollux - 12 Years Ago
That's very cool.  I'm sure Vidi and Cherry will try it in Unity.  Wait and See :)
By Ascensi - 12 Years Ago
I definitely do not want to pay 1499 every year for Face Plus, not unless I were a big studio being flooded by money in all directions. There is faceshift too but it's not as clean, needs more calibration but at least you have the software for the rest of your life, no money drainage in particular for Indie Artists & hobbyists or new studios on a tight budget. 
By animagic - 12 Years Ago
My point is that markerless facial animation using a webcam is possible, so it is something for RL to look into.
By Pollux - 12 Years Ago
kurtjn (9/13/2013)
I definitely do not want to pay 1499 every year for Face Plus

WOW.... didn't see the price...!  :w00t:

By Ascensi - 12 Years Ago
Pollux (9/13/2013)
kurtjn (9/13/2013)
I definitely do not want to pay 1499 every year for Face Plus

WOW.... didn't see the price...!  :w00t:

Yes if only companies realize that if they over price things, the potential customer is likely to get extremely frustrated and create a free version of the same kind of product that may be superior and ultimately ends that company. I think some of them now realize like Autodesk that if they overprice, consumers get together and create open source that may rival their own. Blender is such an example that now the big companies are feeling it and are lowering prices and working to adapt appeasing potential customers by including export and import preset/options compatible with Blender etc.

By fclone.bluegarage - 8 Years Ago
Hi we created webcam facial motion capture software named f-clone.
http://f-clone.com/
Hope this will helps your creative
Thanks
By mr_vidius - 8 Years Ago
why stop at just the facial.. why not do the whole thing? perception neuron is way too expensive. and way too touchy. the current mocap RL has
is basically useless. when it comes right down to it. that was a total waste of $500 (capture unit + the plug in) if you ask me. I looked into other
motion capture systems.. same thing.. over priced, and way too sensitive electronics. since I am  a professional videographer/photographer.. I was
trying to find something that I could take motion from a video clip. but no luck so far. so I'm doing it all manual. either taking purchased RL motions and modifying
them.. or the old fashioned way of making motions from a video clip, frame by frame.. oh boy... so yeah. do it all folks. go all the way. none of this half butt stuff.

vidius
By animagic - 8 Years Ago
fclone.bluegarage (5/27/2017)
Hi we created webcam facial motion capture software named f-clone.
http://f-clone.com/
Hope this will helps your creative
Thanks

With a name like f-clone you should be compatible with iClone...:D
By GOETZIWOOD STUDIOS - 8 Years Ago
mr_vidius (5/27/2017)
../.. perception neuron is way too expensive. and way too touchy../.. since I am  a professional videographer/photographer..
vidius

If you are a professional then you should find Perception Neuron way too cheap, as there is no equivalent under 10k or 15k yet regular systems are more in the 20K/30K with fingers and all *and* with a capture surface similar to Neuron.
Touchy ? mtakerkart from the forum here seems to make a pretty good and successful uses out of Neuron, check his videos.

I really don't understand why peoples complain about the price of Perception Neuron. It is a revolution in itself, in terms of features and price. Now accessible to hobbyists (photography hobbyists do not hesitate to buy $2K (and more) camera from Canon or Nikon, so a full mocap suit including fingers at $1.5k seems pretty cheap to me).

And thanks to Reallusion to have understood this and to have created the Neuron plugin for iClone.
By yoyomaster - 8 Years Ago
Looks interesting, price is nice, but the only thing that seems to work in the demo is head rotation, so I guess I am doing something wrong here.
I am also wondering what this means in the buy section? "One Month (no autorenew)"
Also wondering if it can be use with iClone?
By mtakerkart - 8 Years Ago
Downloaded it. Doesn't works. Just a grey window.
I sended them an email .
For the price  it cost 199$ for a personal comercial liscence

https://forum.reallusion.com/uploads/images/b740d73e-d6d6-40d1-904e-6d55.jpg
By mtakerkart - 8 Years Ago
Little bit strange that there is only an email to contact. No website , no facebook or a personal name , no number phone or adress....
When I launch the App I can see a very furtive "MADE WITH UNITY" then a empty gray window... . I deduce it use the dead Unity facePlus .
By wendyluvscatz - 8 Years Ago
they do have a Facebook, I "liked" it
but the name F-clone and similar logo got me a bit suspicious
the whole site a bit vague, cannot find a EULA or TOS page.
By animagic - 8 Years Ago
The program starts for me, but then gets stuck at the initial window; clicking Next has no effect.

.https://forum.reallusion.com/uploads/images/9c08274b-e70c-4661-915a-8a92.png
By Zeronimo - 8 Years Ago
wendyluvscatz (5/28/2017)
they do have a Facebook, I "liked" it
but the name F-clone and similar logo got me a bit suspicious
the whole site a bit vague, cannot find a EULA or TOS page.



To find the EULA you have to click on PURCHASE  --> CHECKOUT
https://forum.reallusion.com/uploads/images/c83412ae-0839-4ec2-8cbe-9a00.jpg
By yoyomaster - 8 Years Ago
It is sort of working here, but only head rotation works, I might install my Kinect and see if it does better, you really need clean and well lit video with a webcam, tried it last night in bad lighting and the video capture was very flickery, the Kinect might solve that!
https://forum.reallusion.com/uploads/images/a8e608f2-a44c-4808-b94b-2b49.jpg
By Ascensi - 8 Years Ago
Did the Kinect  work any better for you? Also I'm wondering if the eyes are tracked as well.
By Archie Andrews - 8 Years Ago
I dont get if this is a working commercial package or something that is redundant just to play with? Ideally I'm looking for a package that will take custom avatars and be able to animate the heads in realtime.  I notice it was developed in Unity which I use but can't find any contact or support info? Anyone any wiser? Thanks
By Archie Andrews - 8 Years Ago
I dont get if this is a working commercial package or something that is redundant just to play with? Ideally I'm looking for a package that will take custom avatars and be able to animate the heads in realtime.  I notice it was developed in Unity which I use but can't find any contact or support info? Anyone any wiser? Thanks
By Archie Andrews - 8 Years Ago
Ignore last posts found these pages will take a look. - http://f-clone.com/how-to-use-f-clone

By Kelleytoons - 8 Years Ago
Wait -- this product was created 4 YEARS AGO, and is called f-clone and doesn't work with iClone?  Hmmm, RL's lawyers ought to at the very least look into copyright name infringement (it's a bit too on the money).

And am I the only one who wonders why in heck, in four years, no one at that company (or elsewhere) tried to get it to work directly with iClone.  I can understand at least one reason -- the head morphs/bones/whatever might not have been extensive enough until around iClone 6, but they easily match anything needed now in 7 and CC 2.0.  

I guess one thing is for sure -- if the RL stuff comes in at an insane price tag, as some are insinuating, at least there might be a viable alternate (even if it requires more cleanup and a convoluted pipeline -- what, bring it into Maya with an exported CC head and then imported back in?  What other approach could be done to directly animate CC characters?)
By animagic - 8 Years Ago
We discussed some time ago, and I believe the consensus was the f-clone is not a viable alternative.
By freerange - 8 Years Ago
If you are looking for cheap face tracking you could try Brekel Pro Face, it uses a depth sensor camera like Faceshift did. So for those of you who still have your Kinect camera you can try it out, if it works for you it is only $79.

Faceware at full retail is about the same as doing a full neuron setup and the plugin so it isn't super crazy though definitely not hobbyist or toying around pricing. They are actually pretty cheap compared to a lot of their competition and is a very solid markerless system that can use just about any video source. Considering the current characters include the default Faceware blendshape set, plus a bunch more the time savings of having a push button solution might make up for any price difference. 

Intel Realsense cameras include face tracking and you can tie into the SDK to roll your own. 

There is a lot more going on to face tracking though than just feature tracking. They usually tie into a FACS or FACS type database and use machine learning algorithms for matching to the correct FACS pose (blendshape selection and strength of selection) then a retargeting system to apply that to your character. It is actually quite a bit more complicated than body tracking.

It is a real shame Faceshift is gone, it was pretty reasonable and worked great. Once Apple snatched it up it was impossible to get ahold of. Hao Li's new research looks promising though.