Profile Picture

First WIP Faceware

Posted By mtakerkart 7 Years Ago
Rated 5 stars based on 1 vote.
Author
Message
mtakerkart
mtakerkart
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)

Group: Forum Members
Last Active: 13 hours ago
Posts: 3.1K, Visits: 28.2K
My first faceware WIP.

Connecting Faceware to Iclone is very very easy. Very stable.

Using Faceware is for me very very tricky for now.
I used a c270 logitech camera so only 30 fps and 640x480
Loooooot of trouble to have a descent tracking of my ..... MOUTH!!!
lost every time ,always djerky even when I was stanstill, lot of failed calibration. 
Because my lips are thin I must use red lipstick because blue doesn't work at all. 
It is CRUCIAL to have perfect lighting , not good , perfect with poor webcam.
I really doesn't recommand to use low webcam.
The algorythm of the facial mocap make confusion with opening mouth and moving jaw.
When I open my mouth without open jaw , my avatar open the jaw anyway.
When I smile  with open mouth , no open jaw and my eyes open , my avatar smile with open jaw and close
eyelids. 
The only way to avoid this I found, is to desabled the Jaw move whith the faceware
mocap then let the audio viseme drive the Jaw after .
It is crucial also to not have your camera too down because faceware will do
your eyelids closing.3Dtest mentioned it but it's seems to be very precise and it depend of my 
morphologie , I have falling eyelid in a natural way. forcing me to watch up helps.

You really need a solid grip of your headcam to avoid shaking moves but having
a solid grip on your forhead make your brows moves difficult to perform Wink
I use a climbing helmet. 

I didn't understand why there is 2 settings with the grid guide regarding the 3dtest 
tutorial , it's very confusing.


https://forum.reallusion.com/uploads/images/806b2724-5b7e-4cc9-9d9d-93ca.jpg



https://forum.reallusion.com/uploads/images/f5243cbd-d530-4169-92de-e67e.jpg


The result is completely different dependant the character's face with the same mocap. 
Some faces always open the mouth ,some face completely frown the brows...

I also used a PS3 Eye camera to have a 60 fps , same messy tracking result 
and faceware never handle the 60 fps image sequence.
It plays the sequence at the speed x2 when 60fps is selected....

So I don't have choice to use a better camera. Faceware use Gopro  and not webcam  so what about this one??

 All of the following videos is the exact same animation apply to different character. 











Kelleytoons
Kelleytoons
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 9.2K, Visits: 21.8K
While the face may not be tracking all that well (although, given my command of French -- high school from nearly a half century ago -- I can't really tell much) but what sells this is what I was afraid of -- your body capture is really good.

It's what I'm struggling with now.  I have much better face capture (I can get 60fps with no issues and it works perfectly in iClone) but because I don't have a suit I have to capture body separately, and I haven't found a good workflow for that.  I suppose a helmet cam *could* be synced up (by importing the sequence into FW using a virtual webcam) but I'd still need a sync point (the traditional way of a keyboard "clap" might work IF I could sync the audio up in iClone -- I'm not so sure about this).  I'm pretty sure I can't run my iPisoft mocap at the same time I run FW -- that would be stressing even the highest end machine.

Anyway, it looks good, and if you get better FW keying you'll be home free.  Best of luck to you.



Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley
Skuzzlebutt
Skuzzlebutt
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)Distinguished Member (4.3K reputation)

Group: Forum Members
Last Active: Last Year
Posts: 580, Visits: 8.6K
to be honest, i see jerky eyebrow movement and nothing beyond normal lipsync here
you could have just face keyed this and got the same result if not better,
this is no reflection on you or your work, but dropping cash on this doesn't seem to be a good idea right now,
perhaps when the curve editor is out and they force you to buy popvideo to use it.
and... a week later nothing even remotely impressive posted by anyone ........



.
Edited
7 Years Ago by nabbler
mtakerkart
mtakerkart
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)

Group: Forum Members
Last Active: 13 hours ago
Posts: 3.1K, Visits: 28.2K
Thank you Kelly . Yes today I know how to use Neuron that's why I'm confident with Faceware . I clearly have a camera probleme.
My pipeline is this homemade suit :

https://forum.reallusion.com/uploads/images/4e81442b-40f9-4319-b46d-6de7.jpg


I open at the same time Audacity (audio) , Axis Neuron (body) , Faceware and Iclone.
I start Audacity , when starting Axis it make a BIP wich is recording in the audio file then when started the faceware record I said a Bip recording in the audio too.
Then in Iclone it's easy to synchronise wich each bip . One for mocap , one for faceware. Then I import the audio speach.

https://forum.reallusion.com/uploads/images/4819ce43-1c02-4a87-81f4-0080.jpg
Edited
7 Years Ago by mtakerkart
Kelleytoons
Kelleytoons
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 9.2K, Visits: 21.8K
Yeah, I would have to do something along those lines.  I'll have to see what shakes out -- part of my problem is a lot of my talent will not be able to mocap, only do the facial video (which I'll then mocap).  So I will have a disconnect between face and body anyway.  I just need to see what works -- perhaps miming along with the facial audio (and you French are known for mimes, right? <g>Wink.



Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley
mtakerkart
mtakerkart
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)

Group: Forum Members
Last Active: 13 hours ago
Posts: 3.1K, Visits: 28.2K
and you French are known for mimes, right?


No joke ! thrat's true!! That why Faceware can't handle my face!!!! Every person I meet (no joke) said that I'm talking and moving like "Louis de Funes"!!
And I want to use this fact for my story.... Tongue
justaviking
justaviking
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (20.4K reputation)Distinguished Member (20.4K reputation)Distinguished Member (20.4K reputation)Distinguished Member (20.4K reputation)Distinguished Member (20.4K reputation)Distinguished Member (20.4K reputation)Distinguished Member (20.4K reputation)Distinguished Member (20.4K reputation)Distinguished Member (20.4K reputation)

Group: Forum Members
Last Active: 3 Weeks Ago
Posts: 8.2K, Visits: 26.5K
a) Thank you for sharing your results, including struggles and discoveries.

b) I love the picture of you in your mocap suit.  The new splash screen for iClone should remind everyone to close their curtains before doing any facial or full-body motion capture.  I am sure our neighbors already think we are strange, and if they see us creating our next dance or fight video, they might call the police or the insane asylum to come and pick us up.  We have a strange hobby, no?



iClone 7... Character Creator... Substance Designer/Painter... Blender... Audacity...
Desktop (homebuilt) - Windows 10, Ryzen 9 3900x CPU, GTX 1080 GPU (8GB), 32GB RAM, Asus X570 Pro motherboard, 2TB SSD, terabytes of disk space, dual  monitors.
Laptop - Windows 10, MSI GS63VR STEALTH-252, 16GB RAM, GTX 1060 (6GB), 256GB SSD and 1TB HDD

illusionLAB
illusionLAB
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (3.9K reputation)Distinguished Member (3.9K reputation)Distinguished Member (3.9K reputation)Distinguished Member (3.9K reputation)Distinguished Member (3.9K reputation)Distinguished Member (3.9K reputation)Distinguished Member (3.9K reputation)Distinguished Member (3.9K reputation)Distinguished Member (3.9K reputation)

Group: Forum Members
Last Active: 2 Years Ago
Posts: 393, Visits: 4.8K
Thanks for being the one to 'finally' post an actual RL FaceWare capture - I'm actually surprised that we haven't seen dozens already as it's been available over a week!  I'd steer clear of the 'knock off' camera on Amazon... that sort of product only sells because it 'looks good' at a quarter of the price.  I imagine beyond the lighting, you're probably getting some serious 'jello effect' from your webcam - they are optimized for static capture, so rolling shutter is the least of their priorities.  As always, being at the 'bleeding edge' of technology means you're going to spend 'a little more' than you budgeted for ;-)  If I was making a face capture 'head rig' I'd opt for the GoPro Hero4 Session... it's smaller and lighter than a regular GoPro, costs $200cdn, and shoots HD at 60fps - so, more than enough for what you're intending... and you'll always get 60fps regardless if the lighting drops.

Oh, and power to you for posting a pic of yourself wearing lipstick!  ;-)  I can't tell you how many compromising videos I've made in pursuit of my art.
mtakerkart
mtakerkart
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)Distinguished Member (15.7K reputation)

Group: Forum Members
Last Active: 13 hours ago
Posts: 3.1K, Visits: 28.2K
We have a strange hobby, no?

Oh, and power to you for posting a pic of yourself wearing lipstick!  ;-)


Hahaha! Thank you guys!! Yes my chidren said at home, it is dad who makes up makeup.....

Thank you IllusionLAB , yep I think I'll opte for a Gopro.

Kelleytoons
Kelleytoons
Posted 7 Years Ago
View Quick Profile
Distinguished Member

Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)Distinguished Member (35.7K reputation)

Group: Forum Members
Last Active: Yesterday
Posts: 9.2K, Visits: 21.8K
illusionLAB (10/2/2017)
Thanks for being the one to 'finally' post an actual RL FaceWare capture - I'm actually surprised that we haven't seen dozens already as it's been available over a week!  


But if you note what I said at the beginning, this is exactly why I haven't (can't speak for anyone else).  Facial mocap on its own is, well, uncompelling.  It's only part of the story, and frankly you can see the demos that show just this.  What is needed is to combine it all, and for those of us without mocap suits it usually means a lot more effort.

I was JUST able to get the final piece of hardware for my own efforts, so tomorrow I'll be fooling around with the full range and should be able to start posting.  But, again, as I said, it's not just about talking heads (talking heads is the most boring kind of *anything* -- television news is all about talking heads).





Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley



Reading This Topic