Kinect Mocap Software -- my take on these three


https://forum.reallusion.com/Topic260329.aspx
Print Topic | Close Window

By Kelleytoons - 9 Years Ago
I've just spent the last two days playing around with three Kinect V2 mocap software products and I thought I'd share my opinion of them.  Sorry for no examples -- the stuff I thought was crap I erased before I thought about it.

1) iClone's own mocap module.  Obviously the most integrated, it has a couple of features the other two lack, like the ability to *only* record upper body animation, and the ability to record direct on the iClone timeline (so you can play back an audio, for example, and sync to it).  All my tests were pretty piss poor, though -- it suffered (even at the smoothest setting level) from a ton of jitters and never looked very convincing.  It looked, to me, like what I expected from mocap stuff about a decade ago.  I will not be using it.

2) Brekel's Pro Body 2.  This guy, who has written a TON of mocap software (covering everything from face to body to hands) doesn't have a demo that lasts longer than 4 seconds, which made it difficult to fully test, but after doing dozens of recordings I think I pretty well have a grip on it.  For upper body animation it does a very good job: not great, but very good, better than iClone's.  Lower body is a hot mess, however, sliding feet putting it even (slightly) worse than iClone's offering.  It's also more expensive than iClone's.  I will not be using it (although I may use his Leap motion finger capture software, which works well).

3) iPisoft's Mocap Studio 3 - By far the best I tested, this software produced results that looked as good as any mocap suit stuff I've seen, as long as you work within its limitations.  While you *can* use more than one Kinect with it (and thus eliminate those limitations) for now I'm choosing not to do so, as my mocap needs are modest (I have so many mocap files there isn't any action or major movements I really need -- what I want out of mocap is the subtle things that this does *extremely* well).  Perhaps the biggest drawback is it's not Real time, and the processing involved is a bit tedious (not horrible, but certainly a certain amount of work).  For those efforts, though, you are rewarded with pristine files that are truly indistinguisable from even the PN suit stuff I've seen.

While it's a subscription model, at less than $100 it's not horrible.  A year from now I may well be buying a $500 mocap suit, so this will definitely serve as a good stopgap measure in the meantime.  I have to admit I was surprised to find this one was the one I wanted to use, as I was sure it was going to be Brekel's (until I had more time to work with it).  

There are free trials of both of the last two, and in the case of the iPi Studio it's fairly unrestricted for 30 days, so there's really no excuse for anyone with a Kinect V2 not to at least try it.  I think you'll be pleasantly surprised (particularly if you've been using RL's plugin).
By animagic - 9 Years Ago
Thanks, for doing this, it was very helpful.

I'm sorry to hear that the iClone offering is still disappointing, but it's good to know that even the $100/year offering from IPI is usable. My needs seem to be similar to yours in that I need something to
spruce up various gestures that go with dialog.
By toonarama - 9 Years Ago
Thanks Kelleytoons - that was very informative and interesting and - like you - I was surprised not only by what you thought was best, but also that you thought it was better than PN.

I have similar MOCAP needs in that I am after subtle movements for animating dialogue rather than anything else. So many of the demonstrations of MOCAP software have people flying around the place which is clearly not practical or sensible at my age and with my available space! 
By Kelleytoons - 9 Years Ago
If I implied that ipi was better than PN I did not mean to do so -- I mean the movements are every bit as good, within the limitations of the unit.  Which is to say subtle body movements that look like real people.

The advantage over PN is that there is no wearing of any rig, nor any setup process -- you simply press record.  But you also have to process the captured video -- it's a two step process, in that you first record the video and then you process it.  While the processing is not onerous, once you figure it out (as usual, I was pretty stupid in assuming I knew how it worked, although the instructions weren't all that clear) it certainly is not real time, which could be a factor for many.

As to that latter point, though, I will say that after doing a few days of tests of the various systems I found as a one-man shop that trying to watch what I was doing in a monitor was detrimental to any performance I was giving.  I don't actually think the real time aspects of mocap are that significant in that regard (and, let's face it, the best performers in the world, like Andy Serkis, certainly don't watch themselves as they do their mocap).  I'm not saying I could never see an advantage in this -- interacting with some environmental objects might be important -- but by and large it is not any kind of factor for me.

Let me also point out that with the iPi you could certainly have body parts pass in front of each other with no issues -- it recognizes arms and legs quite well.  With the one Kinect setup it *will* have some issues if you put your arms behind your back, or try turning around, although you can even get away with some of these moves if you are quick or patient enough for the post processing involved to fix the problems.  

Finally, I would be remiss if I didn't say that after thinking it over late last night I am leaning towards getting the two Kinect solution.  My own position may be rather unique in this regard but let me explain it fully in case it is of any help at all.  I think the one sensor version is as good as one sensor can possibly be and, as I said, as long as you live within those limitations (no arms behind the body, no turning around) you will have as excellent a no-finger mocap solution as you can possibly get.  It's also extremely cost effective.

I also firmly believe that a year from now the mocap landscape will be significantly different.  I can see the handwriting on the wall -- if we don't have under $500 mocap suits and/or Kinect type solutions that do precise capture of BOTH the face and body (iPi hints they are working on face stuff now) I will be amazed.  That's one reason I'm not going to plunge into a PN rig at the moment -- I think it's something folks are going to be kicking themselves very soon.

I have a rather significant birthday in January, though, and at my age I don't have much time left in life to wait for prices to drop and things to improve.  I want to be able to do things like turns and arms behind the body or at least not worry if I do want them.  Plus there's that birthday and a sort of "reward yourself for living this long" mentality.  Since I already have one Kinect V2, and since I can get another (plus the cable) for around $150 on Amazon, and finally, since I'd be paying about $100 anyway for the one version, I think I'll pony up the extra $250 and go ahead and get the two sensor version.  The examples for two sensor are VERY good -- PN is not better in any way other than finger capture (which I can do with my Leap if I need it -- my tests so far reveal that finger capture isn't nearly as important as I thought it might be).   My concern about the subscription model kind of vanishes then because, a year from now, I think the landscape will be very different and if it's not I can just reup (my hunch is, to remain competitive, iPi will have to lower prices.  Or I'll get something cheaper and sell my Kinects on eBay :>Wink

The other concern I had about the two sensor version is you need two computers -- not a huge issue for most (certainly not for me -- I have more computers around the house than I know what to do with) and the mechanism to record from both at the same time looks straightforward.  But that's what I'll be testing next.  If anyone is interested in THAT process I'll be glad to report on it.

(Oh, and both the iPi and Brekel's software produce Human/IK files that import really well into XChange and map perfectly with the Maya model -- sorry if I didn't mention that earlier but that was a big factor to me.  So does Brekel's Leap finger capture software but I might post separately about that sometime if anyone's interested).
By animagic - 9 Years Ago
I certainly will look forward to your reports. I'm a youngster in comparison to you, but money is a bit tight at the moment, so I think field experiences will be very valuable to help me decide what to get.
By Rampa - 9 Years Ago
I'm really interested in hearing your Leap Motion hand capture experiences. Hands are almost as important as face for me.
By pmaina - 9 Years Ago
I share your views Kelleytoons. Thanks for the excellent review!

Perhaps I might add that it looks like its possible to cut down costs even further and get better results(!) if you do away with KINECT and get the cheap Sony PS Eye cameras instead.
By Kelleytoons - 9 Years Ago
You should read all about the Sony cameras -- at first blush they look like a great solution, but there are (as always) some tradeoffs to consider.

Also, for those of you who are on the fence about the Kinect V2, Microsoft has them for $99 right now (until the end of the year, I think) so no better time to jump in and get one (I *almost* pulled the trigger on a refurbished one from Amazon for $89, which would have been a Bad Mistake).  Some online retailers have reduced their prices to match, but most are still offering them at a far higher price so buying direct from MS isn't a Bad Idea.

And, yes, I'll be reporting on all this (I think until the 3D face thingee is released in a month my focus will be on motion capture of various sorts).
By Rampa - 9 Years Ago
I was just up at the local Best Buy, and Kinect 2's were $99.00 there as well. No PC adapters there though.

I know that Microsoft did develop hand tracking for the Kinect 2, but never made it a product.

By Kelleytoons - 9 Years Ago
That's great about the Best Buys -- I would have bought there if I had known (I have to wait a week for them from MS -- oh well...)  But MS has the adapters too, so perhaps a wash.

I'll do some more stuff with the Leap hand mocap and I'll post some videos about workflow.  I really think it's a great solution in terms of ease of use and accuracy -- while it's a bit of a PITA because it's a separate capture process, that also has some advantages (in being able to separate hands from body motions).  What I haven't quite figured out is if it's possible to separate right from left, because right now it doesn't appear so (so you need to duplicate your hands if you want, for example, someone giving the finger and doing it on both sides and saving it out separately).
By pmaina - 9 Years Ago
rampa (11/11/2015)

I know that Microsoft did develop hand tracking for the Kinect 2, but never made it a product.

Amazing!