I have been playing with the motion capture, and although my first experiments produced rather disappointing results, practice and experimentation, as well as laying down animations in multiple layers for different body parts has improved dramatically the results I have received. This practice has forced me to face a few limitations of the device and software, but also given me a list of relatively minor improvements that would lead to a substantially more usable product. In no particular order:

1. Time delay before recording. One of the things I love about iClone is the fact that I, as an individual, can do things that just years ago would take a team of people to accomplish. Working by myself, however, leaves me oftentimes with nobody to press the space key for me to begin recording. As a result, every captured animation begins with my avatar reaching over to press the space key. While you can cut this out of the animation, it would be substantially easier to just have a countdown timer that would allow you time to position yourself/your avatar before the recording starts.

2. Smoothing. The realtime nature of the capture and the limitations of the capture device/software tends to make captured motions extremely "twitchy" compared to manually animated motions. It would be really nice to have some sort of smoothing filter that could be applied to captured animations after the capture to provide for variable smoothing of the actions to make the motions appear less spastic.

3. Limb calibration: Maybe it's just my body shape/proportions, but I find that when I capture myself with my arms straight at my sides, my avatar's arms protrude at about 30 degrees from the torso. While this can be manually corrected by editing the motion layer, it makes capture a bit of a challenge. Just like the driver captures an H pose, it should be possible to have a character calibration, where your character appears on the screen, and presents several different poses for you to mimic, and the captured information could be used to adjust your animation prior to capture so that the avatar's 1) arms go straight down/up when your arms are straight down/up, that your avatars fingers touch when you touch your fingers together, that your legs come together when you stand with your feet together, etc. This simple act of calibration would make interaction with a virtual environment much more intuitive, and would eliminate a great deal of motion layer editing.

4. Time dilation. Sometimes there is a complicated motion that needs to be captured (for example, some dance steps), that the sad human operator (i.e. me) is incapable of adequately performing in realtime. A feature that would allow the capture to be made in "slow motion" and then played realtime would be useful for capturing complicated or precise motions.


The mocap wishlist is endless, but these seemed like four improvements that could be made relatively easily.

NOT easy to implement, but would be awesome:

5. Multiple camera/Kinect support. The ability to simultaneously use two or three kinects or cameras to create more accurate 360 degree motion capture

6. Gesture track capture. A hand/gesture capture made using closeup views of the hands would substantially speed the animation process where hands play a large part of the animation, and would even open the doors for using the product for producing sign-language animations (although two captures would need to be made, one for the arms, and the other to lay the gestures down.

7. Wheelchair support. Some people cannot enter the H pose. It would be really nifty if an alternate capture mode could be created to calibrate and track only the upper body. Leg animation could be captured in a second pass using arms.

I've been using iClone since version 2, and I've had more fun playing during the last couple of weeks with Version 5 and the motion capture plugin than I had with any of the then-new features of the prior versions. I hope that some/all of these features make their way into the motion capture product soon.

You want dual Kinects? OK

General Picture - animating now to life itself(tm)
Member of Content Wizards

Actually, the multiple camera idea was inspired after having played with the IPi Soft capture software. I didn't know that they now supported multiple Kinects because the tech specs say only one, so I appreciate the link. The trick that I would like to see iClone develop is either a similar ability, or the ability to link into a solution like this in realtime as opposed to going through the BVH format. When working properly, realtime animation in iClone makes it much easier to do things like navigating around virtual props, or interacting items using the physics engine that would require a lot of additional planning if capturing to BVH and then importing.

Edited to add the following for the benefit of those reading this thread following the mass deletion of posts, to add what I gained from the exchange:

Following the post below this one, a very long and sometimes contentious discussion of the iPiSoft markerless motion capture system took place. Demands for proof that it worked were made by some, peaklevels, whose banner used to be about a 1/3 page ad for the iPiSoft product supporting 2 Kinects that linked to his sales page participated extensively. Peaklevels stated he does not use iClone, however provided a number of motions and examples to show that the iPiSoft product will integrate rather nicely into the iClone production pipeline through BVH export. A number of back and forth exchanges took place, occasionally due to the misunderstanding of the parties as to what would constitute "proof" that the product worked as advertised, as well as to its $595 price tag.

What I gained from this discussion was that the iPiSoft product is not directly comparable to the realtime mocap plugin because it is not realtime. The best thing about the realtime plugin is that, if it were improved a bit, would provide a very quick and easy way to do animations, but even then would not be able to achieve the same level of quality of a multi-camera non-realtime markerless motion capture system (the non-realtime system can expend a huge number of CPU/GPU cycles crunching the input data, a luxury realtime does not have). I believe the realtime plugin, if it worked a little better, would be ideal to do quick and dirty animations, realtime demonstrations, scene blocking/testing, and for introducing kids, friends, and family to animation . To do high-quality, close-to-professional quality motion capture, especially of more complex motions requires an aftermarket solution.

Thank you CallNeg for the suggestions. We appreciate you taking the time to share your views on the Mocap Device Plugin. Your suggestions have now been passed to the iClone development team.

*PLEASE NOTE* All the posts that followed this have been removed. This was done for the following reasons.

1) They were completely off-topic. This is the Wishful Features forum.

2) They descended into arguments and insults including the use of profanity which is against forum rules.

                                                                

Peter
Forum Administrator

www.reallusion.com

Actually, they were likely removed because they emphasized a non Reallusion product (iPi) that is superior to the Mocap product sold by Reallusion.

As I wrote in one of those posts that was deleted, but perhaps could have been moved, the mentioning of other products is healthy for everyone. It provides the necessary wake up call to Reallusion to make their product usable and perhaps even better than the competitor.

Honestly Peter, the posts that included some insults were in the minority and THOSE were the only posts that should have been removed. The other posts you felt were off topic could have been moved to another category. I put time into writing my posts, and I know other people do too. Our feedback and free testing is what makes your products stronger.

Thank you Peter.

Martin Munthe
Filmmaker
http://munthe.biz
http://youtube.com/watch?v=2ntlOSGrRrI

I still like Peter, I don't think he's bad at all. I don't agree with some of his decisions, and I certainly don't mind openly sharing these opinions. I also know some things are likely out of his hands...things he is told from above.

You sound frustrated Peaklevels, and I can understand that. Keep in mind that there are Reallusion users who are interested in your product, but we also like iClone. We won't jump ship on iClone too easily, so please be careful about putting Reallusion down across the board.

And to be honest, your new graphic is much slicker and better designed than the large one. IMHO. Cheers.

Peaklevels, your last post is either sarcastic, or very confusing.

Also, my last post was responding to the original post of this thread...a list of what would make iClone mocap better. My list shared similar features to iPi, sure, and that just means these should be standard features in any mocap system.

However, I'm not sure iPi records head tilts and hand tilts off the human's motion...these should also be standard features regardless of the mocap company :)

I'm not going to get into a discussion with peaklevels this time but I want to add a few notes on how to view the Kinect Plugin compared to other systems of motion capture out there.

The Kinect plugin rests on the OpenNI driver. This is an open source driver used by the Brekel software as well. The OpenNI driver reads the Kinect just like the Xbox does and provides the option to get a real time read of a person standing in front of the Kinect. What other systems - like the on from iPi Soft - does is to do an optical analysis track that is then post processed. It actually takes a while before you get to view your recording. There is absolutely no feedback during the actual recording.

Post processing of visual tracking provides a better capture. It can be likened to iClone using a real time game engine to achieve realtime renders - you either find that to be an advantage or you start screaming about how superior Renderman is to the iClone game engine - they shouldn't be compared. It's all a matter of priority in what your trying to achieve with your work flow. Using two Kinects brings more details to the track. Keep in mind that professional optical motion capture systems used on big blockbusters uses several hundreds of cameras. So you will never get that kind of detail using two Kinects. That being said - iPi is a really cool garage solution for the independent producer. The reason we wont have a two Kinect setup for realtime work for a while is that it will demand double the performance of our computer systems. And it's already quite intense work to work with the OpenNI on a regular desktop. Not all iClone customers use Tesla setups I would expect.

The downside to post processing systems is that you loose the real time feature of the Kinect plugin based on OpenNI. I work a lot with previz work with paying customers. They have come to expect realtime, draft quality renders of different camera setups and actor/dummy motions. iClone gives me this possibility. I can't do it with post process based systems. After all the previz work I can move on and use something better for the actual performance capture - like the Vicon system or the OptiTrack (or iPi if a two Kinect setup is sufficient for what I'm doing).

I also view the Kinect plugin as a really interesting experimental feature for no-budget film making and interactive stuff.

It's all about real time. That's why the flame war on what's a better system and then we're talking about a real time system vs a post processing system is so strange to me. Why compare them? They are two interesting systems and tools available for us to use for different purposes. It's a non argument and for some reason it seems to trigger a lot of emotions. I have no idea why.

Just my two cents.

Martin Munthe
Filmmaker
http://munthe.biz
http://youtube.com/watch?v=2ntlOSGrRrI

If you connect a virtual camera to an avatar and use the Kinect plugin on that avatar and add camera smoothing to the camera you actually get a good looking camera path. So the smoothing feature is there and ready for Reallusion to work with.

Fature request number one on my list, guys. Pretty please? :)

Martin Munthe
Filmmaker
http://munthe.biz
http://youtube.com/watch?v=2ntlOSGrRrI