I think about motion capture with an iphone. One the one hand, the demo look very promising. However, what completely destroys the mocap is the noise and jitter that I see in all the videos I found so far. Is it possible to remove that noise and jitter in iclone?
Are there any videos that demonstrate this?

Good question I’d definitely like the answer to. In some of the videos using it, perhaps the person wasn’t capturing while in minimal mode? When I’ve captured still in High mode I’ve got a substantial amount of jitter. Plus I think it also relies on your internet connection. Kelly toons has managed to get it working via USB. I’m not sure exactly the method, but seems like that would be a faster way to transfer data? I use it frequently and sometimes it’s pretty good and sometimes I do wish I could smooth the capture.

My "NEWEST" Latest:)  Movie Made With Amaaaazing iClone


—> High Noon”ish” in Narrow Valley<—

Nowadays I capture using Ethernet but, honestly, I don't see the jitter you're referring to -- then again, everyone has a different eye and expects different things.  Can you point us towards a video where you see this jitter?  If I don't see it there, I can't help you further.

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley

I think it is quite obvious here:
https://youtu.be/89dW2LFR07I?t=18

As I am just a beginner, I can only guess what causes the problem. To me, it seems that the problem is mostly caused by noise, but sometimes it appears that the sampling rate is to low which creates jumps. If the sampling rate of the iphone is limited, the only solution is to move more slowly. 
However, the noise could be improved by iclone. Does it have some form of low-pass filter to smoothen the movements and remove the jitter?

Um, no -- about 98% of that video is just showing the live capture.  Has very little to do with the animation results (there is about a one second animation at the end that looks fine to me).

Show me an ANIMATION made with this that has the jitter (you can't compare the actual live capture -- that's like assuming iClone does real time rendering, which it clearly does not.  It takes at least three or four times as long -- sometimes MUCH longer -- to render something, at which point a lot of smoothing occurs).

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley

Thanks for pointing that out. My assumption was that this is the result, as I have not yet found any marketing video that shows the final output. 

So my point is that I cannot show you what I am asking for. All the demos I found were either jittery or movements were so overexaggerated that it was hard to make a judgment about the quality of the recording.

So far, I mostly used Character Creator. Now I get to the stage where I want do do animation for unreal. 

I’ve gotten jitter before but my internet does suck not sure if that’s what’s caused it. The last Time I recorded facial in hi mode (which was stupid) and I did get jitter. I’ve stopped doing that with good results, but the last trailer I did used a weird Shark character. I set the facial capture at about 110 strength overall and there was definite jitter. Was on a time deadline so didn’t really have to redo it and see if I could reduce the effect. I do toons so I crank up the strength. During the quarantine my WiFi has been awful. I’m guessing that has had an effect on my captures.

Mike do you have a tute for the setup? I’ll do a search on the forum.

@Mikay you’ll quickly find KT is an invaluable source of help on the forum. Lots of helpful members that don’t mind sharing their learnings here which is a welcome change from most forums.

My "NEWEST" Latest:)  Movie Made With Amaaaazing iClone


—> High Noon”ish” in Narrow Valley<—

I was asking to see examples because, honestly, everyone sees things differently.  You definitely will see live capture not as smooth as animation.

I'll post some examples of my own video -- if you see jitter in these I cannot help you, as I do not (and like what I've done).  I'm not saying these are the best you can do, but they are pretty representative of what can be achieved with Live Face (however -- be aware of one important thing.  Two of these weren't done "live" -- which is to say I lipsynced to the actual voice(s).  So the sync may be off on those a bit at times.  But the sync isn't the issue here, the smoothness of what can get captured is, and these demonstrate how something actually animates):

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley

Haha, "Never Too Late" is damn funny ^^


I know, I subscribed to his Youtube channel before I even bought iclone.

For me, this is related to the uncanny valley. People like the increase in realism until it gets fairly realistic but not realistic enough to nail it. Using cartoon characters is a way to avoid this. However, if you strive for realism, then it's getting more difficult. I give you an example of very smooth mocap:
https://www.youtube.com/watch?v=pySDflhRMs4

Apart from the animated wrinkle maps, there is obviously a difference in the capture quality. Yes, it is true, with 20% of the effort you get 80% of the outcome, but I am willing to go for more, and I just want to know what my options are.
I found this feature for lip sync and tested it: https://www.youtube.com/watch?v=cb3jvt4H2Rs
This vastly improves the lip sync. Is there a similar feature for smoothing motion capture data from the iphone?

Yeah, uncanny valley indeed.

For your example that you posted I would suggest that has nothing to do with the mocap per se, but the model itself.  It is FAR more detailed than an iClone/CC3 model.  I think THAT'S where the smoothness you are seeing comes in.  We are told that a new CC (and, I assume, iClone) model will be coming with improved facial features and that this will also give us better visemes and thus I suspect that will get us closer to that if that's what you want (not me but, as you say, some folks strive for "realism").  But, again, nothing to do with Live Face.

I suppose the way to prove this would be to take some Live Face animation and export it to, what, Unreal (was that the software used?) and somehow apply it to that higher detailed figure, but I doubt that's going to work (right now, for example, we can't use Daz HD avatars -- well, we CAN but they are scaled down in realism to the CC3 one.  My gut tells me the same thing would happen in reverse there, that any added detail in the avatar wouldn't translate the motion well, if at all.  But I know nothing about Unreal so perhaps someone else will chime in here.

So, again -- I don't think it's a limitation of Live Face -- that is easily just as smooth as the animation you show.   But if you want to achieve that level of detail in an avatar, iClone/CC3 ain't the thing for you (at least right now.  If you are patient then by this fall perhaps we'll know a lot more).

---

Alienware Aurora R16, Win 11, i9-149000KF, 3.20GHz CPU, 64GB RAM, RTX 4090 (24GB), Samsung 870 Pro 8TB, Gen3 MVNe M-2 SSD, 4TBx2, 39" Alienware Widescreen Monitor
Mike "ex-genius" Kelley