|
By 4u2ges - 2 Years Ago
|
And of course I made a typo in the Subject (why can't we edit the subject??)
This is a WIP of the project I am currently working on.
It is a mechanical piano with physics enabled keys which can be used for a "true" piano playing animations.
There are 2 aspects of piano playing simulations. Animating keys and animating fingers.
I had a choice to make a full 88 keys MIDI piano and then animate finger to follow the keys, or to make a piano with physics enabled keys and have keys to follow animated fingers.
Each method has advantages and disadvantages. I have chosen the latter.
The piano is practically ready. It has a keyboard comprised from 266 props including dummies and springs.
Piano keys are activated by tiny rigid spheres attached to each 3rd bone of player's finger. Here is a schematic including dummies.
Now, animating fingers is a different story. I tried playing in neuron suite gloves just to find out that it is difficult to say the least.
Taking wide voicing chords clean is virtually impossible. And then I can imagine what the animation clean up and syncing with keyboard task would take.
So I decided to animate manually. I recorded the piano playing and currently animating fingers/hands much like lip syncing.
Reach effectors for hands do the job for independently animating body parts without disrupting fingers animation but what makes it hard to achieve fluid hand/fingers animation, is lack of fingers pinning.
Hopefully iClone 8 would have a better pinning system in place.
So far just a render as I only animated a few measures. I will share the piano (for someone crazy enough like me  ) and an animated clip when it's ready.
And hopefully I'd have some tips to facilitate process of animation.
Any tips/comments are welcome.
|
|
By Amper Sand - 2 Years Ago
|
Jesus! 
|
|
By wires - 2 Years Ago
|
A work of art Beethoven.  
|
|
By Zeronimo - 2 Years Ago
|
Excellent,
I had thought for a moment to reproduce a video like this one with iclone, but in front of the difficulty I quickly gave up. But maybe with your plugin it will become possible.
|
|
By mark - 2 Years Ago
|
I'm all in Zero' if you can animate Chico!!!  Amazing work 4u2'!!!!
|
|
By toystorylab - 2 Years Ago
|
Wow...
I think the decision to go for a piano with physics enabled keys and have keys to follow animated fingers,
instead of a full 88 keys MIDI piano and then animate finger to follow the keys is a good one.
The midi version would probably have accurate piano keys according to music but animate fingers to follow keys is tricky.
Sure, to make this look real with the audio will be tricky too but is less important to me than the overall animation effect.
So I'm looking forward to try this out!
|
|
By JIX - 2 Years Ago
|
|
This is awesome!
I´m very interested, because I was thinking about a video clip featuring a piano player lately. Besides my GF is a pro pianist.
|
|
By Rampa - 2 Years Ago
|
Looks like this does a pretty good job. Says they will have 3D export and import in a later version.
https://www.concertcreator.ai
|
|
By Zeronimo - 2 Years Ago
|
Rampa (1/30/2022)
Looks like this does a pretty good job. Says they will have 3D export and import in a later version. https://www.concertcreator.ai
yes, but with a subscription $179.99 per year ... it becomes much less attractive.
|
|
By 4u2ges - 2 Years Ago
|
Thanks folks!
@Toystory – precisely what I though initially.
@Rampa Interesting collaboration between RL and those folks. $15 bucks month is not bad for serious projects, that is when they’d have an export version (with original CC characters I suppose). Otherwise…
I guess there are enough people willing to try it out. Thus, I will share a beta version of the piano project.
At least I am not going to be alone in my success (or failure)  . https://drive.google.com/file/d/17IqVFiuSkvFgKs7P2dbLifV-u9fk-v7k/view?usp=sharing
(renders are Iray as you might guess)
I downloaded original piano from 3D Warehouse. It was stripped/reworked in Blender to make it fully operational.
Aside from keyboard, I made most other parts functional like lids, fallboard, etc (still need to morph pedals).
Keyboard and keys measuring correspond approximately to the real mid-sized grand piano.
When morphing your piano player, make sure they have thin and long fingers for easy animation.
All keys have been properly named - just in case someone come up with some amazing script to automate animation (though I doubt it’s possible with current implementation of Python). AI is another story as it was pointed already.
I have included dummy spheres for player fingers inside the project.
To begin, animate your character for the corresponding pose, realign spheres with tip of the fingers as shown in screenshot and LINK each sphere to the corresponding 3rd finger bone. Then check dummy box for each.
Regarding animation I’ll share my experience. You may come up with your own ideas – do share.
Hands, feet and torso have reach effectors enabled (original limbs pinning is not reliable and cause slipping).
The only inconvenience is, that I have to switch back and force between Motion Layer Editor and Edit transform for reach effectors (it would be nice to have those under a single panel in IC8).
Most of the time I animate hand reach effectors for a music measure first and then animate fingers only.
Whenever I animate finger or two, a hallow key is created, I always solidify it by double clicking on a key to prevent blending issues down the road.
So, while animating fingers you have to literally align and stick them inside the keys.
At runtime the keys would obey physics and go down and up as you release the fingers from keys.
Try not to place fingers at the very edge of the keys, it might cause keys to fall out of place.
Rigid body tends to bake animation on the fly, so I always record the animation in By Frame mode (otherwise keys might slip out of place sometimes).
Each consecutive run would override the previous animation. If you need to clear animation from ALL keys, then run animation without character touching the keyboard.
And to test the real speed I disable Rigid Body in Project and run animation in Realtime (after running in By Frame mode first).
Although there is a 4 degrees spring limit, the keys might go down further.
So got to try not to stick fingers into the keys too deep (I am sill tweaking it).
Unfortunately, there is no way to tweak springs physics simultaneously (like for Rigid Body) by selecting all springs at once.
3 values could generally be tweaked for springs: Spring (strengths I suppose), Damping, and Range Rotation by X.
After a few measures I can now animate body parts: spine, head, elbows, legs and because of reach effectors, that will not disrupt previously animated fingers and hands position.
Good Luck!
|
|
By toystorylab - 2 Years Ago
|
Thanx for Beta!
As I'm very busy with commissioned work, not sure how soon I can test this...
|
|
By wires - 2 Years Ago
|
Hi Gurgen,
I've downloaded the Beta and I'll see what I can come up with. I don't know if the Hand Gestures plugin will be of any use, but time will tell. 
|
|
By rosuckmedia - 2 Years Ago
|
A great project👍👍
Greetings rosuckmedia
|
|
By Data Juggler - 2 Years Ago
|
That is cool! Now we just need an AI powered plug-in that listens to a song, and animates the appropriate hand motions.
That sounds easier than a self driving car.
I bought this one piano earlier, and it was real hard to work with.
Can you play the piano in real life? I can't play the guitar, but I read enough books as a kid on how to play the guitar, so I just guessed where fingers should be for notes.
|
|
By Data Juggler - 2 Years Ago
|
JIX (1/30/2022)
This is awesome!
I´m very interested, because I was thinking about a video clip featuring a piano player lately. Besides my GF is a pro pianist.
I knew my grandfather was a talented piano player as he played all his life, but I didn't find out until after he died, that at 7 years old he was a conductor in an orchestra and played 5 instruments. He was going to be a musician, but he married my grandmother, then worked for Exxon for 45 years. I found a perfect attendance plaque, stating he never missed a day of work in 45 years.
|
|
By Roundtree - 2 Years Ago
|
Definitely off to a good start. I thought about it at one time. But that's as far as I got. :-)
|
|
By Roundtree - 2 Years Ago
|
|
Awesome story about your grandfather!
|
|
By 4u2ges - 2 Years Ago
|
No rush Toystory, Gerry. I going slow myself. Not sure if Hand Gesture plugin would work. I mean it should, but then hands have to animated separately.
Or rather hands have to be animated first and then gestures. I suppose it might give some good impression of someone playing and hitting keys right.
Thanks rosuckmedia and Earl-1
@Data Juggler First, the story about your grandpa was indeed awesome Now, I am not sure about AI. I have looked at the link Rampa provided and I wish to hear a demo - not only see what they have there on main page.
I mean I see she hits the notes right and hand movement is fluid, but it's kind of basic. I want to see how they would play more complex stuff, like arpeggiate, playing chromatics, soloing, etc.
I do play piano and guitar a bit. Certainly not at the pro level, but enough to record something for animation. First 30 sec would come soon I hope
Again, if we only had an FK for fingers with option to pin them....
|
|
By 4u2ges - 2 Years Ago
|
Finally started some animation today. One more tip:
It is a lot easier to animate from the front. Just need to hide all parts of the piano and leave only keys visible:
|
|
By wires - 2 Years Ago
|
|
Good tip on hiding parts to get better views of the animation Gurgen.
|
|
By 4u2ges - 2 Years Ago
|
Thanks Gerry!
|
|
By 4u2ges - 2 Years Ago
|
So here is, two of my hobbies in one. Fusion Blues in F (Intro - first 25 seconds).
Performed on Roland FP-90. Rendered with Iray. (I am still a lousy piano player though  )
Got to admit, animating fingers and hands is a PITA.
Although I felt more comfortable at the end, when got a hang of it. Not 100% the way I'd like it to be though.
Still 1.5 min to go (probably a few more weekends). Stay tuned....
This is my timeline thus far:
BTW, physics keyboard could be wrapped into any other acoustic or electronic piano shell.
I will probably model some electronic keyboard with it for the marketplace.
|
|
By animagic - 2 Years Ago
|
Sounds good to me. My piano playing is so rusty I would have a hard time...
The animation is quite decent too. A piano teacher would probably say that you have your wrists too low...
One thought I had is whether it would make sense to save chords as gestures. That is chords where all notes are played at the same time. It would be different depending on the key, but it might save time.
|
|
By 4u2ges - 2 Years Ago
|
animagic (2/8/2022)
Sounds good to me. My piano playing is so rusty I would have a hard time... The animation is quite decent too. A piano teacher would probably say that you have your wrists too low... One thought I had is whether it would make sense to save chords as gestures. That is chords where all notes are played at the same time. It would be different depending on the key, but it might save time.
Thanks. Ha ha, yes, that is how I actually play. My wrists are always low. I got an excuse for the left hand though.
Those stretched chords (Gm7 - GFB-flat for instance) is not possible to take from the top for most people.
I could barely reach outside keys with tip of the fingers laying hand flat. 
Regarding Gestures for chords, I also came out with the idea close to the end, when I tired animating the same, or similar chords over and over.
But I almost never used gestures for animation. It has a separate track and I remember once I had some problem adding gesture and then animating fingers after that on motion track.
Need to do some more studying in regards to this matter.
|
|
By mark - 2 Years Ago
|
Super excellent!!!! VERY NICE!!!
|
|
By 4u2ges - 2 Years Ago
|
|
mark (2/8/2022)
Super excellent!!!! VERY NICE!!!
Thank you Mark
|
|
By toystorylab - 2 Years Ago
|
Cool... Quite impressive. And nice render!
I did first test but gave up after a short time. 
My hands are quite a mess...
|
|
By gordryd - 2 Years Ago
|
|
The piano playing animation is very convincing - kudos for all the work you put in.
|
|
By 4u2ges - 2 Years Ago
|
Thanks guys!
Sorry Pieter, got to have lots of patience with it
|
|
By 4u2ges - 2 Years Ago
|
I have now a Photo-realistic Digital Piano available at marketplace (modeled my own setup )
https://marketplace.reallusion.com/photorealistic-digital-piano-354574
|
|
By rosuckmedia - 2 Years Ago
|
Hi 4u2ges
Your works are great.👍👍👍👍
Greetings rosuckmedia
|
|
By toystorylab - 2 Years Ago
|
|
|
By 4u2ges - 2 Years Ago
|
Thanks rosuckmedia!
@toystorylab  👍
|
|
By Rampa - 2 Years Ago
|
So Concert Creator is temporarily free!
https://www.concertcreator.ai/download
|
|
By Data Juggler - 2 Years Ago
|
Have you seen this: I should have read to the end to see Rampa's post.
This look so cool.
|
|
By 4u2ges - 2 Years Ago
|
This is what Rampa has posted a while back. I have tried it since it's free for now.
Animation
I do not see the way to export out the animation/character and the piano.
All renders should be done within the app (at least for now).
Player (at least the girl which I have tried) has kind of short fingers. She probably has to play 5.5" octave piano (which hard to come by in real life).
9's and 10's are getting played, but her fingers do not reach the keys .
But overall it's pretty much impressive for the AI.
Sound
It sounds with too much ambience and sustain, like player does not remove a foot from a damper pedal.
It's not critical if you would replace audio track with real life playing, but for import from MIDI file generated audio track would be retained.
And I did not find any settings to control the sound.
So here I have uploaded an mp3 file (Georgia on my Mind - original interpretation by 7notemode) into Concert Creator.
It took a while to AI process it, but the result was pretty much OK. There are little hiccups here and there, arpeggios are messed up a bit and there is a desynchronization with the original payer.
I then rendered, removed generated audio track (I did not like it) and replaced it with the original mp3 played by 7notemode.
As I said, there are some asynchronicity, but again it's still pretty much good for fairly complex tune animation (specially that it does not come from MIDI but just from an audio file).
Now, if it only could be exported (I guess you'd have to pay for it).
|
|
By 4u2ges - 2 Years Ago
|
And this is me playing Fusion Blues in F (which I never had a chance to fully animate in iClone )
Again, I have replaced generated audio track with my own audio recording.
This time is a view from the top (small fingers length deficiencies can be seen here clearly).
|
|
By animatedbeing - 2 Years Ago
|
I made a test video from a classical midi file I found on the web. The program is a blast. I would love to be able to export the animations.
|
|
By deerob - 2 Years Ago
|
wowo wow wow!
PHENOMENAL work!
I would LOVE to be able to use the piano.
I am expanding in making my dance videos, with quick blips of "the band" playing the music.
Thanks for sharing!
Dan
|
|
By 4u2ges - 2 Years Ago
|
|
deerob (6/14/2022)
wowo wow wow!
PHENOMENAL work!
I would LOVE to be able to use the piano.
I am expanding in making my dance videos, with quick blips of "the band" playing the music.
Thanks for sharing!
Dan
Thanks, I have updated the free piano project with some of the piano panels converted to mirror reflective surfaces: https://drive.google.com/file/d/116RbUvGjGeUSGb4PFwLKfVNObxwp4F3A/view?usp=sharing
(as per: https://forum.reallusion.com/FindPost516437.aspx)
|
|
By ReiniWien - 2 Years Ago
|
Hi 4u2ges,
many many thx, great work, scenes will be more realistic at least 100 times.
I just want to ask, is it also ok to use it in commercial projects? The free piano version and the keyboard from the marketplace?
Thx again, best regards, ReiniWien
|
|
By 4u2ges - 2 Years Ago
|
|
ReiniWien (8/24/2022)
Hi 4u2ges,
many many thx, great work, scenes will be more realistic at least 100 times.
I just want to ask, is it also ok to use it in commercial projects? The free piano version and the keyboard from the marketplace?
Thx again, best regards, ReiniWien
Hi ReiniWien,
Keyboard purchased from marketplace can be used in commercial project freely as it is an extended license by default.
And the free one from this thread, it's OK to use as well.
Share your work outcome if possible and Good Luck!
|