If your hat is Attached to the head, then Detach it and Link back to the head instead.
Then on at the timeline when it's time to lift it, link it to the hand, raise the hand and lower it back. And then again link the hat to the head.
I have this old, improvised tutorial to hide the hair under the hat and link it. You may start watching from 3:55:
https://youtu.be/corIwEyFqU4?si=dkW5LY01OVE2G87k&t=235