I've always thought it would be cool to have gesture animation that is being controlled by speech, in addition to lip-synching.

Nvidia has been working on just that and introduced Audio2Gesture: https://www.nvidia.com/en-us/on-demand/session/omniverse2020-om1573/.

---