[sdiy] AI-steered "copilot" piano controller

[cheater cheater]

Here's a guy pressing buttons on a controller that just has 8 large
arcade style buttons and that translates into chords and chromatic
progressions on a physical grand piano. Pressing the same button
multiple times does not play the same note again, instead it
progresses the notes according to what will fit the harmony. I haven't
seen a more involved demo yet that "is a full song" but it's obvious
that this stuff has interesting potential. Personally I'd love to see
what could be done with outputs that provide more than 10 fingers or
impossible-to-hit combinations of notes. At that point this new
synthesis method will stop trying to imitate and will come unto its
own as an endeavor that stands on its own merit.

Quote:

What’s behind the magic? “Piano Genie” is a discrete autoencoder
architecture that uses LSTM to map piano notes to low-res controller
buttons, then decode back to the piano space. It’s trained on 1400
virtuosic performances from the International Piano-e-Competition.

Video / thread:

https://twitter.com/DrJimFan/status/1625167688883646464

More videos:

https://www.youtube.com/playlist?list=PLBUMAYA6kvGVOmhAwLRP4i_L15D7AoWDJ