Piano Genie stands as a remarkable innovation in the realm of musical interfaces, offering users an intelligent controller that seamlessly maps 8-button input to a full 88-key piano in real-time. Unlike traditional musical interfaces, Piano Genie presents a unique opportunity for users to freely improvise and explore musical creativity, reminiscent of popular video games like Rock Band and Guitar Hero, but with the added freedom of improvisation rather than adherence to a fixed repertoire.
But how does Piano Genie achieve this remarkable feat? The key lies in its sophisticated architecture, which utilizes a discrete autoencoder framework trained on a dataset of existing piano performances. Here’s how it works:
- Mapping Sequences: Piano Genie establishes a 1-to-1 mapping between button presses and notes, granting users precise control over timing and polyphony. However, rather than employing a fixed mapping, Piano Genie learns a time-varying mapping through a bidirectional LSTM encoder and a unidirectional LSTM decoder.
- Training Process: During training, the bidirectional LSTM encoder maps sequences of piano notes to corresponding sequences of controller buttons, while a unidirectional LSTM decoder decodes these controller sequences back into piano performances. This process allows Piano Genie to learn the intricate relationship between button inputs and musical outputs.
- Encouraging Musical Contours: Piano Genie incentivizes the preservation of pitch contours in piano performances by penalizing the encoder for deviating from relative pitch ordering. This ensures that the generated performances closely mirror the contours of the button sequences, enhancing the musical expressiveness of the interface.
- Dataset and Style: Piano Genie is trained on a dataset of approximately 1400 virtuosic performances from the International Piano-e-Competition, resulting in generated performances that reflect the style of the training data. However, the model is open-sourced, allowing users to train Piano Genie on their own MIDI files and customize its output to suit their preferences.
In essence, Piano Genie represents a fusion of cutting-edge AI technology and musical innovation, offering users an intuitive and versatile platform for musical expression. Whether for novice musicians exploring their creativity or seasoned performers seeking new avenues of improvisation, Piano Genie opens up exciting possibilities in the world of music interfaces.
