This is a submission for the Google AI Studio Multimodal Challenge
What I Built
Piano Hero is an innovative AI - powered keyboard piano tool. You can enter a song title or describe a melody, and its AI will generate the corresponding piano key sequence. It can either play automatically on the virtual keyboard or prompt users to press the corresponding keys. Additionally, it supports exporting and importing data in JSON file format, making it convenient for users to save and share their musical creations or imported content.
Demo
https://piano-hero-334600858897.us-west1.run.app/
How I Used Google AI Studio
I utilized Google AI Studio to create Piano Hero, leveraging its multimodal capabilities throughout the process.
Initially, I simply prompted Google AI Studio to generate a piece of music based on a song title and prompt users to press the corresponding keys. Back then, the virtual keyboard was quite rudimentary, and the generated songs were short, only having a dozen or twenty keys.
Subsequently, I started prompting Google AI Studio to improve aspects like the number and appearance of keyboard keys, as well as the duration of generated songs. When necessary, I provided reference piano images, taking advantage of the multimodal ability to process visual information.
Later, I wondered if Google AI Studio could directly generate a melody based on users' input prompts. I gave it a try, and to my delight, it could!
Finally, I introduced autoplay and import/export features with the help of further prompts to Google AI Studio, enhancing the functionality of Piano Hero.
Multimodal Features
Piano Hero offers a range of impressive features that deliver a great user experience.
Firstly, it supports users in generating melodies by entering prompts. This means users can simply type in a song title or describe a melody, and the AI will create a corresponding musical piece, allowing for creative freedom.
Secondly, it has an autoplay function. When a melody is generated, the virtual piano keys light up in sync with the music, guiding users to press the right keys if they want to play along. This visual cue makes learning and playing music intuitive and engaging.
Thirdly, it features import and export capabilities based on JSON files. Users can save their created melodies as JSON files and share them, or import JSON files to play melodies made by others. This enhances the reusability and sharing of musical creations, making the experience more versatile and collaborative.
All these features work together to provide users with a seamless, creative, and interactive piano - playing experience.
Top comments (0)