Skip to content

Latest commit

 

History

History
 
 

ai-duet

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

A.I. Duet

A piano that responds to you.

About

This experiment lets you make music through machine learning. A neural network was trained on many MIDI examples and it learned about musical concepts, building a map of notes and timings. You just play a few notes, and see how the neural net responds.

https://aiexperiments.withgoogle.com/ai-duet

This is not an official Google product.

CREDITS

Built by Yotam Mann with friends on the Magenta and Creative Lab teams at Google. It uses TensorFlow, Tone.js and open-source tools from the Magenta project. Check out more at A.I. Experiments.

OVERVIEW

A.I. Duet is composed of two parts, the front-end which is in the static folder and the back-end which is in the server folder. The front-end client creates short MIDI files using the players's input which is sent to a Flask server. The server takes that MIDI input and "continues" it using Magenta and TensorFlow which is then returned back to the client.

INSTALLATION

Install Flask, Magenta and TensorFlow. Also download the attention_rnn bundle and put it in the server folder.

Then run the server from the command line:

cd server
python server.py

Then to build and install the front-end Javascript code, first make sure you have Node.js 6 or above and webpack installed. And then install of the dependencies of the project and build the code by typing the following in the terminal:

cd static
npm install
webpack -p

You can now play with A.I. Duet at localhost:8080.

MIDI SUPPORT

The A.I. Duet supports MIDI keyboard input using Web Midi API and the WebMIDI library.

PIANO KEYBOARD

The piano can also be controlled from your computer keyboard thanks to Audiokeys. The center row of the keyboard is the white keys.

AUDIO SAMPLES

Multisampled piano from Salamander Grand Piano V3 by Alexander Holm (Creative Commons Attribution 3.0).

String sounds from MIDI.js Soundfonts generated from FluidR3_GM.sf2 (Creative Commons Attribution 3.0).