Hello. This system is called LSTM-512. It also has 512 units inside its RNN block.
Welcome to this repository which contains work being done on modelling Irish Folk Music using Deep Learning. The task of interest here is to apply language modelling techniques to a corpus of Irish Folk tunes from TheSession. The ABC Notation is an important musical notation tool used by various folk musicians around the world to annotate their tunes. So the goal of this little project is to apply language modelling techniques from deep learning on a corpus of Irish Folk Tunes transcribed in the ABC Music Notation Format.
Since the above task is that of music generation through text, it also requires thoughts for efficient ways to evaluate the learnt models. Thus, there are small attempts here at trying to evaluate a corpus of computer generated tunes against a smaller corpus of Irish Double Jigs.
I would like to mention that I am not an expert at Irish Music, so this is just me looking at the world of Irish Music from the outside, and trying to understand its structures using Deep Learning. Hence, any feedback is welcome. Thanks!
This project is heavily influenced by the work and ideas of Bob Sturm and the Folk-RNN project. Thanks for all your insights and research!
python3 -m venv venv
source venv/bin/activate
git clone https://github.com/richhiey1996/AI-Music-Generation-Challenge-2020
pip install --user --upgrade pip
pip install -r requirements.txt
After completing the setup, you will have to collect a processed set of tunes to create a Tensorflow Dataset. I have already processed some tunes that I extracted from theSession Data Dump as CSV, so you can find them here. Since we are trying to model the sequences of notes which make up music, extra metadata fields are irrelevant and are thus removed.
Once you have downloaded the above tune collection, you can create a Tensorflow Dataset to get started with the Deep Learning process. The root directory of this project contains Preprocessing.ipynb which holds instructions for converting a set of tunes into a TFRecord Dataset. The details involved with PreProcessing of the data can be found here. You will also have to configure the directory paths according to instructions mentioned in the Preprocessing notebook.
If everything works well, by the end of this, you end up with directory containing the tune vocabulary (for mapping musical symbols to numbers), tfrecord file (dataset of tunes for the deep learning models) and the tunes csv file (file downloaded during data collection
Processed TFRecord Dataset files can be found here.
Lets run some deep learning experiments now. Since we have a dataset of Irish Folk Tunes in a textual format, it is easy to feed data to various language models and see how well they are able to model the language of music. In order to do this, I provide the models with a basic vocabulary of musical words required to generate fluent ABC text. You can find the vocabulary used over here. It is automatically generated during the preprocessing stage.
First you will have to train a Model. Currently, there are two types of models implemented here - Transformers and RNN Models. There are default configs stored within the configs folder for both these models, but you can modify it to create your own models with little hassle. More information is provided in the notebooks. Training can be monitored by looking at your tensorboard dashboard.
After the training process comes to an end and checkpoints are saved, you can restore the model to generate new tunes from what it has already learnt. I generate 10000 tunes from the trained Folk LSTM like model. However, this is a general purpose learning system and the generation process can be tweaked by changing things around within the Model notebooks.
In context of the AI Music Generation Challenge 2020, the tunes generated by such models will be evaluated against a small corpus of Irish double jigs. So with that in mind, I tried simulating something similar.
I created two sets of tunes. One set contained the 365 double jigs. The second corpus contained 100 tunes generated by the model. After this, I converted tunes in both these sets to MIDI using ABC2MIDI. You can find a sample run of my evaluation here.
These MIDI files were then moved into Ableton Live and connected to an instrument. I felt that evaluation by listening on a small set of generated content would be a good first step to understand how well the model has trained. After this, I converted these tunes into sheet music to understand how well the visual patterns match across both sets of tunes. EasyABC makes it easy to do this. So I just feed it both the ABC files, and inspect the visual patterns in the dataset.
I did think of coming up with metrics and numbers for evaluating the models, but I couldn't imagine how such a metric would be useful in assessing the quality of a set of tunes. I do plan to implement edit distance matrix calculations between the set of generated tunes and the 365 double jigs. But again, I do not know how assessing the similarity of tunes could tell us much about the musical quality of a set of tunes.
I really enjoyed working with the ABC Music Notation, and Irish Folk Music in general. I love how the music is constantly moving, but it can still make you feel at ease. I also learnt that though none of these methods are novel or state-of-the-art in any way, this task can be used as a tool for learning about the possibilites that lie between the worlds of Folk Music (or music in general) and Deep Learning.