NeuralPi

NeuralPi is a guitar pedal using neural networks to emulate real amps and pedals on a Raspberry Pi 4. The NeuralPi software is a VST3 plugin built with JUCE, which can be run as a normal audio plugin or cross-compiled to run on the Raspberry Pi 4 with Elk Audio OS. The NeuralPi includes model selection, EQ, and gain/volume controls from a remote instance of the plugin over WiFi. The pedal runs high quality amp/pedal models on an economical DIY setup, costing around $120 for hardware to build yourself.
Check out a video demo on YouTube
Check out the step by step build guide published on Towards Data Science

NeuralPi can sound like an amplifier or distortion/overdrive pedal using the power of neural networks. Models trained from recordings of real amps and pedals can be loaded into the plugin for endless possiblities on your guitar. Create your own models or use custom tones from GuitarML.

WARNING: The audio output of the HiFiBerry DAC + ADC is at line level. Guitar amplifiers expect a low level electric guitar signal (instrument level). Use NeuralPi only where line level inputs are expected.

There are four main components to the guitar pedal:

1. Raspberry Pi 4b
2. HiFiBerry DAC + ADC
3. Elk Audio OS
4. NeuralPi VST3 plugin

This is the normal plugin (v1.2.0), available for Windows (Standalone, VST3) and Mac (Standalone, AU, VST3). After connencting the Raspberry Pi and remote computer to the same local WiFi network, enter the RaspberryPi's IP address (keep the default ports) to enable control over WiFi. The Win/Mac plugins are fully functional guitar plugins that allow you to try out GuitarML's most advanced amp/pedal models without building the Raspberry Pi pedal.

Note: The plugin must be restarted after using the Import Tone button for changes to take effect.

Installing the plugin

See the Release page for the cross-compiled Raspberry Pi / Elk Audio OS compatible VST3 plugin and Win/Mac installers.

After running the plugin or standalone for the first time, the two default models will be copied to the following locations. Any imported models will be copied here as well. Model files must be manually removed from these locations to perform model clean-up.

Mac/Linux: /home/<username>/Documents/GuitarML/NeuralPi/tones 
Windows: C:/Users/<username>/Documents/GuitarML/NeuralPi/tones
Elk Audio OS: /home/mind/Documents/GuitarML/NeuralPi/tones

Adding New Models

Once your NeuralPi is set up, you can add new models from a remote computer using the following steps:

1. From the remote computer, run the plugin and add new models using the "Import Tone" button. Optionally, you can manually add new json files to the Documents/GuitarML/Chameleon/tones directory. Note: The "tones" directory is created the first time you run NeuralPi.
2. Turn on your WiFi enabled NeuralPi (see Elk documentation for connecting the Raspberry Pi to a local WiFi network)
3. Download the update_models.bat(Windows) or update_models.sh(Mac/Linux) to your remote computer. These scripts are located in the "scripts/" directory of this repository. You must change the rpi_ip_address and host_model_path to the Raspberry Pi's IP address and path to your json tones (on remote computer). The json files will be first copied from the remote computer to the NeuralPi, and then back from the NeuralPi to the remote computer. This allows updating models from the NeuralPi when you connect a new remote computer.
4. From the remote computer connected to the same local WiFi network as NeuralPi, run the update_models.bat(Windows) or update_models.sh(Mac/Linux) from a cmd terminal.
   
   Note: It is important that all models files have unique names with no spaces.
   Note: Ensure from the terminal output that you were able to connect over WiFi, and that the model files were copied properly.
   
   
5. Restart both the NeuralPi and the remote instance of the NeuralPi plugin. From the remote NeuralPi GUI, enter the Raspberry Pi's IP address. As long as both devices are connected to the local WiFi network, you will be able select models from the NeuralPi plugin dropdown list to change models running on the Raspberry Pi.

IMPORTANT: The plugin uses a sort() function to order the models alphabetically. Due to differences in the behaviour of this function on Linux (Elk OS) vs. Win/Mac, you must start json filenames with a capital letter, otherwise the NeuralPi on Elk will sort models starting with a lowercase letter at the end of the list and the controller will be out of sync with the NeuralPi pedal.

MIDI control of NeuralPi parameters

The “config_neuralpi_MIDI.json” file contains MIDI mapping of NeuralPi parameters.

The names of parameters are: "Gain", "Master", "Bass", "Mid", "Treble", "Presence", "Delay", "Reverb", "Model", "Ir". In that json file, you can see that those parameters have been asigned to incoming MIDI CC# messages "1", "2", "3", "4, "5", "6", "7", "8", "9" and "10" respectively. But editing the file allows you to chose whatever CC# to whatever parameter, by just changing values in the “cc_number” and “parameter_name” commmands.

Sushi will listen to incoming MIDI CC# messages, will normalize (0, 127) MIDI values range to (0, 1) Sushi range, and will set that value to correspondent parameter. For instance, if your MIDI controller sends a CC2 message with value "127", Sushi will receive that message and set "Master" parameter (“Master” is assigned to “CC2”) to be "1" (MIDI “127” value normalized to “1”).

You´ll need to copy the config file to the Raspberry, for instance through ssh over Wifi (login as root):

scp -r config_neuralpi_MIDI.json root@:/home/mind/config_files/

For connecting a MIDI device:

1 - Plug your MIDI device into any Raspberry USB port.

2 - Login as “mind” user, “elk” password, and run Sushi with the MIDI config:

sushi -r --multicore-processing=2 –c ~/config_files/config_neuralpi_MIDI.json &

3 – To list MIDI devices connected to the Raspberry, run:

aconnect –l

4 – You can now connect your MIDI device to Sushi either by their listed ports, or by their names. Run:

aconnect "your-listed-device-name" "Sushi"

NOTE 1: Currentlly, "Model" and "Ir" parameters are a little tricky to control. NeuralPi asigns a value to each file saved in "tones" or "Ir" directory. It divides the (0, 1) range of values by the number of files available, so for instance, if you had just 2 tone files in the directory, one of them would respond to any value in the (0, 0.49) range, and the other would respond to any value in the (0.5, 1) range. When a lot of tones/IR files are stored in the directories, it´s difficult to guess which value corresponds to which tone/IR... so for the moment, if you need to control tones or IRs with MIDI, you´d have to work a little to find out values, or just reduce the number of tones/IRs in your directories to make the task easier.

NOTE 2: When NeuralPi is started, it seeks for tones in the directory. If it doesn´t find the default ones (“BluesJR_FullD.json” and “TS9_FullD.json” tones), it will create them again. So if you had only 2 tones because yo wanted to control them easily by MIDI, and neither of them are the BluesJr nor TS9, bear in mind that at Sushi startup, suddenly BluesJr and TS9 files will be created again in the directory, so there´ll be 4 models/files stored and the values assigned for each one might have changed. If you want only 2 models, you can cheat a little and rename your desired model files as “BluesJR_FullD.json” and “TS9_FullD.json” (make sure the name is identical). This way, NeuralPi thinks that default models are already in the directory and doesn´t add any files.

To Do

Elk Audio OS also supports physical controls through Sensei. Gain/Volume and EQ knobs can be added, as well as a LCD screen for selecting different models. One could build an actual guitar pedal with NeuralPi and any number of other digital effects and controls.

While running PyTorch locally on the Raspberry Pi might be a stretch, it is fully capable of recording high quality audio with the HiFiBerry hat. Implement a capture feature by automating the recording of input/output samples, pushing to remote computer for training, then updating the Pi with the newly trained model.

Info

The neural network is a re-creation of the LSTM inference model from Real-Time Guitar Amplifier Emulation with Deep Learning

The Automated-GuitarAmpModelling project was used to train the .json models.
GuitarML maintains a fork with a few extra helpful features, including a Colab training script. IMPORTANT: When training models for NeuralPi, ensure that a LSTM size of 20 is used. NeuralPi is optimized to run models of this size, and other sizes are not currently compatible.

The plugin uses RTNeural, which is a highly optimized neural net inference engine intended for audio applications.

The HiFiBerry DAC+ADC card used for this project provides 192kHz/24bit Analog-to-Digital and Digital to Analog, which is industry standard for high quality audio devices. The plugin processes at 44.1kHz (specified in config file) for the neural net DSP.

Build Instructions

To build the plugin for use on the Raspberry Pi with Elk Audio OS, see the official Elk Audio Documentation

Build with Cmake

# Clone the repository
$ git clone https://github.com/GuitarML/NeuralPi.git
$ cd NeuralPi

# initialize and set up submodules
$ git submodule update --init --recursive

# build with CMake
$ cmake -Bbuild
$ cmake --build build --config Release

The binaries will be located in NeuralPi/build/NeuralPi_artefacts/

Build with Projucer

1. Clone or download this repository.

2. Download and install JUCE This project uses the "Projucer" application from the JUCE website.

3. Download the RTNeural submodule (cd into the NeuralPi repo first):

   git submodule update --remote --recursive

4. Download and extract: json Json for c++.

5. Open the NeuralPi.jucer file and in the appropriate Exporter Header Search Path field, enter the appropriate include paths. For example:

  <full-path-to>/json-develop/include
  <full-path-to>/NeuralPi/modules/RTNeural
  <full-path-to>/NeuralPi/modules/RTNeural/modules/xsimd/include

6. Build NeuralPi from the Juce Projucer application for the intended build target.

Note: Make sure to build in Release mode unless actually debugging. Debug mode will not keep up with real time playing.