Add support for NAM files

[christoph-hart]

Hi Jatin,

how hard would it be to add support for parsing the NAM file format?

https://github.com/sdatkinson/NeuralAmpModelerCore

Just from a quick peek at both sources the required layers are almost there (except for the wavenet layer which seems like a high level abstraction of existing low level layers).

I would love to avoid adding too many neural network engines to my project so if you think it‘s doable I‘ll give it a shot.

[brummer10]

@christoph-hart
I've created a project were I implemented both engines. So users could load nam and json/aidax files without take care which one to load. Implementation of both engines is straight forward.
https://github.com/brummer10/Ratatouille.lv2

[christoph-hart]

Thanks for the input and adding both engines is definitely an option but I would love to avoid adding the big fat Eigen library and RTNeural is already in there with what looks to me 95% of the required feature set.

[jatinchowdhury18]

Hi All!

I think it should be possible to construct a NAM-style model using RTNeural's layers. If I remember correctly NAM uses a "Temporal Convolutional Network", and I have implemented a couple of those in the past using RTNeural's layers, although there are sometimes variations between those types of networks. Here's an example of a "micro-TCN" implementation that we use as part of RTNeural's test suite. Probably the best route forward would be to use that implementation as a starting point, add whatever might be missing from the NAM model architecture, and probably adapt the mechanism for loading model weights to match whatever format NAM models use to store their weight. I'd be happy to help with this process as my time allows.

That said, I'm not sure it would make sense to add support for NAM models directly to RTNeural, since I think it falls a little bit outside the scope of what RTNeural does. I do have some future plans for a sort of "model library" which would have example implementations of several neural network architectures that are commonly used in real-time audio (and maybe other real-time domains as well), and I think having NAM models as part of the model library would be great. However, there's some other changes I want to make to RTNeural before starting on that, so it may be a while before I get there.

[olilarkin]

Also interested in this. sdatkinson/NeuralAmpModelerCore#49

[RustoMCSpit]

maybe relevant? Chowdhury-DSP/BYOD#363

[brummer10]

Thanks for the input and adding both engines is definitely an option but I would love to avoid adding the big fat Eigen library and RTNeural is already in there with what looks to me 95% of the required feature set.

Just out of curiosity I checked if we could build NeuralAmpModelerCore against the Eigen library comes with RTNeural, and yes, it works flawless. We could even share the jsaon header.

[MaxPayne86]

My 2 cents: we want to avoid boilerplate code on engine side or even inside plugin, i.e. we don't really want RTNeural to have methods to parse .nam (or .aidax or whatever) model files (torch weights), we want to adjust the model file that is coming out of a training repo and port to the format used by RTNeural.
For example: Automated-GuitarAmpModelling repo uses torch and creates a model file that is not directly supported by RTNeural, mostly because RTNeural is using keras implementation as a reference. I've created this script that simply adapt the model file from Automated-GuitarAmpModelling into what's expected on RTNeural side. The same could be done for .nam.