-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for NAM files #143
Comments
@christoph-hart |
Thanks for the input and adding both engines is definitely an option but I would love to avoid adding the big fat Eigen library and RTNeural is already in there with what looks to me 95% of the required feature set. |
Hi All! I think it should be possible to construct a NAM-style model using RTNeural's layers. If I remember correctly NAM uses a "Temporal Convolutional Network", and I have implemented a couple of those in the past using RTNeural's layers, although there are sometimes variations between those types of networks. Here's an example of a "micro-TCN" implementation that we use as part of RTNeural's test suite. Probably the best route forward would be to use that implementation as a starting point, add whatever might be missing from the NAM model architecture, and probably adapt the mechanism for loading model weights to match whatever format NAM models use to store their weight. I'd be happy to help with this process as my time allows. That said, I'm not sure it would make sense to add support for NAM models directly to RTNeural, since I think it falls a little bit outside the scope of what RTNeural does. I do have some future plans for a sort of "model library" which would have example implementations of several neural network architectures that are commonly used in real-time audio (and maybe other real-time domains as well), and I think having NAM models as part of the model library would be great. However, there's some other changes I want to make to RTNeural before starting on that, so it may be a while before I get there. |
Also interested in this. sdatkinson/NeuralAmpModelerCore#49 |
maybe relevant? Chowdhury-DSP/BYOD#363 |
Just out of curiosity I checked if we could build NeuralAmpModelerCore against the Eigen library comes with RTNeural, and yes, it works flawless. We could even share the jsaon header. |
My 2 cents: we want to avoid boilerplate code on engine side or even inside plugin, i.e. we don't really want RTNeural to have methods to parse .nam (or .aidax or whatever) model files (torch weights), we want to adjust the model file that is coming out of a training repo and port to the format used by RTNeural. |
Hi Jatin,
how hard would it be to add support for parsing the NAM file format?
https://github.com/sdatkinson/NeuralAmpModelerCore
Just from a quick peek at both sources the required layers are almost there (except for the wavenet layer which seems like a high level abstraction of existing low level layers).
I would love to avoid adding too many neural network engines to my project so if you think it‘s doable I‘ll give it a shot.
The text was updated successfully, but these errors were encountered: