A plugin to create joint embeddings from text and audio using CLAP by LAION.
- Create joint embeddings from text and audio
- Use the embeddings in combination with nendo's vector search functionality
Please make sure you have the correct version of Pytorch installed. Go to pytorch.org and select your OS, package manager and CUDA version to get the correct installation command.
- Install Nendo
pip install nendo-plugin-embed-clap
Take a look at a basic usage example below. For more detailed information, please refer to the documentation.
>>> from nendo import Nendo
>>> nd = Nendo(plugins=["nendo_plugin_embed_clap"])
>>> track = nd.library.add_track(file_path="path/to/file.mp3")
>>> embedding = nd.plugins.embed_clap(track=track)Visit our docs to learn all about how to contribute to Nendo: Contributing
Nendo: MIT License
Pretrained models: The weights are released by LAION under the Apache 2.0 license.