New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Export mamba models #626

Open

bhack opened this issue Nov 19, 2024 · 3 comments

bhack commented Nov 19, 2024

Can you add a small example on how to export mamba models?

See pytorch/pytorch#130150

https://pytorch.org/tutorials/intermediate/torch_export_tutorial.html#custom-ops

Thanks

bhack mentioned this issue

Mamba is ExecuTorch compatible huggingface/transformers#33839

Open

Contributor

vasqu commented Nov 19, 2024 •

edited

Loading

Iirc, I think the team at meta showed it in a pretty nice way over here: https://github.com/facebookresearch/lingua/tree/main/apps/mamba/component

Edit: at least for Mamba2

Edit 2: I was too quick, i mistook it for compiling 🤦 ignore this

Author

bhack commented Nov 19, 2024

I am asking this cause huggingface/transformers#33839.

But yes after exporting the model it could be also compiled so a complete example of export+compile it will be very helpful.

Author

bhack commented Nov 26, 2024

@mathuvu Do you plan to contribute your https://github.com/facebookresearch/lingua/tree/main/apps/mamba/component upstream?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment