New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[New Model]: Mistral-Nemo #6563

Closed

Hambaobao opened this issue Jul 19, 2024 · 2 comments

Labels

new model

Hambaobao commented Jul 19, 2024 •

edited

Loading

The model to consider.

https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407

The closest model vllm already supports.

What's your difficulty of supporting the model you want?

No response

Hambaobao added the new model label

Hambaobao changed the title ~~[New Model]: mistralai/Mistral-Nemo-Instruct-2407~~ [New Model]: Mistral-Nemo

jonzhep commented Jul 19, 2024

we just need to pull head_dim from config i think
https://github.com/huggingface/transformers/commit/4c040aba02b0283619a06bdc40ecf868508b9e52

Collaborator

simon-mo commented Jul 19, 2024

Closed by #6548. It will be released next week. Please try build from source for now. Thank you!

simon-mo closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment