Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vLLM Pascal architecture fix no longer works #8

Open
elabz opened this issue Jun 6, 2024 · 1 comment
Open

vLLM Pascal architecture fix no longer works #8

elabz opened this issue Jun 6, 2024 · 1 comment

Comments

@elabz
Copy link

elabz commented Jun 6, 2024

Hello, what a great project! Unfortunately, vLLM fix for Pascal arch no longer works on the main branch.
vLLM changed the way it checks for compute capability, but I was unable to find how it's done in the current version
Would you be able to refactor the Pascal patch to make TinyLLM work with recent version of vLLM? Would much oblige!

@jasonacox
Copy link
Owner

Thanks @elabz ! You are right. I need to update it.

We have a PR to vLLM to add pascal support to the main project: vllm-project/vllm#4409 - here is the fix: https://github.com/vllm-project/vllm/pull/4409/files

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants