-
-
Notifications
You must be signed in to change notification settings - Fork 5.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
enforce eager mode with bnb quantization temporarily #6846
enforce eager mode with bnb quantization temporarily #6846
Conversation
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge). To run full CI, you can do one of these:
🚀 |
Co-authored-by: Michael Goin <[email protected]>
@mgoin Thanks for comments. Updated. |
/ready |
bitsandbytes-foundation/bitsandbytes#1308 this issue got fixed , this will enable not to enforce eager mode ? @chenqianfzh @kylesayrs |
@devlup correct but we need a new release of bitsandbytes to be made since we use the pypi package |
Signed-off-by: Alvant <[email protected]>
Temporarily enforce eager_mode in bitsandbytes quantization before the known issue (#5569) is fixed.