-
-
Notifications
You must be signed in to change notification settings - Fork 10.4k
[Bugfix] Fix MoE BNB version #22260
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bugfix] Fix MoE BNB version #22260
Conversation
Signed-off-by: Jee Jee Li <[email protected]>
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request updates the minimum required version of the bitsandbytes
library for MoE models from 0.45.3
to 0.46.1
. I recommend refactoring the duplicated version check logic to improve code maintainability and prevent future inconsistencies.
if bitsandbytes.__version__ < "0.46.1": | ||
raise ImportError("bitsandbytes version is wrong. Please " | ||
"install bitsandbytes>=0.45.3.") | ||
"install bitsandbytes>=0.46.1.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This version check logic is duplicated in BitsAndBytesLinearMethod.__init__
(lines 172-174). This duplication makes maintenance harder, as a version bump requires changes in two places. This could lead to inconsistencies if one location is missed during an update. To improve maintainability and adhere to the DRY (Don't Repeat Yourself) principle, I recommend extracting this logic into a shared helper function.
import bitsandbytes
if bitsandbytes.__version__ < "0.46.1":
raise ImportError("bitsandbytes version is wrong. Please install bitsandbytes>=0.46.1.")
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Jinzhen Lin <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Noam Gat <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Paul Pak <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Diego-Castan <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Essential Elements of an Effective PR Description Checklist
supported_models.md
andexamples
for a new model.Purpose
Test Plan
Test Result
(Optional) Documentation Update