-
-
Notifications
You must be signed in to change notification settings - Fork 10.4k
[doc] Update x86 CPU-inference installation doc to reflect optionality of AVX512f #22707
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…tructions AVX512f is optional, see vllm-project#5452 updating the docs to reflect this. Signed-off-by: Sooraj S <[email protected]>
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
Signed-off-by: Sooraj S <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request updates the x86 CPU inference installation documentation to reflect that the avx512f
flag is now optional. The changes correctly mark the flag as optional and update the warning and build instructions. I've included a suggestion to improve the clarity and grammar of the updated warning message.
docker build -f docker/Dockerfile.cpu \ | ||
--build-arg VLLM_CPU_AVX512BF16=false (default)|true \ | ||
--build-arg VLLM_CPU_AVX512VNNI=false (default)|true \ | ||
--build-arg VLLM_CPU_DISABLE_AVX512=false (default)|true \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Sooraj S <[email protected]>
Hi @sooraj-satheesh In fact the AVX2 version is not recommend because it only has basic feature support. We recommend AVX512 version and only provides AVX512 pre-built image. |
@bigPYJ1151 yes but the dockerfile supports building without it and it would reduce confusion to document this. The recommendation against it can also be added to the doc. |
Signed-off-by: Sooraj S <[email protected]>
Co-authored-by: Li, Jiang <[email protected]> Signed-off-by: Sooraj S <[email protected]>
…y of AVX512f (vllm-project#22707) Signed-off-by: Sooraj S <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Li, Jiang <[email protected]> Signed-off-by: Paul Pak <[email protected]>
…y of AVX512f (vllm-project#22707) Signed-off-by: Sooraj S <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Li, Jiang <[email protected]> Signed-off-by: Diego-Castan <[email protected]>
…y of AVX512f (vllm-project#22707) Signed-off-by: Sooraj S <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Li, Jiang <[email protected]>
…y of AVX512f (vllm-project#22707) Signed-off-by: Sooraj S <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Li, Jiang <[email protected]>
…y of AVX512f (vllm-project#22707) Signed-off-by: Sooraj S <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Li, Jiang <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
…y of AVX512f (vllm-project#22707) Signed-off-by: Sooraj S <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Li, Jiang <[email protected]>
Essential Elements of an Effective PR Description Checklist
supported_models.md
andexamples
for a new model.Purpose
AVX512f is optional, see #5452 . updating the docs to reflect this. Solves #18660
Test Plan
Build the docker image with --build-arg VLLM_CPU_DISABLE_AVX512=true
Test Result
Container does not crash with
Illegal Instruction
on a cpu without avx512f.(Optional) Documentation Update