[Doc] Add max_lora_rank configuration guide

[chi2liu]

Purpose

Add documentation to help users properly configure max_lora_rank parameter to avoid performance issues.

Context

As discussed with @jeejeelee , in multi-LoRA serving scenarios, max_lora_rank must be set to accommodate the largest rank among all LoRA adapters that might be loaded. However, setting it unnecessarily high wastes memory and can cause performance issues.

Changes

Added a new section "Configuring max_lora_rank" at the end of docs/features/lora.md that explains:

• How to set the parameter correctly for multi-LoRA scenarios
• Performance implications of setting it too high
• Examples of good vs bad configuration
• What happens when a LoRA's rank exceeds the configured limit

Case:

• Default max_lora_rank is 16
• When set higher than actual adapter rank, causes memory over-allocation and performance loss
• Users often don't realize this misconfiguration is causing slowdowns

Why this causes performance degradation:

• vLLM pre-allocates GPU memory tensors based on max_lora_rank for all LoRA operations
• When max_lora_rank > actual rank, the system allocates more tensors
• During inference, Punica kernels (lora_shrink/lora_expand) compute on the entire pre-allocated tensor
• This results in some wasted computation on zero-padded values
• Large tensors also cause L2 cache thrashing (hit rate drops)
• Memory bandwidth becomes saturated moving unnecessary zeros

Performance comparison (production workload A100-80GB):

• Gemma-3-27B + rank=16 LoRA
• With max_lora_rank=256: 3.649s per request
• With max_lora_rank=16: 2.292s per request
• 59% performance improvement after fixing configuration

Memory usage:

• With max_lora_rank=256: 3.18GB GPU memory for LoRA
• With max_lora_rank=16: 0.20GB GPU memory for LoRA
• 15.9x memory reduction

(Optional) Documentation Update

Not required - this is a diagnostic improvement only.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request adds a crucial warning for a common LoRA misconfiguration where max_lora_rank is set higher than the adapter's actual rank, which can lead to significant performance degradation and memory over-allocation. The implementation is clean and placed correctly within the adapter loading logic. The warning message is informative and provides a clear, actionable solution to the user. This is an excellent diagnostic improvement that will help users avoid silent performance issues.

[jeejeelee]

We support multi-LoRA, where each LoRA has a different rank - it could be 16, or it could be 256. The max_lora_rank needs to be set to the maximum rank. So I think the warning in this PR is unreasonable

[chi2liu]

We support multi-LoRA, where each LoRA has a different rank - it could be 16, or it could be 256. The max_lora_rank needs to be set to the maximum rank. So I think the warning in this PR is unreasonable

@jeejeelee You're right - I didn't think through the multi-LoRA dynamic loading case properly. When serving multiple
LoRAs with different ranks, max_lora_rank needs to handle the largest one, and having smaller ranks is completely
normal.

Instead of runtime warnings, would it make sense to add a brief doc note? Something like:

• Set max_lora_rank to your highest LoRA rank
• Don't set it way higher than needed - wastes memory and can cause hard-to-debug perf issues
• Example: For LoRAs with ranks [16, 32, 64], use --max-lora-rank 64, not 256

What do you think?

[jeejeelee]

Nice, we can add the related notes in the end of lora doc

[chi2liu]

Nice, we can add the related notes in the end of lora doc

Thanks for the feedback! I've removed the code changes and only kept the documentation addition as you suggested.
The new section explains how to properly configure max_lora_rank and warns about performance implications of setting it too high.
Please take a look and let me know if you have any suggestions for the docs.

[jeejeelee]

Suggested change

	## Configuring `max_lora_rank`
	## Configuring `max_lora_rank`
	## Using Tips
	### Configuring `max_lora_rank`

[chi2liu]

updated

[jeejeelee]

Suggested change

	- **Example**: If your LoRA adapters have ranks [16, 32, 64], use `--max-lora-rank 64` rather than 256
	For example, if your LoRA adapters have ranks [16, 32, 64], use `--max-lora-rank 64` rather than 256

[chi2liu]

updated

[jeejeelee]

LGTM, thank you for contribution