chore: support pytorch format in lora

[KilJaeeun]

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

• Hello, I would like to make the .pt file format compatible with LoRA.
• This is my first time submitting a PR..! so there may be some missing parts. Please feel free to leave any comments.

Test Plan

Test Result

(Optional) Documentation Update

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request adds support for loading LoRA adapters from .pt files. The implementation correctly adds the file path and loading logic. However, it misses a critical validation step to check for unexpected modules, which is already present for .bin files. I've added a comment to include this check to prevent potential issues with incompatible adapters.

[gemini-code-assist]

The logic to check for unexpected modules is missing for .pt files. This check is present for .bin files and is crucial to ensure that the loaded LoRA adapter is compatible with the model, preventing potential errors and incorrect behavior. The same validation should be applied here.

While this suggestion duplicates code, a follow-up refactoring could consolidate the logic for both .bin and .pt files to improve maintainability.

        elif os.path.isfile(lora_pt_file_path):
            # When a pt file is provided, we rely on config to find unexpected
            # modules.
            unexpected_modules = []
            target_modules = peft_helper.target_modules
            if not isinstance(target_modules, list):
                target_modules = [target_modules]
            for module in target_modules:
                # Compatible with more modules,
                # such as:layers.11.self_attn.k_proj
                part_name = module.split(".")[-1]
                if part_name not in expected_lora_modules:
                    unexpected_modules.append(module)
            # loaded lora's target modules must be a subset of
            # expected_lora_modules. It is not reliable. See
            # https://github.com/vllm-project/vllm/pull/5909. But there's no
            # other better mechanism.
            if unexpected_modules and not is_regex_target_modules(
                    peft_helper.target_modules, expected_lora_modules):
                raise ValueError(
                    f"While loading {lora_dir}, expected"
                    f" target modules in {expected_lora_modules}"
                    f" but received {unexpected_modules}."
                    f" Please verify that the loaded LoRA module is correct")
            tensors = torch.load(lora_pt_file_path,
                                 map_location=device,
                                 weights_only=True)

[jeejeelee]

QQ：does the .pt format exist? Can you provide an example?

[KilJaeeun]

Wow, thank you for your reply..
I found the following lora models on Hugging Face:
And I can't disclose it, but I'm also using it for work.

• https://huggingface.co/jtromero/qwen2-0.5b-lora-single-device/blob/main/adapter_model.pt
• https://huggingface.co/shariar076/Llama-3.1-8B-Instruct-DPO-75R25L-PoliTune/blob/main/adapter_model.pt
• https://huggingface.co/openSUSE/Cavil-Qwen3-4B/blob/main/adapter_model.pt

[KilJaeeun]

@jeejeelee
Hello, I’ve added an example to your comment. Could you please leave an additional review? I’m waiting for your feedback :)

[jeejeelee]

The processing logic for lora_pt_file_path and lora_bin_file_path looks the same. Can we merge them to avoid redundant code?

[jeejeelee]

Please fix the pre-commit failure, and then LGTM

[jeejeelee]

Thank you for contribution