Skip to content

Conversation

bigPYJ1151
Copy link
Member

@bigPYJ1151 bigPYJ1151 commented Aug 26, 2025

Purpose

  • Add _EventPlaceholder to workaround CUDA event usage with CPUModelRunner
  • Replace GPU tensor in CpuGpuBuffer.

Test Plan

CI tests

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: jiang1.li <[email protected]>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a fix for using CUDA events with the CPU model runner by introducing a _torch_cuda_wrapper to monkey-patch torch.cuda.Event. The changes look good overall, but there is a critical issue in the implementation of _torch_cuda_wrapper that could lead to a crash in environments without CUDA support. I've provided a suggestion to make it more robust. The other changes, including using self.pin_memory and refactoring tensor handling with CpuGpuBuffer, are solid improvements.

@DarkLight1337 DarkLight1337 enabled auto-merge (squash) August 26, 2025 13:39
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 26, 2025
@DarkLight1337 DarkLight1337 merged commit 9b01870 into vllm-project:main Aug 26, 2025
48 checks passed
tc-mb pushed a commit to tc-mb/vllm that referenced this pull request Aug 27, 2025
epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025
xiao-llm pushed a commit to xiao-llm/vllm that referenced this pull request Aug 28, 2025
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Sep 3, 2025
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready ONLY add when PR is ready to merge/full CI is needed v1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants