Skip to content

Conversation

davidxia
Copy link
Contributor

@davidxia davidxia commented May 16, 2025

  • make it apparent the set up instructions are for Nvidia CUDA
  • the pip install command requires the PyTorch index
  • install the python3-dev Debian package that's required for building mamba-ssm wheel from source
The output of python collect_env.py
$ python vllm/collect_env.py
INFO 05-16 18:10:30 [__init__.py:248] Automatically detected platform cuda.
Collecting environment information...
PyTorch version: 2.7.0+cu128
Is debug build: False
CUDA used to build PyTorch: 12.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 24.04.2 LTS (x86_64)
GCC version: (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
Clang version: Could not collect
CMake version: version 3.28.3
Libc version: glibc-2.39

Python version: 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] (64-bit runtime)
Python platform: Linux-6.11.0-1014-gcp-x86_64-with-glibc2.39
Is CUDA available: True
CUDA runtime version: 12.9.41
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA A100-SXM4-40GB
Nvidia driver version: 575.51.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Address sizes:                        46 bits physical, 48 bits virtual
Byte Order:                           Little Endian
CPU(s):                               12
On-line CPU(s) list:                  0-11
Vendor ID:                            GenuineIntel
Model name:                           Intel(R) Xeon(R) CPU @ 2.20GHz
CPU family:                           6
Model:                                85
Thread(s) per core:                   2
Core(s) per socket:                   6
Socket(s):                            1
Stepping:                             7
BogoMIPS:                             4400.43
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat avx512_vnni md_clear arch_capabilities
Hypervisor vendor:                    KVM
Virtualization type:                  full
L1d cache:                            192 KiB (6 instances)
L1i cache:                            192 KiB (6 instances)
L2 cache:                             6 MiB (6 instances)
L3 cache:                             38.5 MiB (1 instance)
NUMA node(s):                         1
NUMA node0 CPU(s):                    0-11
Vulnerability Gather data sampling:   Not affected
Vulnerability Itlb multihit:          Not affected
Vulnerability L1tf:                   Not affected
Vulnerability Mds:                    Not affected
Vulnerability Meltdown:               Not affected
Vulnerability Mmio stale data:        Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI SW loop, KVM SW loop
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.4
[pip3] nvidia-cublas-cu12==12.8.3.14
[pip3] nvidia-cuda-cupti-cu12==12.8.57
[pip3] nvidia-cuda-nvrtc-cu12==12.8.61
[pip3] nvidia-cuda-runtime-cu12==12.8.57
[pip3] nvidia-cudnn-cu12==9.7.1.26
[pip3] nvidia-cufft-cu12==11.3.3.41
[pip3] nvidia-cufile-cu12==1.13.0.11
[pip3] nvidia-curand-cu12==10.3.9.55
[pip3] nvidia-cusolver-cu12==11.7.2.55
[pip3] nvidia-cusparse-cu12==12.5.7.53
[pip3] nvidia-cusparselt-cu12==0.6.3
[pip3] nvidia-nccl-cu12==2.26.2
[pip3] nvidia-nvjitlink-cu12==12.8.61
[pip3] nvidia-nvtx-cu12==12.8.55
[pip3] pyzmq==26.4.0
[pip3] sentence-transformers==3.2.1
[pip3] torch==2.7.0+cu128
[pip3] torchaudio==2.7.0+cu128
[pip3] torchvision==0.22.0+cu128
[pip3] transformers==4.51.3
[pip3] transformers-stream-generator==0.0.5
[pip3] triton==3.3.0
[pip3] tritonclient==2.51.0
[pip3] vector-quantize-pytorch==1.21.2
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.8.5.dev708+g451da4bcb (git sha: 451da4bcb)
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
	GPU0	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	0-11	0		N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

LD_LIBRARY_PATH=/usr/local/cuda-12.9/lib64:/usr/local/cuda-12.9/lib64
NCCL_CUMEM_ENABLE=0
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1
CUDA_MODULE_LOADING=LAZY

Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@mergify mergify bot added the documentation Improvements or additions to documentation label May 16, 2025
Copy link
Contributor Author

@davidxia davidxia May 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without the --torch-backend=auto I got this error

ERROR: Ignored the following versions that require a different python version: 1.10.0 Requires-Python <3.12,>=3.8; 1.10.0rc1 Requires-Python <3.12,>=3.8; 1.10.0rc2 Requires-Python <3.12,>=3.8; 1.10.1 Requires-Python <3.12,>=3.8; 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11; 1.6.2 Requires-Python >=3.7,<3.10; 1.6.3 Requires-Python >=3.7,<3.10; 1.7.0 Requires-Python >=3.7,<3.10; 1.7.1 Requires-Python >=3.7,<3.10; 1.7.2 Requires-Python >=3.7,<3.11; 1.7.3 Requires-Python >=3.7,<3.11; 1.8.0 Requires-Python >=3.8,<3.11; 1.8.0rc1 Requires-Python >=3.8,<3.11; 1.8.0rc2 Requires-Python >=3.8,<3.11; 1.8.0rc3 Requires-Python >=3.8,<3.11; 1.8.0rc4 Requires-Python >=3.8,<3.11; 1.8.1 Requires-Python >=3.8,<3.11; 1.9.0 Requires-Python >=3.8,<3.12; 1.9.0rc1 Requires-Python >=3.8,<3.12; 1.9.0rc2 Requires-Python >=3.8,<3.12; 1.9.0rc3 Requires-Python >=3.8,<3.12; 1.9.1 Requires-Python >=3.8,<3.12
ERROR: Could not find a version that satisfies the requirement torch==2.7.0+cu128 (from versions: 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.7.0)
ERROR: No matching distribution found for torch==2.7.0+cu128

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without this package, I got this error when building mamba from source. This is with uv pip install, but the error with pip install is similar.

full error
$ uv pip install --no-build-isolation "git+https://github.com/state-spaces/[email protected]"
Resolved 42 packages in 190ms
  × Failed to build `mamba-ssm @ git+https://github.com/state-spaces/mamba@95d8aba8a8c75aedcaa6143713b11e745e7cd0d9`
  ├─▶ The build backend returned an error
  ╰─▶ Call to `setuptools.build_meta.build_wheel` failed (exit status: 1)

      [stdout]


      torch.__version__  = 2.7.0+cu126


      running bdist_wheel
      Guessing wheel URL:  https://github.com/state-spaces/mamba/releases/download/v2.2.4/mamba_ssm-2.2.4+cu12torch2.7cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
      Precompiled wheel not found. Building from source...
      running build
      running build_py
      copying mamba_ssm/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm
      copying mamba_ssm/distributed/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/distributed
      copying mamba_ssm/distributed/distributed_utils.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/distributed
      copying mamba_ssm/distributed/tensor_parallel.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/distributed
      copying mamba_ssm/models/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/models
      copying mamba_ssm/models/mixer_seq_simple.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/models
      copying mamba_ssm/models/config_mamba.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/models
      copying mamba_ssm/utils/generation.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/utils
      copying mamba_ssm/utils/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/utils
      copying mamba_ssm/utils/hf.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/utils
      copying mamba_ssm/utils/torch.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/utils
      copying mamba_ssm/ops/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops
      copying mamba_ssm/ops/selective_scan_interface.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops
      copying mamba_ssm/modules/mha.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules
      copying mamba_ssm/modules/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules
      copying mamba_ssm/modules/mamba2_simple.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules
      copying mamba_ssm/modules/mamba_simple.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules
      copying mamba_ssm/modules/ssd_minimal.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules
      copying mamba_ssm/modules/mamba2.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules
      copying mamba_ssm/modules/block.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules
      copying mamba_ssm/modules/mlp.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/modules
      copying mamba_ssm/ops/triton/selective_state_update.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/__init__.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/ssd_chunk_scan.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/softplus.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/layer_norm.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/ssd_combined.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/layernorm_gated.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/ssd_chunk_state.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/ssd_bmm.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/k_activations.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      copying mamba_ssm/ops/triton/ssd_state_passing.py -> build/lib.linux-x86_64-cpython-312/mamba_ssm/ops/triton
      running egg_info
      writing mamba_ssm.egg-info/PKG-INFO
      writing dependency_links to mamba_ssm.egg-info/dependency_links.txt
      writing requirements to mamba_ssm.egg-info/requires.txt
      writing top-level names to mamba_ssm.egg-info/top_level.txt
      reading manifest file 'mamba_ssm.egg-info/SOURCES.txt'
      adding license file 'LICENSE'
      adding license file 'AUTHORS'
      writing manifest file 'mamba_ssm.egg-info/SOURCES.txt'
      running build_ext
      building 'selective_scan_cuda' extension
      [1/1] c++ -MMD -MF /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o.d -fno-strict-overflow -Wsign-compare -DNDEBUG
      -g -O2 -Wall -fPIC -I/home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan -I/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include
      -I/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/usr/local/cuda-12.9/include
      -I/home/dxia/src/github.com/vllm-project/vllm/.venv/include -I/usr/include/python3.12 -c -c /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan/selective_scan.cpp
      -o /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o -O3 -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H
      '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1016"' -DTORCH_EXTENSION_NAME=selective_scan_cuda -D_GLIBCXX_USE_CXX11_ABI=1
      FAILED: /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o
      c++ -MMD -MF /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o.d -fno-strict-overflow -Wsign-compare -DNDEBUG
      -g -O2 -Wall -fPIC -I/home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan -I/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include
      -I/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/usr/local/cuda-12.9/include
      -I/home/dxia/src/github.com/vllm-project/vllm/.venv/include -I/usr/include/python3.12 -c -c /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan/selective_scan.cpp
      -o /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/csrc/selective_scan/selective_scan.o -O3 -std=c++17 -DTORCH_API_INCLUDE_EXTENSION_H
      '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1016"' -DTORCH_EXTENSION_NAME=selective_scan_cuda -D_GLIBCXX_USE_CXX11_ABI=1
      In file included from /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/Device.h:4,
                       from /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/api/include/torch/python.h:8,
                       from /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/csrc/selective_scan/selective_scan.cpp:7:
      /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/include/torch/csrc/python_headers.h:12:10: fatal error: Python.h: No such file or directory
         12 | #include <Python.h>
            |          ^~~~~~~~~~
      compilation terminated.
      ninja: build stopped: subcommand failed.

      [stderr]
      /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated
      !!

              ********************************************************************************
              Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0).

              By 2026-Feb-18, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.

              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************

      !!
        corresp(dist, value, root_dir)
      /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsWarning: `install_requires` overwritten in `pyproject.toml`
      (dependencies)
        corresp(dist, value, root_dir)
      /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/config/_apply_pyprojecttoml.py:61: SetuptoolsDeprecationWarning: License classifiers are deprecated.
      !!

              ********************************************************************************
              Please consider removing the following classifiers in favor of a SPDX license expression:

              License :: OSI Approved :: BSD License

              See https://packaging.python.org/en/latest/guides/writing-pypro
      /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
      !!

              ********************************************************************************
              Please consider removing the following classifiers in favor of a SPDX license expression:

              License :: OSI Approved :: BSD License

              See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
              ********************************************************************************

      !!
        self._finalize_license_expression()
      /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py:480: UserWarning: The detected CUDA version (12.9) has a minor version mismatch with the version
      that was used to compile PyTorch (12.6). Most likely this shouldn't be a problem.
        warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
      /home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py:490: UserWarning: There are no x86_64-linux-gnu-g++ version bounds defined for CUDA version 12.9
        warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
      Emitting ninja build file /home/dxia/.cache/uv/git-v0/checkouts/b8d3779b7264c4d1/95d8aba/build/temp.linux-x86_64-cpython-312/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      Traceback (most recent call last):
        File "<string>", line 317, in run
        File "/usr/lib/python3.12/urllib/request.py", line 240, in urlretrieve
          with contextlib.closing(urlopen(url, data)) as fp:
                                  ^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen
          return opener.open(url, data, timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/urllib/request.py", line 521, in open
          response = meth(req, response)
                     ^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/urllib/request.py", line 630, in http_response
          response = self.parent.error(
                     ^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/urllib/request.py", line 559, in error
          return self._call_chain(*args)
                 ^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain
          result = func(*args)
                   ^^^^^^^^^^^
        File "/usr/lib/python3.12/urllib/request.py", line 639, in http_error_default
          raise HTTPError(req.full_url, code, msg, hdrs, fp)
      urllib.error.HTTPError: HTTP Error 404: Not Found

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2506, in _run_ninja_build
          subprocess.run(
        File "/usr/lib/python3.12/subprocess.py", line 571, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "<string>", line 11, in <module>
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 432, in build_wheel
          return _build(['bdist_wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 423, in _build
          return self._build_with_temp_dir(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir
          self.run_setup()
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 317, in run_setup
          exec(code, locals())
        File "<string>", line 336, in <module>
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 186, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
          dist.run_commands()
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
          self.run_command(cmd)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command
          super().run_command(command)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
        File "<string>", line 334, in run
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
          self.run_command("build")
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
          self.distribution.run_command(command)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command
          super().run_command(command)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
          self.distribution.run_command(command)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 1104, in run_command
          super().run_command(command)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 99, in run
          _build_ext.run(self)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run
          self.build_extensions()
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 1010, in build_extensions
          build_ext.build_extensions(self)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 484, in build_extensions
          self._build_extensions_serial()
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 510, in _build_extensions_serial
          self.build_extension(ext)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 264, in build_extension
          _build_ext.build_extension(self, ext)
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 565, in build_extension
          objects = self.compiler.compile(
                    ^^^^^^^^^^^^^^^^^^^^^^
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 815, in unix_wrap_ninja_compile
          _write_ninja_file_and_compile_objects(
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2159, in _write_ninja_file_and_compile_objects
          _run_ninja_build(
        File "/home/dxia/src/github.com/vllm-project/vllm/.venv/lib/python3.12/site-packages/torch/utils/cpp_extension.py", line 2522, in _run_ninja_build
          raise RuntimeError(message) from e
      RuntimeError: Error compiling objects for extension

      hint: This usually indicates a problem with the package or the build environment.

@davidxia davidxia marked this pull request as ready for review May 16, 2025 18:10
Copy link

mergify bot commented May 23, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @davidxia.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Copy link

mergify bot commented May 24, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @davidxia.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@davidxia
Copy link
Contributor Author

@hmellor I rebased the conflicts away if you get a chance to review. Thanks! 🙏

@mergify mergify bot removed the needs-rebase label May 28, 2025
Copy link

mergify bot commented Jun 2, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @davidxia.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@davidxia
Copy link
Contributor Author

@hmellor lmk how the PR looks. Thanks!

@DarkLight1337
Copy link
Member

FYI there is no need to workaround mamba-ssm since #21421

@hmellor
Copy link
Member

hmellor commented Jul 25, 2025

@davidxia sorry about the delay on this one. If you reverted the change about mamba-ssm I'd be happy to accept the other parts of this PR

@davidxia
Copy link
Contributor Author

@hmellor Thanks, removed that part.

@davidxia davidxia requested a review from hmellor July 29, 2025 16:53
Copy link
Member

@hmellor hmellor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of comments, but other than that this looks good to me

* make it apparent the set up instructions are for Nvidia CUDA
* the `pip install` command requires the PyTorch index
* install the `python3-dev` Debian package that's required for building
  mamba-ssm wheel from source

Signed-off-by: David Xia <[email protected]>
@simon-mo simon-mo merged commit 7b49cb1 into vllm-project:main Jul 29, 2025
5 of 8 checks passed
@davidxia davidxia deleted the docs2 branch July 29, 2025 17:48
liuyumoye pushed a commit to liuyumoye/vllm that referenced this pull request Jul 31, 2025
x22x22 pushed a commit to x22x22/vllm that referenced this pull request Aug 5, 2025
npanpaliya pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Aug 6, 2025
jinzhen-lin pushed a commit to jinzhen-lin/vllm that referenced this pull request Aug 9, 2025
noamgat pushed a commit to noamgat/vllm that referenced this pull request Aug 9, 2025
paulpak58 pushed a commit to paulpak58/vllm that referenced this pull request Aug 13, 2025
diegocastanibm pushed a commit to diegocastanibm/vllm that referenced this pull request Aug 15, 2025
epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation force-merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants