[llm] bump vllm to 0.9.0.1

[lk-chen]

Why are these changes needed?

Many important reasons

1. P/D disaggregation feature is added in 0.9.0
2. There's this dependency chain: vllm0.8.5 -> opentelemetry1.26.0 -> protobuf3, ray wants to upgrade to protobuf4

• vllm0.9.0 allows opentelemetry 1.27, which accepts protobuf4

3. some users want vllm 0.9.0

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[eicherseiji]

@lk-chen Can we change the import path here:

# python/ray/llm/_internal/serve/deployments/llm/vllm/vllm_engine.py
        if self.llm_config.log_engine_metrics:
            from ray.llm._internal.serve.deployments.llm.vllm.vllm_loggers import (
                RayPrometheusStatLogger,
            )

            # V1 AsyncLLMEngine does not yet support add_logger
            # For now, assume folks enabling log_engine_metrics do not require LoggingStatLogger, PrometheusStatLogger
            custom_stat_loggers = [RayPrometheusStatLogger]

From

            from ray.llm._internal.serve.deployments.llm.vllm.vllm_loggers import (

to

            from vllm.v1.metrics.ray_wrappers import (

and delete python/ray/llm/_internal/serve/deployments/llm/vllm/vllm_loggers.py?

release/llm_tests/serve/test_llm_serve_integration.py would have to be changed as well. Lmk if it's easier for me to just upload a patch.

[eicherseiji]

Taking this over

[eicherseiji]

Workarounds in this PR:

• DeepSeek models (flashMLA) fail to load due to Ray setting CUDA_VISIBLE_DEVICES="", ephemerally
  • vLLM PR: [Bugfix] Allow CUDA_VISIBLE_DEVICES='' in Platform.device_id_to_physical_device_id vllm-project/vllm#18979
  • Workaround: clear CUDA_VISIBLE_DEVICES env variable before launching the vLLM engine
• RayPrometheusStatLogger is broken upstream
  • vLLM PR: Fix ValueError: Missing value for tag key(s): model_name,engine. vllm-project/vllm#19113
  • Workaround: Keep our copy of RayPrometheusStatLogger in Ray until the vLLM PR lands/the next vLLM version bump.

[kouroshHakha]

Just a couple of qs before we merge.

[kouroshHakha]

Does this mean that if I run the script using compiled graph twice in a row the second one will die because of this hythersis effect?

[eicherseiji]

Yeah; I think pytest isn't letting the engine/pytorch clean up correctly between parameterizations so this is the workaround.