add an env var for path to pre-downloaded flashinfer cubin files

[842974287]

Summary:
Previously #21893 added a check for network access to NV cubin artifact endpoint. And if the check failed, we would opt out from trtllm attn backend.

This PR added a env var FLASHINFER_CUBIN_DIR to allow people specify a cubin dir which contains pre-downloaded cubin files. When FLASHINFER_CUBIN_DIR is specified, we will directly return True in has_nvidia_artifactory().

Differential Revision: D80035005

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D80035005

[gemini-code-assist]

Code Review

This pull request adds a new environment variable FLASHINFER_CUBIN_DIR to allow specifying a local directory for pre-downloaded flashinfer cubin files. This is a useful feature for environments without network access to NVIDIA's artifactory. My review focuses on improving the robustness of the implementation and ensuring it aligns with the project's conventions for managing environment variables. I've provided two comments: one to centralize the new environment variable definition for better maintainability, and another to add a more robust check for the provided directory path to prevent potential runtime errors.

[gemini-code-assist]

The current check for FLASHINFER_CUBIN_DIR is not robust enough. It will return True even if the environment variable is set to an empty string (''), which is not a valid path and will likely cause failures later. It's better to also verify that the path exists and is a directory.

Suggested change

	# Since FLASHINFER_CUBIN_DIR defines the pre-downloaded cubins path, when
	# it's true, we could assume the cubins are available.
	if FLASHINFER_CUBIN_DIR is not None:
	return True
	# If FLASHINFER_CUBIN_DIR is set and points to a valid directory,
	# we can assume the cubins are available.
	if FLASHINFER_CUBIN_DIR and os.path.isdir(FLASHINFER_CUBIN_DIR):
	return True

[gemini-code-assist]

For consistency and maintainability, environment variables should be managed centrally in vllm/envs.py. Please add FLASHINFER_CUBIN_DIR to vllm/envs.py and import it from there. This aligns with how other environment variables are handled in the project.

After adding it to vllm/envs.py, you can change this line to:

Suggested change

	FLASHINFER_CUBIN_DIR = os.environ.get("FLASHINFER_CUBIN_DIR", None)
	FLASHINFER_CUBIN_DIR = envs.FLASHINFER_CUBIN_DIR

[842974287]

Having an RFC PR on flashinfer to add the same env var as well flashinfer-ai/flashinfer#1462.

[pavanimajety]

Thanks for the PR - Idea looks good to me, please address gemini's comments, they seem reasonable.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D80035005

[842974287]

@pavanimajety updated the PR, please take another look!

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D80035005

[pavanimajety]

LGTM, please rebase your branch for the CI

@mgoin for CI and further comments

[houseroad]

Looks good, thanks

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D80035005

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D80035005