[XPU] IPEX-optimized Punica Wrapper on XPU

[chaojun-zhang]

Test Plan

XPU_USE_TRITION_KERNEL=0 VLLM_USE_V1=1 CCL_ZE_IPC_EXCHANGE=drmfd VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 VLLM_WORKER_MULTIPROC_METHOD=spawn python3 benchmarks/benchmark_throughput.py --model meta-llama/Llama-2-7b-hf --backend vllm --dataset ./ShareGPT_V3_unfiltered_cleaned_split.json --num-prompts 1000 --max-loras 1 --max-lora-rank 8 --enable-lora --lora-path "yard1/llama-2-7b-sql-lora-test" --enforce_eager

1. xpu kernel + XPU_USE_TRITION_KERNEL=0

Adding requests: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [00:00<00:00, 1147.75it/s]
Processed prompts: 74%|████████████████████████████████████████████████████████████████████████████████████████▋ | 745/1000 [02:13<00:36, 6.92it/s, est. speed input: 1484.93 toks/s, output: 987.98 toks/s]WARNING 07-27 20:30:06 [_logger.py:68] Encountered invalid prefix detokenization error for request 623, resetting decode stream.
Processed prompts: 86%|█████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 861/1000 [02:27<00:13, 10.22it/s, est. speed input: 1523.34 toks/s, output: 1096.47 toks/s]WARNING 07-27 20:30:20 [_logger.py:68] Encountered invalid prefix detokenization error for request 623, resetting decode stream.
Processed prompts: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [03:08<00:00, 5.31it/s, est. speed input: 1306.31 toks/s, output: 1249.40 toks/s]
Throughput: 5.29 requests/s, 2543.90 total tokens/s, 1243.62 output tokens/s
Total num prompt tokens: 245995
Total num output tokens: 235278

2. trition kernel + XPU_USE_TRITION_KERNEL=1

Adding requests: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [00:00<00:00, 1356.26it/s]
Processed prompts: 38%|█████████████████████████████████████████████▋ | 384/1000 [01:20<02:13, 4.62it/s, est. speed input: 1260.77 toks/s, output: 625.78 toks/s]WARNING 07-27 20:33:58 [_logger.py:68] Encountered invalid prefix detokenization error for request 183, resetting decode stream.
Processed prompts: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [03:50<00:00, 4.34it/s, est. speed input: 1067.74 toks/s, output: 1021.23 toks/s]
Throughput: 4.33 requests/s, 2082.29 total tokens/s, 1017.96 output tokens/s
Total num prompt tokens: 245995
Total num output tokens: 235278

The XPU kernel achieved a 1.22x throughput improvement (from 2082.29 to 2543.90) over the Triton kernel

[gemini-code-assist]

Code Review

This pull request introduces a new Punica wrapper for XPU using IPEX kernels, which shows significant performance improvement. The implementation looks solid, but I've found a few issues that need to be addressed. There's a critical typo in an environment variable name that would prevent it from working as intended. Additionally, there are a couple of high-severity issues related to deprecated API usage and a return type violation that should be fixed for code quality and future compatibility.

[gemini-code-assist]

There's a typo in the environment variable name. It should be XPU_USE_TRITON_KERNEL instead of XPU_USE_TRITION_KERNEL. This typo will prevent users from correctly enabling the Triton kernel path.

Suggested change

	xpu_use_triton_kernel = os.getenv("XPU_USE_TRITION_KERNEL", "0") == "1"
	xpu_use_triton_kernel = os.getenv("XPU_USE_TRITON_KERNEL", "0") == "1"

[gemini-code-assist]

The msg attribute of ImportError is deprecated since Python 3.10 and will be removed in Python 3.12. Use e.args[0] to access the error message for better future compatibility.

Suggested change

	logger.warning("Import error msg: %s", e.msg)
	logger.warning("Import error msg: %s", e.args[0])

[gemini-code-assist]

This function is type-hinted to return None, but it returns a value. The operations on y are performed in-place, so this return statement is unnecessary and contradicts the function's signature. Remove this line.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[yma11]

Maybe we keep this env in our internal code but for upstream, let's just use ipex kernel path.

[chaojun-zhang]

Currently, my internal tests show that the Triton kernel is better than the XPU kernel in torch.compile mode. Keeping this option allows for its use in torch.compile?

[jikunshang]

I prefer name it to XPU_PUNICA_USE_TRITON

[chaojun-zhang]

Remove this env for internal code now .

[jeejeelee]

LGTM, please address the pre-commit failure

[xuechendi]

@DarkLight1337 @mgoin , may we get a quick review on this PR?