KVCrush method for cache eviction [Updated]

[gopikrishnajha]

Creating new and updated PR for KVCrush as I was having a tough time resolving merge conflicts on the existing PR (#2211). Please consider this as the official PR and ignore the old one.

• I have addressed ALL the comments apart from a few for which I have added explanation in the old PR.
• Documentation and accuracy evaluation on LongBench is added here.
• KV cache budget is in terms of blocks now, not tokens.
• For all the comments in the older PR where I have clarifications to make, I have added them as my comment, and have marked others as resolved (after making changes here.)

[vshampor]

Please make the CI checks green and we'll merge this. You will probably have to update your local version of pybind11-stubgen and regenerate the .pyi file (part of the build process anyway) to resolve some of the CI issues

[l-bat]

I don't want to slow down the merge, but the test should be fixed in the following PR.

[l-bat]

Suggested change

	@pytest.mark.nightly
	@pytest.mark.precommit

[Copilot]

Pull Request Overview

This PR implements the KVCrush method for cache eviction, an enhancement to the existing H2O/SnapKV cache eviction algorithms. KVCrush selects representative blocks from the evictable cache area using clustering analysis rather than simply evicting low-score blocks.

Key changes include:

• Implementation of the KVCrush algorithm with configurable anchor point modes (RANDOM, ZEROS, ONES, MEAN, ALTERNATE)
• Integration of KVCrush configuration into the existing CacheEvictionConfig system
• Comprehensive test coverage including unit tests and performance evaluation on LongBench datasets

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tools/continuous_batching/benchmark/continuous_batching_benchmark.cpp	Updates benchmark configuration to include new KVCrush parameters
tests/python_tests/test_kv_cache_eviction.py	Adds KVCrush vs SnapKV baseline comparison tests and new test configurations
tests/cpp/kvcrush.cpp	Comprehensive unit tests for KVCrush algorithm components
tests/cpp/cache_eviction.cpp	Updates existing cache eviction tests to support KVCrush configuration
src/python/py_continuous_batching_pipeline.cpp	Python bindings for KVCrush configuration classes
src/python/openvino_genai/py_openvino_genai.pyi	Type hints for new KVCrush Python API
src/python/openvino_genai/__init__.pyi	Export declarations for KVCrush classes
src/python/openvino_genai/__init__.py	Import statements for KVCrush classes
src/cpp/src/continuous_batching/kvcrush.hpp	Header file defining KVCrush algorithm interface
src/cpp/src/continuous_batching/kvcrush.cpp	Core KVCrush algorithm implementation
src/cpp/src/continuous_batching/cache_eviction.hpp	Integration of KVCrush into cache eviction system
src/cpp/src/continuous_batching/cache_eviction.cpp	Implementation of KVCrush integration logic
src/cpp/include/openvino/genai/cache_eviction.hpp	Public API definitions for KVCrush configuration
site/docs/concepts/optimization-techniques/kvcache-eviction-algorithm.md	Documentation and performance evaluation results