Enable CDPruner #2714
Draft
Enable CDPruner #2714
+3,729
−35
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…r requests for performance optimization
…e configuration. Refactor CDPruner to use visual tokens percentage instead of count for pruning configuration
…ode and remove unused visual token pruning methods
…e arguments and update GenerationConfig structure
…e vision config handling
…genai into ywang2/vlm-cdpruner
…ross codebase for consistency in CDPruner configuration
…in_percentage" - Updated Python scripts to reflect the corrected parameter name in argument parsing and configuration settings. 2. Added unit tests for the FastGreedyDPP class to ensure proper functionality and selection behavior based on the visual tokens retention percentage.
…genai into ywang2/vlm-cdpruner
…elated configurations
…r improved cache performance
…nhance cache performance with manual loop unrolling
…mproved readability
…lves for improved relevance computation and DPP selection
…ing invalid values Refactor performance logging in CDPruner: consolidate timing summary and metrics output
…ent vector subtraction and multiplication functions
…n for improved performance
…uration for performance metrics
…model compilation and enhance token selection with visual token splitting for improved performance Add support for OpenVINO ops in CDPruner configuration: introduce 'use_ops_model' flag for enhanced pruning computation
github-actions
bot
added
category: llm_bench
…r function for efficient token selection across kernel matrices
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Integrating a CDPruner module for visual token pruning into VLM pipeline.
Paper: CDPruner (arXiv)
Code: GitHub Repository
Tickets: CVS-173220