preserve properties from allowlist in Tokenizer

[pavel-esir]

No description provided.

[Copilot]

Pull Request Overview

This PR updates the tokenizer implementation to preserve selected properties from a whitelist instead of clearing all properties. The change allows specific OpenVINO hints and configuration options to be passed through to the tokenizer/detokenizer models on CPU.

• Introduces a filter_properties method that maintains only whitelisted properties
• Replaces the previous behavior of clearing all properties with selective filtering
• Updates the comment to reflect the new filtering approach

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Wovchena]

This overload doesn't have a path so enable_save_ov_model should result in failure. I think you shouldn't filter properties

[Wovchena]

You shouldn't filter*

[pavel-esir]

If we don't filter, and allow all properties to pass to 'CPU' then it fails because cpu plugin

2025-08-12T13:35:28.4948146Z E       Exception from src/inference/src/dev/plugin.cpp:53:
2025-08-12T13:35:28.4948893Z E       Exception from src/plugins/intel_cpu/src/config.cpp:457:
2025-08-12T13:35:28.4949480Z E       NotFound: Unsupported property NPUW_DEVICES by CPU plugin.
...
2025-08-12T13:35:28.4924214Z E           Exception from src/plugins/intel_cpu/src/config.cpp:457:
2025-08-12T13:35:28.4924878Z E           NotFound: Unsupported property prompt_lookup by CPU plugin.
...

and so on. We should leave only properties which core.compile_model(ov_[de]tokenizer, properties) can handle.

I also didn't quite understand bout saved model. At this point regardless of whether i filter or we don't have 'enable_save_ov_model' in properties. Could you please elaborate what did you mean?

[Wovchena]

If there's no enable_save_ov_model, what property leads to the exception?

[pavel-esir]

At least 3 arguments above NPUW_DEVICES, prompt_lookup, draft_model, probably some more

[Wovchena]

I guess that should do for now. But this is not a full list of valid properties. It will need to be converted to a black list instead of white list in the next PR because it's easier to resolve a problem reported as an error rather then just observe perf degradation.

[pavel-esir]

Agree with you

[Wovchena]

Ticket 172236