[Performance]: Benchmark performance (CPU utilization) mismatch on 2 different CPU

[cesarpgouveia]

Hi,

Two target platforms were tested:

Platform 1:

• 13th Gen Intel(R) Core(TM) i7-1370P 1.90 GHz
• Windows 10 Enterprise
• OpenVINO 2024.4 installed via PyPi
• Python 3.9.5

Platform 2:

• Intel(R) Core(TM) Ultra 7 165H 3.80 GHz
• Windows 11 Pro
• OpenVINO 2024.4 installed via PyPi
• Python 3.9.5

Performance issue description

We ran the following benchmark on the two target platforms:

benchmark_app -m C:\resnet50.onnx -d CPU -hint throughput -niter 10000

And we got two behaviors:

Platform 1:

Platform 2:

As you can see, on Plat1 all the CPU cores are fully utilized when running the benchmark app, but on Plat2 we don’t observe the same behavior. Do you know why this might be happening? It seems that we are not getting the full advantage of the CPU on Plat2, which could negatively impact performance. Do you have any insights or explanations for this? Could there be a configuration on the CPU, operating system, or OpenVINO side that we can adjust to ensure full utilization of all CPU cores on Plat2?

Step-by-step reproduction

$ conda create --name vino python=3.9.5
$ conda activate vino
$ pip install openvino==2024.4
$ benchmark_app -m C:\resnet50.onnx -d CPU -hint throughput -niter 10000

[Wan-Intel]

Could you please install the latest version of OpenVINO™ 2025.3.0 and see if the issue persists? You may install OpenVINO™ 2025.3.0 via pip install openvino==2025.3.0.

[cesarpgouveia]

Hi @Wan-Intel thanks for the response. We tested using the latest version and we got the same performance, so the issue persists