Apertus - Error while loading model

[BlueskyFR]

System Info

Latest VLLM Docker image with latest git transformers, running on an NVIDIA GPU

Who can help?

@ArthurZucker @EduardDurech @Cyrilvallez

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

ValueError: There is no module or parameter named 'model.layers.0.mlp.act_fn.beta' in TransformersForCausalLM

To reproduce, load swiss-ai/Apertus-8B-Instruct-2509 with the latest git version of Transformers for inference.

In my case, I used the VLLM Docker image and upgraded transformers manually:

docker run --rm -it --gpus all -e HF_TOKEN=$HF_TOKEN vllm/vllm-openai:latest --model swiss-ai/Apertus-8B-Instruct-2509 --port 8080 --async-scheduling

Expected behavior

The model should load without any issue. I don't know what this missing module is.

[Cyrilvallez]

Hey! This is a vllm issue, and should be open directly on their repo (though it's been fixed already!) 🤗

[BlueskyFR]

Many thanks for the quick answer!

[EduardDurech]

Yea check vllm-project/vllm#24100