[Bug] AutoModelForMaskGeneration fails to load facebook/sam-vit-base due to config mismatch

[TristanLecourtois]

[Bug] AutoModelForMaskGeneration fails to load facebook/sam-vit-base due to config mismatch

Hi,

I'm trying to run the Jupyter notebook https://github.com/NielsRogge/Transformers-Tutorials/blob/master/Grounding%20DINO/GroundingDINO_with_Segment_Anything.ipynb but I encountered an error when loading the SAM model with AutoModelForMaskGeneration I'm trying to use the model facebook/sam-vit-base in a Python script using:

from transformers import AutoModelForMaskGeneration
model = AutoModelForMaskGeneration.from_pretrained("facebook/sam-vit-base")

But I get the following error:

ValueError: Unrecognized configuration class <class 'transformers.models.sam.configuration_sam.SamConfig'> for this kind of AutoModel: AutoModelForMaskGeneration.
Model type should be one of SamHQConfig.

It seems that facebook/sam-vit-base is not compatible with AutoModelForMaskGeneration, which expects a SamHQConfig.

[NielsRogge]

Thanks for reporting, looks like indeed there's a bug here due to it being defined twice: https://github.com/huggingface/transformers/blob/593e29c5e2a9b17baec010e8dc7c1431fed6e841/src/transformers/models/auto/modeling_auto.py#L1529-L1539. Will submit a PR.