transformer_lens.model_bridge.supported_architectures.qwen2_5_vl module

Qwen2.5-VL architecture adapter.

Alibaba’s Qwen2.5-VL (Qwen2_5_VLForConditionalGeneration): a windowed ViT at model.visual (window attention with a few full-attention blocks, RMS block norms, gated vision MLP, 2D rotary) whose patch merger feeds a Qwen2-layout text decoder at model.language_model. Text attention uses mRoPE — three position streams (temporal/height/width) split across rotary channels — so the generic RoPE reconstruction would be text-only-correct but wrong for image runs; attention stays HF-native.

class transformer_lens.model_bridge.supported_architectures.qwen2_5_vl.Qwen2_5_VLArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for Qwen2_5_VLForConditionalGeneration models.

__init__(cfg: Any) → None

Initialize the Qwen2.5-VL architecture adapter.

component_mapping: ComponentMapping | None

required_libraries: list[str] = ['torchvision']

required_libraries_group: str = 'multimodal'

uses_split_attention: bool

weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None