transformer_lens.model_bridge.supported_architectures.qwen3_vl_moe module

Qwen3-VL-MoE architecture adapter.

Alibaba’s Qwen3-VL-MoE (Qwen3VLMoeForConditionalGeneration): the Qwen3-VL layout (DeepStack vision tower, interleaved-mRoPE text decoder) with a sparse-MoE MLP — batched experts plus a parameter-only top-k router returning a (logits, scores, indices) tuple, so the router stays unwrapped and MoEBridge delegates the block. Layers listed in mlp_only_layers hold a dense gated MLP under the same name.

class transformer_lens.model_bridge.supported_architectures.qwen3_vl_moe.Qwen3VLMoeArchitectureAdapter(cfg: Any)

Bases: Qwen3VLArchitectureAdapter

Architecture adapter for Qwen3VLMoeForConditionalGeneration models.