transformer_lens.model_bridge.supported_architectures.qwen3 module

Qwen3 architecture adapter.

Base adapter for the Qwen3 model family. Provides shared config setup, attention bridge construction, and setup_component_testing used by Qwen3, Qwen3.5, and Qwen3Next variants.

class transformer_lens.model_bridge.supported_architectures.qwen3.Qwen3ArchitectureAdapter(cfg: Any, *, hybrid: bool = False)

Bases: ArchitectureAdapter

Architecture adapter for Qwen3 dense models.

RMSNorm, RoPE, GQA, Q/K head norms, gated MLP. No biases. Serves as base class for Qwen3.5 and Qwen3Next hybrid variants.

setup_component_testing(hf_model: Any, bridge_model: Any = None) → None

Set eager attn on HF model and rotary_emb on attention bridges.