transformer_lens.model_bridge.supported_architectures.qwen2 module

Qwen2 architecture adapter.

class transformer_lens.model_bridge.supported_architectures.qwen2.Qwen2ArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for Qwen2 models.

Qwen2 hardcodes q/k/v biases (o_proj, MLP, and norms are bias-free); the include_biases conversions keep GQA K/V biases in the per-head (n_kv_heads, d_head) layout weight processing expects.

__init__(cfg: Any) → None

Initialize the Qwen2 architecture adapter.