transformer_lens.model_bridge.supported_architectures.glm_moe_dsa module

GLM-MoE-DSA architecture adapter.

class transformer_lens.model_bridge.supported_architectures.glm_moe_dsa.GlmMoeDsaArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for Z.ai GLM-5 / GLM-5.1 DSA models.

GLM-MoE-DSA combines MLA-style latent attention, a learned sparse-attention indexer, dense early MLP layers, and sparse MoE later layers.

setup_component_testing(hf_model: Any, bridge_model: Any = None) → None

Set up rotary embedding references for component testing.