transformer_lens.model_bridge.supported_architectures.glm_moe_dsa module¶
GLM-MoE-DSA architecture adapter.
- class transformer_lens.model_bridge.supported_architectures.glm_moe_dsa.GlmMoeDsaArchitectureAdapter(cfg: Any)¶
Bases:
DeepSeekMLAFamilyArchitectureAdapterArchitecture adapter for Z.ai GLM-5 / GLM-5.1 DSA models.
GLM-MoE-DSA combines MLA-style latent attention, a learned sparse-attention indexer, dense early MLP layers, and sparse MoE later layers. Checkpoints always compress Q, so the family’s required-Q default holds.
- attention_cls¶
alias of
GlmMoeDsaAttentionBridge
- component_mapping: ComponentMapping | None¶
- eager_attention: bool = True¶
- prepend_bos: bool | None = False¶
- uses_split_attention: bool¶
- weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None¶