transformer_lens.model_bridge.supported_architectures.glm4 module

GLM-4-0414 architecture adapter.

Z.ai’s GLM-4-0414 family (Glm4ForCausalLM: GLM-4-32B-0414, GLM-Z1): the dense GLM decoder (adjacent-pair partial RoPE, joint gate_up_proj MLP) with Gemma-2-style sandwich norms — extra RMS norms applied to the attention and MLP outputs before their residual adds.

class transformer_lens.model_bridge.supported_architectures.glm4.Glm4ArchitectureAdapter(cfg: Any)

Bases: GlmArchitectureAdapter

Architecture adapter for Glm4ForCausalLM models.