transformer_lens.model_bridge.supported_architectures.ernie4_5_moe module

ERNIE 4.5 MoE architecture adapter.

Baidu’s ERNIE 4.5 MoE (Ernie4_5_MoeForCausalLM): the dense ERNIE attention (GLM-style interleaved RoPE, config-gated biases) with a sparse MoE MLP — sigmoid-corrected top-k router, batched fused gate_up experts, optional shared experts, and a dense-MLP prefix before moe_layer_start_index.

class transformer_lens.model_bridge.supported_architectures.ernie4_5_moe.Ernie4_5_MoeArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for Ernie4_5_MoeForCausalLM models.

__init__(cfg: Any) → None

Initialize the ERNIE 4.5 MoE architecture adapter.