transformer_lens.model_bridge.supported_architectures.afmoe module

AFMoE (Arcee Trinity, AfmoeForCausalLM) adapter: sandwich norms, QK-norm attention with sigmoid gating and NoPE/sliding RoPE (so attention delegates to HF), dense + sparse-MoE MLP layers split at num_dense_layers.

class transformer_lens.model_bridge.supported_architectures.afmoe.AfmoeArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for AfmoeForCausalLM models.

__init__(cfg: Any) → None

Initialize the AFMoE architecture adapter.

component_mapping: ComponentMapping | None

supports_fold_ln = False

uses_split_attention: bool

weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None