transformer_lens.model_bridge.supported_architectures.exaone module

EXAONE architecture adapter.

Supports LG AI Research’s EXAONE-3.0 / 3.5 / Deep families (ExaoneForCausalLM, trust_remote_code checkpoints). Llama-style RMSNorm + RoPE + GQA + gated MLP under GPT-2-flavored module names: transformer.wte, transformer.h[i], transformer.ln_f, and a double-nested attention (attn.attention). EXAONE-4.0 is a separate native-transformers architecture (Exaone4ForCausalLM) and is not covered here.

class transformer_lens.model_bridge.supported_architectures.exaone.ExaoneArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for ExaoneForCausalLM (EXAONE-3.x) models.

The remote modeling code follows current HF conventions (Cache API, position_embeddings tuples), so the standard bridges delegate cleanly. Naming quirks: attention projections live one level deeper than usual (attn.attention.q_proj), the gated MLP uses c_fc_0 (gate) / c_fc_1 (up) / c_proj (down), and rotary sits at transformer.rotary.

__init__(cfg: Any) → None

Initialize the EXAONE architecture adapter.

prepare_model(hf_model: Any) → Any

Shim the EXAONE-3.x remote module for transformers >= 5.13, which renamed create_causal_mask’s input_embeds kwarg to inputs_embeds.