transformer_lens.model_bridge.supported_architectures.nemotron module

Nemotron architecture adapter.

NVIDIA’s dense Nemotron-3/4 and Minitron families (NemotronForCausalLM): llama-shaped GQA decoder with three quirks — LayerNorm1P normalization (zero-centered gamma, applied as weight + 1), a non-gated squared-ReLU MLP (up_proj/down_proj only), and partial rotary embeddings (rotary factor 0.5).

class transformer_lens.model_bridge.supported_architectures.nemotron.NemotronArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for NemotronForCausalLM models.

__init__(cfg: Any) → None

Initialize the Nemotron architecture adapter.