transformer_lens.model_bridge.supported_architectures.t5 module

T5 architecture adapter.

class transformer_lens.model_bridge.supported_architectures.t5.T5ArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for T5 models.

T5 is an encoder-decoder model with: - Shared embeddings - Encoder stack (self-attention + FFN) - Decoder stack (self-attention + cross-attention + FFN) - Language modeling head

Supports both standard T5 (DenseReluDense with wi/wo) and gated variants like Flan-T5 (T5DenseGatedActDense with wi_0/wi_1/wo).

__init__(cfg: Any) → None

Initialize the T5 architecture adapter.

Parameters:

cfg – The configuration object.