transformer_lens.model_bridge.supported_architectures.bert module

BERT architecture adapter.

This module provides the architecture adapter for BERT models.

class transformer_lens.model_bridge.supported_architectures.bert.BertArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for BERT models.

__init__(cfg: Any) → None

Initialize the BERT architecture adapter.

Parameters:

cfg – The configuration object.

prepare_model(hf_model: Any) → None

Adjust component mapping based on the actual HF model variant.

BertForMaskedLM has cls.predictions (MLM head). BertForNextSentencePrediction has cls.seq_relationship (NSP head) and no MLM-specific LayerNorm.