transformer_lens.model_bridge.supported_architectures.bert module

BERT architecture adapter.

This module provides the architecture adapter for BERT models.

class transformer_lens.model_bridge.supported_architectures.bert.BertArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for BERT models.

__init__(cfg: Any) → None

Initialize the BERT architecture adapter.

Parameters:

cfg – The configuration object.

component_mapping: ComponentMapping | None

prepare_model(hf_model: Any) → None

Adjust component mapping based on the actual HF model variant.

BertForMaskedLM has cls.predictions (MLM head). BertForNextSentencePrediction has cls.seq_relationship (NSP head) and no MLM-specific LayerNorm.

supports_generation: bool = False

uses_split_attention: bool

weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None