transformer_lens.model_bridge.supported_architectures.roberta module

RoBERTa architecture adapter.

class transformer_lens.model_bridge.supported_architectures.roberta.RobertaArchitectureAdapter(cfg: Any)

Bases: BertArchitectureAdapter

Architecture adapter for RobertaForMaskedLM: BERT’s layer, MLM head under lm_head.*.

Positions start at padding_idx + 1 = 2, so W_pos drops the table’s first two rows.

__init__(cfg: Any) → None

Initialize the RoBERTa architecture adapter.

prepare_model(hf_model: Any) → None

Read the usable context off the loaded position table.