transformer_lens.model_bridge.supported_architectures.modernbert_decoder module

ModernBERT Decoder (ModernBertDecoderForCausalLM) adapter: ModernBERT run causally. Attention delegates (per-layer sliding windows) and LN folding is disabled (layer-0 Identity attention norm).

class transformer_lens.model_bridge.supported_architectures.modernbert_decoder.ModernBertDecoderArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for ModernBertDecoderForCausalLM models.

__init__(cfg: Any) → None

Initialize the ModernBERT Decoder architecture adapter.

component_mapping: ComponentMapping | None

setup_component_testing(hf_model: Any, bridge_model: Any = None) → None

Delegated attention computes rotary inside HF; nothing to wire.

supports_center_writing_weights = False

supports_fold_ln = False

uses_split_attention: bool

weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None