transformer_lens.model_bridge.supported_architectures.mpt module¶

MPT (MPTForCausalLM) adapter — ALiBi, fused Wqkv, weight-only LayerNorm, no biases.

class transformer_lens.model_bridge.supported_architectures.mpt.MPTArchitectureAdapter(cfg: Any)¶

Bases: ArchitectureAdapter

MPT adapter: ALiBi bias; all layers bias-free (no b_Q/b_K/b_V/b_O/b_in/b_out/ln bias).

bool_4d_mask_is_keep: bool = False¶

component_mapping: ComponentMapping | None¶

uses_split_attention: bool¶

validate_output_logits_transform() → None¶: Reject ambiguous remote-code logit scaling not used by integrated HF MPT.

weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None¶