transformer_lens.model_bridge.supported_architectures.mpt module¶

MPT (MPTForCausalLM) adapter — ALiBi, fused Wqkv, weight-only LayerNorm, no biases.

class transformer_lens.model_bridge.supported_architectures.mpt.MPTArchitectureAdapter(cfg: Any)¶

MPT adapter: ALiBi bias; all layers bias-free (no b_Q/b_K/b_V/b_O/b_in/b_out/ln bias).