transformer_lens.model_bridge.supported_architectures.bitnet module

BitNet b1.58 (BitNetForCausalLM) adapter: llama layout plus attn/ffn sub-layer RMSNorms (attn_sub_norm reapplied by an adapter-local attention bridge).

class transformer_lens.model_bridge.supported_architectures.bitnet.BitNetArchitectureAdapter(cfg: Any)

Bases: LlamaArchitectureAdapter

Architecture adapter for BitNetForCausalLM models.

__init__(cfg: Any) → None

Initialize the BitNet architecture adapter.

applicable_phases: list[int] = [1, 2, 4]

component_mapping: ComponentMapping | None

uses_split_attention: bool

weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None