transformer_lens.model_bridge.supported_architectures.phi module

Phi architecture adapter.

class transformer_lens.model_bridge.supported_architectures.phi.PhiArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for Phi models.

__init__(cfg: Any) → None

Initialize the Phi architecture adapter.

Parameters:

cfg – The configuration object.

component_mapping: ComponentMapping | None
default_cfg: dict[str, Any] = {'use_fast': False}
setup_component_testing(hf_model: Any, bridge_model: Any = None) → None

Set up rotary embedding references for Phi component testing.

Phi uses RoPE (Rotary Position Embeddings). We set the rotary_emb reference on all attention bridge instances for component testing.

Parameters:

  • hf_model – The HuggingFace Phi model instance

  • bridge_model – The TransformerBridge model (if available, set rotary_emb on actual instances)

uses_split_attention: bool
weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None