transformer_lens.model_bridge.supported_architectures.deepseek_v3 module

DeepSeek V3 architecture adapter.

Supports DeepSeek V3 and DeepSeek-R1 models (both use DeepseekV3ForCausalLM).

class transformer_lens.model_bridge.supported_architectures.deepseek_v3.DeepSeekV3ArchitectureAdapter(cfg: Any)

Bases: DeepSeekMLAFamilyArchitectureAdapter

Architecture adapter for DeepSeek V3 / R1 models.

Uses RMSNorm, MLA with compressed Q/KV projections, partial RoPE, MoE on most layers (dense MLP on first few), and no biases. HF builds the two-stage LoRA Q path only when q_lora_rank is set and a single q_proj otherwise (ai-sage/GigaChat3-10B-A1.8B), so both Q paths are mapped optional.

component_mapping: ComponentMapping | None

q_lora_optional: bool = True

uses_split_attention: bool

weight_processing_conversions: Dict[str, ParamProcessingConversion | str] | None