transformer_lens.model_bridge.supported_architectures.switch_transformers module

Switch Transformers (SwitchTransformersForConditionalGeneration) adapter: T5 encoder-decoder with top-1-routed sparse-MoE feed-forwards. Blocks delegate wholesale (v5’s plain tensor protocol), FF is a delegated MoEBridge with an optional router.

class transformer_lens.model_bridge.supported_architectures.switch_transformers.SwitchTransformersArchitectureAdapter(cfg: Any)

Bases: T5ArchitectureAdapter

Architecture adapter for SwitchTransformersForConditionalGeneration models.

__init__(cfg: Any) → None

Initialize the Switch Transformers architecture adapter.

prepare_loading(model_name: str, model_kwargs: dict) → None

The google/switch-base-* repos ship pytorch_model.bin only; skip v5’s Hub-side safetensors auto-conversion (it needs a conversion PR).