transformer_lens.model_bridge.supported_architectures.hubert module

HuBERT architecture adapter.

Supports HubertModel (bare encoder) and HubertForCTC (with CTC head). Encoder blocks are structurally identical to BERT (post-LN by default, pre-LN when do_stable_layer_norm=True).

class transformer_lens.model_bridge.supported_architectures.hubert.HubertArchitectureAdapter(cfg: Any)

Bases: ArchitectureAdapter

Architecture adapter for HuBERT audio models.

HubertForCTC nests HubertModel under a ‘hubert.’ prefix; prepare_model() detects this and adjusts component paths.

prepare_loading(model_name: str, model_kwargs: dict) → None

Propagate HuBERT-specific HF config attributes to bridge config.

Prevents silent-default bugs where adapter reads from bridge config but the attribute was never propagated from HF config.

prepare_model(hf_model: Any) → None

Detect HubertForCTC (has ‘hubert.’ prefix) and add CTC head.