transformer_lens.model_bridge.sources.vllm.overlays.base module

Base class for vLLM overlays.

class transformer_lens.model_bridge.sources.vllm.overlays.base.AdapterOverlay

Bases: object

A vLLM overlay for a single architecture family.

capture_specs() is called BEFORE LLM(...) to register the dot-paths and output widths the plugin should pre-allocate GPU buffers for. The plugin reads these during Worker.load_model so capture hooks are present when torch.compile traces the model.

nonfiring_hooks() enumerates hooks that vLLM’s fused kernels prevent from firing; surfaced as a single boot-time warning.

capture_specs(hf_config: Any) → Dict[str, Tuple[str, int]]

Return {canonical_hook_name: (dot_path_in_vllm_model, output_width)}.

nonfiring_hooks() → List[str]

Canonical hook names that vLLM’s fused kernels cannot expose.