transformer_lens.model_bridge.sources.vllm.internals module

Single chokepoint for vLLM internal API access.

vLLM rearranges its internal class paths every 4-6 weeks. Centralize every llm.llm_engine.… walk here so version drift is patched in one place.

Validated against ``vllm==0.20.2`` (also the version pinned in demos/vLLM_Bridge_Integration_Test.ipynb). The patched-load-model path in plugin.py and the hf_config walk below have been confirmed on that release; newer releases may move attributes — re-validate before bumping.

transformer_lens.model_bridge.sources.vllm.internals.extract_hf_config(llm: Any) → Any

Return the HF config that vLLM loaded the model from.

transformer_lens.model_bridge.sources.vllm.internals.segment_by_request(model_runner: Any) → Any

Return (query_offsets_cpu, req_ids); request i = rows offsets[i]:offsets[i+1].

Only valid inside a forward. req_ids is row order, NOT submission order — join on it. Reads model_runner.query_start_loc (backend-agnostic; the runner builds it before any attention backend, whereas FlashInfer buries its offsets in an opaque C++ wrapper), falling back to attn metadata for backends that surface them directly. (None, req_ids) ⇒ caller single-slices.

transformer_lens.model_bridge.sources.vllm.internals.verify_hook_coverage(llm: Any) → None

Raise if any configured capture hook installed on NO rank.

Per-rank absence is legal (pipeline-parallel ranks own layer subsets), so hook installation skips missing modules instead of raising — this boot-time check restores the fail-loud contract: a hook absent everywhere is a broken overlay dot-path and would otherwise read back as silent zeros.