transformer_lens.benchmarks.component_benchmark module

Component-level benchmarks to compare individual model pieces.

This module provides benchmarks for comparing individual model components (attention, MLP, embedding, etc.) between HuggingFace and TransformerBridge.

transformer_lens.benchmarks.component_benchmark.benchmark_all_components(bridge, hf_model, atol: float = 0.0001, rtol: float = 0.0001, reference_model: Any | None = None) → BenchmarkResult

Comprehensive benchmark of all model components.

This function systematically tests every component in the model using the architecture adapter to find and compare equivalent components.

Parameters:

  • bridge – The TransformerBridge model

  • hf_model – The HuggingFace model to compare against

  • atol – Absolute tolerance for comparison

  • rtol – Relative tolerance for comparison

  • reference_model – Optional reference model (unused, for API consistency)

Returns:

BenchmarkResult summarizing all component tests