transformer_lens.tools.model_registry.relevancy module¶
Relevancy scoring for unsupported architectures.
Computes a composite relevancy score (0-100) for each architecture gap, combining demand (model count), usage (downloads), and benchmarkability (smallest model size).
- Formula:
relevancy = 0.45 * demand + 0.35 * usage + 0.20 * benchmarkability
- transformer_lens.tools.model_registry.relevancy.compute_relevancy_score(model_count: int, total_downloads: int, min_param_count: int | None, max_model_count: int, max_downloads: int) float¶
Compute composite relevancy score for an architecture gap.
- Parameters:
model_count – Number of models using this architecture.
total_downloads – Aggregate downloads across all models of this architecture.
min_param_count – Parameter count of the smallest model (None if unknown).
max_model_count – Max model count across all gap architectures (for normalization).
max_downloads – Max total downloads across all gap architectures (for normalization).
- Returns:
Relevancy score from 0 to 100.
- transformer_lens.tools.model_registry.relevancy.compute_scores_for_gaps(gaps: list[dict]) list[dict]¶
Compute relevancy scores for a list of architecture gap dicts.
Mutates each gap dict in-place by adding a ‘relevancy_score’ field, then returns the list sorted by score descending.
- Parameters:
gaps – List of gap dicts with ‘architecture_id’, ‘total_models’, ‘total_downloads’, and ‘min_param_count’ fields.
- Returns:
The same list, sorted by relevancy_score descending (total_models as tiebreaker).