transformer_lens.model_bridge.buffer_restore module

Repair non-persistent buffers destroyed by meta-device loading.

transformers 5.x replaces every non-persistent buffer with torch.empty_like (modeling_utils._move_missing_keys_from_meta_to_device) and only restores rotary tables for modules exposing original_inv_freq. Remote code predating that attribute silently keeps uninitialized memory — internlm2 loads a table of zeros, which collapses RoPE to the identity transform.

The defect is invisible to the phase benchmarks: they compare the bridge against an HF reference loaded the same way, so both sides are wrong together and agree. Shared by adapters (prepare_model) and the benchmark’s HF reference so the two can never drift apart.

transformer_lens.model_bridge.buffer_restore.restore_rotary_inv_freq(hf_model: Any) → int

Recompute invalid rotary inv_freq tables in place; returns the count.

Only tables failing _is_valid_inv_freq() are touched, so legitimately scaled tables are never clobbered.