transformer_lens.config package

Submodules

• transformer_lens.config.hooked_transformer_config module
  • HookedTransformerConfig
    • HookedTransformerConfig.NTK_by_parts_factor
    • HookedTransformerConfig.NTK_by_parts_high_freq_factor
    • HookedTransformerConfig.NTK_by_parts_low_freq_factor
    • HookedTransformerConfig.NTK_original_ctx_len
    • HookedTransformerConfig.act_fn
    • HookedTransformerConfig.attention_dir
    • HookedTransformerConfig.attn_only
    • HookedTransformerConfig.attn_scale
    • HookedTransformerConfig.attn_scores_soft_cap
    • HookedTransformerConfig.attn_types
    • HookedTransformerConfig.checkpoint_index
    • HookedTransformerConfig.checkpoint_label_type
    • HookedTransformerConfig.checkpoint_value
    • HookedTransformerConfig.d_vocab_out
    • HookedTransformerConfig.decoder_start_token_id
    • HookedTransformerConfig.dtype
    • HookedTransformerConfig.eps
    • HookedTransformerConfig.experts_per_token
    • HookedTransformerConfig.final_rms
    • HookedTransformerConfig.from_checkpoint
    • HookedTransformerConfig.from_dict()
    • HookedTransformerConfig.gated_mlp
    • HookedTransformerConfig.init_mode
    • HookedTransformerConfig.init_weights
    • HookedTransformerConfig.initializer_range
    • HookedTransformerConfig.is_layer_norm_activation()
    • HookedTransformerConfig.load_in_4bit
    • HookedTransformerConfig.model_name
    • HookedTransformerConfig.n_devices
    • HookedTransformerConfig.n_params
    • HookedTransformerConfig.norm_topk_prob
    • HookedTransformerConfig.normalization_type
    • HookedTransformerConfig.num_experts
    • HookedTransformerConfig.original_architecture
    • HookedTransformerConfig.output_logits_soft_cap
    • HookedTransformerConfig.parallel_attn_mlp
    • HookedTransformerConfig.post_embedding_ln
    • HookedTransformerConfig.relative_attention_max_distance
    • HookedTransformerConfig.relative_attention_num_buckets
    • HookedTransformerConfig.rotary_adjacent_pairs
    • HookedTransformerConfig.rotary_base
    • HookedTransformerConfig.rotary_base_local
    • HookedTransformerConfig.rotary_dim
    • HookedTransformerConfig.rotary_scaling_factor
    • HookedTransformerConfig.scale_attn_by_inverse_layer_idx
    • HookedTransformerConfig.seed
    • HookedTransformerConfig.set_seed_everywhere()
    • HookedTransformerConfig.tie_word_embeddings
    • HookedTransformerConfig.to_dict()
    • HookedTransformerConfig.tokenizer_name
    • HookedTransformerConfig.tokenizer_prepends_bos
    • HookedTransformerConfig.trust_remote_code
    • HookedTransformerConfig.ungroup_grouped_query_attention
    • HookedTransformerConfig.unwrap()
    • HookedTransformerConfig.use_NTK_by_parts_rope
    • HookedTransformerConfig.use_attn_in
    • HookedTransformerConfig.use_attn_scale
    • HookedTransformerConfig.use_hook_mlp_in
    • HookedTransformerConfig.use_hook_tokens
    • HookedTransformerConfig.use_local_attn
    • HookedTransformerConfig.use_normalization_before_and_after
    • HookedTransformerConfig.use_qk_norm
    • HookedTransformerConfig.use_yarn_rope
    • HookedTransformerConfig.window_size
    • HookedTransformerConfig.yarn_attention_factor
    • HookedTransformerConfig.yarn_beta_fast
    • HookedTransformerConfig.yarn_beta_slow
    • HookedTransformerConfig.yarn_factor
    • HookedTransformerConfig.yarn_original_max_position_embeddings
• transformer_lens.config.transformer_bridge_config module
  • TransformerBridgeConfig
    • TransformerBridgeConfig.__init__()
    • TransformerBridgeConfig.head_dim
• transformer_lens.config.transformer_lens_config module
  • TransformerLensConfig
    • TransformerLensConfig.act_fn
    • TransformerLensConfig.attn_only
    • TransformerLensConfig.d_head
    • TransformerLensConfig.d_mlp
    • TransformerLensConfig.d_model
    • TransformerLensConfig.d_vocab
    • TransformerLensConfig.default_prepend_bos
    • TransformerLensConfig.device
    • TransformerLensConfig.dtype
    • TransformerLensConfig.eps
    • TransformerLensConfig.experts_per_token
    • TransformerLensConfig.final_rms
    • TransformerLensConfig.from_dict()
    • TransformerLensConfig.gated_mlp
    • TransformerLensConfig.layer_norm_folding
    • TransformerLensConfig.n_ctx
    • TransformerLensConfig.n_heads
    • TransformerLensConfig.n_key_value_heads
    • TransformerLensConfig.n_layers
    • TransformerLensConfig.normalization_type
    • TransformerLensConfig.num_experts
    • TransformerLensConfig.positional_embedding_type
    • TransformerLensConfig.to_dict()
    • TransformerLensConfig.unwrap()
    • TransformerLensConfig.use_attn_result
    • TransformerLensConfig.use_split_qkv_input
    • TransformerLensConfig.uses_rms_norm

Module contents

Configuration classes for TransformerLens.