Add Falcon-H1 parallel hybrid architecture adapter (FalconH1ForCausalLM)

[udapy]

Description

Adds TransformerBridge support for FalconH1ForCausalLM (TII Falcon-H1 family).

• Falcon-H1 is a parallel hybrid: every block runs GQA attention and a Mamba-2 mixer side-by-side on the same normalized input, then a SwiGLU MLP.
• Both branches are mapped as independently hookable streams so researchers can ablate one path and compare

Fixes # (issue)

// summary

Aspect	Choice
Block container	BlockBridge — input_layernorm + pre_ff_layernorm, real hook_resid_mid
Parallel branches	attn (GQA) + mamba (SSM2MixerBridge) on every block
Positional embeddings	Model-level rotary_emb + setup_component_testing
Mamba inner norm	Optional (mamba_rms_norm=false on released checkpoints)
Generation	Standard KV-cache path — is_stateful not set
Weight folding	weight_processing_conversions = {} (HF applies multipliers in passthrough forward)
verify_models	applicable_phases = [] — hybrid SSM precedent; parity gated by integration tests

Key design decision: no is_stateful

• With is_stateful=True, greedy generation diverges from HF after the first decode step,
• the bridge’s stateful loop uses the pure-Mamba cache_params convention, but Falcon-H1 (transformers ≥5.x) carries attention KV and Mamba state in one unified past_key_values / DynamicCache.
• The default generate path matches HF bit-for-bit.

Files changed (10 files)

File	Change
transformer_lens/model_bridge/supported_architectures/falcon_h1.py	New adapter
tests/unit/.../test_falcon_h1_adapter.py	Structural unit tests (47)
tests/integration/.../test_falcon_h1_adapter.py	Real-model parity tests (26)
supported_architectures/__init__.py	Registration 1/4
factories/architecture_adapter_factory.py	Registration 2/4
tools/model_registry/__init__.py	Registration 3/4 (HF_SUPPORTED_ARCHITECTURES, CANONICAL_AUTHORS_BY_ARCH)
tools/model_registry/generate_report.py	Registration 4/4
sources/transformers.py + sources/_bridge_builder.py	Mamba config attrs in both _HF_PASSTHROUGH_ATTRS lists
tools/model_registry/data/supported_models.json	6 canonical tiiuae/Falcon-H1-* entries (status 0)

Verification (local)

Model	Forward	Generation
tiiuae/Falcon-H1-Tiny-90M-Instruct	max_diff == 0.0	Bit-for-bit greedy
tiiuae/Falcon-H1-0.5B-Base	max_diff == 0.0	Bit-for-bit greedy

• Integration tests also cover HF delegation, parallel-branch hooks (both attn + mamba in the same block), attention and Mamba branch ablation, and SSM config propagation.

CI gating

• Integration tests are marked pytest.mark.slow only (same as test_nemotron_h_adapter.py).
• CI and make integration-test deselect them via -m "not slow".
• No skipif(CI) — redundant given current makefile/CI wiring.

Unit tests + TestRegistrySyncedWithFactory run in normal CI.

Tokenizer policy

• Verified on Falcon-H1-0.5B-Base: bos_token_id=None, no BOS prepended → default_prepend_bos correctly left unset.

Not in scope

• preprocess_weights() / compatibility-mode multiplier folding
• verify_models registry score updates (applicable_phases=[])

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

Checklist:

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
  • (No separate documentation page change)
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have not rewritten tests relating to key interfaces which would affect backward compatibility

Related

• Closes [Proposal] Add Falcon-H1 parallel hybrid adapter (FalconH1ForCausalLM) #1403
• Builds on patterns from [Proposal] Add Nemotron-H hybrid Mamba2-Transformer adapter #1402 / feat: add NemotronH hybrid Mamba2-Transformer architecture adapter #1434 (NemotronH hybrid adapter)
• Paper: https://arxiv.org/abs/2507.22448

[jlarson4]

This looks great! I had to do a conflict resolution for the supported models file, so once this CI run passes, I will get this merged. Thanks for your hard work @udapy

[udapy]

@jlarson4
Thank you for resolving supported model file conflicts.
Appreciate it.

[jlarson4]

@udapy No problem! Merging now