Skip to content

Return type op fast path results without the full fulfillment machinery [-0.83% avg]#30

Draft
xmakro wants to merge 2 commits into
perf/base-0703from
perf/typeop-fastpath-hoist
Draft

Return type op fast path results without the full fulfillment machinery [-0.83% avg]#30
xmakro wants to merge 2 commits into
perf/base-0703from
perf/typeop-fastpath-hoist

Conversation

@xmakro

@xmakro xmakro commented Jul 4, 2026

Copy link
Copy Markdown
Owner

Query type ops check their fast paths (trivial sizedness proofs, alias free normalizations) inside the closure passed to scrape_region_constraints. Every fast path hit therefore still paid for an inference snapshot, an ObligationCtxt with a boxed fulfillment engine, RefCell take and reset cycles for region constraints, and the region constraint scraping itself, all over empty data. MIR type check issues several type ops per statement, normalizing the place and rvalue types and proving sizedness of the place type, and the overwhelming majority of these hit a fast path.

Fast path results are final. They register no obligations and produce no region constraints, so the fast path is now checked once at the top of fully_perform and the result returned directly, skipping all of that machinery. The two inner fast path checks become dead and are removed. resolve_vars_if_possible is kept on the returned value because the normalize fast path can return values that still contain inference variables, matching what scrape_region_constraints did before.

Local measurement: from scratch ThinLTO stage2 with jemalloc, instructions:u, full rustc-perf suite, both sides built from scratch.

  • primary benchmarks: mean -0.83% across all cells; check profile mean -1.34%, debug -0.71%, opt -0.45%
  • check full on large primaries: image -3.99%, ripgrep -3.84%, cargo -3.81%, syn -3.48%, cranelift-codegen -3.42%, hyper -3.13%, diesel -2.16%
  • statement heavy stress benchmarks: ucd -7.1%, unicode-normalization -6.5%, tuple-stress -6.1%, deep-vector -5.4%, coercions -5.3%
  • 256 cells improved by at least 0.25%, 1 regressed (a single known noisy incr-patched cell)

tests/ui/borrowck, tests/ui/nll, tests/ui/sized-hierarchy, tests/ui/associated-types and tests/ui/higher-ranked pass (1752 tests, 0 failures). x check compiler is clean.


Update: review found that skipping scrape_region_constraints also skipped its role as the unconditional drain point for region state pending on the InferCtxt. The old-solver hidden type equation in insert_hidden_type registers region constraints eagerly and could leave partial state pending when it fails partway (the ? in relate_opaques bails before register_goals scrapes). Reworked in two commits:

  1. relate_opaques now rolls back partial hidden type equations via commit_if_ok, making "no region state pending between type ops" a real invariant.
  2. The hoisted fast path debug-asserts that invariant (plus not-in-snapshot) via a new InferCtxt::has_pending_region_state helper instead of relying on it silently. Release behavior and the perf numbers above are unchanged.

Note: the next solver's solver_region_constraint_storage (-Zassumptions-on-binders) is cross-op by design and intentionally outside both the scrape and the assertion.

Full tests/ui (21304 tests) passes on a stage1 build with debug-assertions = true, exercising the new assertions.

xmakro and others added 2 commits July 26, 2026 05:57
Equating a hidden type in insert_hidden_type registers region
constraints on the inference context eagerly and returns only the
non-region goals. When the equation fails partway, those partial
region constraints stayed pending on the inference context: nothing
drained them until the next query type op scraped and attributed them
to itself, whichever op that happened to be. Roll them back instead.
This makes it an invariant that no region state is pending on the
inference context between type ops.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Query type ops check their fast paths (trivial sizedness proofs, alias
free normalizations) inside the closure passed to
scrape_region_constraints. Every fast path hit therefore still paid for
an inference snapshot, an ObligationCtxt with a boxed fulfillment
engine, RefCell take and reset cycles for region constraints, and the
region constraint scraping itself, all over empty data. MIR type check
issues several type ops per statement, and the overwhelming majority of
these hit a fast path.

Fast path results are final: they register no obligations and produce
no region constraints, so the fast path is now checked once at the top
of fully_perform and the result returned directly. The two inner fast
path checks become dead and are removed. resolve_vars_if_possible is
kept on the returned value, matching what scrape_region_constraints did
before.

Skipping the machinery also skips its role as the unconditional drain
point for region state pending on the inference context. With the
hidden type equation rollback in the previous commit, nothing registers
such state outside a type op; the fast path debug-asserts that
invariant (and not-in-snapshot, matching the machinery's own
assertions) via the new InferCtxt::has_pending_region_state helper,
which is documented to stay in sync with what
scrape_region_constraints drains. Borrowck's leftover-constraints debug
check polices the same invariant after every op.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@xmakro
xmakro force-pushed the perf/typeop-fastpath-hoist branch from 5190b9a to 03969d2 Compare July 26, 2026 05:57
@xmakro xmakro changed the title Return type op fast path results without the full fulfillment machinery Return type op fast path results without the full fulfillment machinery [-0.83% avg] Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant