Support limiting prefixes probed for MySQL to constrain runtime - #38164
Open
peterdukelarsen wants to merge 1 commit into
Open
Support limiting prefixes probed for MySQL to constrain runtime#38164peterdukelarsen wants to merge 1 commit into
peterdukelarsen wants to merge 1 commit into
Conversation
peterdukelarsen
changed the title
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 11, 2026 17:26
f3a2461 to
90dc1b3
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 11, 2026 17:38
90dc1b3 to
f3a000c
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
8 times, most recently
from
August 11, 2026 19:40
48cfbb4 to
f071ccd
Compare
peterdukelarsen
changed the title
peterdukelarsen
marked this pull request as ready for review
August 11, 2026 19:48
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 11, 2026 20:15
f071ccd to
1d68e03
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 11, 2026 22:41
1d68e03 to
ecefb45
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 11, 2026 23:02
ecefb45 to
f5d757b
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 11, 2026 23:35
f5d757b to
9304f11
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 11, 2026 23:42
9304f11 to
bcc67c6
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 12, 2026 00:12
bcc67c6 to
1b0861e
Compare
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 12, 2026 01:07
1b0861e to
a097f1c
Compare
Bound the prefix partitioner to a per-table probe query budget scaled to the estimated row count, 2500 requests per billion rows by default via the mysql_source_snapshot_partition_requests_per_billion_rows dyncfg, with a floor of 256 so small tables can afford their handful of splits. Probes run sequentially, so this caps the wall-clock that partitioning can add to snapshot start in proportion to the snapshot work it optimizes. An exhausted budget stops splitting early and leaves coarser buckets, never incorrect ones. Adds a mock test asserting the partitioner never exceeds its budget and still returns a valid ordered boundary list when truncated.
peterdukelarsen
force-pushed
the
pl/mysql-partition-request-budget
branch
from
August 12, 2026 01:23
a097f1c to
c33afc9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Part of SS-97.
In degenerate cases with high cardinality of first characters, many short prefixes (i.e. "a", "aa", "aaa", "aaaa"...), or unexpected bugs, we want to ensure partitioning terminates in a reasonable amount of time.
Description
Bounds the partitioner's probe traffic with a request budget derived from the table's estimated row count, via the new
mysql_source_snapshot_partition_requests_per_billion_rowsdyncfg (floored at 256 requests). When the budget runs out the walk stops splitting and emits coarser partitions, so cost is capped without losing correctness. The budget also serves as the hard bound on the walk for degenerate key spaces.Adds mock tests for budget-bounded probing and for termination on non-advancing prefixes.
🤖 Generated with Claude Code