Handle null and null-like characters sorting before spaces in pad-spaced columns - #38147
Draft
peterdukelarsen wants to merge 2 commits into
Draft
Conversation
Restores the broader live test suite on top of the probe interface PR: case-insensitive traversal, LIKE wildcard and metacharacter data, multibyte and emoji keys, ULID and UUID primary keys, collation behavior, stale table statistics, and sargability via Handler_read session counters.
Tests default to utf8mb4_bin, with utf8mb4_general_ci for the case-insensitive cases, matching the collations snapshot splitting accepts. New tests pin the edge cases that rule out other collations: the Czech ch contraction silently drops prefixes, and NUL keys and eszett expansions return prefixes that sort below the range they came from. Under PAD SPACE collations a key continuing with a character below space sorts below its own prefix, so bare prefix bounds hide those keys. Range bounds are now NUL-padded to sit at the infimum of the prefix extension space, with the original prefix kept alongside to exclude the exact key. A None lower bound means the start of the key space.
peterdukelarsen
added a commit
that referenced
this pull request
Aug 11, 2026
### Motivation Split out some testing from #38022 to keep that PR manageable. Part of: [SS-97](https://linear.app/materializeinc/issue/SS-97) ### Description Adds testing for more diverse character types, collations, and for asserting searches use the B-tree rather than scanning the full table. ### For Context I've caught a couple of bugs in the later PR from discussion with Marty and a review with Dennis: #38093. 1. For PAD SPACE collations certain characters will sort before a shorter string. Our algorithm still works cleanly for that case at a high level, just throwing out the lower bounds which for utf8mb4_bin are all things like null, carriage return, and tab. The one tricky spot we needed to handle correctly was the upper bound, for which we're able to append NUL characters to get behavior like what we would have expected initially. I tried fixing this directly initially, but thought the fix was more complex than the perf issue was worth (see [here](#38147)). 2. Certain collations can't be supported with this approach because multiple characters in a row can in fact sort differently from it's prefix. There are tests here highlighting that and then most of the tests have been moved to utf8mb4_bin to match the first collation we intend to support.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove these sections if your commit already has a good description!
Motivation
Why does this change exist? Link to a GitHub issue, design doc, Slack
thread, or explain the problem in a sentence or two. A reviewer who has
no context should understand why after reading this section.
If this implements or addresses an existing issue, it's enough to link to that:
Closes
Fixes
etc.
Description
What does this PR actually do? Focus on the approach and any non-obvious
decisions. The diff shows the code --- use this space to explain what the
diff can't tell a reviewer.
Verification
How do you know this change is correct? Describe new or existing automated
tests, or manual steps you took.