Skip to content

feat(like): support LIKE over Binary arrays with byte semantics - #9497

Open
moshap-firebolt wants to merge 1 commit into
developfrom
like-on-binary
Open

feat(like): support LIKE over Binary arrays with byte semantics#9497
moshap-firebolt wants to merge 1 commit into
developfrom
like-on-binary

Conversation

@moshap-firebolt

@moshap-firebolt moshap-firebolt commented Aug 19, 2026

Copy link
Copy Markdown

Summary

LIKE was restricted to Utf8 inputs — Like::return_dtype rejected Binary haystacks. But SQL LIKE over binary/bytea is well defined as a byte-wise match, and the matcher already runs over the haystack bytes. This adds LIKE support for Binary arrays with byte semantics.

Changes

  • Like::return_dtype accepts Binary input for case-sensitive LIKE. ILIKE over Binary is still rejected, since case folding is only well defined over text.
  • LikePattern regex fallback compiles with .unicode(false) when the haystack is Binary, so _/. match a single byte and %/.* span arbitrary bytes (including invalid UTF-8). Utf8 haystacks keep Unicode codepoint semantics — unchanged. The literal fast paths (Eq/StartsWith/EndsWith/Contains) were already byte-based.

Semantics

For a multi-byte haystack aЖb (Ж = 2 bytes):

pattern Utf8 (codepoint) Binary (byte)
a_b match no match
a__b no match match

% and the literal patterns match byte-for-byte on Binary, including across invalid-UTF-8 byte sequences.

Tests

  • test_like_binary_byte_semantics — covers the byte-vs-codepoint distinction on multi-byte and invalid-UTF-8 data.
  • test_ilike_over_binary_is_rejected — asserts ILIKE over Binary errors.
  • All existing Utf8 LIKE/ILIKE tests continue to pass (no behavior change for Utf8).

LIKE was restricted to Utf8 inputs: `Like::return_dtype` rejected Binary
haystacks. But SQL LIKE over binary/bytea is well defined as a byte-wise
match, and the matcher already runs over the haystack bytes.

- `Like::return_dtype` accepts Binary input for case-sensitive LIKE. ILIKE
  over Binary is still rejected, since case folding is only well defined
  over text.
- The regex fallback in `LikePattern` compiles with `.unicode(false)` when
  the haystack is Binary, so `_`/`.` match a single byte and `%`/`.*` span
  arbitrary bytes (including invalid UTF-8). Utf8 haystacks keep Unicode
  codepoint semantics, unchanged. The literal fast paths
  (Eq/StartsWith/EndsWith/Contains) were already byte-based.

Utf8 LIKE behavior is unchanged. Added tests covering the byte-vs-codepoint
distinction on multi-byte and invalid-UTF-8 data, and the ILIKE-over-Binary
rejection. On `aЖb` (Ж = 2 bytes), `a_b` matches under Utf8 (one codepoint)
but not under Binary (one byte); `a__b` matches under Binary.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@connortsui20 connortsui20 added the changelog/feature A new feature label Aug 19, 2026
@codspeed-hq

codspeed-hq Bot commented Aug 19, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 12.87%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 2 improved benchmarks
✅ 1954 untouched benchmarks
⏩ 517 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime words_gather_scalar[65536] 9.4 µs 8.2 µs +14.04%
Simulation compress_fsst[(500, 64, 8)] 585.9 µs 524.5 µs +11.71%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing like-on-binary (721b719) with develop (b81420e)

Open in CodSpeed

Footnotes

  1. 517 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@connortsui20 connortsui20 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

This causes an issue with how we rewrite LIKE, which assumes that its input is only UTF8.

struct LikeStatsRewrite;
impl StatsRewriteRule for LikeStatsRewrite {
fn scalar_fn_id(&self) -> ScalarFnId {
Like.id()
}
fn falsify(
&self,
expr: &BoundExpression,
ctx: &StatsRewriteCtx<'_>,
) -> VortexResult<Option<BoundExpression>> {
let like_options = expr.as_::<Like>();
if like_options.negated || like_options.case_insensitive {
return Ok(None);
}
let Some(pattern) = expr.child(1).as_opt::<Literal>() else {
return Ok(None);
};
let Some(pattern) = pattern.as_utf8().value() else {
return Ok(None);
};
let source = expr.child(0);
Ok(match LikeVariant::from_str(pattern) {
Some(LikeVariant::Exact(text)) => {
min(source, ctx)
.zip(max(source, ctx))
.map(|(source_min, source_max)| {
or(
gt(source_min, lit(text.as_ref())),
lt(source_max, lit(text.as_ref())),
)
})
}
Some(LikeVariant::Prefix(prefix)) => {
let Some(successor) = prefix.to_string().increment().ok() else {
return Ok(None);
};
min(source, ctx)
.zip(max(source, ctx))
.map(|(source_min, source_max)| {
or(
gt_eq(source_min, lit(successor)),
lt(source_max, lit(prefix.as_ref())),
)
})
}
None => None,
})
}
}

This approximately translates the LIKE to min > UTF8_LITERAL OR max < UTF8_LITERAL, where min and max are stats with the same dtype as the array/column. So once we allow binary columns as input to LIKE, we will try to compare a binary scalar (min/max) with a Utf8 literal scalar.

I think that there are a few possible solutions here, and to be honest neither of them seem the most ideal:

  • We could just not use stats rewriting / pruning for binary LIKE, and this is probably the simplest thing
  • We could coerce the Utf8 pattern into Binary when we see this pattern, and then the comparison comes for free (we are trying to move away from implicit coercions in Vortex)
  • I also thought about requiring exact-typed patterns (we could say that you must prefix a LIKE on a binary with a b, so col_a LIKE b"prefix%", but that is a larger change.

Interested in what you are thinking?

/// Unicode mode, matching identically to a `&str` regex on valid UTF-8 input. For Binary haystacks
/// (`byte_mode`) Unicode is disabled, so `.`/`.*` operate on single bytes and match arbitrary byte
/// sequences (including invalid UTF-8), giving SQL byte semantics for `_`/`%`.
fn regex_like(pattern: &str, case_insensitive: bool, byte_mode: bool) -> VortexResult<Regex> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this take a struct of options

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is byte_mode ascii mode?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants