Skip to content

parser: fold set-op keywords case-insensitively (mixed-case arms were dropped) - #55

Merged
chiradip merged 1 commit into
mainfrom
fix-mixed-case-setop-keywords
Jul 27, 2026
Merged

parser: fold set-op keywords case-insensitively (mixed-case arms were dropped)#55
chiradip merged 1 commit into
mainfrom
fix-mixed-case-setop-keywords

Conversation

@chiradip

@chiradip chiradip commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Problem

fold_set_operations decided the set operator by comparing the case-preserved token text against only the UPPER and lower spellings:

if (keyword == "UNION" || keyword == "union") { ... }
else if (keyword == "EXCEPT" || keyword == "except" || keyword == "MINUS" || keyword == "minus") { ... }
// and for INTERSECT: if (!(keyword == "INTERSECT" || keyword == "intersect")) break;

SQL keywords are case-insensitive, so a mixed-case spelling like Union / Intersect / ExCePt matched neither branch. The fold loop then breaks — and the operator plus the entire right operand are silently dropped:

Input Before After (correct)
SELECT 1 Union SELECT 2 lone SELECT 1 (right arm gone) UNION(SELECT 1, SELECT 2)
SELECT 1 UNION SELECT 2 Intersect SELECT 3 UNION(1, 2) — INTERSECT arm gone, wrong shape UNION(1, INTERSECT(2, 3))
SELECT 1 ExCePt SELECT 2 lone SELECT 1 EXCEPT(SELECT 1, SELECT 2)

This is a silently-wrong result: a legal multi-arm set query is truncated to its first branch with no diagnostic. Uppercase and all-lowercase both worked; only mixed case failed. Found by the repeatable frontend-audit's regression sweep of the INTERSECT-precedence set-op fold.

Fix

Match on the case-insensitive current_token_->keyword_id (the tokenizer already sets Keyword::UNION / INTERSECT / EXCEPT for any casing) instead of the case-preserved spelling. The original spelling is still copied into primary_text, so the node's displayed keyword keeps the user's casing.

MINUS is not a set-op keyword in this tokenizer (there is no Keyword::MINUS), so it is never a Keyword-typed token and never reached this Keyword-gated loop — the old "MINUS" string branch was dead code. Dropping it changes nothing (verified: ... users MINUS SELECT ... still parses exactly as before, with MINUS consumed as a table alias).

Tests

Adds three regression tests to test_precedence_regression.cpp:

  • MixedCaseUnionFoldsUnion keeps both arms
  • MixedCaseExceptFoldsExCePt folds as EXCEPT
  • MixedCaseIntersectBindsTighterThanUnion — mixed-case inner Intersect preserves both the fold and the precedence

Full suite: 37/37 green.

… dropped)

fold_set_operations matched the case-PRESERVED token text against only the
UPPER and lower spellings (keyword == "UNION" || keyword == "union", and
likewise for INTERSECT/EXCEPT). SQL keywords are case-insensitive, so a
mixed-case spelling such as `Union`/`Intersect`/`ExCePt` matched neither
arm: the fold loop broke, and the operator together with the entire right
operand were SILENTLY dropped. `SELECT 1 Union SELECT 2` parsed to a lone
SELECT 1; `SELECT 1 UNION SELECT 2 Intersect SELECT 3` dropped the INTERSECT
arm (and with it the correct precedence).

Match on the case-insensitive current_token_->keyword_id instead (the
tokenizer already sets Keyword::UNION/INTERSECT/EXCEPT for every casing).
The case-preserved value is still copied into primary_text, so the node's
displayed keyword keeps the user's original spelling.

MINUS is not a set-op keyword in this tokenizer (no Keyword::MINUS), so it is
never a Keyword-typed token and never reached this loop; the old
`keyword == "MINUS"` branch was dead and dropping it changes nothing
(verified: `... users MINUS SELECT ...` still parses as before, MINUS as an
alias).

Adds MixedCaseUnionFolds / MixedCaseExceptFolds /
MixedCaseIntersectBindsTighterThanUnion to test_precedence_regression.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FyKDAbMWGwiZq4iJx3imsg
@chiradip
chiradip merged commit bdb6b26 into main Jul 27, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant