parser: fold set-op keywords case-insensitively (mixed-case arms were dropped) - #55
Merged
Merged
Conversation
… dropped) fold_set_operations matched the case-PRESERVED token text against only the UPPER and lower spellings (keyword == "UNION" || keyword == "union", and likewise for INTERSECT/EXCEPT). SQL keywords are case-insensitive, so a mixed-case spelling such as `Union`/`Intersect`/`ExCePt` matched neither arm: the fold loop broke, and the operator together with the entire right operand were SILENTLY dropped. `SELECT 1 Union SELECT 2` parsed to a lone SELECT 1; `SELECT 1 UNION SELECT 2 Intersect SELECT 3` dropped the INTERSECT arm (and with it the correct precedence). Match on the case-insensitive current_token_->keyword_id instead (the tokenizer already sets Keyword::UNION/INTERSECT/EXCEPT for every casing). The case-preserved value is still copied into primary_text, so the node's displayed keyword keeps the user's original spelling. MINUS is not a set-op keyword in this tokenizer (no Keyword::MINUS), so it is never a Keyword-typed token and never reached this loop; the old `keyword == "MINUS"` branch was dead and dropping it changes nothing (verified: `... users MINUS SELECT ...` still parses as before, MINUS as an alias). Adds MixedCaseUnionFolds / MixedCaseExceptFolds / MixedCaseIntersectBindsTighterThanUnion to test_precedence_regression. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FyKDAbMWGwiZq4iJx3imsg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
fold_set_operationsdecided the set operator by comparing the case-preserved token text against only the UPPER and lower spellings:SQL keywords are case-insensitive, so a mixed-case spelling like
Union/Intersect/ExCePtmatched neither branch. The fold loop thenbreaks — and the operator plus the entire right operand are silently dropped:SELECT 1 Union SELECT 2SELECT 1(right arm gone)UNION(SELECT 1, SELECT 2)SELECT 1 UNION SELECT 2 Intersect SELECT 3UNION(1, 2)— INTERSECT arm gone, wrong shapeUNION(1, INTERSECT(2, 3))SELECT 1 ExCePt SELECT 2SELECT 1EXCEPT(SELECT 1, SELECT 2)This is a silently-wrong result: a legal multi-arm set query is truncated to its first branch with no diagnostic. Uppercase and all-lowercase both worked; only mixed case failed. Found by the repeatable frontend-audit's regression sweep of the INTERSECT-precedence set-op fold.
Fix
Match on the case-insensitive
current_token_->keyword_id(the tokenizer already setsKeyword::UNION/INTERSECT/EXCEPTfor any casing) instead of the case-preserved spelling. The original spelling is still copied intoprimary_text, so the node's displayed keyword keeps the user's casing.MINUSis not a set-op keyword in this tokenizer (there is noKeyword::MINUS), so it is never aKeyword-typed token and never reached thisKeyword-gated loop — the old"MINUS"string branch was dead code. Dropping it changes nothing (verified:... users MINUS SELECT ...still parses exactly as before, withMINUSconsumed as a table alias).Tests
Adds three regression tests to
test_precedence_regression.cpp:MixedCaseUnionFolds—Unionkeeps both armsMixedCaseExceptFolds—ExCePtfolds as EXCEPTMixedCaseIntersectBindsTighterThanUnion— mixed-case innerIntersectpreserves both the fold and the precedenceFull suite: 37/37 green.