Skip to content

Keep keyword before leading-dot float literal (issue #601)#868

Merged
andialbrecht merged 1 commit into
andialbrecht:masterfrom
deepakganesh78:fix/issue601-between-leading-dot-float
Jul 25, 2026
Merged

Keep keyword before leading-dot float literal (issue #601)#868
andialbrecht merged 1 commit into
andialbrecht:masterfrom
deepakganesh78:fix/issue601-between-leading-dot-float

Conversation

@deepakganesh78

Copy link
Copy Markdown
Contributor

Fixes #601.

Problem

Parsing a BETWEEN bound written as a float without a leading zero mis-tokenizes the statement:

sqlparse.parse("... where l_discount between .03 and .06")

BETWEEN gets demoted from a keyword to a Name, the WHERE body is grouped wrong, and .03 / .06 stop being standalone number tokens.

Cause

The lexer rule that supports schema . name member access:

(r'[A-ZÀ-Ü]\w*(?=\s*\.)', tokens.Name),  # 'Name'.

matches any word followed by whitespace + a period — including a keyword sitting in front of a float literal like .03. So between matched this rule and was tagged as a Name.

Fix

Add a (?!\d) negative lookahead so the rule ignores a period that begins a numeric literal:

(r'[A-ZÀ-Ü]\w*(?=\s*\.(?!\d))', tokens.Name),

schema.name / schema . name / a.b.c member access is unchanged; only the keyword-before-float case is corrected.

Tests

  • test_between_leading_dot_float_issue601BETWEEN stays a keyword, AND stays a keyword, and .03 / .06 are Number.Float tokens (upper and lower case).
  • test_keyword_before_qualified_name_still_grouped — guards that qualified-name grouping keeps working.

Full suite: 494 passed, 2 xfailed, 1 xpassed.

The SQL_REGEX rule that tags a word followed by a period as a Name (to
support ``schema . name`` member access) also matched a keyword sitting in
front of a float literal written without a leading zero, e.g. the ``BETWEEN``
in ``x BETWEEN .03 AND .06``. As a result BETWEEN was demoted to a Name, the
statement was grouped incorrectly, and the bounds were no longer standalone
number tokens.

Add a ``(?!\d)`` negative lookahead so the Name-before-period rule ignores a
period that starts a numeric literal, while ``schema . name`` grouping is
preserved. Adds regression tests covering both the fix and the member-access
behavior.
@codecov

codecov Bot commented Jul 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.16%. Comparing base (111b35c) to head (267f571).
⚠️ Report is 8 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #868      +/-   ##
==========================================
+ Coverage   97.13%   97.16%   +0.03%     
==========================================
  Files          31       31              
  Lines        3663     3704      +41     
  Branches      328      331       +3     
==========================================
+ Hits         3558     3599      +41     
  Misses         63       63              
  Partials       42       42              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@andialbrecht
andialbrecht merged commit a194d31 into andialbrecht:master Jul 25, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error paring BETWEEN .. AND .. clause when decimal's '0' is omitted

2 participants