Emit SEMI join inner conditions in the ON clause - #347
Draft
JoshDreamland wants to merge 1 commit into
Draft
Conversation
JoshDreamland
force-pushed
the
fix-semi-join-inner-conds
branch
2 times, most recently
from
August 14, 2026 17:16
60b40d1 to
068b462
Compare
JoshDreamland
marked this pull request as draft
August 14, 2026 17:20
JoshDreamland
force-pushed
the
fix-semi-join-inner-conds
branch
from
August 17, 2026 15:55
068b462 to
a03e55b
Compare
A SEMI or ANTI join uses its ON clause as the match test, so every condition that references the inner relation must appear there. ClickHouse's LEFT SEMI JOIN exposes one arbitrary matching row to the WHERE clause. Testing an inner condition against that row can miss another qualifying match and return too few rows. Do not rely on PostgreSQL's `is_pushed_down` flag to place these conditions. PostgreSQL sets it for semijoin conditions because their placement does not affect its executor, but the distinction matters to a remote ClickHouse join. Move every condition that references the inner relation into the remote ON clause. ClickHouse versions before 26.3 reject non-equality ON conditions that span both sides when join_use_nulls is enabled; 23.x rejects them regardless of that setting. Keep affected joins local on those servers. Resolve the user mapping as the executor would, and disable pushdown if no mapping is available. Add `EXISTS` and `NOT EXISTS` tests that verify results whether the join runs remotely or locally.
JoshDreamland
force-pushed
the
fix-semi-join-inner-conds
branch
from
August 17, 2026 20:55
a03e55b to
f49ddb0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A SEMI or ANTI join uses its ON clause as the match test, so every condition that references the inner relation must appear there. ClickHouse's LEFT SEMI JOIN exposes one arbitrary matching row to the
WHEREclause. Testing an inner condition against that row can miss another qualifying match and return too few rows.Do not rely on PostgreSQL's
is_pushed_downflag to place these conditions. PostgreSQL sets it for semijoin conditions because their placement does not affect its executor, but the distinction matters toa remote ClickHouse join. Move every condition that references the inner relation into the remote
ONclause.ClickHouse versions before 26.3 reject non-equality ON conditions that span both sides when join_use_nulls is enabled; 23.x rejects them regardless of that setting. Keep affected joins local on those servers. Resolve the user mapping as the executor would, and disable pushdown if no mapping is available.
Add
EXISTSandNOT EXISTStests that verify results whether the join runs remotely or locally.Claude Summary
An
EXISTSwhose correlation condition references both sides was deparsed into theWHEREclause instead of the join'sONclause, silently returning too few rows. ClickHouse'sLEFT SEMI JOINsurfaces the column values of one arbitrary matching row, so the condition interrogated that row rather than asking whether any matching row satisfied it.EXISTS-only shapeNo NULLs are involved — every column above is
NOT NULL, verified viaattnotnull. It is purely the scope of the existential quantifier, demonstrable in ClickHouse with no nullable types at all: