Pr extend coverage - #38157
Draft
def- wants to merge 3 commits into
Draft
Conversation
DELETE, UPDATE and INSERT ... SELECT run on the coordinator, which holds a write lock on the target table across the read and the write. Every such statement therefore serializes against every other one on that table, and the coordinator loop is occupied for the duration. This sequences them from the session task instead, using optimistic concurrency control. The selection is read through an internal subscribe, which streams the mutation's diffs directly rather than peeking every matched row and recomputing them. The write is submitted at the timestamp the diffs were observed at, and the group committer refuses it if another writer got there first. A refusal is a retry with a fresh snapshot, not a lost update, and the retry budget is `max_occ_retries`. Concurrency is bounded by a semaphore of `max_concurrent_occ_writes` permits, acquired before the read holds so that queued operations do not pin compaction on their read dependencies while they wait. That parameter carries a domain constraint of at least 1, since zero permits would leave every read-then-write waiting out its `statement_timeout`, which is what the new `U32InRange` constraint expresses. `statement_timeout` is enforced in one place, the `select!` that owns the whole operation, because every phase can block: permit acquisition, linearization against a far-future `as_of`, and the retry loop itself. The path is off by default and gated by `enable_adapter_frontend_occ_read_then_write`, which is read once at startup and fixed for the life of the process. A mixed-mode window would be unsound: the lock-based path excludes concurrent writers, the OCC path detects them afterwards, and the two do not synchronize. The `recursion_limit` sqllogictest golden drops from 418 to 388 UNION branches, because sequencing a read-then-write from the session task adds frames to the planning path that test measures. Large mutations get faster because the subscribe streams diffs, small ones get slower because each installs a dataflow where the old path used a fast-path peek. That trade is deliberate and recorded in the design doc.
The OCC path linearizes its `as_of` before subscribing, but the subscribe then follows Persist, and Persist runs ahead of the timestamp oracle. Group commit appends to the txns shard first, which makes the write readable and advances the table's upper, and calls `oracle.apply_write` only after. A read-then-write whose selection consolidates to empty inside that window concluded from state no oracle-timestamped read could reach yet, and returned immediately because it had nothing to write. A strict-serializable read issued after that response then still saw the rows the response said were not there. Real time orders the read last and no serial order explains the history. Only the zero-row exits are exposed. A write is linearized for free, because group commit answers the writer after applying the timestamp, and the coordinator's path takes its read timestamp from the oracle and so cannot observe state the oracle has not reached. `OccOutcome` now separates the two cases rather than encoding "nothing to write" as a `Committed` with no timestamp. `NoRowsMatched` carries the timestamp emptiness was concluded at, and the caller waits for the oracle to reach it before responding. The timestamp is absent only for a selection that reads no persisted state, which has nothing to linearize against. Waiting is what the existing `ensure_read_linearized` does, and it is preferable to applying the write timestamp ourselves: forcing the oracle forward off another writer's timestamp would stall subsequent reads of unrelated collections until they caught up. The `group_commit_before_apply_write` failpoint holds the window open, and is the same one a second `environmentd` opens on its own with no ordering against local Persist visibility. Two mzcompose tests come with this. `workflow_test_occ_zero_row_write_linearization` is the regression test. It parks the winning DELETE inside the window, confirms from a serializable read that the delete is visible in Persist and from a strict-serializable read that the oracle cannot serve it yet, and only then runs the UPDATE that has to report zero rows. Attempts where the subscribe reported progress before the append, told apart by the write conflict count, prove nothing and are retried. `workflow_mixed_mode_read_then_write` is a guard rather than a regression test. It runs as its own nightly step rather than from the composition's default workflow, which iterates workloads rather than workflows, so the file joins the linter's list of compositions whose extra workflows are run separately. The dyncfg is sampled once per process, so a rolling restart can leave one instance sequencing on the coordinator behind in-process write locks and another under OCC. Those locks never excluded a second process, so this is not new, but what keeps the two from corrupting a row is the catalog fence, and nothing states that. The test asserts the fence keeps the commit windows disjoint, so it goes red if that stops holding.
def-
force-pushed
the
pr-extend-coverage
branch
from
August 11, 2026 11:04
a6ce43d to
538d19d
Compare
def-
force-pushed
the
pr-extend-coverage
branch
from
August 11, 2026 12:18
538d19d to
c1f3987
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Targeting #37924