Skip to content

fix(table): assert default spec id when the default partition spec changes - #1834

Open
ryanworl wants to merge 1 commit into
apache:mainfrom
ryanworl:pr/update-spec-assert-default-spec-id
Open

fix(table): assert default spec id when the default partition spec changes#1834
ryanworl wants to merge 1 commit into
apache:mainfrom
ryanworl:pr/update-spec-assert-default-spec-id

Conversation

@ryanworl

Copy link
Copy Markdown
Contributor

What

UpdateSpec.BuildUpdates fenced spec evolution with assert-last-assigned-partition-id only. Two racing remove-only spec evolutions assign no new field ids, so both pass that assertion and both commit — last writer wins and the other spec change is silently lost.

This adds the missing requirement: AssertDefaultSpecID, pinned to the base default spec id, whenever the default partition spec changes.

Why

Java parity: UpdateRequirements.forUpdateTable registers AssertDefaultSpecID whenever a SetDefaultPartitionSpec update is present, alongside the last-assigned-partition-id assertion. With this in place, the second of two concurrent remove-only evolutions fails assert-default-spec-id instead of clobbering the first.

Tests

  • TestUpdateSpecBuildChanges now expects both requirements on add / remove / rename, with assert-default-spec-id pinning the base spec id.
  • go test ./table/... and golangci-lint run are clean.

Made with Cursor

…anges

UpdateSpec.BuildUpdates only sent assert-last-assigned-partition-id, so
two racing remove-only spec evolutions (which assign no new field ids)
could both succeed. Match Java UpdateRequirements.forUpdateTable: a
SetDefaultPartitionSpec update also registers AssertDefaultSpecID on
the base default spec id (see UpdateRequirements.Builder.update and
requireDefaultPartitionSpecNotChanged).

Co-authored-by: Cursor <cursoragent@cursor.com>
@ryanworl
ryanworl requested a review from zeroshade as a code owner August 13, 2026 20:21

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against Java's UpdateRequirements.forUpdateTable: it does register AssertDefaultSpecID whenever a SetDefaultPartitionSpec update is present, and this is exactly the gap that lets two racing remove-only spec evolutions (no new field ids assigned) both slip past assert-last-assigned-partition-id. The fix in table/update_spec.go:170-174 is correctly placed inside the DefaultPartitionSpec() != newSpec.ID() branch, so both remove-only paths (SetDefaultSpecUpdate to an existing spec id) and add paths are covered, and it pins to us.txn.tbl.Metadata().DefaultPartitionSpec() — the base table metadata — consistent with the neighboring AssertLastAssignedPartitionID call. Multi-operation transactions are fine too: requirementSemanticKey in transaction.go dedupes identical requirements by their JSON encoding. Full ./table/... suite passes on the PR head. One optional nit: a test that actually exercises the race (validate the new requirement against metadata whose default spec id has moved, expecting failure) would be nice, but assertDefaultSpecId.Validate is already covered in requirement_test.go, so I don't consider it blocking.

@zeroshade

Copy link
Copy Markdown
Member

This is approved, but it picked up a conflict when #1833 landed just ahead of it in today's merge queue. Could you rebase on latest main? Happy to merge once it's green again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants