fix(table): assert default spec id when the default partition spec changes - #1834
fix(table): assert default spec id when the default partition spec changes#1834ryanworl wants to merge 1 commit into
Conversation
…anges UpdateSpec.BuildUpdates only sent assert-last-assigned-partition-id, so two racing remove-only spec evolutions (which assign no new field ids) could both succeed. Match Java UpdateRequirements.forUpdateTable: a SetDefaultPartitionSpec update also registers AssertDefaultSpecID on the base default spec id (see UpdateRequirements.Builder.update and requireDefaultPartitionSpecNotChanged). Co-authored-by: Cursor <cursoragent@cursor.com>
zeroshade
left a comment
There was a problem hiding this comment.
Verified against Java's UpdateRequirements.forUpdateTable: it does register AssertDefaultSpecID whenever a SetDefaultPartitionSpec update is present, and this is exactly the gap that lets two racing remove-only spec evolutions (no new field ids assigned) both slip past assert-last-assigned-partition-id. The fix in table/update_spec.go:170-174 is correctly placed inside the DefaultPartitionSpec() != newSpec.ID() branch, so both remove-only paths (SetDefaultSpecUpdate to an existing spec id) and add paths are covered, and it pins to us.txn.tbl.Metadata().DefaultPartitionSpec() — the base table metadata — consistent with the neighboring AssertLastAssignedPartitionID call. Multi-operation transactions are fine too: requirementSemanticKey in transaction.go dedupes identical requirements by their JSON encoding. Full ./table/... suite passes on the PR head. One optional nit: a test that actually exercises the race (validate the new requirement against metadata whose default spec id has moved, expecting failure) would be nice, but assertDefaultSpecId.Validate is already covered in requirement_test.go, so I don't consider it blocking.
|
This is approved, but it picked up a conflict when #1833 landed just ahead of it in today's merge queue. Could you rebase on latest main? Happy to merge once it's green again. |
What
UpdateSpec.BuildUpdatesfenced spec evolution withassert-last-assigned-partition-idonly. Two racing remove-only spec evolutions assign no new field ids, so both pass that assertion and both commit — last writer wins and the other spec change is silently lost.This adds the missing requirement:
AssertDefaultSpecID, pinned to the base default spec id, whenever the default partition spec changes.Why
Java parity:
UpdateRequirements.forUpdateTableregistersAssertDefaultSpecIDwhenever aSetDefaultPartitionSpecupdate is present, alongside the last-assigned-partition-id assertion. With this in place, the second of two concurrent remove-only evolutions failsassert-default-spec-idinstead of clobbering the first.Tests
TestUpdateSpecBuildChangesnow expects both requirements on add / remove / rename, withassert-default-spec-idpinning the base spec id.go test ./table/...andgolangci-lint runare clean.Made with Cursor