feat(table): allow RewriteFiles to add delete files

[fallintoplace]

Summary

Allow rewrite operations to add position and equality delete files atomically.

Why

A rewrite can produce delete files together with replacement data files, but RewriteFiles currently rejects delete files as additions.

What changed

Add delete-file routing to RewriteFiles and a transaction replacement path that validates and appends delete files in the same snapshot. Validate file types, paths, references, format version, and equality field IDs.

Tests

• go test ./table -run TestRewriteFiles_AddsDeleteFile -count=1
• go test ./table -run TestRewriteFiles -count=1

[zeroshade]

The spec-level validation here is thorough (DV uniqueness per data file, applicability by sequence number, surviving pos-delete suppression, v2/v3 gating, dropped-equality-field tolerance), and the test coverage is genuinely strong. I built the branch and ran the full ./table suite — all green. A few concerns before this leaves draft:

1. No commit-time conflict validation for the new delete-file invariants. All of the new checks in Transaction.replaceFiles (transaction.go: "deletion vector for data file %s already exists and must be replaced", "would survive", referenced-data-file liveness in validateAddedDeletionVectorTargets) run against meta.currentSnapshot() at staging time only. RewriteFiles.Commit (rewrite_files.go:376-378) still registers rewriteValidator(r.dataFilesToDelete) — which only guards the rewritten data files — and registers nothing at all for a pure delete-file rewrite (dataFilesToDelete empty), while skipDefaultValidator suppresses the overwrite isolation validator. So two concurrent transactions that each rewrite pos-deletes into a DV for the same referenced data file can both commit on retry, producing two DVs for one data file (a v3 spec violation); similarly a concurrent RowDelta can land a new delete on the DV's target that the staged "must be replaced" check would have rejected. I think the rewrite validator needs to also cover the data files referenced by added DVs (and be registered even when no data files are rewritten), re-checking the single-DV/no-surviving-delete invariants against concurrent snapshots the way validateNoNewDeletesForRewrittenFiles does.

2. RewriteFiles.DataSequenceNumber has no upper bound. It's validated >= 0 (rewrite_files.go:180) but nothing prevents passing a value greater than the new snapshot's sequence number, which would stamp ADDED data entries with a sequence number above the snapshot's own — breaking the spec invariant and making future deletes mis-apply. I'd validate seq <= meta.nextSequenceNumber() in replaceFiles where you already compute addedDataSequenceNumber.

3. Membership-only sequence validation for added pos-deletes is a documented footgun. For pos-delete rewrites, an added file's explicit sequence number only has to be one of the removed files' sequence numbers (transaction.go, "expected one of the replaced delete sequence numbers"). If a caller merges content from sources with seqs {3, 7} into one output and assigns 3, deletes silently stop applying to data files with seq in (3, 7] — data resurrection with no error. The DeleteFileAddition doc acknowledges grouping isn't representable in the API, but given the failure mode is silent, it may be worth either representing the group explicitly or at least calling this out more loudly. Question rather than a demand — what's your take?

Minor: writeAddedDeleteManifest (snapshot_producers.go:996) skips the writerClosed guard that writeAddedManifest uses; it works because ManifestWriter.Close is idempotent, but matching the surrounding pattern would be more consistent. Also, the for path := range setToDelete { if _, replacement := setToAdd[path]; ... } branch in replaceFiles is dead code — the entries loop already rejects any setToAdd path referenced by the table, so a path can never be in both sets.

Since this is a draft I'm leaving comments only; happy to re-review once the conflict-validation story for delete additions is settled.