fix: fence artifact namespaces across restart - #456
Open
wchwawa wants to merge 2 commits into
Open
Conversation
Bind every root to one immutable provider-neutral object namespace across etcd routing, Holt fences, recovery records, CLI, SDK, and Python clients. Preserve legacy fence and outbox bytes behind an explicit one-way adoption path. Keep temporary S3 read failures retryable and redacted, and add independent real-etcd/RustFS gates for wrong-prefix rejection, kill/reopen epochs, outage recovery, and a deterministic materials workflow. Signed-off-by: wchwawa <wch19961116@gmail.com>
Signed-off-by: wchwawa <wch19961116@gmail.com>
wchwawa
marked this pull request as ready for review
August 15, 2026 13:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue
Fixes #450
Summary
The root cause was not RustFS or etcd health. Metadata ownership and artifact storage had no shared durable identity, so a restart could combine a valid metadata root with a different but healthy object prefix. This change makes that identity part of the authority contract at every mutation and recovery boundary.
Startup now requires a namespace-marker read. Existing roots require explicit operator-confirmed adoption, roots owned by one shard must use one namespace, and the workspace RPC protocol advances to v3.
Scope
Code Contract (Code Changes Only)
docs/development/code_contract.md.errors.rsand carry stable error kinds when crossing package boundaries.RootId -> LogicalShardIdaffinity, one active writer per shard, owner-epoch fencing, and explicit shard-local atomicity; no split root or cross-shard transaction is implied.nokv-agent, routing and workflows innokv-client, and the concrete backend plus MCP wiring in thenokvCLI.Claims and Evidence
Validation
cargo fmt --all -- --check: passcargo clippy --workspace --all-targets -- -D warnings: passcargo test --target-dir /private/tmp/nokv-object-namespace-fence-target -q --workspace: passobject-namespace-recovery: pass on a clean Ubuntu runner with retained terminal evidencenokv-workspace: pass, including the independent real-etcd E2 fence gate and RustFS volume cleanupThe first clean-runner attempt exposed a Linux-only harness bug before NoKV started: RustFS runs as non-root UID/GID
10001:10001, while the host bind directory was root-owned. A focused RED test now requires a Docker-managed volume and fail-fast container-exit diagnostics. The corrected gate passed without changing product recovery semantics.Contributor Sign-off
Signed-off-bytrailer.