From 743942815e43763635d8c5bdbc8c76929db1615f Mon Sep 17 00:00:00 2001 From: Samuel Laferriere <9342524+samlaf@users.noreply.github.com> Date: Fri, 7 Aug 2026 17:52:12 -0400 Subject: [PATCH] feat(tee-cli)!: assemble the founding validator set from the harvest MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The founding reorder's assemble half: `manifest assemble` now derives the founding validator set from the harvested cohort instead of shipping the transitional empty set #87 left behind. With this the founding order is whole — init → up → harvest → assemble → configure — and `network_id` pins the founders' keys by construction. ## What assemble does now - Pairs each harvested box (`inputs/harvest/.json`) with an authored withdrawal-credentials address and the box's current IP from `nodes/.json`, and emits the completed `[[validators]]` through `summit genesis set-validators`. The empty-set prepend is gone. - Re-verifies every archived quote against the policy compiled from `inputs/measurements.json` before pinning anything. This is the second of the two gates: root_key admission gates privacy, assemble-time DCAP gates consensus membership — summit never talks to the custodian, so an unverified founding pubkey could otherwise vote from outside a TEE. - Refuses a harvest carrying a repeated node or consensus pubkey: summit keys validator accounts by node pubkey, so a duplicate silently collapses the set, and a shared consensus key is accidental-equivocation material. ## Withdrawal credentials are a list, not a node-name map `inputs/founder-withdrawal-credentials.json` is now a JSON array of addresses, one per founding node, paired to the boxes in node-name order at assemble time. Keying it by cohort node name forced the file to be authored after `up`, which is backwards — the founders' addresses are a fact about the founders, not about the infrastructure. `manifest init` scaffolds it as the fourth authored input, and `--founders N` fills in `0x00…0` placeholders, which is all a throwaway founding needs. The count check that replaces the exact name match runs twice: at harvest before any quote is fetched, and again at assemble. The pairing is positional and nothing downstream can tell a swapped pair from an intended one, so assemble logs each pairing as it builds the set. ## up - `--count` is optional with `--network`: the authored credentials are the founding set, so they size the cohort. A contradicting `--count` is an error rather than a silent override. - The preflight image-pin check reads the authored `inputs/measurements.json` rather than the assembled manifest and policy. Provisioning precedes assembly now, so at `up` time the inputs are all a network directory holds. The success banner walks harvest → assemble → configure instead of pointing at the ceremony. ## Renames, docs, and the throwaway path - `measurement-policy.json` → `measurement-policy-bootstrap.json`: the file is only the *founding* allowlist — what `bootstrap_policy_hash` pins and registry genesis storage compiles from. The live policy is the registry contract's state, which the authority can mutate after genesis. - `manifest init --force` re-authors inputs for a re-found, matching the `assemble --force` that already existed. - `tee/docs/runbook-devnet.md` is the copy-pasteable throwaway recipe (init → provision → harvest → assemble → configure → smoke-test → down), and `networks/.gitignore` ignores `tmp-*/` wholesale so a scratch founding never dirties git. - `example-devnet` is documented as a schema example, not a runnable founding: no artifact set can be assembled without a live harvest. - Every path the CLIs print is absolute, so it resolves from any cwd. The seismic_node Pulumi program path and its default stack config move to `tee/cli/common/repo.py`, letting `manifest init` point the founder at the stack config to review (vhd_blob_url, operator_ip_cidr) without common importing the network CLI. - The dev stack's `vhd_blob_url` moves to a 2026-08-07 image, built after the summit-key-holder unit landed (seismic-images #48). Breaking for any existing network directory: the policy filename and the credentials file shape both changed, and `up --network` now expects `inputs/`. Nothing is founded on the old layout. Tests stay offline throughout (verifier subprocess and summit emitter mocked). Next: configure's per-boot IP splice, the launch-time pubkey and block-0 assertions, and deleting the genesis ceremony. --- tee/README.md | 171 +++-- tee/cli/common/manifest.py | 588 ++++++++++++++++-- tee/cli/common/repo.py | 23 + tee/cli/common/tests/test_manifest.py | 297 ++++++++- tee/cli/network/harvest.py | 138 ++-- tee/cli/network/orchestrator.py | 220 ++++--- tee/cli/network/tests/test_harvest.py | 42 +- tee/cli/network/tests/test_orchestrator.py | 135 ++-- tee/docs/runbook-devnet.md | 185 ++++++ tee/networks/.gitignore | 6 + tee/networks/README.md | 82 ++- ...json => measurement-policy-bootstrap.json} | 0 tee/pulumi/seismic_node/Pulumi.dev.yaml | 2 +- 13 files changed, 1494 insertions(+), 395 deletions(-) create mode 100644 tee/cli/common/repo.py create mode 100644 tee/docs/runbook-devnet.md rename tee/networks/example-devnet/{measurement-policy.json => measurement-policy-bootstrap.json} (100%) diff --git a/tee/README.md b/tee/README.md index 5dea7fe4..0e55954a 100644 --- a/tee/README.md +++ b/tee/README.md @@ -11,8 +11,8 @@ This README is intentionally architectural. By the end you should understand the layering and the two CLIs split by audience — `seismic-tee-node` (operator-facing: act on your own node) and `seismic-tee-network` (Seismic-internal: found a network — provision a -cohort and run the genesis ceremony) — what each does vs. what the -operator does, and what's still pending cleanup. +cohort, harvest its founding keys, assemble its identity) — what each +does vs. what the operator does, and what's still pending cleanup. ## What gets deployed @@ -29,9 +29,11 @@ for its [InitConfig](https://github.com/SeismicSystems/enclave/blob/seismic/crat never provisions; it consumes a descriptor of an already-running node. VHD upload to the Azure image registry is a seismic-images concern (`make push-azure-*`), not this tool. -- **For `seismic-tee-network genesis-ceremony`:** the `summit` binary on - PATH (its `genesis` subcommands). (The summit genesis defaults to the - artifact-set copy beside `--manifest`.) +- **For founding (`seismic-tee-network harvest` / `manifest assemble`):** + the enclave repo's `verify-quote` on PATH — DCAP verification of the + harvested founding quotes — plus, for assemble's derivations, `summit` + (its `genesis` subcommands), `seismic-reth` (`genesis-hash`), and + `seismic-measurement-admission`. - **For `seismic-tee-network up`** (cohort provisioning): the `pulumi` CLI on PATH. The `seismic_node` project pins ephemeral bootstrap stacks to the operator-local `file://~` backend; durable shared networks use a separate @@ -62,15 +64,16 @@ The split is "act on my own node" vs "bring a whole network into existence." Only the first is ever external, so it's its own CLI; the second is Seismic-internal. The boundary between them is the **node descriptor** file: `seismic-tee-network` *produces* descriptors (by -provisioning) and *consumes* them (genesis ceremony), while `seismic-tee-node` -only consumes one to reach an already-running node. +provisioning) and *consumes* them (harvest, assemble, configure), while +`seismic-tee-node` only consumes one to reach an already-running node. | CLI | Command | Run by | Purpose | |---|---|---|---| | `seismic-tee-node` | `configure` | any operator, on first boot | POST the node TOML to tdx-init. Reads the node address from `--node `. (Attestation verification is disabled pending attested-tls — see below.) | | `seismic-tee-network` | `up` / `down` | Seismic, internal | Provision / tear down a cohort of TDX nodes, one independent Pulumi stack each (drives Pulumi via the Automation API). | -| `seismic-tee-network` | `genesis-ceremony` | Seismic, once per network | Gather the cohort's summit pubkeys, build `genesis.toml`, POST it to each node. One-shot: `send_genesis` only exists in the pre-genesis node API. | -| `seismic-tee-network` | `manifest` | Seismic | Assemble / validate the network manifest. | +| `seismic-tee-network` | `harvest` | Seismic, once per network | Collect + DCAP-verify the founding cohort's summit keys into `inputs/harvest/` — the provenance `manifest assemble` pins the validator set from. | +| `seismic-tee-network` | `manifest` | Seismic | Scaffold (`init`) / `assemble` / `validate` a network directory. `assemble` pins the harvested founding set and mints `network_id`. | +| `seismic-tee-network` | `genesis-ceremony` | — | Unused: the assembled summit genesis already carries the validator set, so summit boots straight from the delivered file and nothing waits on `send_genesis`. Kept only until its deletion lands. | Only `seismic-tee-node` is cloud-agnostic and **never wraps Pulumi**; `seismic-tee-network` is the one allowed to. A **node descriptor** is a @@ -94,8 +97,8 @@ the same audience line: - **Public (operator):** `seismic-tee-node` + the `pulumi/seismic_node/` program — the primitives for configuring and provisioning *one* node. - **Private (this repo):** `seismic-tee-network` + `pulumi/playground/` - — founding a *network* (orchestrating a cohort, the genesis ceremony), - which only Seismic does. + — founding a *network* (orchestrating a cohort, harvesting and pinning + its founding keys), which only Seismic does. The package layout mirrors this split: `cli/node/` + `cli/common/` + `pulumi/seismic_node/` are the public half, `cli/network/` + @@ -138,40 +141,40 @@ artifact-set layout); tdx-init writes them for reth's `--chain` and summit's `--genesis-path`. You never run `seismic-tee-network` — that's Seismic-internal network creation. -### Creating a new network (genesis ceremony) +### Creating a new network (founding) The network-shape decisions live entirely in step 1's authored inputs; -everything after is derivation (`manifest assemble`) or per-node plumbing. -A node joining later re-runs only steps 2–3 (via the operator CLI — see -above); the manifest assembly and the ceremony are one-time founding acts. -A committed worked example lives in `networks/example-devnet/` — a -template identity that can also found a throwaway test devnet (see +everything after is harvesting facts from the live cohort, derivation +(`manifest assemble`), or per-node plumbing. A node joining later re-runs +only steps 2 and 5 (via the operator CLI — see above); the harvest and +the manifest assembly are one-time founding acts. A committed example of +the resulting directory lives in `networks/example-devnet/` — it +documents the artifact-set shape, not a runnable founding (see `networks/README.md`). +The walkthrough below teaches the workflow; the copy-pasteable recipe — +parameterized, with the gitignored `tmp-*` naming for throwaway +networks, a smoke test, and teardown — is +[docs/runbook-devnet.md](docs/runbook-devnet.md). + ```bash # 0. (Upstream) Build the image in seismic-images (CI or `make build`) # and publish it + its measurements. This tool does not build. -# 1. Create the network directory: `init` scaffolds the three authored -# inputs (reth genesis, summit genesis params, measurements) under inputs/; -# you edit them; then `assemble --dir` derives the artifact set -# (manifest, injected genesis, completed summit genesis, promoted -# policy) into the directory's top level. Everything top-level is -# hash-pinned by the manifest; everything under inputs/ is provenance. -# Edits go to the *inputs* + re-assemble — the manifest itself is pure -# output, never hand-edited. -# Commit the directory for any network that matters: it is everything -# needed to (re)configure, join, or debug the network later, and its -# manifest is the network's immutable identity. assemble needs -# `seismic-reth` on PATH for the offline genesis-hash gate and `summit` -# for the genesis config digest. +# 1. Create the network directory: `init` scaffolds the four authored +# inputs (reth genesis, summit genesis params, measurements, the +# founders' withdrawal credentials) under inputs/ and you edit them. +# --founders N fills the credentials with N placeholder addresses, which +# is all a throwaway needs; a real founding writes the founders' own. +# The artifact set is derived later (step 4) — after the cohort exists, +# because the founding validator set is harvested from it. uv run seismic-tee-network manifest init tee/networks/devnet-3 \ --reth-genesis ../seismic-reth/crates/seismic/chainspec/res/genesis/dev.json \ --measurements ../seismic-images/build/measurements.json \ - --measurement-id seismic_2026-06-11.abc123.vhd + --measurement-id seismic_2026-06-11.abc123.vhd \ + --founders 2 # → edit tee/networks/devnet-3/inputs/summit-genesis.toml # (namespace, …) -uv run seismic-tee-network manifest assemble tee/networks/devnet-3 # 2. Provision the cohort — one independent Pulumi stack per node — and # capture each node's descriptor. Shared settings inherit from the dev @@ -179,16 +182,45 @@ uv run seismic-tee-network manifest assemble tee/networks/devnet-3 # from it (→ dev-bootstrap-node), so each node's stack, resource group, # VM, and DNS record are dev-bootstrap-node-. # --network ties the cohort to the network directory: it cross-checks the -# config's vhd_blob_url basename against the measurement policy's -# measurement_id records, refusing a stale image pin before any resource -# exists (name check only — attestation is the cryptographic gate), and -# writes the descriptors into /nodes/ (gitignored: infra state, not -# identity). Without --network they land in a shared gitignored -# descriptors/ dir next to the Pulumi project; --out-dir overrides either. -uv run seismic-tee-network up --count 2 --network tee/networks/devnet-3 +# config's vhd_blob_url basename against the measurement_id stamped into +# inputs/measurements.json, refusing a stale image pin before any +# resource exists (name check only — attestation is the cryptographic +# gate), and writes the descriptors into /nodes/ (gitignored: infra +# state, not identity). Without --network they land in a shared +# gitignored descriptors/ dir next to the Pulumi project; --out-dir +# overrides either. The cohort size comes from the withdrawal +# credentials authored in step 1 — they are the founding set — so +# --count is only needed without --network. +uv run seismic-tee-network up --network tee/networks/devnet-3 # → tee/networks/devnet-3/nodes/dev-bootstrap-node-{1,2}.json -# 3. Configure the whole cohort in one command: exactly one genesis node +# 3. Harvest the founding keys. Each box boots identity-free: its +# summit-key-holder generates summit keypairs in RAM and serves +# {pubkeys, quote} until the box takes its config POST. Harvest polls +# every holder, fetches its pubkeys plus a TDX quote over a fresh +# per-box nonce, DCAP-verifies each quote against the intended image +# measurements (verify-quote on PATH), and archives the verified facts +# under inputs/harvest/. Any anomaly burns the whole harvest — a box +# that already took a POST, a failed verification, repeated pubkeys: +# re-found (down + fresh up), never retry around it. +uv run seismic-tee-network harvest tee/networks/devnet-3 + +# 4. Assemble the artifact set (manifest, injected reth genesis, completed +# summit genesis, promoted bootstrap policy) into the directory's top +# level: re-verifies the archived quotes against the compiled policy, +# pairs each harvested box with its authored withdrawal credentials, +# fills the validator set into the summit genesis via summit's own +# emitter, and mints network_id. Everything top-level is hash-pinned by +# the manifest; everything under inputs/ is provenance. Edits go to the +# *inputs* + re-assemble — the manifest itself is pure output, never +# hand-edited. Commit the directory for any network that matters: it is +# everything needed to (re)configure, join, or debug the network later, +# and its manifest is the network's immutable identity. Needs +# seismic-reth, summit, seismic-measurement-admission, and verify-quote +# on PATH. +uv run seismic-tee-network manifest assemble tee/networks/devnet-3 + +# 5. Configure the whole cohort in one command: exactly one genesis node # (mints root_key) plus N joiners pointed at it (each fetches root_key # from node 1 over the attested handshake); every first-boot disk wipe # is watched in parallel. On a greenfield cohort this runs in two stages @@ -201,29 +233,11 @@ uv run seismic-tee-network configure \ --genesis tee/networks/devnet-3/nodes/dev-bootstrap-node-1.json \ --join tee/networks/devnet-3/nodes/dev-bootstrap-node-2.json \ --manifest tee/networks/devnet-3/network-manifest.json - -# 4. Run the genesis ceremony once: builds genesis.toml from the cohort -# (needs the `summit` binary on PATH) and fans it out to every summit. -# Waits for cohort readiness itself in two stages: each node's reth must -# serve the manifest's genesis hash as block 0, then every node's Summit -# pubkeys must be readable. `configure` normally waits for root-key bootstrap -# and encrypted-disk initialization; if that watch was skipped, interrupted, -# or followed by service recovery, this barrier may still take several -# minutes. Each stage has a 15-minute readiness timeout; the reth stage -# reuses configure's per-node disk progress display and pauses its timeout -# while provisioning is active. Elapsed and remaining time are shown while -# waiting, and the command can run straight after step 3. -# One-shot: send_genesis only exists in the pre-genesis node API, so this -# can never be re-run against a live network. --summit-genesis defaults -# to the artifact-set copy beside --manifest (verified against the -# manifest's summit.genesis_config_digest); --node defaults to the -# descriptors in the nodes/ dir beside it. -uv run seismic-tee-network genesis-ceremony \ - --manifest tee/networks/devnet-3/network-manifest.json ``` -After step 3 each node is up; after step 4 they produce blocks. RPC is at -`https:///rpc` (the `fqdn` from each descriptor). +After step 5 each node is up and produces blocks: summit boots straight +from the delivered genesis, which already carries the founding validator +set. RPC is at `https:///rpc` (the `fqdn` from each descriptor). ### The measurement-admission pipeline @@ -241,8 +255,9 @@ seismic-images: make measure │ register + event log (build artifact, kept for audit) ▼ assemble: promote [rust: seismic-measurement-admission promote] - │ measurement-policy.json — the human-reviewed allowlist, exactly - │ the admission-schema registers (Azure v1: pcr4/pcr9/pcr11) + │ measurement-policy-bootstrap.json — the human-reviewed + │ allowlist, exactly the admission-schema registers + │ (Azure v1: pcr4/pcr9/pcr11) ▼ assemble: compile + inject [rust: seismic-measurement-admission compile] │ admission IDs + registry genesis-storage slots, written verbatim @@ -253,7 +268,10 @@ assemble: pin + render ├─ measurements.bootstrap_policy_hash = SHA-256(policy bytes) (python) └─ render network-manifest.json; network_id = SHA-256(bytes) (python) ▼ -validate / up --network — re-run the same derivations as acceptance gates +validate — re-runs the same derivations as acceptance gates (and + up --network checks the image pin against the measurements input + before provisioning; harvest + assemble verify the founding quotes + against the promoted policy) ▼ chain: MeasurementRegistry live from block 0 — no init transaction; empty storage fails closed @@ -375,11 +393,28 @@ spawns Lima to compute a security-critical value itself. ## Other operations -### Delete a VM +### Tearing down + +`seismic-tee-network down` destroys stacks in parallel — each stack's +VM, disks, IP, NSG, and DNS record together — removes the stack, and +deletes its descriptor. Pass the same `--network` (and, if used, +`--stack-prefix`) as `up` so it finds the descriptors under +`/nodes/`: + +```bash +# whole cohort +uv run seismic-tee-network down --count 2 --network tee/networks/devnet-3 +# or specific node(s) +uv run seismic-tee-network down --stack dev-bootstrap-node-2 \ + --network tee/networks/devnet-3 +``` -VM teardown is a provisioning concern — `pulumi destroy` against that -node's stack removes the VM, disks, IP, NSG, and DNS record together. -There is no CLI delete path. +`pulumi destroy --stack ` (from `pulumi/seismic_node/`) tears down +the same resources — the orchestrator and the plain CLI are front-ends +over the same state — but leaves the stack registration and descriptor +behind; `down` cleans up all three. For a throwaway network, also +`rm -rf` its network directory afterward — a founding that was never +meant to persist shouldn't leave a committed-looking identity around. ### Building images diff --git a/tee/cli/common/manifest.py b/tee/cli/common/manifest.py index 7437552f..85855e7c 100644 --- a/tee/cli/common/manifest.py +++ b/tee/cli/common/manifest.py @@ -28,26 +28,36 @@ (the accepted admission IDs are a per-network fact); eth.genesis_hash summit-genesis.toml the complete summit genesis every node - boots from: the input with - eth_genesis_hash filled and a validators - set; summit.genesis_config_digest - measurement-policy.json Flashbots-compatible measurement + boots from: the input completed with + eth_genesis_hash and the founding + validator set pinned from the harvest; + summit.genesis_config_digest + measurement-policy-bootstrap.json + Flashbots-compatible measurement allowlist promoted from the raw measurements; bootstrap_policy_hash Each artifact is its input with derived fields filled in at assemble time; -the raw measurements become measurement-policy.json because promotion is a -format transformation. +the raw measurements become the bootstrap policy because promotion is a +format transformation. `assemble` also reads the cohort descriptors under +nodes/ (runtime infra state, written by `up --network`) for each founding +validator's IP — delivered in the genesis file but excluded from its +config digest, so IPs never enter network_id. Usage (one directory per network: `init` gathers the authored inputs — the only command that takes loose files — then `assemble`/`validate` operate on -the directory): +the directory; between `init` and `assemble` the founding cohort is +provisioned and harvested, since assemble pins the harvested validator +set): uv run python -m tee.cli.common.manifest init tee/networks/seismic-devnet-3 \ --reth-genesis dev.json \ --measurements ../seismic-images/build/measurements.json \ - --measurement-id seismic_2026-06-11.abc123.vhd - # edit tee/networks/seismic-devnet-3/inputs/summit-genesis.toml: + --measurement-id seismic_2026-06-11.abc123.vhd --founders 4 + # edit tee/networks/seismic-devnet-3/inputs/summit-genesis.toml and + # inputs/founder-withdrawal-credentials.json, then: + # seismic-tee-network up --network tee/networks/seismic-devnet-3 --count N + # seismic-tee-network harvest tee/networks/seismic-devnet-3 uv run python -m tee.cli.common.manifest assemble tee/networks/seismic-devnet-3 uv run python -m tee.cli.common.manifest validate tee/networks/seismic-devnet-3 """ @@ -58,6 +68,7 @@ import json import logging import re +import shutil import subprocess import sys import tempfile @@ -69,7 +80,9 @@ from eth_utils import keccak +from tee.cli.common.descriptor import load_descriptor, require from tee.cli.common.logging_setup import setup_logging +from tee.cli.common.repo import DEFAULT_STACK_CONFIG logger = logging.getLogger(__name__) @@ -94,21 +107,39 @@ # a second one in Python. DEFAULT_ADMISSION_BIN = "seismic-measurement-admission" -# Summit's node binary; its `genesis digest` subcommand computes -# summit.genesis_config_digest. The digest is SHA-256 over summit's -# domain-prefixed SSZ serialization of the complete genesis — summit's own -# definition of chain identity (its P2P and signing domains derive from it) — -# so deploy shells out to the one implementation instead of mirroring the -# SSZ layout in Python. +# Summit's node binary. Its `genesis digest` subcommand computes +# summit.genesis_config_digest: SHA-256 over summit's domain-prefixed SSZ +# serialization of the complete genesis — summit's own definition of chain +# identity (its P2P and signing domains derive from it). Its `genesis +# set-validators` subcommand emits the completed genesis the digest is +# computed over. Both are shell-outs to the one implementation instead of +# mirroring the SSZ layout / canonical rendering in Python. DEFAULT_SUMMIT_BIN = "summit" +# The DCAP verifier from the enclave repo (bin/verify-quote): exit 0 plus +# one JSON report on stdout ⇔ verified. `network harvest` runs it when the +# founding keys are collected, and `manifest assemble` re-runs it over the +# archived evidence before the harvested set is pinned. Verification-only; +# runs natively on any dev platform (verification is pure computation over +# the evidence bytes — no TEE hardware involved). +DEFAULT_VERIFY_QUOTE_BIN = "verify-quote" + +# Summit's consensus (BLS) port: each validator entry in the completed +# summit genesis pins ":". IPs are operational data — the config +# digest excludes them — so they are delivered but never pinned. +SUMMIT_CONSENSUS_PORT = 18551 + # Today's hardcoded summit BLS domain separator. # two chains sharing it can cross-replay BLS signatures. # TODO: make it configurable _SUMMIT_DEFAULT_NAMESPACE = "_SUMMIT" MANIFEST_FILENAME = "network-manifest.json" -POLICY_FILENAME = "measurement-policy.json" +# "bootstrap" because this file is only the *founding* allowlist (what the +# manifest's bootstrap_policy_hash pins and registry genesis storage is +# compiled from); the live policy is the registry contract's state, which +# the authority can mutate after genesis. +POLICY_FILENAME = "measurement-policy-bootstrap.json" RETH_GENESIS_FILENAME = "reth-genesis.json" # Both the authored input (under inputs/) and the shipped artifact (at the # network directory top level) use this basename: same format, the @@ -128,7 +159,8 @@ NODES_DIRNAME = "nodes" # The founding cohort's inputs: founder-withdrawal-credentials.json is -# authored (node name -> withdrawal credentials); harvest/ holds what +# authored (one address per founding node, paired in node-name order by +# load_founding_set — authorable before any box exists); harvest/ holds what # `network harvest` collected from the live cohort (pubkeys, quotes, # verification reports) — provenance like measurements.json, but harvested # rather than authored. @@ -300,7 +332,8 @@ def promote_measurements( attestation_type: str = DEFAULT_ATTESTATION_TYPE, admission_bin: str = DEFAULT_ADMISSION_BIN, ) -> bytes: - """Promote `make measure` output into measurement-policy.json bytes. + """Promote `make measure` output into measurement-policy-bootstrap.json + bytes. Shells out to the admission CLI's `promote`, which selects exactly the admission-schema registers from the raw measured-boot output, normalizes @@ -439,6 +472,344 @@ def _check_hex_or_gate(value: Any, nbytes: int, fieldname: str) -> None: raise GateError(str(e)) from None +def _check_bare_hex(value: Any, nbytes: int, fieldname: str) -> None: + """Bare lowercase hex — summit's keystore wire spelling, the form the + genesis config_digest commits to. Any other spelling is rejected, never + normalized, so nothing non-canonical is laundered into the pinned set.""" + if not isinstance(value, str) or not re.fullmatch( + rf"[0-9a-f]{{{2 * nbytes}}}", value + ): + raise GateError( + f"{fieldname}: expected {nbytes}-byte lowercase bare hex, got {value!r}" + ) + + +def load_founder_credentials(path: Path) -> list[str]: + """Load the authored founder-withdrawal-credentials.json: one + 0x-prefixed address per founder, in a JSON array. + + A list rather than a node-name mapping so the founders' addresses are + authorable before any box exists — they are a fact about the founders, + not about the infrastructure. `load_founding_set` pairs the i-th + address with the i-th founding validator in node-name order. + """ + if not path.is_file(): + raise GateError( + f"{path} not found — author it as a JSON array of the founders' " + "withdrawal credentials (0x-prefixed addresses), one per " + "founding node" + ) + try: + data = json.loads(path.read_text()) + except json.JSONDecodeError as e: + raise GateError(f"{path}: not valid JSON: {e}") from None + if not isinstance(data, list) or not all(isinstance(v, str) for v in data): + raise GateError( + f"{path}: expected a JSON array of withdrawal credentials " + "(0x-prefixed addresses), one per founding node" + ) + bad = sorted({addr for addr in data if not _is_address(addr)}) + if bad: + raise GateError( + f"{path}: withdrawal credentials must be 0x + 40 hex chars; bad " + f"entr(ies): {', '.join(bad)}" + ) + return data + + +def _is_address(value: str) -> bool: + try: + _check_hex(value, 20, "withdrawal credentials") + except ManifestSchemaError: + return False + return True + + +def load_harvest_records(harvest_dir: Path) -> dict[str, dict[str, Any]]: + """Read the harvested founding records (inputs/harvest/.json). + + Validates the fields the founding set is built from — nonce, both + pubkeys, the evidence object — and rejects a pubkey repeated across + boxes: summit's genesis keys validator accounts by node pubkey, so a + repeated key silently collapses the set, and a shared consensus key is + accidental-equivocation material. The records are plain committed + files, so everything is re-checked here even though the harvest + validated it at collection time. + """ + paths = sorted(harvest_dir.glob("*.json")) if harvest_dir.is_dir() else [] + if not paths: + raise GateError( + f"no harvest records in {harvest_dir} — assemble pins the " + "founding validator set from them; provision the cohort " + "(`up --network`) and run `seismic-tee-network harvest` first" + ) + records: dict[str, dict[str, Any]] = {} + for path in paths: + try: + record = json.loads(path.read_text()) + except json.JSONDecodeError as e: + raise GateError(f"{path}: not valid JSON: {e}") from None + if not isinstance(record, dict): + raise GateError(f"{path}: expected a JSON object") + _check_bare_hex(record.get("harvest_nonce"), 32, f"{path}: harvest_nonce") + _check_bare_hex(record.get("node_public_key"), 32, f"{path}: node_public_key") + _check_bare_hex( + record.get("consensus_public_key"), 48, f"{path}: consensus_public_key" + ) + if not isinstance(record.get("evidence"), dict): + raise GateError( + f"{path}: no evidence object — without the archived quote the " + "record cannot be re-verified, so it must not be pinned" + ) + records[path.stem] = record + for key_field in ("node_public_key", "consensus_public_key"): + seen: dict[str, str] = {} + for name in sorted(records): + key = records[name][key_field] + if key in seen: + raise GateError( + f"{seen[key]} and {name} carry the same {key_field} " + f"({key}); the harvest is not the distinct founder set " + "being pinned — re-found and re-harvest" + ) + seen[key] = name + return records + + +@dataclass +class FoundingSet: + """The founding cohort as assemble pins it: the summit validator + entries (harvested keys + authored credentials + current IPs) and the + harvest records they came from (for quote re-verification).""" + + validators: list[dict[str, str]] + records: dict[str, dict[str, Any]] + + +def load_founding_set(network_dir: Path) -> FoundingSet: + """Pair the harvested cohort with its authored withdrawal credentials + and current IPs into summit validator entries. + + The credentials are positional: the i-th authored address goes to the + i-th harvested box in node-name order, and the counts must match + exactly — one address short means a box can't be pinned, one too many + means the harvest isn't the cohort the founders authored for, and + either way assembling would pin a set other than the intended one. The + pairing is logged and lands visibly in the emitted genesis, since + nothing downstream can tell a swapped pair from an intended one. IPs + come from the cohort descriptors under nodes/ (":"): delivered in the genesis file but excluded from its config + digest, so the committed file is a founding-era snapshot and IP churn + never re-founds. + """ + inputs_dir = network_dir / INPUTS_DIRNAME + founders = load_founder_credentials(inputs_dir / FOUNDERS_FILENAME) + records = load_harvest_records(inputs_dir / HARVEST_DIRNAME) + if len(founders) != len(records): + raise GateError( + f"{inputs_dir / FOUNDERS_FILENAME} carries {len(founders)} " + f"withdrawal credential(s) but {len(records)} box(es) were " + f"harvested into {inputs_dir / HARVEST_DIRNAME} " + f"({', '.join(sorted(records))}) — author one address per " + "founding node" + ) + nodes_dir = network_dir / NODES_DIRNAME + validators = [] + for name, credentials in zip(sorted(records), founders, strict=True): + descriptor_path = nodes_dir / f"{name}.json" + if not descriptor_path.is_file(): + raise GateError( + f"{descriptor_path} not found — the cohort descriptors from " + "`up --network` supply each founding validator's IP. A " + "harvested box whose descriptor is gone means the cohort " + "changed under the harvest: re-found rather than assembling" + ) + try: + ip = require(load_descriptor(descriptor_path), "public_ip", descriptor_path) + except (json.JSONDecodeError, ValueError) as e: + raise GateError(f"{descriptor_path}: {e}") from None + logger.info("founding validator %s: withdrawals to %s", name, credentials) + validators.append( + { + "node_public_key": records[name]["node_public_key"], + "consensus_public_key": records[name]["consensus_public_key"], + "ip_address": f"{ip}:{SUMMIT_CONSENSUS_PORT}", + "withdrawal_credentials": credentials, + } + ) + return FoundingSet(validators=validators, records=records) + + +def _verify_quote_bin_not_found(verify_quote_bin: str) -> GateError: + return GateError( + f"{verify_quote_bin!r} not found; build the enclave repo's " + "bin/verify-quote and put it on PATH, or pass --verify-quote-bin" + ) + + +def verify_quote_evidence( + evidence: dict[str, Any], + *, + nonce: str, + node_pubkey: str, + consensus_pubkey: str, + policy_path: Path, + verify_quote_bin: str = DEFAULT_VERIFY_QUOTE_BIN, + pccs_url: str | None = None, + override_azure_outdated_tcb: bool = False, +) -> dict[str, Any]: + """DCAP-verify one founding quote via the enclave repo's `verify-quote`. + + Its contract: exit 0 plus one JSON report on stdout ⇔ the evidence + verifies cryptographically, its report_data binds this nonce + these + pubkeys, and its measurements satisfy the policy. The evidence goes + over stdin, byte-exact with the harvest archive. + """ + cmd = [ + verify_quote_bin, + "--evidence", + "-", + "--policy", + str(policy_path), + "--nonce", + nonce, + "--node-pubkey", + node_pubkey, + "--consensus-pubkey", + consensus_pubkey, + ] + if pccs_url: + cmd += ["--pccs-url", pccs_url] + if override_azure_outdated_tcb: + cmd.append("--override-azure-outdated-tcb") + try: + result = subprocess.run( + cmd, + input=json.dumps(evidence).encode("utf-8"), + capture_output=True, + # Generous — DCAP verification fetches collateral over the + # network (PCCS) — but a hung fetch must not stall the + # harvest/assemble forever. + timeout=300, + ) + except FileNotFoundError: + raise _verify_quote_bin_not_found(verify_quote_bin) from None + except subprocess.TimeoutExpired: + raise GateError(f"`{' '.join(cmd)}` timed out") from None + if result.returncode != 0: + detail = result.stderr.decode("utf-8", "replace").strip() + raise GateError(f"quote verification failed:\n{detail}") + try: + report = json.loads(result.stdout) + except json.JSONDecodeError: + report = None + if not isinstance(report, dict) or report.get("verified") is not True: + raise GateError( + f"`{verify_quote_bin}` exited 0 without a verified report: " + f"{result.stdout!r}" + ) + return report + + +def verify_harvest_records( + records: dict[str, dict[str, Any]], + policy_bytes: bytes, + verify_quote_bin: str = DEFAULT_VERIFY_QUOTE_BIN, + pccs_url: str | None = None, + override_azure_outdated_tcb: bool = False, + verify_fn: Callable[[str, dict[str, Any], Path], dict[str, Any]] | None = None, +) -> None: + """Re-verify every archived founding quote against the compiled policy. + + The harvest verified these quotes when it collected them, but assemble + is the step that pins the validator set into network_id — so it re-runs + the same check over the archived evidence rather than trusting an + earlier run's verdict (the records are plain files that may have been + copied, committed, and edited between harvest and assemble). + """ + if verify_fn is None and shutil.which(verify_quote_bin) is None: + # Tooling, not evidence: a missing verifier fails here, before the + # loop whose failures carry burned-founding advice. + raise _verify_quote_bin_not_found(verify_quote_bin) + with tempfile.NamedTemporaryFile( + prefix="measurement-policy-", suffix=".json" + ) as policy_file: + policy_file.write(policy_bytes) + policy_file.flush() + policy_path = Path(policy_file.name) + run_verify = verify_fn or ( + lambda _name, record, path: verify_quote_evidence( + record["evidence"], + nonce=record["harvest_nonce"], + node_pubkey=record["node_public_key"], + consensus_pubkey=record["consensus_public_key"], + policy_path=path, + verify_quote_bin=verify_quote_bin, + pccs_url=pccs_url, + override_azure_outdated_tcb=override_azure_outdated_tcb, + ) + ) + for name in sorted(records): + try: + run_verify(name, records[name], policy_path) + except GateError as e: + raise GateError( + f"{name}: {e}\nA founding key whose archived quote does " + "not verify must not be pinned — re-found (or re-harvest " + "an unchanged cohort) rather than assembling around it" + ) from None + logger.info("%s: archived founding quote verified", name) + + +def summit_set_validators( + template_bytes: bytes, + validators: list[dict[str, str]], + summit_bin: str = DEFAULT_SUMMIT_BIN, +) -> bytes: + """Emit the completed summit genesis via `summit genesis set-validators`. + + Emission belongs to summit: the subcommand parses the template into + summit's own Genesis type, replaces its validator set with + `validators`, sorts them by node key (the order config_digest hashes), + renders the whole file canonically — summit's hex spellings, no + authored comments — and reloads what it emits, so a genesis no node + could load fails here rather than at boot. The returned bytes are what + the artifact set ships and the manifest's digest commits to. + """ + with ( + tempfile.NamedTemporaryFile(suffix=".toml") as template_file, + tempfile.NamedTemporaryFile(suffix=".json") as validators_file, + ): + template_file.write(template_bytes) + template_file.flush() + validators_file.write((json.dumps(validators, indent=2) + "\n").encode()) + validators_file.flush() + cmd = [ + summit_bin, + "genesis", + "set-validators", + "-i", + template_file.name, + "-v", + validators_file.name, + ] + try: + result = subprocess.run(cmd, capture_output=True, timeout=120, check=True) + except FileNotFoundError: + raise GateError( + f"{summit_bin!r} not found; build summit (the `genesis " + "set-validators` subcommand) or pass --summit-bin" + ) from None + except subprocess.TimeoutExpired: + raise GateError(f"`{' '.join(cmd)}` timed out") from None + except subprocess.CalledProcessError as e: + detail = (e.stderr or e.stdout or b"").decode("utf-8", "replace").strip() + raise GateError(f"`{' '.join(cmd)}` failed: {detail}") from None + if not result.stdout: + raise GateError(f"`{' '.join(cmd)}` emitted nothing on stdout") + return result.stdout + + @dataclass class GateContext: """Artifact set a manifest is validated against (deploy-side gates).""" @@ -668,7 +1039,7 @@ def _validate_registry_account( f"registry {addr} genesis storage is empty: the admission policy " "must be genesis-pinned. Seed the account with the compiled " "registry_genesis_storage (`seismic-measurement-admission " - "compile measurement-policy.json`)" + "compile measurement-policy-bootstrap.json`)" ) if actual != expected: problems = [ @@ -712,12 +1083,13 @@ def fill_eth_genesis_hash(genesis_bytes: bytes, eth_genesis_hash: str) -> bytes: value. The hash is derived from reth-genesis.json — never authored — but summit's - genesis-binary parser requires the field to be present in the TOML it - reads, so the shipped copy must carry it. Any declared value is dropped - (it can only be stale copy-paste, e.g. summit's example_genesis.toml) and - the computed one is prepended — always valid TOML for a top-level key, and - deterministic, so the filled copy is what `genesis_config_digest` commits - to and the artifact set ships. + genesis parser requires the field to be present in the TOML it reads, so + the template fed to `summit genesis set-validators` must carry it. Any + declared value is dropped (it can only be stale copy-paste, e.g. summit's + example_genesis.toml) and the computed one is prepended — always valid + TOML for a top-level key; set-validators re-renders the completed file, + which is what `genesis_config_digest` commits to and the artifact set + ships. """ lines, in_table = [], False for line in genesis_bytes.splitlines(keepends=True): @@ -744,6 +1116,7 @@ def assemble( reth_genesis: Path, summit_genesis: Path, policy_bytes: bytes, + validators: list[dict[str, str]], registry: str = DEFAULT_REGISTRY, authority: str = DEFAULT_AUTHORITY, reth_bin: str = "seismic-reth", @@ -752,17 +1125,19 @@ def assemble( genesis_hash_fn: Callable[[Path], str] | None = None, compile_fn: Callable[[bytes], dict[str, Any]] | None = None, digest_fn: Callable[[Path], str] | None = None, + set_validators_fn: Callable[[bytes, list[dict[str, str]]], bytes] | None = None, ) -> AssembledManifest: """Assemble, render, and gate-check a v1 network manifest. The summit genesis's `eth_genesis_hash` is derived from reth-genesis.json, never authored: whatever the input declares (if anything) is replaced with - the computed value in the completed copy the artifact set ships — - committed bytes never carry a stale hash. The manifest pins that - completed genesis via summit's own config digest (`summit genesis - digest`), which covers the consensus parameters and the validator set; - an input without validators ships an empty placeholder set - (`validators = []`). + the computed value — committed bytes never carry a stale hash. The + `validators` set (the founding cohort's harvested keys, paired with + authored credentials and current IPs — see load_founding_set) is filled + in by `summit genesis set-validators`, which re-renders the whole file + canonically; the artifact set ships summit's emission, and the manifest + pins it via summit's own config digest (`summit genesis digest`), which + covers the consensus parameters and the validator set but not the IPs. The registry account's genesis storage is likewise derived, not authored: the policy document is compiled and its registry_genesis_storage injected @@ -798,12 +1173,25 @@ def assemble( authored["eth_genesis_hash"], eth_hash, ) + if not validators: + raise GateError( + "no founding validators — the validator set is pinned from the " + "harvest, and a founding with an empty set is not a network" + ) if "validators" not in authored: - # summit requires the field to *parse* a genesis (its Genesis type has - # no serde default); an input authored without a validator set ships - # an empty placeholder. + # summit requires the field to *parse* a genesis (its Genesis type + # has no serde default), and set-validators loads the template + # before replacing whatever set it declares — so an input authored + # without one gets an empty placeholder purely to make the template + # loadable. The shipped set always comes from `validators`. authored_bytes = b"validators = []\n" + authored_bytes - summit_genesis_bytes = fill_eth_genesis_hash(authored_bytes, eth_hash) + template_bytes = fill_eth_genesis_hash(authored_bytes, eth_hash) + emit = set_validators_fn or ( + lambda template, vals: summit_set_validators( + template, vals, summit_bin=summit_bin + ) + ) + summit_genesis_bytes = emit(template_bytes, validators) resolve_digest = digest_fn or ( lambda p: summit_config_digest(p, summit_bin=summit_bin) ) @@ -895,10 +1283,11 @@ def starter_summit_genesis(name: str) -> str: """ # json.dumps emits a valid TOML basic string for these simple values. return f"""\ -# Summit network parameters. `manifest assemble` prepends the two derived -# fields above this header — eth_genesis_hash (from reth-genesis.json) and -# validators (the founding validator set — TEE-born keys gathered from the -# live cohort) — and ships the completed file as summit-genesis.toml. +# Summit network parameters. `manifest assemble` completes this input with +# the two derived fields — eth_genesis_hash (from reth-genesis.json) and +# validators (the founding set: TEE-born keys harvested from the live +# cohort) — and ships summit's own rendering of the completed file as +# summit-genesis.toml, so comments here never reach the artifact. # Review every value before founding a real network. leader_timeout_ms = 2000 notarization_timeout_ms = 4000 @@ -947,20 +1336,26 @@ def init_network_dir( measurements: Path, summit_genesis: Path | None = None, measurement_id: str | None = None, + founders: int = 0, + force: bool = False, ) -> list[Path]: - """Scaffold a network directory's three authored inputs under inputs/. + """Scaffold a network directory's four authored inputs under inputs/. Copies the genesis and measurements in (stamping measurement_id into the latter when given), and writes a starter summit genesis - (namespace = name) unless one is supplied to copy. The founder edits - these in place, then `assemble --dir` derives the artifact set into the - directory's top level — inputs and the committed artifacts live - together, so the directory is the whole network (commit it for networks - that matter). + (namespace = name) unless one is supplied to copy, plus `founders` + placeholder withdrawal credentials (`0x00…0` — obviously fake, so a + set that survives into a network anyone cares about shows on sight). + The founder edits all four in place, then provisions and harvests the + cohort (assemble pins the harvested validator set) before + `assemble --dir` derives the artifact set into the directory's top + level — inputs and the committed artifacts live together, so the + directory is the whole network (commit it for networks that matter). """ measurements_bytes = measurements.read_bytes() if measurement_id is not None: measurements_bytes = stamp_measurement_id(measurements_bytes, measurement_id) + credentials = [f"0x{i:040x}" for i in range(1, founders + 1)] contents = { RETH_GENESIS_FILENAME: reth_genesis.read_bytes(), MEASUREMENTS_FILENAME: measurements_bytes, @@ -969,13 +1364,15 @@ def init_network_dir( if summit_genesis is not None else starter_summit_genesis(name).encode() ), + FOUNDERS_FILENAME: (json.dumps(credentials, indent=2) + "\n").encode(), } inputs_dir = out_dir / INPUTS_DIRNAME existing = [n for n in contents if (inputs_dir / n).exists()] - if existing: + if existing and not force: raise GateError( f"refusing to overwrite existing input(s) in {inputs_dir}: " - f"{', '.join(existing)}" + f"{', '.join(existing)} — pass --force to re-author them " + "(re-assembling from changed inputs is a new network identity)" ) inputs_dir.mkdir(parents=True, exist_ok=True) written = [] @@ -1134,8 +1531,26 @@ def add_summit_bin(p: argparse.ArgumentParser) -> None: "--measurement-id", default=None, help="image artifact filename the measurements belong to; stamped " - f"into {INPUTS_DIRNAME}/{MEASUREMENTS_FILENAME} so assemble needs " - "no --measurement-id", + f"into {INPUTS_DIRNAME}/{MEASUREMENTS_FILENAME}. Only needed when " + "the measurements file carries no measurement_id of its own " + "(seismic-images' make measure stamps one)", + ) + ini.add_argument( + "--founders", + type=int, + default=0, + metavar="N", + help="how many placeholder withdrawal credentials to scaffold into " + f"{INPUTS_DIRNAME}/{FOUNDERS_FILENAME} (one per founding node, " + "paired in node-name order at assemble time). The placeholders are " + "all a throwaway needs; a real founding replaces them with the " + "founders' addresses. Default: an empty list to fill in", + ) + ini.add_argument( + "--force", + action="store_true", + help="overwrite existing authored inputs (re-authoring them and " + "re-assembling is a new network identity)", ) asm = sub.add_parser( @@ -1172,6 +1587,24 @@ def add_summit_bin(p: argparse.ArgumentParser) -> None: action="store_true", help="overwrite an existing manifest (a new network identity)", ) + asm.add_argument( + "--verify-quote-bin", + default=DEFAULT_VERIFY_QUOTE_BIN, + help="DCAP verifier CLI from the enclave repo (bin/verify-quote), " + "used to re-verify the archived harvest quotes before the founding " + "set is pinned", + ) + asm.add_argument( + "--pccs-url", + default=None, + metavar="URL", + help="forwarded to verify-quote: PCCS URL for DCAP collateral", + ) + asm.add_argument( + "--override-azure-outdated-tcb", + action="store_true", + help="forwarded to verify-quote: allow the Azure outdated-TCB override path", + ) add_reth_bin(asm) add_admission_bin(asm) add_summit_bin(asm) @@ -1191,10 +1624,13 @@ def add_summit_bin(p: argparse.ArgumentParser) -> None: args = parser.parse_args(argv) + # Absolute from here on, so every path this CLI prints is clickable in a + # terminal and names one directory unambiguously. + args.dir = args.dir.resolve() if args.command == "init": - args.name = args.name or args.dir.resolve().name + args.name = args.name or args.dir.name elif args.command == "assemble": - args.name = args.dir.resolve().name + args.name = args.dir.name inputs_dir = args.dir / INPUTS_DIRNAME args.reth_genesis = inputs_dir / RETH_GENESIS_FILENAME args.summit_genesis = inputs_dir / SUMMIT_GENESIS_FILENAME @@ -1220,18 +1656,39 @@ def main() -> None: args.measurements, args.summit_genesis, args.measurement_id, + founders=args.founders, + force=args.force, ) for path in written: logger.info("wrote %s", path) - id_hint = ( - "" - if args.measurement_id - else " --measurement-id " + # Suggest --measurement-id only when the copied measurements + # actually lack one (make measure stamps it at the source; a + # promoted policy carries one per record). + stamped = json.loads( + (args.dir / INPUTS_DIRNAME / MEASUREMENTS_FILENAME).read_bytes() ) + needs_id = isinstance(stamped, dict) and "measurement_id" not in stamped + id_hint = " --measurement-id " if needs_id else "" + inputs_dir = args.dir / INPUTS_DIRNAME + founders_hint = ( + "update the placeholder addresses in" + if args.founders + else "fill in one address per founding node in" + ) + # No --count on `up`: the authored credentials size the cohort. print( - f"Scaffolded {args.dir}. Edit the inputs (at minimum review " - f"{INPUTS_DIRNAME}/{SUMMIT_GENESIS_FILENAME}), then:\n" - f" seismic-tee-network manifest assemble {args.dir}{id_hint}" + f"Scaffolded {args.dir}. Next:\n" + f" 1. review {inputs_dir / SUMMIT_GENESIS_FILENAME}\n" + f" 2. {founders_hint}\n" + f" {inputs_dir / FOUNDERS_FILENAME}\n" + f" 3. review the stack config the cohort boots from\n" + f" {DEFAULT_STACK_CONFIG}\n" + " (vhd_blob_url must name the image the measurements " + "describe;\n" + " region, VM size, and operator_ip_cidr live there too)\n" + f" 4. seismic-tee-network up --network {args.dir}\n" + f" 5. seismic-tee-network harvest {args.dir}\n" + f" 6. seismic-tee-network manifest assemble {args.dir}{id_hint}" ) elif args.command == "assemble": missing = [ @@ -1246,17 +1703,31 @@ def main() -> None: + f" — authored inputs live under {INPUTS_DIRNAME}/; " "scaffold them with `manifest init`" ) + founding = load_founding_set(args.dir) + logger.info( + "founding set: %d validator(s) from %s", + len(founding.validators), + args.dir / INPUTS_DIRNAME / HARVEST_DIRNAME, + ) policy_bytes = promote_measurements( args.measurements.read_bytes(), args.measurement_id, args.attestation_type, admission_bin=args.admission_bin, ) + verify_harvest_records( + founding.records, + policy_bytes, + verify_quote_bin=args.verify_quote_bin, + pccs_url=args.pccs_url, + override_azure_outdated_tcb=args.override_azure_outdated_tcb, + ) assembled = assemble( name=args.name, reth_genesis=args.reth_genesis, summit_genesis=args.summit_genesis, policy_bytes=policy_bytes, + validators=founding.validators, registry=args.registry, authority=args.authority, reth_bin=args.reth_bin, @@ -1267,6 +1738,7 @@ def main() -> None: logger.info("wrote %s", args.out / MANIFEST_FILENAME) logger.info("wrote %s", args.out / POLICY_FILENAME) logger.info("wrote %s", args.out / RETH_GENESIS_FILENAME) + logger.info("wrote %s", args.out / SUMMIT_GENESIS_FILENAME) print(f"network_id: {assembled.network_id}") else: manifest_bytes = args.manifest.read_bytes() diff --git a/tee/cli/common/repo.py b/tee/cli/common/repo.py new file mode 100644 index 00000000..0a38cbd6 --- /dev/null +++ b/tee/cli/common/repo.py @@ -0,0 +1,23 @@ +"""Where this checkout keeps the things the CLIs point at. + +Layout facts, not configuration. Both front-ends name them — the +orchestrator provisions from the Pulumi program, `manifest init` points +the founder at its stack config — so they are spelled once here rather +than derived from `__file__` in each module that needs them. +""" + +from pathlib import Path + +# tee/cli/common/repo.py -> tee/ +_TEE_DIR = Path(__file__).parents[2] + +# The single-node program the orchestrator fans out over. A local-program +# workspace points at this dir, so the project name / runtime / venv all +# come from its Pulumi.yaml — identical to running `pulumi` in that dir. +SEISMIC_NODE_DIR = _TEE_DIR / "pulumi" / "seismic_node" + +# Shared settings are inherited from this stack config unless --config +# overrides: the image pin every cohort boots (vhd_blob_url), the VM +# shape and region, and operator_ip_cidr — the CIDR that may reach the +# operator-only ports. +DEFAULT_STACK_CONFIG = SEISMIC_NODE_DIR / "Pulumi.dev.yaml" diff --git a/tee/cli/common/tests/test_manifest.py b/tee/cli/common/tests/test_manifest.py index accaea38..52f04f32 100644 --- a/tee/cli/common/tests/test_manifest.py +++ b/tee/cli/common/tests/test_manifest.py @@ -18,6 +18,7 @@ from tee.cli.common import manifest as manifest_mod from tee.cli.common.manifest import ( DEFAULT_ADMISSION_BIN, + SUMMIT_CONSENSUS_PORT, AssembledManifest, GateContext, GateError, @@ -27,13 +28,17 @@ compute_network_id, init_network_dir, inject_registry_genesis_storage, + load_founding_set, + load_harvest_records, promote_measurements, render_manifest, render_network_section, run_validation_gates, + summit_set_validators, validate_manifest_schema, validate_reth_genesis_matches, validate_summit_genesis_matches, + verify_harvest_records, write_artifact_set, ) @@ -44,6 +49,31 @@ def _content_digest(path: Path) -> str: return "0x" + hashlib.sha256(path.read_bytes()).hexdigest() +# A founding validator entry as load_founding_set builds it (harvested keys +# in summit's bare-lowercase-hex keystore spelling, authored credentials, +# descriptor IP + consensus port). +VALIDATOR = { + "node_public_key": "ab" * 32, + "consensus_public_key": "cd" * 48, + "ip_address": f"203.0.113.7:{SUMMIT_CONSENSUS_PORT}", + "withdrawal_credentials": "0x" + "f3" * 20, +} + + +def _fake_set_validators(template: bytes, validators: list[dict[str, str]]) -> bytes: + """Test stand-in for `summit genesis set-validators`: replaces the + template's placeholder set with inline-table entries, sorted by node key + like summit's canonical emission. Line-level splice (not a re-render), so + byte-oriented assertions about the rest of the template stay meaningful.""" + entries = ", ".join( + "{ " + ", ".join(f"{k} = {json.dumps(v[k])}" for k in sorted(v)) + " }" + for v in sorted(validators, key=lambda v: v["node_public_key"]) + ) + return template.replace( + b"validators = []\n", f"validators = [{entries}]\n".encode(), 1 + ) + + # The shared policy-compiler CLI from the enclave repo. Tests of the # subprocess boundary run only where it is built (everything else injects # compile_fn / crafts policy bytes directly, mirroring genesis_hash_fn). @@ -493,9 +523,11 @@ def _assemble(self, **overrides) -> AssembledManifest: "reth_genesis": self.reth_genesis, "summit_genesis": self.summit_genesis, "policy_bytes": self.policy_bytes, + "validators": [dict(VALIDATOR)], "genesis_hash_fn": lambda _p: self.ETH_HASH, "compile_fn": self._report, "digest_fn": _content_digest, + "set_validators_fn": _fake_set_validators, } kwargs.update(overrides) # ty can't verify a **kwargs dict-splat against typed params. @@ -695,12 +727,28 @@ def test_assemble_fills_eth_genesis_hash(self): # validate-style round trip: gates re-pass over the written copy. run_validation_gates(assembled.manifest, self._ctx(summit_genesis=written)) - def test_assemble_fills_empty_validators_placeholder(self): - # summit requires the key to *parse* a genesis (no serde default); a - # template authored without a validator set ships an empty placeholder. + def test_assemble_ships_the_founding_validator_set(self): + # The validator set the artifact ships is exactly the one passed in + # (the founding cohort from load_founding_set), filled by the + # set-validators emission. assembled = self._assemble() genesis = tomllib.loads(assembled.summit_genesis_bytes.decode()) - self.assertEqual(genesis["validators"], []) + self.assertEqual(genesis["validators"], [VALIDATOR]) + + def test_assemble_rejects_an_empty_validator_set(self): + with self.assertRaisesRegex(GateError, "no founding validators"): + self._assemble(validators=[]) + + def test_digest_commits_to_the_emitted_genesis(self): + # The manifest's config digest is computed over set-validators' + # output (validator set included), not over the pre-emission + # template. + assembled = self._assemble() + emitted = "0x" + hashlib.sha256(assembled.summit_genesis_bytes).hexdigest() + self.assertEqual(assembled.manifest["summit"]["genesis_config_digest"], emitted) + self.assertIn( + VALIDATOR["node_public_key"].encode(), assembled.summit_genesis_bytes + ) def test_assemble_replaces_declared_genesis_hash(self): # A declared value (e.g. from summit's example_genesis.toml) is stale @@ -769,7 +817,7 @@ def test_write_artifact_set_refuses_overwrite(self): write_artifact_set(self.out_dir, assembled) for name in ( "network-manifest.json", - "measurement-policy.json", + "measurement-policy-bootstrap.json", "reth-genesis.json", "summit-genesis.toml", ): @@ -804,11 +852,16 @@ def test_scaffolds_inputs_with_starter_template(self): self.assertEqual( sorted(p.name for p in written), [ + "founder-withdrawal-credentials.json", "measurements.json", "reth-genesis.json", "summit-genesis.toml", ], ) + # No --founders: an empty list to fill in, not a guessed cohort size. + self.assertEqual( + json.loads((inputs / "founder-withdrawal-credentials.json").read_text()), [] + ) self.assertEqual( (inputs / "reth-genesis.json").read_bytes(), self.reth_genesis.read_bytes(), @@ -829,10 +882,29 @@ def test_copies_supplied_genesis_verbatim(self): src.read_bytes(), ) - def test_refuses_overwrite(self): + def test_refuses_overwrite_unless_forced(self): init_network_dir(self.out, "t", self.reth_genesis, self.measurements) with self.assertRaisesRegex(GateError, "refusing to overwrite"): init_network_dir(self.out, "t", self.reth_genesis, self.measurements) + # The re-found/re-author path: --force overwrites the inputs. + self.reth_genesis.write_text('{"config": {"chainId": 9999}}') + init_network_dir( + self.out, "t", self.reth_genesis, self.measurements, force=True + ) + rewritten = (self.out / "inputs" / "reth-genesis.json").read_text() + self.assertIn("9999", rewritten) + + def test_founders_scaffolds_placeholder_credentials(self): + init_network_dir( + self.out, "testnet-1", self.reth_genesis, self.measurements, founders=3 + ) + path = self.out / "inputs" / "founder-withdrawal-credentials.json" + self.assertEqual( + json.loads(path.read_text()), + [f"0x{1:040x}", f"0x{2:040x}", f"0x{3:040x}"], + ) + # Placeholders are a usable founder set, not a stub to be rewritten. + self.assertEqual(len(manifest_mod.load_founder_credentials(path)), 3) def test_stamps_measurement_id(self): init_network_dir( @@ -865,26 +937,206 @@ def test_rejects_measurement_id_for_promoted_policy(self): ) +class FoundingSetTests(unittest.TestCase): + """load_founding_set / load_harvest_records: pairing the harvest with + the authored credentials and the cohort descriptors into the validator + entries assemble pins.""" + + ADDRESS = "0x" + "f3" * 20 + + def setUp(self): + tmp = tempfile.TemporaryDirectory() + self.addCleanup(tmp.cleanup) + self.net = Path(tmp.name) + self.inputs = self.net / manifest_mod.INPUTS_DIRNAME + self.harvest = self.inputs / manifest_mod.HARVEST_DIRNAME + self.nodes = self.net / manifest_mod.NODES_DIRNAME + self.harvest.mkdir(parents=True) + self.nodes.mkdir() + self._write_founders([self.ADDRESS]) + self._write_record("node-1") + self._write_descriptor("node-1", "203.0.113.7") + + def _write_founders(self, obj) -> None: + path = self.inputs / manifest_mod.FOUNDERS_FILENAME + path.write_text(json.dumps(obj)) + + def _write_record( + self, + name: str, + node_key: str = "ab" * 32, + consensus_key: str = "cd" * 48, + **overrides, + ) -> None: + record: dict = { + "harvest_nonce": "11" * 32, + "node_public_key": node_key, + "consensus_public_key": consensus_key, + "evidence": {"attestation_type": "azure-tdx"}, + } + record.update(overrides) + record = {k: v for k, v in record.items() if v is not None} + (self.harvest / f"{name}.json").write_text(json.dumps(record)) + + def _write_descriptor(self, name: str, ip: str) -> None: + (self.nodes / f"{name}.json").write_text( + json.dumps({"public_ip": ip, "fqdn": f"{name}.example.com"}) + ) + + def test_builds_validator_entries_sorted_by_node_name(self): + # The authored credentials are positional: the i-th address pairs + # with the i-th box in node-name order. + self._write_founders([self.ADDRESS, "0x" + "aa" * 20]) + self._write_record("node-2", node_key="ef" * 32, consensus_key="ab" * 48) + self._write_descriptor("node-2", "203.0.113.8") + founding = load_founding_set(self.net) + self.assertEqual( + founding.validators, + [ + { + "node_public_key": "ab" * 32, + "consensus_public_key": "cd" * 48, + "ip_address": f"203.0.113.7:{SUMMIT_CONSENSUS_PORT}", + "withdrawal_credentials": self.ADDRESS, + }, + { + "node_public_key": "ef" * 32, + "consensus_public_key": "ab" * 48, + "ip_address": f"203.0.113.8:{SUMMIT_CONSENSUS_PORT}", + "withdrawal_credentials": "0x" + "aa" * 20, + }, + ], + ) + self.assertEqual(sorted(founding.records), ["node-1", "node-2"]) + + def test_missing_harvest_burns_with_harvest_hint(self): + for path in self.harvest.glob("*.json"): + path.unlink() + with self.assertRaisesRegex(GateError, "harvest"): + load_founding_set(self.net) + + def test_harvested_box_without_credentials(self): + self._write_record("node-2", node_key="ef" * 32, consensus_key="ab" * 48) + self._write_descriptor("node-2", "203.0.113.8") + with self.assertRaisesRegex(GateError, r"1 withdrawal credential\(s\)"): + load_founding_set(self.net) + + def test_more_credentials_than_harvested_boxes(self): + self._write_founders([self.ADDRESS, self.ADDRESS]) + with self.assertRaisesRegex(GateError, r"2 withdrawal credential\(s\)"): + load_founding_set(self.net) + + def test_missing_descriptor_burns(self): + (self.nodes / "node-1.json").unlink() + with self.assertRaisesRegex(GateError, "re-found"): + load_founding_set(self.net) + + def test_malformed_credentials_rejected(self): + self._write_founders(["0x1234"]) + with self.assertRaisesRegex(GateError, "0x1234"): + load_founding_set(self.net) + + def test_credentials_mapping_rejected(self): + self._write_founders({"node-1": self.ADDRESS}) + with self.assertRaisesRegex(GateError, "expected a JSON array"): + load_founding_set(self.net) + + def test_non_canonical_key_spelling_rejected(self): + # Uppercase hex would digest differently under summit's v1 spelling + # rules — rejected, never normalized. + self._write_record("node-1", node_key="AB" * 32) + with self.assertRaisesRegex(GateError, "node_public_key"): + load_harvest_records(self.harvest) + + def test_record_without_evidence_rejected(self): + self._write_record("node-1", evidence=None) + with self.assertRaisesRegex(GateError, "evidence"): + load_harvest_records(self.harvest) + + def test_duplicate_node_key_across_boxes_rejected(self): + self._write_record("node-2", consensus_key="ab" * 48) + with self.assertRaisesRegex(GateError, "node_public_key"): + load_harvest_records(self.harvest) + + +class VerifyHarvestRecordsTests(unittest.TestCase): + """The assemble-time re-verification driver (the verify-quote shell-out + itself is exercised through the harvest tests, which mock the same + subprocess boundary).""" + + RECORD = { + "harvest_nonce": "11" * 32, + "node_public_key": "ab" * 32, + "consensus_public_key": "cd" * 48, + "evidence": {"attestation_type": "azure-tdx"}, + } + + def test_verifies_every_record_against_the_policy_file(self): + calls: list[tuple[str, bytes]] = [] + + def verify_fn(name, record, policy_path): + calls.append((name, policy_path.read_bytes())) + return {"verified": True} + + records = {"node-2": dict(self.RECORD), "node-1": dict(self.RECORD)} + verify_harvest_records(records, b"policy bytes", verify_fn=verify_fn) + # Every record, deterministic order, against exactly the promoted + # policy bytes. + self.assertEqual( + calls, [("node-1", b"policy bytes"), ("node-2", b"policy bytes")] + ) + + def test_failure_names_the_box_and_burns(self): + def verify_fn(name, record, policy_path): + if name == "node-2": + raise GateError("quote verification failed") + return {"verified": True} + + records = {"node-1": dict(self.RECORD), "node-2": dict(self.RECORD)} + with self.assertRaisesRegex(GateError, "node-2.*\n.*not be pinned"): + verify_harvest_records(records, b"policy", verify_fn=verify_fn) + + def test_missing_verifier_binary_is_a_gate_error(self): + with self.assertRaisesRegex(GateError, "not found") as ctx: + verify_harvest_records( + {"node-1": dict(self.RECORD)}, + b"policy", + verify_quote_bin="no-such-verify-quote", + ) + # Tooling, not evidence: the preflight fails before the loop, so a + # missing verifier never carries the burned-founding advice. + self.assertNotIn("re-found", str(ctx.exception)) + + +class SetValidatorsTests(unittest.TestCase): + """The `summit genesis set-validators` subprocess boundary (emission + semantics — sorting, canonical rendering, reload-what-it-wrote — are + pinned by summit's own tests).""" + + def test_missing_binary_is_a_gate_error(self): + with self.assertRaisesRegex(GateError, "not found"): + summit_set_validators( + b"validators = []\n", [dict(VALIDATOR)], summit_bin="no-such-summit" + ) + + class DirCliTests(unittest.TestCase): """assemble/validate take the network directory as their sole positional - argument (`_parse_args`); only init handles loose files.""" + argument (`_parse_args`); only init handles loose files. Every derived + path is absolute — the paths this CLI prints have to be clickable.""" + + NET = Path("networks/testnet-1").resolve() def test_assemble_dir_resolution(self): args = manifest_mod._parse_args(["assemble", "networks/testnet-1"]) self.assertEqual(args.name, "testnet-1") self.assertEqual(args.admission_bin, DEFAULT_ADMISSION_BIN) + self.assertEqual(args.verify_quote_bin, manifest_mod.DEFAULT_VERIFY_QUOTE_BIN) # assemble reads the authored inputs under inputs/. - self.assertEqual( - args.reth_genesis, Path("networks/testnet-1/inputs/reth-genesis.json") - ) - self.assertEqual( - args.summit_genesis, - Path("networks/testnet-1/inputs/summit-genesis.toml"), - ) - self.assertEqual( - args.measurements, Path("networks/testnet-1/inputs/measurements.json") - ) - self.assertEqual(args.out, Path("networks/testnet-1")) + self.assertEqual(args.reth_genesis, self.NET / "inputs/reth-genesis.json") + self.assertEqual(args.summit_genesis, self.NET / "inputs/summit-genesis.toml") + self.assertEqual(args.measurements, self.NET / "inputs/measurements.json") + self.assertEqual(args.out, self.NET) def test_assemble_requires_dir(self): with self.assertRaises(SystemExit): @@ -901,17 +1153,18 @@ def test_init_dir_positional_defaults_name(self): "m.json", ] ) - self.assertEqual(args.dir, Path("networks/testnet-1")) + self.assertEqual(args.dir, self.NET) self.assertEqual(args.name, "testnet-1") def test_validate_dir_resolution(self): + net = Path("networks/t").resolve() args = manifest_mod._parse_args(["validate", "networks/t"]) - self.assertEqual(args.manifest, Path("networks/t/network-manifest.json")) + self.assertEqual(args.manifest, net / "network-manifest.json") self.assertEqual(args.admission_bin, DEFAULT_ADMISSION_BIN) # validate reads the *shipped* summit genesis, not the authored input. - self.assertEqual(args.summit_genesis, Path("networks/t/summit-genesis.toml")) + self.assertEqual(args.summit_genesis, net / "summit-genesis.toml") self.assertEqual( - args.measurement_policy, Path("networks/t/measurement-policy.json") + args.measurement_policy, net / "measurement-policy-bootstrap.json" ) diff --git a/tee/cli/network/harvest.py b/tee/cli/network/harvest.py index 0ea64aac..eb407877 100644 --- a/tee/cli/network/harvest.py +++ b/tee/cli/network/harvest.py @@ -32,7 +32,7 @@ Any anomaly burns the whole harvest: a quote window already closed (HTTP 410 — the box accepted a config POST), a failed verification, or a -cohort that doesn't match the authored +cohort whose size doesn't match the authored `inputs/founder-withdrawal-credentials.json` all abort the run. A harvested key is trustworthy only if the same box later accepts the real configure cleanly — never retry around a burned harvest; re-found @@ -44,7 +44,6 @@ import re import secrets import shutil -import subprocess import tempfile import time from dataclasses import dataclass @@ -71,11 +70,10 @@ WAIT_LOG_INTERVAL_SECONDS = 30 # The DCAP verifier from the enclave repo (bin/verify-quote), expected on -# PATH like the admission CLI. Verification-only, Linux-only at runtime — -# macOS callers run it in a Linux container. -DEFAULT_VERIFY_QUOTE_BIN = "verify-quote" +# PATH like the admission CLI. Shared constant with `manifest assemble`, +# which re-verifies the archived quotes before pinning the founding set. +DEFAULT_VERIFY_QUOTE_BIN = manifest_mod.DEFAULT_VERIFY_QUOTE_BIN -_ADDRESS_RE = re.compile(r"^0x[0-9a-fA-F]{40}$") # Holder pubkeys are summit's keystore wire format: lowercase bare hex, # exactly as `commonware_utils::hex` renders — the spelling summit's # genesis config_digest commits to, so any other form is rejected here @@ -90,9 +88,9 @@ class QuoteWindowClosed(Exception): @dataclass(frozen=True) class HarvestTarget: - """One cohort box: descriptor stem (= its key in the authored - founder-withdrawal-credentials.json), its IP, and the fresh 32-byte - nonce (hex) minted for this run's quote request.""" + """One cohort box: descriptor stem (its name in inputs/harvest/), its + IP, and the fresh 32-byte nonce (hex) minted for this run's quote + request.""" name: str public_ip: str @@ -163,6 +161,8 @@ def _parse_args(argv: list[str] | None = None) -> argparse.Namespace: if not args.dir.is_dir(): raise SystemExit(f"network directory not found: {args.dir}") + # Absolute from here on, so every path this CLI prints is clickable. + args.dir = args.dir.resolve() inputs_dir = args.dir / manifest_mod.INPUTS_DIRNAME args.measurements = inputs_dir / manifest_mod.MEASUREMENTS_FILENAME args.founders = inputs_dir / manifest_mod.FOUNDERS_FILENAME @@ -173,9 +173,9 @@ def _parse_args(argv: list[str] | None = None) -> argparse.Namespace: ) if not args.founders.is_file(): raise SystemExit( - f"{args.founders} not found — author it as a JSON object mapping " - "each cohort node name (descriptor filename stem) to that " - "founder's withdrawal credentials (0x-prefixed address)" + f"{args.founders} not found — author it as a JSON array of the " + "founders' withdrawal credentials (0x-prefixed addresses), one " + "per founding node" ) if args.node is None: nodes_dir = args.dir / manifest_mod.NODES_DIRNAME @@ -197,10 +197,10 @@ def _parse_args(argv: list[str] | None = None) -> argparse.Namespace: # cohort list callers expect. args.node = [path for group in args.node for path in group] # Descriptor filename stems are the harvest's node names (the - # founder-credentials keys and the inputs/harvest/ filenames), so - # compare stems, not paths: two spellings of one file or two files - # sharing a stem would otherwise silently collapse into one - # harvested box. + # inputs/harvest/ filenames, and the order the authored withdrawal + # credentials pair against), so compare stems, not paths: two + # spellings of one file or two files sharing a stem would otherwise + # silently collapse into one harvested box. stems = [p.stem for p in args.node] dupes = sorted({s for s in stems if stems.count(s) > 1}) if dupes: @@ -214,40 +214,26 @@ def _parse_args(argv: list[str] | None = None) -> argparse.Namespace: return args -def load_founders(path: Path, cohort: list[str]) -> dict[str, str]: - """Load inputs/founder-withdrawal-credentials.json and pair it against +def load_founders(path: Path, cohort: list[str]) -> list[str]: + """Load inputs/founder-withdrawal-credentials.json and count it against the live cohort. - The authored founder list and the harvested cohort must agree exactly: - a box with no credentials can't be pinned, and an entry with no box - means the cohort is incomplete — either way `assemble` would pin a set - other than the intended one, so the mismatch aborts the harvest. + `assemble` pairs the i-th authored address with the i-th box in + node-name order, so a count that doesn't match the cohort would leave a + box unpinnable or pin a set other than the one the founders authored + for. Checked here too, before any quote is fetched, so the fix costs + nothing. """ try: - data = json.loads(path.read_text()) - except json.JSONDecodeError as e: - raise SystemExit(f"{path}: not valid JSON: {e}") from None - if not isinstance(data, dict) or not all(isinstance(v, str) for v in data.values()): - raise SystemExit( - f"{path}: expected a JSON object mapping node name -> withdrawal " - "credentials (0x-prefixed address)" - ) - bad = sorted(name for name, addr in data.items() if not _ADDRESS_RE.match(addr)) - if bad: + data = manifest_mod.load_founder_credentials(path) + except manifest_mod.GateError as e: + raise SystemExit(str(e)) from None + if len(data) != len(cohort): raise SystemExit( - f"{path}: withdrawal credentials must be 0x + 40 hex chars; bad " - f"entr(ies): {', '.join(bad)}" + f"{path} carries {len(data)} withdrawal credential(s) but the " + f"cohort has {len(cohort)} box(es) ({', '.join(sorted(cohort))}) " + "— author one address per founding node" ) - missing = sorted(set(cohort) - set(data)) - extra = sorted(set(data) - set(cohort)) - if missing or extra: - lines = [] - if missing: - listing = ", ".join(missing) - lines.append(f" cohort box(es) with no founder entry: {listing}") - if extra: - lines.append(f" founder entr(ies) with no cohort box: {', '.join(extra)}") - raise SystemExit(f"{path} does not match the cohort:\n" + "\n".join(lines)) return data @@ -382,51 +368,28 @@ def verify_quote( pccs_url: str | None, override_azure_outdated_tcb: bool, ) -> dict[str, Any]: - """DCAP-verify one harvested quote via the enclave repo's `verify-quote`. - - Its contract: exit 0 plus one JSON report on stdout ⇔ the evidence - verifies cryptographically, its report_data binds this nonce + these - pubkeys, and its measurements satisfy the policy. The evidence goes - over stdin — the same parsed evidence object the archive records. A - failure burns the harvest: a founding key whose quote doesn't verify - must never reach `assemble`. + """DCAP-verify one harvested quote via the enclave repo's `verify-quote` + (the shared shell-out in manifest.py — `assemble` re-runs the same check + over the archived evidence before pinning the set). A failure burns the + harvest: a founding key whose quote doesn't verify must never reach + `assemble`. """ - cmd = [ - verify_bin, - "--evidence", - "-", - "--policy", - str(policy_path), - "--nonce", - target.nonce, - "--node-pubkey", - quote["node_public_key"], - "--consensus-pubkey", - quote["consensus_public_key"], - ] - if pccs_url: - cmd += ["--pccs-url", pccs_url] - if override_azure_outdated_tcb: - cmd.append("--override-azure-outdated-tcb") - result = subprocess.run( - cmd, input=json.dumps(quote["evidence"]).encode("utf-8"), capture_output=True - ) - if result.returncode != 0: - detail = result.stderr.decode("utf-8", "replace").strip() - raise SystemExit( - f"{target.name}: quote verification failed — the harvest is " - f"burned (re-found rather than retrying):\n{detail}" - ) try: - report = json.loads(result.stdout) - except json.JSONDecodeError: - report = None - if not isinstance(report, dict) or report.get("verified") is not True: - raise SystemExit( - f"{target.name}: `{verify_bin}` exited 0 without a verified " - f"report: {result.stdout!r}" + return manifest_mod.verify_quote_evidence( + quote["evidence"], + nonce=target.nonce, + node_pubkey=quote["node_public_key"], + consensus_pubkey=quote["consensus_public_key"], + policy_path=policy_path, + verify_quote_bin=verify_bin, + pccs_url=pccs_url, + override_azure_outdated_tcb=override_azure_outdated_tcb, ) - return report + except manifest_mod.GateError as e: + raise SystemExit( + f"{target.name}: {e}\nThe harvest is burned: re-found rather " + "than retrying around it." + ) from None def check_overwrite(harvest_dir: Path, names: list[str], force: bool) -> None: @@ -469,8 +432,7 @@ def main() -> None: if verify_bin is None: raise SystemExit( f"`{args.verify_quote_bin}` not found on PATH. Build the enclave " - "repo's bin/verify-quote and put it on PATH. It is Linux-only at " - "runtime; on macOS run it in a Linux container." + "repo's bin/verify-quote and put it on PATH." ) targets = [] diff --git a/tee/cli/network/orchestrator.py b/tee/cli/network/orchestrator.py index 06c8a2eb..6ed9522f 100644 --- a/tee/cli/network/orchestrator.py +++ b/tee/cli/network/orchestrator.py @@ -37,7 +37,6 @@ import argparse import getpass -import hashlib import json import os import re @@ -50,14 +49,8 @@ from pulumi import automation as auto from tee.cli.common import manifest as manifest_mod - -# The single-node program this orchestrator fans out over. A local-program -# workspace points at this dir, so the project name / runtime / venv all -# come from its Pulumi.yaml — identical to running `pulumi` in that dir. -SEISMIC_NODE_DIR = Path(__file__).parents[2] / "pulumi" / "seismic_node" - -# Shared settings are inherited from this stack config unless --config overrides. -DEFAULT_CONFIG = SEISMIC_NODE_DIR / "Pulumi.dev.yaml" +from tee.cli.common.repo import DEFAULT_STACK_CONFIG as DEFAULT_CONFIG +from tee.cli.common.repo import SEISMIC_NODE_DIR # Per-node descriptors land here when no --network ties the cohort to a # network directory (gitignored either way) so a cohort's outputs stay @@ -69,8 +62,9 @@ def _resolve_out_dir(out_dir: str | None, network: Path | None) -> Path: """Descriptor destination: explicit --out-dir wins; a --network cohort's - descriptors live under the network directory's nodes/ subdir (where - genesis-ceremony finds them by default); else the shared descriptors/.""" + descriptors live under the network directory's nodes/ subdir (where the + founding steps — harvest, assemble, configure — find them by default); + else the shared descriptors/.""" if out_dir: return Path(out_dir) if network is not None: @@ -111,15 +105,20 @@ def _node_config(template: Mapping, name: str) -> dict[str, auto.ConfigValue]: def _check_vhd_matches_network(template: Mapping, network_dir: Path) -> str: """Refuse to provision when the image pin names an artifact the network's - measurement policy doesn't cover. + measurements input doesn't cover. Name-level tripwire only: the basename of the config's `vhd_blob_url` is - the image artifact filename (`seismic[-dev]_..vhd`), which is - also the policy records' `measurement_id`. Comparing the two catches a - stale or typo'd image pin before any cloud resource exists — hours before - the genesis ceremony's block-0 assert would surface it. No VHD bytes are - inspected; the *running* VM is verified cryptographically at attestation - time, never here. + the image artifact filename (`seismic[-dev]_..vhd`), which + is also the `measurement_id` seismic-images' `make measure` stamps into + its measurements output (`manifest init --measurement-id` overrides it; + an already-promoted policy carries it per record). Comparing the two + catches a stale or typo'd image pin before any cloud resource exists. + The *authored input* is checked, not the assembled artifact set, + because provisioning precedes assembly — the founding order is + up → harvest → assemble, so at `up` time the inputs are all a network + directory holds. No VHD bytes are + inspected; the running VM is verified cryptographically at harvest and + attestation time, never here. """ vhd_url = next( ( @@ -133,34 +132,84 @@ def _check_vhd_matches_network(template: Mapping, network_dir: Path) -> str: raise SystemExit( "--network given but the stack config carries no vhd_blob_url to check" ) - manifest_path = network_dir / manifest_mod.MANIFEST_FILENAME - policy_path = network_dir / manifest_mod.POLICY_FILENAME - for path in (manifest_path, policy_path): - if not path.is_file(): - raise SystemExit( - f"--network {network_dir}: missing {path.name} " - "(run `manifest assemble` first)" - ) - manifest = manifest_mod.validate_manifest_schema(manifest_path.read_bytes()) - policy_bytes = policy_path.read_bytes() - policy_hash = "0x" + hashlib.sha256(policy_bytes).hexdigest() - if policy_hash != manifest["measurements"]["bootstrap_policy_hash"]: + measurements_path = ( + network_dir / manifest_mod.INPUTS_DIRNAME / manifest_mod.MEASUREMENTS_FILENAME + ) + if not measurements_path.is_file(): + raise SystemExit( + f"--network {network_dir}: missing {manifest_mod.INPUTS_DIRNAME}/" + f"{manifest_mod.MEASUREMENTS_FILENAME} — scaffold the network " + "directory with `manifest init` before provisioning its cohort" + ) + try: + measurements = json.loads(measurements_path.read_bytes()) + except json.JSONDecodeError as e: + raise SystemExit(f"{measurements_path}: not valid JSON: {e}") from None + if isinstance(measurements, list): + # An already-promoted policy: each record names its own image. + ids = [ + record["measurement_id"] + for record in measurements + if isinstance(record, dict) + and isinstance(record.get("measurement_id"), str) + ] + elif isinstance(measurements, dict) and isinstance( + measurements.get("measurement_id"), str + ): + ids = [measurements["measurement_id"]] + else: + ids = [] + if not ids: raise SystemExit( - f"--network {network_dir}: {policy_path.name} does not hash to the " - "manifest's bootstrap_policy_hash — stale or edited artifact set " - "(re-run `manifest assemble`, or `manifest validate` to diagnose)" + f"{measurements_path} carries no measurement_id to check the " + "image pin against — regenerate it with seismic-images' `make " + "measure` (which stamps the field) or re-run `manifest init " + "--measurement-id ` so a stale VHD " + "pin can be caught before provisioning" ) - ids = [record["measurement_id"] for record in json.loads(policy_bytes)] vhd_name = vhd_url.rsplit("/", 1)[-1] if vhd_name not in ids: raise SystemExit( - f"vhd_blob_url points at {vhd_name!r} but the network's measurement " - f"policy covers only: {', '.join(ids)}. Stale image pin or wrong " + f"vhd_blob_url points at {vhd_name!r} but {measurements_path} " + f"covers only: {', '.join(ids)}. Stale image pin or wrong " "--network dir; refusing to provision." ) return vhd_name +def _cohort_size(count: int | None, network: Path | None) -> int: + """How many nodes to provision: the authored founder set, or --count. + + A network directory already states the cohort size — one withdrawal + credential per founding node — and harvest and assemble both refuse a + cohort that doesn't match it, so taking the count from the inputs is + the only spelling that can't drift. --count stays accepted (and + required without --network), but must agree. + """ + if network is None: + if count is None: + raise SystemExit("--count is required without --network") + return count + path = network / manifest_mod.INPUTS_DIRNAME / manifest_mod.FOUNDERS_FILENAME + try: + authored = len(manifest_mod.load_founder_credentials(path)) + except manifest_mod.GateError as e: + raise SystemExit(str(e)) from None + if not authored: + raise SystemExit( + f"{path} is empty — it decides the cohort size, so author one " + "withdrawal-credentials address per founding node (`manifest " + "init --founders N` scaffolds placeholders)" + ) + if count is not None and count != authored: + raise SystemExit( + f"--count {count} contradicts the {authored} withdrawal " + f"credential(s) in {path}, which are the founding set: drop " + "--count, or re-author the credentials for the cohort you want" + ) + return authored + + def _descriptor_from_outputs(outputs: Mapping[str, auto.OutputValue]) -> dict: # Just the handoff fields (DESCRIPTOR_KEYS), not every stack output. return {key: outputs[key].value for key in DESCRIPTOR_KEYS if key in outputs} @@ -218,7 +267,13 @@ def _parse_up_args() -> argparse.Namespace: description="Provision a cohort of TDX nodes (one Pulumi stack each)." ) parser.add_argument( - "--count", type=int, required=True, help="Number of nodes to provision." + "--count", + type=int, + default=None, + help=( + "Number of nodes to provision. Optional with --network, which " + "takes the count from the authored withdrawal credentials." + ), ) parser.add_argument( "--config", @@ -253,37 +308,41 @@ def _parse_up_args() -> argparse.Namespace: default=None, metavar="DIR", help=( - "Network directory (from `manifest assemble`) this cohort is for: " - "refuse to provision unless the config's vhd_blob_url basename is " - "one of the measurement policy's measurement_id records (catches a " - "stale image pin before any resource exists; name check only — " - "attestation is the cryptographic gate), and write the descriptors " - "to /nodes/, where genesis-ceremony finds them by default." + "Network directory (from `manifest init`) this cohort is for: " + "refuse to provision unless the config's vhd_blob_url basename " + "matches the measurement_id in the directory's " + "inputs/measurements.json (catches a stale image pin before any " + "resource exists; name check only — attestation is the " + "cryptographic gate), and write the descriptors to /nodes/, " + "where harvest and assemble find them by default." ), ) - return parser.parse_args() + args = parser.parse_args() + # Absolute from here on, so every path this CLI prints is clickable. + if args.network is not None: + args.network = args.network.resolve() + return args def up_main() -> None: args = _parse_up_args() + count = _cohort_size(args.count, args.network) _ensure_passphrase(confirm=True) with open(args.config) as f: template = yaml.safe_load(f) if args.network is not None: vhd_name = _check_vhd_matches_network(template, args.network) - print( - f"Image pin {vhd_name} is covered by the {args.network} measurement policy." - ) + print(f"Image pin {vhd_name} matches the {args.network} measurements input.") prefix = args.stack_prefix or f"{_env_from_config(args.config)}-bootstrap-node" out_dir = _resolve_out_dir(args.out_dir, args.network) out_dir.mkdir(parents=True, exist_ok=True) + names = [f"{prefix}-{i}" for i in range(1, count + 1)] # Sequential on purpose: clearer logs and gentler on Azure quota for a # first cut. The stacks are independent, so a future --parallel can run # them on a thread pool without changing anything else here. - for i in range(1, args.count + 1): - name = f"{prefix}-{i}" - print(f"\n=== {name}: provisioning node {i}/{args.count} ===") + for i, name in enumerate(names, start=1): + print(f"\n=== {name}: provisioning node {i}/{count} ===") stack = auto.create_or_select_stack( stack_name=name, work_dir=str(SEISMIC_NODE_DIR) @@ -299,45 +358,53 @@ def up_main() -> None: genesis_desc = out_dir / f"{prefix}-1.json" join_flags = "".join( - f" --join {out_dir / f'{prefix}-{i}.json'}" for i in range(2, args.count + 1) + f" --join {out_dir / f'{prefix}-{i}.json'}" for i in range(2, count + 1) ) if args.network is not None: - # The network dir exists and holds the descriptors, so the remaining - # steps need no placeholders — and genesis-ceremony finds the cohort - # in /nodes/ on its own. + # The network dir holds the descriptors, so harvest and assemble + # find the cohort in /nodes/ on their own. manifest_arg = args.network / manifest_mod.MANIFEST_FILENAME print( - f"\nProvisioned {args.count} node(s) for {args.network}. Next:\n" + f"\nProvisioned {count} node(s) for {args.network}. Next:\n" "\n" - "1. Configure the cohort (re-run on every node reboot):\n" + "1. Harvest the founding keys (polls each box's summit-key-holder,\n" + " DCAP-verifies the quotes, archives them under inputs/harvest/):\n" + f" seismic-tee-network harvest {args.network}\n" + "2. Assemble the artifact set (pins the harvested validator set,\n" + f" mints network_id), then commit {args.network}:\n" + f" seismic-tee-network manifest assemble {args.network}\n" + "3. Configure the cohort (re-run on every node reboot):\n" f" seismic-tee-network configure --genesis {genesis_desc}" f"{join_flags} \\\n" - f" --manifest {manifest_arg}\n" - "2. Run the genesis ceremony (one-shot; --node defaults to the\n" - f" descriptors in {out_dir}):\n" - f" seismic-tee-network genesis-ceremony --manifest {manifest_arg}" + f" --manifest {manifest_arg}" ) return - node_flags = " ".join( - f"--node {out_dir / f'{prefix}-{i}.json'}" for i in range(1, args.count + 1) - ) net = "tee/networks/" + moved_genesis = f"{net}/nodes/{prefix}-1.json" + moved_joins = "".join( + f" --join {net}/nodes/{prefix}-{i}.json" for i in range(2, count + 1) + ) print( - f"\nProvisioned {args.count} node(s). To found a network on them:\n" + f"\nProvisioned {count} node(s). To found a network on them:\n" "\n" - "1. Create the network directory (once per network; assemble shells out\n" - " to `seismic-reth genesis-hash`, so seismic-reth must be on PATH):\n" + "1. Create the network directory (once per network):\n" f" seismic-tee-network manifest init {net} \\\n" " --reth-genesis \\\n" - " --measurements --measurement-id \n" - f" # edit {net}/inputs/summit-genesis.toml, then:\n" + " --measurements --measurement-id \\\n" + f" --founders {count}\n" + f" # edit {net}/inputs/summit-genesis.toml and the scaffolded\n" + f" # {net}/inputs/founder-withdrawal-credentials.json\n" + "2. Move the descriptors into the network directory (assemble reads\n" + " each founding validator's IP from them):\n" + f" mkdir -p {net}/nodes && mv {out_dir}/*.json {net}/nodes/\n" + "3. Harvest the founding keys from the live cohort:\n" + f" seismic-tee-network harvest {net}\n" + "4. Assemble the artifact set (seismic-reth, summit, and the\n" + " admission + verify-quote CLIs must be on PATH), then commit it:\n" f" seismic-tee-network manifest assemble {net}\n" - "2. Configure the cohort (re-run on every node reboot):\n" - f" seismic-tee-network configure --genesis {genesis_desc}" - f"{join_flags} \\\n" - f" --manifest {net}/network-manifest.json\n" - "3. Run the genesis ceremony (one-shot) over the same descriptors:\n" - f" seismic-tee-network genesis-ceremony {node_flags} \\\n" + "5. Configure the cohort (re-run on every node reboot):\n" + f" seismic-tee-network configure --genesis {moved_genesis}" + f"{moved_joins} \\\n" f" --manifest {net}/network-manifest.json" ) @@ -390,7 +457,10 @@ def _parse_down_args() -> argparse.Namespace: "its nodes/ subdir is where the descriptors get deleted from." ), ) - return parser.parse_args() + args = parser.parse_args() + if args.network is not None: + args.network = args.network.resolve() + return args def _destroy_one(stack_name: str, out_dir: Path) -> None: diff --git a/tee/cli/network/tests/test_harvest.py b/tee/cli/network/tests/test_harvest.py index 44757a2b..7bf6ac7a 100644 --- a/tee/cli/network/tests/test_harvest.py +++ b/tee/cli/network/tests/test_harvest.py @@ -56,11 +56,13 @@ class ParseArgsTests(unittest.TestCase): def setUp(self): self._tmp = tempfile.TemporaryDirectory() self.addCleanup(self._tmp.cleanup) - self.dir = Path(self._tmp.name) + # Resolved: _parse_args makes every derived path absolute so the + # paths harvest prints are clickable. + self.dir = Path(self._tmp.name).resolve() inputs = self.dir / manifest_mod.INPUTS_DIRNAME inputs.mkdir() (inputs / manifest_mod.MEASUREMENTS_FILENAME).write_text("{}") - (inputs / manifest_mod.FOUNDERS_FILENAME).write_text("{}") + (inputs / manifest_mod.FOUNDERS_FILENAME).write_text("[]") self.nodes = self.dir / manifest_mod.NODES_DIRNAME self.nodes.mkdir() @@ -138,34 +140,35 @@ def setUp(self): def _write(self, obj) -> None: self.path.write_text(json.dumps(obj)) - def test_exact_match_returns_map(self): - self._write({"node-1": ADDRESS, "node-2": ADDRESS}) + def test_matching_count_returns_addresses(self): + self._write([ADDRESS, ADDRESS]) founders = harvest.load_founders(self.path, ["node-1", "node-2"]) - self.assertEqual(founders["node-1"], ADDRESS) + self.assertEqual(founders, [ADDRESS, ADDRESS]) - def test_cohort_box_without_entry_aborts(self): - self._write({"node-1": ADDRESS}) + def test_too_few_credentials_aborts(self): + self._write([ADDRESS]) with self.assertRaises(SystemExit) as ctx: harvest.load_founders(self.path, ["node-1", "node-2"]) - self.assertIn("no founder entry: node-2", str(ctx.exception)) + self.assertIn("1 withdrawal credential(s)", str(ctx.exception)) + self.assertIn("2 box(es)", str(ctx.exception)) - def test_entry_without_cohort_box_aborts(self): - self._write({"node-1": ADDRESS, "node-9": ADDRESS}) + def test_too_many_credentials_aborts(self): + self._write([ADDRESS, ADDRESS]) with self.assertRaises(SystemExit) as ctx: harvest.load_founders(self.path, ["node-1"]) - self.assertIn("no cohort box: node-9", str(ctx.exception)) + self.assertIn("2 withdrawal credential(s)", str(ctx.exception)) def test_malformed_address_aborts(self): - self._write({"node-1": "0x1234"}) + self._write(["0x1234"]) with self.assertRaises(SystemExit) as ctx: harvest.load_founders(self.path, ["node-1"]) - self.assertIn("node-1", str(ctx.exception)) + self.assertIn("0x1234", str(ctx.exception)) - def test_non_object_aborts(self): - self._write([ADDRESS]) + def test_non_list_aborts(self): + self._write({"node-1": ADDRESS}) with self.assertRaises(SystemExit) as ctx: harvest.load_founders(self.path, ["node-1"]) - self.assertIn("expected a JSON object", str(ctx.exception)) + self.assertIn("expected a JSON array", str(ctx.exception)) class FetchQuoteTests(unittest.TestCase): @@ -276,8 +279,11 @@ class VerifyQuoteTests(unittest.TestCase): def _run(self, returncode=0, stdout=b"", stderr=b""): completed = mock.Mock(returncode=returncode, stdout=stdout, stderr=stderr) + # The shell-out lives in manifest.verify_quote_evidence (shared with + # `manifest assemble`'s re-verification); harvest wraps it with the + # burn messaging. with mock.patch.object( - harvest.subprocess, "run", return_value=completed + manifest_mod.subprocess, "run", return_value=completed ) as run: report = harvest.verify_quote( target(), @@ -319,7 +325,7 @@ def test_optional_flags_forwarded(self): returncode=0, stdout=json.dumps(self.REPORT).encode(), stderr=b"" ) with mock.patch.object( - harvest.subprocess, "run", return_value=completed + manifest_mod.subprocess, "run", return_value=completed ) as run: harvest.verify_quote( target(), diff --git a/tee/cli/network/tests/test_orchestrator.py b/tee/cli/network/tests/test_orchestrator.py index cd029798..3a95aecc 100644 --- a/tee/cli/network/tests/test_orchestrator.py +++ b/tee/cli/network/tests/test_orchestrator.py @@ -4,66 +4,68 @@ uv run python -m unittest discover -s tee/tests -v """ -import hashlib import json import tempfile import unittest from pathlib import Path -from tee.cli.common.manifest import render_manifest +from tee.cli.common import manifest as manifest_mod from tee.cli.network.orchestrator import ( DEFAULT_OUT_DIR, _check_vhd_matches_network, + _cohort_size, _resolve_out_dir, ) -POLICY = json.dumps( - [ - { - "measurement_id": "seismic-dev_2026-07-02.5c3b5e.vhd", - "attestation_type": "azure-tdx", - "measurements": {"4": {"expected": "ab" * 24}}, - } - ] -).encode() - - -def _manifest_bytes(policy: bytes) -> bytes: - return render_manifest( - { - "manifest_version": 1, - "name": "t", - "eth": {"chain_id": 5124, "genesis_hash": "0x" + "12" * 32}, - "summit": {"genesis_config_digest": "0x" + "bb" * 32, "namespace": "t"}, - "measurements": { - "bootstrap_policy_hash": "0x" + hashlib.sha256(policy).hexdigest(), - "contracts": { - "registry": "0x" + "10" * 20, - "authority": "0x" + "11" * 20, - }, - }, - } - ) +VHD = "seismic-dev_2026-07-02.5c3b5e.vhd" class VhdNetworkCheckTests(unittest.TestCase): - """`up --network` refuses a VHD pin the measurement policy doesn't cover.""" + """`up --network` refuses a VHD pin the measurements input doesn't cover. - URL = "https://acct.blob.core.windows.net/dev/seismic-dev_2026-07-02.5c3b5e.vhd" + The check reads the authored inputs/measurements.json, not the + assembled artifact set: provisioning precedes assembly (the founding + order is up → harvest → assemble), so at `up` time the inputs are all + a network directory holds. + """ + + URL = f"https://acct.blob.core.windows.net/dev/{VHD}" def setUp(self): self._tmp = tempfile.TemporaryDirectory() self.addCleanup(self._tmp.cleanup) self.net = Path(self._tmp.name) - (self.net / "measurement-policy.json").write_bytes(POLICY) - (self.net / "network-manifest.json").write_bytes(_manifest_bytes(POLICY)) + self.inputs = self.net / manifest_mod.INPUTS_DIRNAME + self.inputs.mkdir() + self._write_measurements( + {"measurement_id": VHD, "measurements": {"4": {"expected": "ab" * 24}}} + ) + + def _write_measurements(self, obj) -> None: + path = self.inputs / manifest_mod.MEASUREMENTS_FILENAME + path.write_text(json.dumps(obj)) def _template(self, url: str) -> dict: return {"config": {"seismic-tee-deploy:vhd_blob_url": url}} - def test_matching_pin_passes(self): + def test_matching_stamped_wrapper_passes(self): + name = _check_vhd_matches_network(self._template(self.URL), self.net) + self.assertEqual(name, VHD) + + def test_matching_promoted_policy_record_passes(self): + # The measurements input may already be a promoted policy; each + # record then names its own image. + self._write_measurements( + [ + { + "measurement_id": VHD, + "attestation_type": "azure-tdx", + "measurements": {"4": {"expected": "ab" * 24}}, + } + ] + ) name = _check_vhd_matches_network(self._template(self.URL), self.net) - self.assertEqual(name, "seismic-dev_2026-07-02.5c3b5e.vhd") + self.assertEqual(name, VHD) def test_uncovered_pin_refused(self): bad = self.URL.replace("5c3b5e", "999999") @@ -71,19 +73,19 @@ def test_uncovered_pin_refused(self): _check_vhd_matches_network(self._template(bad), self.net) self.assertIn("refusing to provision", str(ctx.exception)) - def test_tampered_policy_refused(self): - # Policy bytes not hashing to the manifest's commitment must refuse - # before the measurement_id comparison is even attempted. - (self.net / "measurement-policy.json").write_bytes(POLICY + b"\n") + def test_missing_measurements_input_refused(self): + (self.inputs / manifest_mod.MEASUREMENTS_FILENAME).unlink() with self.assertRaises(SystemExit) as ctx: _check_vhd_matches_network(self._template(self.URL), self.net) - self.assertIn("bootstrap_policy_hash", str(ctx.exception)) + self.assertIn("manifest init", str(ctx.exception)) - def test_missing_artifact_set_refused(self): - (self.net / "network-manifest.json").unlink() + def test_unstamped_wrapper_refused(self): + # A wrapper without a stamped measurement_id leaves nothing to + # compare the pin against — refused, not silently skipped. + self._write_measurements({"measurements": {"4": {"expected": "ab" * 24}}}) with self.assertRaises(SystemExit) as ctx: _check_vhd_matches_network(self._template(self.URL), self.net) - self.assertIn("manifest assemble", str(ctx.exception)) + self.assertIn("measurement_id", str(ctx.exception)) def test_config_without_vhd_url_refused(self): with self.assertRaises(SystemExit): @@ -103,5 +105,52 @@ def test_default_shared_dir(self): self.assertEqual(_resolve_out_dir(None, None), DEFAULT_OUT_DIR) +class CohortSizeTests(unittest.TestCase): + """`up --count` is optional with --network: the authored withdrawal + credentials are the founding set, so they size the cohort.""" + + ADDRESS = "0x" + "c0" * 20 + + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.addCleanup(self._tmp.cleanup) + self.net = Path(self._tmp.name) + self.inputs = self.net / manifest_mod.INPUTS_DIRNAME + self.inputs.mkdir() + self._write_credentials([self.ADDRESS] * 3) + + def _write_credentials(self, addresses) -> None: + (self.inputs / manifest_mod.FOUNDERS_FILENAME).write_text(json.dumps(addresses)) + + def test_count_derived_from_credentials(self): + self.assertEqual(_cohort_size(None, self.net), 3) + + def test_matching_count_accepted(self): + self.assertEqual(_cohort_size(3, self.net), 3) + + def test_contradicting_count_refused(self): + with self.assertRaises(SystemExit) as ctx: + _cohort_size(4, self.net) + self.assertIn("contradicts", str(ctx.exception)) + + def test_empty_credentials_refused(self): + self._write_credentials([]) + with self.assertRaises(SystemExit) as ctx: + _cohort_size(None, self.net) + self.assertIn("--founders", str(ctx.exception)) + + def test_missing_credentials_refused(self): + (self.inputs / manifest_mod.FOUNDERS_FILENAME).unlink() + with self.assertRaises(SystemExit) as ctx: + _cohort_size(None, self.net) + self.assertIn(manifest_mod.FOUNDERS_FILENAME, str(ctx.exception)) + + def test_count_required_without_network(self): + self.assertEqual(_cohort_size(2, None), 2) + with self.assertRaises(SystemExit) as ctx: + _cohort_size(None, None) + self.assertIn("--count is required", str(ctx.exception)) + + if __name__ == "__main__": unittest.main() diff --git a/tee/docs/runbook-devnet.md b/tee/docs/runbook-devnet.md new file mode 100644 index 00000000..9748ad22 --- /dev/null +++ b/tee/docs/runbook-devnet.md @@ -0,0 +1,185 @@ +# Runbook: found a throwaway devnet + +The copy-pasteable recipe for bringing up (and tearing down) a scratch +cohort end to end: init → provision → harvest → assemble → configure → +smoke-test → down. Every command is meant to be run verbatim after +setting the three variables below. + +This doc is operational on purpose. For what each step *means* — the +two-CLI split, the network-directory layout, the measurement-admission +pipeline, the operator footguns — read [../README.md](../README.md) and +[../networks/README.md](../networks/README.md) first. + +## Prerequisites + +Everything in the tee README's "Prerequisites" section, in short: + +- `uv sync` done; `az login`; the `pulumi` CLI on PATH. +- On PATH for harvest + assemble: `verify-quote` (enclave repo), + `seismic-measurement-admission`, `summit`, and `seismic-reth` + (or pass `--verify-quote-bin` / `--admission-bin` / `--summit-bin` / + `--reth-bin`). +- A published dev image whose measurements you have: + `vhd_blob_url` in `pulumi/seismic_node/Pulumi.dev.yaml` must point at + it, and `../seismic-images/build/measurements.json` must be its + measured-boot output (`make measure` stamps the artifact's + `measurement_id` into it) — `up --network` refuses on a basename + mismatch, and harvest fails DCAP verification against the wrong + image. The image + must carry a tdx-init matching the current `[network]`/`[node]` config + schema; an older image 400s the config POST. +- `PULUMI_CONFIG_PASSPHRASE` set, or let `up`/`down` prompt you + (empty = no passphrase). + +## Identifiers for this run + +```bash +NET="tee/networks/tmp-devnet-1" +PREFIX="tmp-devnet-1" +COUNT=4 +``` + +`tmp-*` network directories are gitignored wholesale (see +`networks/.gitignore`), so this run never dirties git and the directory +is `rm -rf`-able at the end. Keep `PREFIX` unique per concurrent cohort — +it names the Pulumi stacks, resource groups, and DNS records. + +## 1. Create the network directory and author its inputs + +```bash +uv run seismic-tee-network manifest init "$NET" \ + --reth-genesis ../seismic-reth/crates/seismic/chainspec/res/genesis/dev.json \ + --measurements ../seismic-images/build/measurements.json \ + --founders "$COUNT" +``` + +No `--measurement-id`: the measurements file already carries the +`measurement_id` that `make measure` stamped into it (the flag exists as +an override for unstamped or re-labeled measurements). + +The reth genesis is required (chain state + contract alloc — an external +fact init can't invent); the dev chain spec from the seismic-reth +checkout is the canonical throwaway choice. The summit genesis is *not* +passed: it holds only per-network parameter choices, so init scaffolds +an editable starter at `$NET/inputs/summit-genesis.toml` — review/edit +it if you care to (the defaults are fine for a throwaway; `namespace` +defaults to the directory basename, which is unique per run and is what +prevents cross-cohort signature replay). To carry in pre-authored summit +parameters instead, pass `--summit-genesis `. + +`--founders "$COUNT"` fills the last authored input, +`inputs/founder-withdrawal-credentials.json`, with `$COUNT` placeholder +addresses (`0x00…0`) — one per founding node, paired to the boxes in +node-name order at assemble time. A network that matters puts the +founders' real addresses there instead; the placeholders are exactly what +a throwaway wants. + +(Assembly comes *after* the cohort exists — step 4 — because the +founding validator set is harvested from the live boxes.) + +## 2. Provision the cohort + +One independent Pulumi stack per node; shared settings inherit from +[`../pulumi/seismic_node/Pulumi.dev.yaml`](../pulumi/seismic_node/Pulumi.dev.yaml) +(override with `--config`). Read it before spending: `vhd_blob_url` is the +image the whole cohort boots (`up` refuses it unless its basename matches +the measurements input), and `operator_ip_cidr`, `vm_size`, +`data_disk_size_gb`, and the region are all set there. The committed dev +values leave `operator_ip_cidr` at `0.0.0.0/0`, which opts out of the NSG +restriction on the operator-only ports — including tdx-init's +unauthenticated first-POST-wins config listener. + +```bash +uv run seismic-tee-network up \ + --stack-prefix "$PREFIX" \ + --network "$NET" +``` + +Descriptors land in `$NET/nodes/$PREFIX-{1..N}.json`. No `--count`: the +withdrawal credentials authored in step 1 are the founding set, so their +number is the cohort size (pass `--count` if you want it stated — it must +agree). + +## 3. Harvest the founding keys + +Each box boots identity-free, its summit-key-holder serving +`{pubkeys, quote}` until the box takes its config POST. Harvest fetches +and DCAP-verifies every box's keys against the intended image +measurements, archiving them under `$NET/inputs/harvest/`: + +```bash +uv run seismic-tee-network harvest "$NET" +``` + +Any anomaly (a box that already took a POST, a failed verification, +repeated pubkeys) burns the whole harvest: `down` + fresh `up`, never +retry around it. + +## 4. Assemble and validate the artifact set + +```bash +uv run seismic-tee-network manifest assemble "$NET" +uv run seismic-tee-network manifest validate "$NET" +``` + +This pins the harvested validator set into the summit genesis, compiles +the measurement policy into the reth genesis, and mints `network_id`. + +## 5. Configure the cohort + +Node 1 is genesis (mints `root_key`); the rest join, fetching `root_key` +from it over the attested handshake: + +```bash +joins=() +for i in $(seq 2 "$COUNT"); do + joins+=(--join "$NET/nodes/$PREFIX-$i.json") +done +uv run seismic-tee-network configure \ + --genesis "$NET/nodes/$PREFIX-1.json" \ + "${joins[@]}" \ + --manifest "$NET/network-manifest.json" +``` + +Greenfield bootstrap is two-stage (the genesis node's enode isn't known +until its reth is up, then the joiners are bootnoded off it) and watches +every node's first-boot LUKS provisioning, so this takes a while. On +success it persists the founding enode set to `$NET/nodes/bootnodes.json`; +a later `configure` run (e.g. after a reboot wiped the tmpfs config dir) +reuses it and reconfigures the whole cohort in one parallel pass. + +## 6. Smoke-test + +Blocks should advance on every node, and all nodes must report the same +`tx_io_pk` (one shared `root_key` — two values means a split network): + +```bash +for descriptor in "$NET"/nodes/"$PREFIX"-*.json; do + fqdn=$(jq -r .fqdn "$descriptor") + echo "=== $(basename "$descriptor") — $fqdn ===" + + curl -fsS "https://$fqdn/rpc" \ + -H 'content-type: application/json' \ + --data '{"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params":[]}' | + jq -r '"block: \(.result)"' + + curl -fsS "https://$fqdn/rpc" \ + -H 'content-type: application/json' \ + --data '{"jsonrpc":"2.0","id":1,"method":"seismic_getTeePublicKey","params":[]}' | + jq -r '"tx_io_pk: \(.result)"' +done +``` + +## 7. Tear down + +```bash +uv run seismic-tee-network down \ + --count "$COUNT" \ + --stack-prefix "$PREFIX" \ + --network "$NET" + +rm -rf "$NET" +``` + +`down` destroys each stack's resources, removes the stack, and deletes +its descriptor; the `rm -rf` discards the throwaway identity itself. diff --git a/tee/networks/.gitignore b/tee/networks/.gitignore index 567b46de..9d8b1b20 100644 --- a/tee/networks/.gitignore +++ b/tee/networks/.gitignore @@ -2,3 +2,9 @@ # regenerated each deploy; `down --network` deletes them), never committed. # Everything else in a network directory is its committable identity. */nodes/ + +# Throwaway foundings (see tee/docs/runbook-devnet.md): the whole directory is +# ignored, so a scratch cohort can be founded, torn down, and deleted +# without touching git. A network that matters gets an unprefixed +# directory — its identity is committed. +tmp-*/ diff --git a/tee/networks/README.md b/tee/networks/README.md index 0318d49f..de6e7c70 100644 --- a/tee/networks/README.md +++ b/tee/networks/README.md @@ -1,12 +1,37 @@ # Network directories -One directory per network: the committable identity of a founded (or -foundable) network. The authored inputs live under `inputs/`; -`seismic-tee-network manifest assemble` derives the artifact set from them -at the top level. Everything top-level is hash-pinned by -`network-manifest.json` — whose SHA-256 is the network's `network_id` — -and everything under `inputs/` is provenance. The founding workflow lives -in the tee README ("Creating a new network"). +One directory per network: the committable identity of a founded +network. The authored inputs live under `inputs/`, joined mid-founding by +the harvested facts (`inputs/harvest/` — the cohort's TEE-born founding +keys and their quotes); `seismic-tee-network manifest assemble` derives +the artifact set from them at the top level. Everything top-level is +hash-pinned by `network-manifest.json` — whose SHA-256 is the network's +`network_id` — and everything under `inputs/` is provenance. + +```text +tee/networks// +├── inputs/ provenance (authored + harvested) +│ ├── reth-genesis.json authored: policy-free EL genesis +│ ├── summit-genesis.toml authored: consensus parameter choices +│ ├── measurements.json authored: raw PCRs from `make measure`, +│ │ measurement_id stamped by `init` +│ ├── founder-withdrawal-credentials.json +│ │ authored: one address per founding +│ │ node, in node-name order +│ └── harvest/ written by `network harvest` +│ └── .json founding pubkeys + quote + verification +├── nodes/ runtime infra state (gitignored) +│ ├── .json descriptor from `up --network` (live IP) +│ └── bootnodes.json founding enode set from `configure` +│ +│ artifact set: derived by `assemble`, every +│ file below hash-pinned by the manifest +├── network-manifest.json SHA-256 of these bytes = network_id +├── reth-genesis.json input + compiled registry storage +├── summit-genesis.toml input + eth_genesis_hash + validator set +└── measurement-policy-bootstrap.json + allowlist promoted from measurements +``` ![How assemble derives the artifact set, what pins what, and where the genesis ceremony picks it up](network-dir.svg) @@ -17,20 +42,33 @@ network's immutable identity — a founded network's `network_id` must never drift. The `nodes/` descriptors are runtime output (live IPs) and stay gitignored. +Throwaway foundings go in a `tmp-*` directory instead — those are +gitignored wholesale, so a scratch cohort can be founded, torn down, and +`rm -rf`'d without touching git (the copy-pasteable recipe is +[../docs/runbook-devnet.md](../docs/runbook-devnet.md)). If a throwaway +turns out to matter, renaming the directory is enough to commit it — +`network_id` is minted from the manifest bytes, not the path — but the +manifest keeps the `tmp-*` name it was assembled under (the name is part +of those bytes), so a network you already suspect will matter deserves a +real directory name from the start. + ## example-devnet -A **template, not a deployed network** — no cohort runs under this -identity. Two intended uses: - -- **Found a throwaway test devnet.** The artifact set is assembled and - valid, so `up --network … --count N` → `configure` → `genesis-ceremony` - brings up a working cohort. If two such cohorts might ever run at once, - give each its own directory with a distinct `namespace` and `chainId` - (cohorts sharing a `network_id` can cross-replay attestation - transcripts). Note `inputs/measurements.json` snapshots a specific - image build — when the deployed VHD moves on, refresh it and - re-assemble (`up --network` refuses on a pin/policy mismatch). -- **Start a real network.** Don't reuse or copy this directory — run - `manifest init ` and author fresh inputs. `namespace` (the BLS - signature domain separator) and `chainId` must be unique per network - that matters. +[example-devnet/](example-devnet/) is a committed example of the shape (minus the +gitignored `nodes/` and, having never been founded from a live cohort, +`inputs/harvest/` and the founder credentials — see below). + +A **schema example, not a runnable founding** — no cohort runs under this +identity, and none can be brought up from it: `assemble` pins the +founding validator set from a live harvest, so producing an artifact set +takes a provisioned cohort, and this directory's committed artifacts +carry an empty validator set no founding produces. It exists to document +the directory shape — the artifacts are internally consistent (every +manifest pin matches its file), so schema-level tooling can be exercised +against it. + +To found any network, throwaway or real, don't reuse or copy this +directory: run `manifest init `, author fresh inputs, and follow +the founding workflow in the tee README. `namespace` (the BLS signature +domain separator) and `chainId` must be unique per network that matters +(cohorts sharing them can cross-replay signatures). diff --git a/tee/networks/example-devnet/measurement-policy.json b/tee/networks/example-devnet/measurement-policy-bootstrap.json similarity index 100% rename from tee/networks/example-devnet/measurement-policy.json rename to tee/networks/example-devnet/measurement-policy-bootstrap.json diff --git a/tee/pulumi/seismic_node/Pulumi.dev.yaml b/tee/pulumi/seismic_node/Pulumi.dev.yaml index 593686c2..bd96d905 100644 --- a/tee/pulumi/seismic_node/Pulumi.dev.yaml +++ b/tee/pulumi/seismic_node/Pulumi.dev.yaml @@ -15,7 +15,7 @@ config: # Required so Azure can authorize the cross-RG blob read for managed disk Import. # Get via: az storage account show -n seismicimages -g seismic-images --query id -o tsv seismic-tee-deploy:vhd_storage_account_id: /subscriptions/214887ea-51a7-4ca7-9cec-29b3cf3d311c/resourceGroups/seismic-images/providers/Microsoft.Storage/storageAccounts/seismicimages - seismic-tee-deploy:vhd_blob_url: https://seismicimages.blob.core.windows.net/dev/seismic-dev_2026-07-22.bcbb14.vhd + seismic-tee-deploy:vhd_blob_url: https://seismicimages.blob.core.windows.net/dev/seismic-dev_2026-08-07.92c343.vhd # Stack-specific deploy config. seismic-tee-deploy:dns_record_name: tee-dev-az-1 # → tee-dev-az-1.seismicdev.net seismic-tee-deploy:resource_group: seismic-tee-dev