EdgeZero full-migration umbrella design spec#839
Draft
aram356 wants to merge 36 commits into
Draft
Conversation
Defines the end-state, current-state gap analysis, and an ordered set of phases (stores, config injection, secret externalization, extractors, legacy-path removal) for moving Trusted Server completely onto EdgeZero. Phase 0 (State<T> extractor + nested #[secret]) is an upstream edgezero prerequisite tracked separately.
6 tasks
Clarify that platform/mod.rs and types.rs are edited (KV re-export and a shrinking RuntimeServices remain), not deleted, in Phase 1.
…tore id - P0-C: Fastly bypasses run_app (multi-value Set-Cookie, logger reinit, JA4/H2 capture) - add EdgeZero dispatch prereq or documented exception - P-BOOT: specify boot-time config/secret store access for Cloudflare/Spin (build_state runs before request context; registry is per-request) - D5: unify the split app-config store id (app_config vs trusted_server_config) - Promote the secret inventory to a spec artifact; array + optional secrets confirmed present, so edgezero #305 must ship ArrayEach + Option<String> - Phase 4 acceptance: per-adapter route parity (EC routes are Fastly-only) - Phase 5: expand deletion ledger (route_tests, viceroy config, fastly.toml, runbook) - Scope the include_str! ban to adapter/runtime app-config only
…iation (D5) - D6: EdgeZero stores are read-only, but KeyRotationManager writes/deletes config+secrets at runtime for /_ts/admin/keys/*. management_api.rs cannot be deleted in Phase 1 unconditionally; gate on a keep/move-to-ops/upstream decision (R10) - D5 expanded: reconcile ALL runtime store ids (app_config, secrets, JWKS, DataDome ts_secrets, S3, fixtures) with edgezero.toml or strict lookup fails - Fastly needs explicit registry injection into its custom oneshot path - Phase 1 plan starts with a store-capability inventory, not deletions
Task 1 is a decision gate (store-id inventory + D5/D6) per review, not deletions. Read-path migration and Fastly custom-dispatch registry injection are fully specified; management_api.rs deletion is gated on the D6 decision.
- Lock plan to D6-a; D6-b/c stop after Task 1 (separate plan) - Kind-aware store-id reconciliation (kv/config/secrets), incl. ec_store as KV - Composite read/write store bridge with a write-delegation test - New task: migrate Fastly/Axum BOOT config read to EdgeZero before deleting impls - Local Fastly registry builders (EdgeZero builders are pub(crate)); R11 - Concrete named tests, files, routes, fixtures (no <test_name> placeholders) - Sync spec D5 (kind-aware), Phase 1 boot-read, R11
…e tests - Task 3: composite holds the whole ConfigRegistry/SecretRegistry and resolves named(store_name) per read (multi-store); strict unknown-id error; test uses 2 ids - Task 1/spec: correct KV ids to ec_store + consent_store (creative/counter/opid are fastly.toml platform stores, not Settings logical ids) - Task 4: core-level loader test with InMemoryConfigStore + ConfigStoreHandle::new - Task 5: add non-default config-id (JWKS) + non-default secret-id (DataDome, S3) tests - Task 6: local EnvConfig runtime-dictionary reader (EdgeZero helper private); R12 - Task 8: tests build registries with >=2 ids and assert unknown id errors strictly - Spec D5/Phase 1: kind-aware incl [stores.kv]; R9 mentions ec_identity_store; R12
- D7: runtime app-config is config-store-only; NO runtime env vars. ts config push reads env at push time and bakes resolved values into the blob. Stores open by logical id (name == id); no runtime EDGEZERO__STORES__*__NAME read. - Task 6: drop local_env_config entirely (D7) - open stores by logical id; resolves the fastly::ConfigStore-has-no-iter and private-helper findings (R12) - creative_store IS a Settings KV id (deprecated); include it, exclude counter_store/opid_store (Fastly-adapter constants) - Task 2/spec: tighten kind-aware KV inventory (ec_store, consent_store, creative_store) - Task 4: run the actual core test name; Task 5: one filter per cargo test invocation
Switch the six edgezero git deps from branch main to worktree-state-nested-secrets-spec-review (stackpop/edgezero#306, stacked on #300) to pick up the Phase 0 State<T> extractor work. cargo check-axum passes.
- DataDome IP-CIDR config store (datadome-ip-bypass) added to store inventory - Explicit app-config decision: store id trusted_server_config, key app_config; rename DEFAULT_CONFIG_STORE_ID + repoint request_signing.config_store_id - Task 3: ConfigRegistry::named returns ConfigStoreBinding -> use binding.handle.get; map EdgeZero Ok(None)/Err to PlatformError - Task 6: exact builder signatures (KV Result<Option<..>,FastlyError>, config/secret Option<..>) + missing-store/open-failure policy - Task 5: DataDome secret read tested on a protected NON-integration route - StoreName reconciled to logical read id (D7); doc + call-site audit step - Task 4: Axum boot reads .edgezero/local-config-trusted_server_config.json, no env override
Blockers: - Hooks::stores() is not overridden on any adapter (empty StoresMetadata); add Task 2 Step 5 to wire stores() from edgezero.toml on all four adapters - Axum uses routes()+AxumDevServer, not run_app; Task 5 Step 0 switches Axum to edgezero_adapter_axum::run_app::<TrustedServerApp>() - request-signing reads use jwks_store/signing_keys but writes use config_store_id/secret_store_id; fix example/fixtures to jwks_store/signing_keys (NOT app_config/secrets); only the app-config store renames to trusted_server_config - PlatformConfigStore/SecretStore mix read+write; Task 3 Step 0 splits write-only PlatformConfigWriter/PlatformSecretWriter so Task 8 can drop reads and compile High/Medium: - D7 softened: EdgeZero builders read EnvConfig but fall back to logical id; we set none - Task 2 Step 6: declare stores in fastly.toml/wrangler.toml/spin.toml/Axum local files - Task 2 test parameterized over example + fixture + all-store-refs config - CI gate adds cargo check-cloudflare + check-spin (wasm surfaces) - Fix stale spec D5/R9 wording
- Axum: exact path = keep AxumDevServer::with_config + .with_{config,kv,secret}_registry
(dev_server::run_app drops custom PORT/axum.toml); add adapter-axum registry builders
- Task 5: extract whole registry via ctx.request().extensions().get::<ConfigRegistry>()
(RequestContext has no whole-registry getter; per-id accessors would wire only default)
- Task 2: enumerate exact file paths; replace brace-glob git add that would hit
non-existent adapter manifest paths
- Spec: state requirement that Fastly management id == runtime logical id for
jwks_store/signing_keys (or supply a mapping, out of Phase 1 scope)
- Task 3: composite writer test asserts (StoreId, key, value) forwarding (D6-a risk)
- Minor: expect("should ...") convention; Task 7 local verify adds check-cloudflare/spin
- Fastly build_per_request_services still read FastlyPlatformConfigStore directly, so injected registries were unused; add Task 6 Step 4b to build RuntimeServices from the composite via registries in extensions - App-config key == store id (trusted_server_config): default_config_key falls back to id and D7 forbids the __KEY env; set CONFIG_BLOB_KEY=trusted_server_config, retire the app_config key (no --key/env needed) - Task 5: rename heading to 'preserve AxumDevServer::with_config + add registries'; list adapter-axum/src/registries.rs (Step 0 already kept with_config) - Task 2 Step 6: exact per-adapter manifest mappings (CF config=KV binding, secrets=flat Worker secrets via wrangler secret put; Spin config/kv=KV labels) - Name S3 secret store id s3-auth explicitly in D5/Task 2 - Fix all-store-refs.toml include_str path (testdata/, not ../testdata/) - Writer traits Send+Sync; expect_err 'should ...' convention
…y file) - Add config_payload.rs (CONFIG_BLOB_KEY) to Task 2 files + git add (was missing) - Task 2 Step 6b: rename app_config in generate-viceroy-config.rs (+ its test), tests/common/config.rs, tests/environments/axum.rs env var (they run in adapter suites and break under the store/key rename) - Task 2 Step 6: Cloudflare/Spin secrets are FLAT (ignore store_name) -> provision the concrete secret KEYS the code reads (signing KIDs, DataDome key name, S3 access_key_id/secret_access_key/session token), not the store id - Task 6: add app.rs to files; test passes only after Step 4b (not 3-4) - Task 1 output table: drop EDGEZERO__STORES__*__NAME column (D7 -> platform resource per adapter, no env mapping)
The prior review's Critical 'APIs do not exist' findings were against a stale cargo-cache checkout (ce6bcf7, edgezero #253 'Add store support for Spin') that predates the registry refactor. Our actual pin is 6ebc29a5 (PR #306), which HAS StoreRegistry/StoresMetadata/Hooks::stores()/dispatch_with_registries/ with_*_registry/[stores.*] ids+default/CF-KV-namespace/Axum-local-file. Added a 'Pinned dependency (verified)' note recording this to prevent re-litigation. Valid fixes: - Cloudflare app.rs settings_from_cloudflare_config_json reads literal value.get("app_config") from the side-channel; change to CONFIG_BLOB_KEY so the key rename doesn't break Cloudflare boot (Phase 2 does the store migration) - Viceroy generator already emits a trusted_server_config store for rollout flags; the app_config->trusted_server_config rename must MERGE the blob into that table, not emit a duplicate table
User chose full convergence on the canonical app-demo wiring. Reshape the
umbrella spec:
- End-state: every adapter is one-line run_app::<App>; App from app! macro;
handlers #[action] + State<Arc<AppState>>; no TS-local registry/dispatch wiring
- Two NEW required upstream edgezero prerequisites (verified gaps in pinned 6ebc29a):
- P0-C: header-preserving Fastly run_app dispatch + pre-dispatch extension hook
(Set-Cookie/JA4/logger) - permanent-exception fallback removed
- P0-D: macro/run_app app-state injection (with_state is builder-only; app!
router + run_app never call it) - a Hooks state hook run_app inserts per request
- D1 retained (load-once Settings) but injected via P0-D state hook, not with_state
- Phase 4 rewritten as full convergence (app! macro, run_app, extractors, catch-all
fallback for integration/publisher dispatch)
- Sequencing note (R14): Phase 1 adapter-registry builders (Tasks 5-6) are throwaway
under run_app; recommend landing P0-C/P0-D early to skip them
- Risks R7 (P0-C required), R13 (P0-D), R14 (sequencing)
…ate) Implementation-ready spec to hand to the edgezero dev, grounded in pinned 6ebc29a: - P0-C: C1 append_header for multi-value Set-Cookie (response.rs:29 set_header bug), C2 Hooks::owns_logging() opt-out, C3 run_app_with_request_extensions pre-dispatch hook for JA4/H2/client_info from the raw fastly::Request - P0-D: Hooks::app_state() injected per request (symmetric with registries) + app! macro state= argument; note P0-D is avoidable via hand-built routes()+with_state
… gaps) - BLOCKER: Task 8 made Fastly read impls write-only, but legacy_main (live until Phase 5) reads through FastlyPlatformConfigStore via build_runtime_services. Fix: only Axum/CF/Spin go write-only in Phase 1; Fastly read impls stay until Phase 5 deletes legacy_main (spec Phase 1/5 boundary + ledger updated) - stores() drift: add Step 5b test asserting stores_metadata() and each adapter's Hooks::stores() equal edgezero.toml (registries build from stores(), not the toml that Step 1 validates) - KV multi-id: CF/Spin build RuntimeServices from kv_store_default() only, so non-default KV ids (consent_store/ec_store) don't resolve; Step 2c decides (registry-backed KV selector vs documented defer + narrowed acceptance) - cloudflare.toml still uses legacy [stores.*].name schema the pinned manifest rejects; Task 2 reconciles/deletes it - Add Cloudflare/Spin non-default config/secret tests (Step 2b)
- KV multi-id DECIDED: registry-backed KV now (not defer). All four adapters resolve named KV via KvRegistry from extensions; remove Fastly consent-store special-casing; per-adapter non-default-KV test. Spec acceptance updated. - Spec: fix stale Phase 1 'delete 4x read impls' bullet to match the corrected Fastly-stays-until-Phase-5 deletion note (no self-contradiction) - Spec: remove duplicate P-BOOT paragraph (kept the full-convergence note) - app_config rename: add tests/environments/cloudflare.rs literals + wrangler.toml placeholder JSON/comment to Task 2 Step 6b + git add
- KV named resolution is a CORE change, not adapter-only: RuntimeServices only has kv_store() single handle and consent (publisher.rs:626) drops the store id via .map(|_| services.kv_store()). Step 2c now adds RuntimeServices::kv_store_named + kv_registry field, updates consent call sites to pass consent_store, removes Fastly special-casing, and lists core files (types.rs, publisher.rs) - Step 2d: test-support helper to build registry-populated RequestContexts + migrate existing direct-context tests (strict registries break them otherwise) - ec_store scope narrowed: Phase 1 only validates it declares/resolves to a KvHandle; NOT EC identity-graph wiring on non-Fastly (separate larger effort)
- KvHandle/consent type path DECIDED: KvRegistry::named yields KvHandle (a wrapper, not a PlatformKvStore impl), and ConsentPipelineInput.kv_store is Option<&dyn PlatformKvStore>. Migrate the consent KV surface to KvHandle: RuntimeServices::kv_handle_named(id) -> Option<KvHandle> (mirrors existing kv_handle()); ConsentPipelineInput.kv_store -> Option<KvHandle>; consent persistence fns take &KvHandle. Files: consent/mod.rs, storage/kv_store.rs added. - Task 5 Step 3/4 now run all four adapters + wasm checks and commit core + Fastly changes (task touches types.rs/publisher.rs/Fastly), per the all-four-green rule
…ral test) - Fastly named-KV sequencing: Task 5 wired kv_registry + removed consent special-casing on Fastly, but Fastly injects registries only in Task 6. Move Fastly named-KV + build_per_request_services composite + runtime_services_for_consent_route removal into Task 6 Step 4b; Task 5 does core + Axum/CF/Spin and leaves Fastly compiling (kv_handle_named returns None) - Behavioral consent test: prove consent_store loads/saves/deletes through the NON-default store and leaves the default untouched (not just handle distinctness) - Convert existing core consent tests (consent/mod.rs kv_store: Some(&store) at ~1450/1481/1492) from &dyn PlatformKvStore to KvHandle - Fix understated KV summaries: spec gap table + plan goal now say default KV is EdgeZero but named/consent-store KV selection is not
… tests) - Interim Fastly regression: Task 5 flipped consent to kv_handle_named but Fastly has no KvRegistry until Task 6 -> consent would silently skip on Fastly. Fix: Task 5 migrates the consent TYPE but keeps the call site on kv_handle() (default, = Fastly's swapped consent store); the named-lookup FLIP + behavioral test are atomic in Task 6 Step 5b once all four inject a KvRegistry - Absent-registry policy: builders stay infallible (return RuntimeServices); build a composite over an EMPTY registry so reads error, not silently fall back - Task 6 Step 5c: Fastly named-KV/consent route test (guards the special-case removal) - Task 3 Step 1b: explicit CompositeSecretStore test (named id, unknown-id error, create/delete StoreId-preserving delegation) - flat CF/Spin secret namespaces mean route tests don't prove store-id binding - Task 5 hygiene: no longer edits/commits Fastly (moved to Task 6)
…+ commit/filter)
- BLOCKER: absent-registry policy said 'composite over empty StoreRegistry', but
StoreRegistry fields are private and from_parts returns None on empty by_id.
Fix: Composite{Config,Secret}Store hold Option<Registry> (reader = the Option
from extensions.get().cloned()); error on None. No empty-registry construction.
- Fastly secret builder: does NOT open per id. One SecretHandle over FastlySecretStore
(stateless; opens per get_bytes) + BoundSecretStore per id. Tightened Task 6 builders
per kind (KV opens/Result, config handle, secret bound-no-open)
- Task 6 commit: add consent/mod.rs (behavioral test file)
- Task 3 test command: single shared-prefix filter 'composite_' (not two filters)
…k 5 scope)
- Task 3 test snippets: pass Some(reader) to Composite{Config,Secret}Store::new
(constructor takes Option<Registry>, snippets passed bare registries)
- Fastly KV builder: FastlyKvStore::open(id) returns Result<FastlyKvStore, KvError>,
not KvHandle; wrap KvHandle::new(Arc::new(store))
- Task 5 stale scope: publisher.rs call site is interim kv_handle() (flip is Task 6);
remove Fastly platform.rs/app.rs from Task 5 files (Task 6 owns them)
- Task 3 Step 4 pass command: cargo test-fastly composite_ (runs both tests)
…Meta P0 (State<T> + nested/array #[secret]) landed on the #306 branch. The nested-secret work reshapes AppConfigMeta: const SECRET_FIELDS -> fn secret_fields() -> Vec<SecretField>. Update TrustedServerAppConfig's impl accordingly (still empty until Phase 3 secret externalization). Verified: cargo check green on core + all four adapters + cli.
…s, D7) - Spec: core PlatformConfigStore/SecretStore read+write traits are NOT deleted in Phase 1 (composite implements them; RuntimeServices exposes them) - only removed in Phase 4. Phase 1 EXTRACTS the write-only sub-traits. Fixed deletion note + ledger. - Writer ordering: Task 3 Step 0 now also impls PlatformConfigWriter/SecretWriter on the existing stores (forwarding to their put/create/delete), so Task 5 has writers before Task 8 converts anything. Explicit per-type impls (not blanket, coherence). - Task 2: add Viceroy integration template + Spin runtime-config.toml (KV labels need a backend); both added to files/commit - D7 consistency: spec Fastly registry text drops EnvConfig (open by logical id); Task 6 config builder default_key = logical id - Task 3 verify: compile axum/cf/spin too (core trait split) - Task 5 Step 2d: Fastly direct-context test migration -> Task 6 (Task 5 excludes Fastly)
- Writer impls can't live in core for adapter-owned types (orphan rule / dep cycle); blanket impl would coherence-conflict with Task 8. Task 3 Step 0 now defines the write-only traits ONLY (core); each adapter impls the writer for its own store as the first step of its composite wiring (Task 5 Axum/CF/Spin; Task 6 Fastly) - fastly.toml holds only [local_server] Viceroy resources; production store links are an operator/provisioning step, not a manifest edit - split the Task 2 bullet - Task 3 verify: cargo check-cloudflare/check-spin (wasm surfaces), not native only - Task 2 Files: add runtime-config.toml + viceroy-template.toml (step/commit had them) - Task 5 Files summary: direct-context test migration is axum/cf/spin (Fastly = Task 6)
- HIGH: consent flip must FAIL CLOSED. Fastly returns 503 today when consent_store is configured but unresolved (tests dispatch_auction_...returns_503 / edgezero_missing_consent_store_breaks_only_consent_routes). .and_then(kv_handle_named) silently -> None (regression). Fix: configured-but-unresolved -> TrustedServerError::KvStore (503 on consent routes); unconfigured -> None; integration routes unaffected. Also fix revocation delete (publisher.rs:885) to use the named handle, not default KV. Add a configured-but-unresolved-errors test. - HIGH: pin note updated 6ebc29a5 -> d8f71a4a (re-verified registry APIs present; AppConfigMeta const->fn compat noted) - Task 4 + Task 6: run all four + check-cloudflare/check-spin (core surfaces change) - Task 5: bounded Axum smoke (timeout/bg+curl), not blocking cargo run - Task 2 Files: add cloudflare.toml; Task 3 Files: add platform/types.rs
…le guard) - HIGH: auction fail-closed gap. Fastly auction route (app.rs:584) got 503-on-missing -consent-store from runtime_services_for_consent_route, removed in Step 4b, but Step 5b only fixed publisher paths. handle_auction builds KvIdentityGraph over consent KV. Fix: shared core resolve_consent_kv (fail-closed) used by BOTH publisher and handle_auction (before KvIdentityGraph); the Fastly 503 tests are mandatory gates. - Task 6 Files: add platform.rs (writer impls), consent/mod.rs, publisher.rs, auction/endpoints.rs, core consent/auction tests, Fastly direct-context test migration - Lockfile guard: branch dep is mutable and upstream advanced past d8f71a4a; execute --locked, no cargo update unless API re-reviewed - Task 8 commit msg: 'Retire NON-Fastly per-adapter read impls' (Fastly stays to Phase 5)
- Auction guard wording corrected: handle_auction does NOT build KvIdentityGraph over consent KV (consent = ec_context.consent(); KvIdentityGraph is the EC-graph input). resolve_consent_kv(...)? is a pure fail-closed guard, no EC-graph semantics change. - Mixed edition: core/fastly 2021, axum 2024 (plan said global 2024). Follow each crate. - --locked on every cargo command (mutable branch dep, upstream advanced past d8f71a4a) - Preflight: cargo fetch/check --locked + confirm resolved rev d8f71a4a before Task 1 - Axum local KV: don't hand-reimplement EdgeZero's private .edgezero/kv-<slug>-<hash>.redb path (silent divergence); reuse/upstream a public constructor. Decision in Task 1. - Spin: declare all kv/config labels + runtime-config backends; drop stale v_ comment; document spin up --runtime-config-file; secrets flat by key - Operator runbook: create docs/internal/store-provisioning.md (mgmt-id==logical-id, provisioning commands) - added to files + commit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the umbrella design spec for moving Trusted Server completely onto EdgeZero primitives: config push, KV, secret store, config injection without an embedded
trusted-server.toml, extractor-based handlers, and deletion of every pre-EdgeZero workaround.This is a docs-only PR (one spec file). It defines the end-state, a verified current-state gap analysis, and an ordered set of phases — each of which will get its own implementation plan/PR.
Spec:
docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.mdPhase map (foundation-first)
State<T>extractor + nested/array#[secret]). Upstream edgezero repo, tracked separately by its own PR.StoreRegistry(delete bespokePlatformConfigStore/PlatformSecretStore,RuntimeServices,management_api.rs,settings_data.rschunk resolver). No upstream dependency — startable immediately.include_str!), dropfrom_toml_and_env+ theconfigcrate.#[secret]onSettings, move inline secrets to the secret store, deleteRedacted<T>. Depends on Phase 0 nested#[secret].#[action]extractors using the upstreamedState<T>; delete per-adapter handler shims.legacy_main/compat.rs/rollout flags). Gated on 100% EdgeZero rollout (Fastly entry point switch (dual-path with flag) #495).Key decisions
Arc<Settings>viaState<Settings>rather than the per-requestAppConfig<C>extractor (avoids re-parsing the wholeSettingson every request).TrustedServerAppConfigwrapper ontoSettings.Status
Draft — for review of scope, phase ordering, and D1 before per-phase implementation plans begin.