diff --git a/agent-context/BACKLOG.md b/agent-context/BACKLOG.md index 2f19be4..e5bcb47 100644 --- a/agent-context/BACKLOG.md +++ b/agent-context/BACKLOG.md @@ -17,7 +17,8 @@ Public build order: [docs/ROADMAP.md](../docs/ROADMAP.md). OpenSpec changes: `op | U-1 + LB-1 overrides | **landed** — S3/console creds sensitive (`8ae7376`, `config/sensitive.go`); loadbalancer `status` Computed (`c38e52b`, `config/loadbalancer.go`); D-021 | | D-020-FU (extensions in signed release) | **landed** (`43294ed`+`60b9e8f`) — `make xpkg.append.extensions` append-then-sign + fail-closed verify in publish workflow; v0.2.2 signed w/ extensions verified | | E5 SonarCloud SECURITY (E5-S11…S17) | **landed 2026-07-25** — 7 lanes Integrated; 9/9 `hack/test/sonar_pg_*` green; SonarCloud SECURITY open: 0 | -| Backlog | **exhausted** — remaining items are operator-only: revoke old classic PAT; nudge/track upstream TF PRs #509/#510/#511 (on merge, re-vendor + drop U-1/LB-1 overrides). Optional Scorecard polish (D-016) non-blocking. | +| E5 SonarCloud MAINT/RELIABILITY (E5-S18…S21) + coverage CI | **landed 2026-08-04** — PR #41 → `f9f4628`; OpenSpec `sonar-maintainability-2026-08` | +| Backlog | **exhausted** for implementable lanes. Operator: wait on upstream TF #509/#510/#511 (on merge, re-vendor + drop U-1/LB-1). Optional Scorecard polish (D-016) non-blocking. No package release for Batch 11 alone (CI/maintainability). | ## E2 test-hardening batch (S06–S10) — ported from kollect's test tooling @@ -179,3 +180,202 @@ OpenSpec (worktree / upcoming PR): `openspec/changes/sonar-security-remediation- **Outcome:** local `bin/…/provider` + `terraformrc.hcl` use `COPY`; remote URL ADD/curl allowed. **Verify:** `bash hack/test/sonar_pg_08_dockerfile_copy_test.sh` · closes **REQ-SONAR-PG-08** **File lock:** `cluster/images/provider-gridscale/Dockerfile`, `hack/test/sonar_pg_08_*` + +--- + +## E5 — SonarCloud MAINTAINABILITY + RELIABILITY remediation (2026-08) + +OpenSpec (create when first lane starts): `openspec/changes/sonar-maintainability-2026-08/` +(REQs **PG-M01…PG-M08**). Source: SonarCloud project `PlatformRelay_provider-gridscale`, +**46 open CODE_SMELL** (Maintainability 38 · Reliability 8). Zero SECURITY / BUG. + +**Out of scope for this batch:** coverage CI wiring (`sonar-coverage-ci` @ `d98f242`); +generated `zz_*.go` smells; SECURITY (already 0). + +**Lane rule:** file locks below are exclusive. Clusters that share a file are **one story / +one lane** (never parallel). Meta tests under `hack/test/sonar_pg_m_*` owned by the same lane +as the production path they guard. + +### agent-loop-local lane plan (N=4, file-disjoint) + +| Lane | Story | Closes (Sonar) | File lock | P | Parallel? | +| --- | --- | --- | --- | --- | --- | +| **L-REG** | E5-S18 | go:S1186 ×2 CRITICAL | `apis/cluster/v1alpha1/register.go`, `apis/namespaced/v1alpha1/register.go`, `hack/test/sonar_pg_m01_*` | P0 | ✅ vs all | +| **L-CFGTEST** | E5-S19 | go:S3776 ×3 CRITICAL + godre:S8205 ×6 MINOR | `config/crd_contract_test.go`, `config/external_name_fuzz_test.go`, `hack/test/sonar_pg_m02_*` (optional structural) | P0 | ✅ vs all **except** any other `config/*_test.go` lane | +| **L-SHELL** | E5-S20 | shelldre:S131 ×1 + S7677 ×3 + S7688 ×8 + S1066 ×4 | `hack/check-docs.sh`, `hack/check-api-docs.sh`, `hack/test/e8_s01_*`, `hack/test/e8_s02_*`, `hack/test/e8_s03_*`, `hack/test/sonar_pg_02_*`, `hack/test/sonar_pg_04_ci_*`, `hack/test/sonar_pg_04_e2e_*`, `hack/test/sonar_pg_m04_*`…`m07_*` | P0/P1 | ✅ vs L-REG / L-CFGTEST / L-DOCKER | +| **L-DOCKER** | E5-S21 | docker:S6570 ×19 MAJOR | `cluster/images/provider-gridscale/Dockerfile`, `hack/test/sonar_pg_m08_*` (extend or sibling of `sonar_pg_08_*`) | P1 | ✅ vs all **except** another Dockerfile lane | + +**Serialization notes** +- Clusters **2 + 8** both touch `config/crd_contract_test.go` → **only E5-S19** (L-CFGTEST). Do not split. +- Clusters **3 + 5 + part of 6** all touch `hack/check-docs.sh` → **only E5-S20** (L-SHELL). +- Cluster **6** e8 scripts + cluster **7** sonar_pg nested-ifs share the `hack/test/` tree but + **disjoint filenames** with each other; kept in **one lane** so shell style stays consistent and + one agent owns all Sonar shell hygiene. +- **L-DOCKER** re-touches the Dockerfile previously owned by landed E5-S17 — preserve `COPY` for + local artefacts; only add quoting for expansions (lines 27–39). +- Base worktrees on a tip that includes the E8 observe meta tests + (`hack/test/e8_s01_*`, `e8_s03_*` — present on `origin/main`; may be missing on stale local + checkouts). Independent of `sonar-coverage-ci`. + +| Story | REQs | Status | +| --- | --- | --- | +| E5-S18 | PG-M01 | ✅ Integrated (PR #41) | +| E5-S19 | PG-M02, PG-M03 | ✅ Integrated (PR #41) | +| E5-S20 | PG-M04, PG-M05, PG-M06, PG-M07 | ✅ Integrated (PR #41) | +| E5-S21 | PG-M08 | ✅ Integrated (PR #41) | +| coverage CI | sonar_pg_09 | ✅ Integrated (PR #41) | + +### E5-S18 — Empty `init()` register funcs: nested comments (P0) +**As a** maintainer **I want** the package `init()` stubs in cluster/namespaced `register.go` to +satisfy go:S1186 **so that** Sonar CRITICAL empty-function smells clear without changing scheme +registration behaviour. + +**Sonar keys / rule** +- `apis/cluster/v1alpha1/register.go:22` — `AZ9magFP_3fq4FMeL0wZ` (go:S1186 CRITICAL) +- `apis/namespaced/v1alpha1/register.go:22` — `AZ9magJP_3fq4FMeL0xy` (go:S1186 CRITICAL) + +**Acceptance criteria** +- Given the Upjet scaffold `func init() {}` at both paths, when the fix lands, then each empty + function body contains a **nested comment** explaining why it is intentionally empty (types + register via `SchemeBuilder` / generated `zz_*.go`), meeting Sonar S1186 — **or** an equivalent + no-op that Sonar accepts without registering types twice. +- Given `make generate` / `make reviewable`, when run after the edit, then neither `register.go` is + overwritten back to a bare empty body (if codegen regenerates them, the intentional comment is + restored in the same change or the comment lives in a non-generated sibling — document which). +- **Edge:** given a mistaken “fix” that calls `SchemeBuilder.AddToScheme` from both package + `init()`s, when controllers start, then we must **not** double-register; prefer nested comment + over wiring registration here. + +**Done when** +- [ ] Meta test asserts both files’ `init` bodies are non-bare-empty (comment or documented pattern). **Level:** M · **Test:** `hack/test/sonar_pg_m01_register_init_not_empty_test.sh` · **Verify:** `bash hack/test/sonar_pg_m01_register_init_not_empty_test.sh` +- [ ] `go test ./apis/...` (or package compile) still green; `make reviewable` clean +- [ ] Closes Sonar keys above (or marked Fixed on next scan) +- [ ] OpenSpec REQ-SONAR-PG-M01 filled under `sonar-maintainability-2026-08` + +**Touches:** `apis/cluster/v1alpha1/register.go`, `apis/namespaced/v1alpha1/register.go`, +`hack/test/sonar_pg_m01_*` +**Depends on:** none · **Parallel-safe-with:** E5-S19, E5-S20, E5-S21 +**Not in scope:** any `zz_*.go`; changing Group/Version; coverage CI + +### E5-S19 — Config test cognitive complexity + named CRD view types (P0) +**As a** maintainer **I want** CRD-contract and external-name fuzz tests under the cognitive-complexity +threshold and anonymous nested structs replaced with named types **so that** Sonar CRITICAL/MINOR +smells in `config/*_test.go` clear without weakening golden/structural assertions. + +**Sonar keys / rules** +- Cognitive complexity (go:S3776 CRITICAL): + - `config/crd_contract_test.go:164` (21→≤15) `AZ9nqLQ5RbNgprH5pBN9` — `TestCRDGoldenContract` + - `config/crd_contract_test.go:201` (28→≤15) `AZ9nqLQ5RbNgprH5pBN-` — `TestCRDStructuralInvariants` + - `config/external_name_fuzz_test.go:18` (20→≤15) `AZ9n-G-75WRcdOwb-njc` — `FuzzGetExternalName` +- Named types (godre:S8205 MINOR) — anonymous structs at `config/crd_contract_test.go:41–53` + keys `AZ9nqLQ5RbNgprH5pBN_` … `AZ9nqLQ5RbNgprH5pBOE` (6 issues) + +> **Must stay one story:** complexity + named types both edit `crd_contract_test.go`. + +**Acceptance criteria** +- Given the three functions above, when complexity is reduced (extract helpers / table helpers / + early continue — outcome-focused), then each reports cognitive complexity **≤ 15** under Sonar + (or golangci equivalent if configured). +- Given the nested anonymous structs inside `type crd`, when refactored, then each flagged nesting + level uses a **named type** (exported not required) with the same JSON tags and behaviour. +- Given existing goldens under `config/testdata/crd-contract/`, when `go test ./config/ -count=1` + runs without `UPDATE_GOLDEN=1`, then all golden + structural invariants still pass (no silent + contract relaxation). +- **Edge:** given `UPDATE_GOLDEN=1`, when regenerating, then goldens remain byte-stable for an + unchanged CRD tree (refactor touches tests only). +- **Edge (fuzz):** given empty / non-string / missing `id` cases already in the fuzzer, when + complexity is reduced, then the stub contract (never error; empty id → `""`) still holds. + +**Done when** +- [ ] `go test ./config/ -count=1` green; fuzz still builds (`go test ./config/ -run=FuzzGetExternalName -count=1`) +- [ ] Closes the 3× S3776 + 6× S8205 keys above +- [ ] REQ-SONAR-PG-M02 (complexity) + REQ-SONAR-PG-M03 (named types) in OpenSpec +- [ ] Gates green (`make test` / reviewable as applicable) + +**Touches:** `config/crd_contract_test.go`, `config/external_name_fuzz_test.go` +**Depends on:** none · **Parallel-safe-with:** E5-S18, E5-S20, E5-S21 +**Not in scope:** changing CRD goldens for product reasons; production `config/*.go`; coverage CI + +### E5-S20 — Shell hygiene: case default, stderr, `[[`, merge nested ifs (P0/P1) +**As a** maintainer **I want** hand-authored `hack/` scripts and meta tests to satisfy Sonar shell +rules **so that** CRITICAL/MAJOR Maintainability and Reliability smells clear without changing +gate pass/fail semantics. + +**Sonar keys / rules (grouped)** +1. **shelldre:S131 CRITICAL** — `hack/check-docs.sh:17` `AZ9oaSNAnB7bKDM9yqGC` (case missing `*)`) +2. **shelldre:S7677 MAJOR** — error messages on stdout → stderr (`>&2`): + - `hack/check-api-docs.sh:22` `AZ9rwTiXTAot3Azg_Ks5` + - `hack/check-docs.sh:33,37` `AZ9oaSNAnB7bKDM9yqGD`, `AZ9oaSNAnB7bKDM9yqGE` +3. **shelldre:S7688 MAJOR RELIABILITY** — prefer `[[` over `[`: + - `hack/check-docs.sh:40` + - `hack/test/e8_s01_observe_docs_test.sh:3` + - `hack/test/e8_s01_observe_yaml_count_test.sh:5,6` + - `hack/test/e8_s02_backuplist_crd_exists_test.sh:5,6` + - `hack/test/e8_s03_publicnetwork_crd_exists_test.sh:5,6` +4. **shelldre:S1066 MAJOR** — merge nested `if`s: + - `hack/test/sonar_pg_02_no_secret_in_run_test.sh:20,38` + - `hack/test/sonar_pg_04_ci_job_scoped_permissions_test.sh:24` + - `hack/test/sonar_pg_04_e2e_job_scoped_permissions_test.sh:25` + +**Acceptance criteria** +- Given `hack/check-docs.sh`’s `case`, when a CRD basename matches none of the skip patterns, then a + `*)` default arm handles it (continue / append — same as today’s implicit fall-through) so S131 + clears. +- Given `::error::` / failure echoes in `check-docs.sh` and `check-api-docs.sh`, when printed, then + they go to **stderr** (`>&2`). +- Given the listed `[ … ]` tests, when updated, then they use bash `[[ … ]]` with equivalent + predicates (counts, `-f`, `-eq`). +- Given nested `if` pairs in the three `sonar_pg_0{2,4}_*` scripts, when merged, then the combined + condition preserves the **same fail messages and fail conditions** (no weaker security grep). +- **Edge:** given `check-docs.sh` with a stale README, when run, then exit non-zero and error text + still appears on stderr; success path still prints the sync confirmation. +- **Edge:** given e8 meta tests after `[[` migration, when CRDs/docs are present, then scripts still + exit 0; when a path is missing, then still exit 1 with FAIL text. + +**Done when** +- [ ] `bash hack/check-docs.sh` behaviour preserved (exit code + stderr on failure) +- [ ] `bash hack/test/e8_s01_*.sh` `e8_s02_*.sh` `e8_s03_*.sh` green on a tip that has those files +- [ ] `bash hack/test/sonar_pg_02_*.sh` `sonar_pg_04_ci_*.sh` `sonar_pg_04_e2e_*.sh` green +- [ ] Meta guards for S131/S7677/S7688/S1066 patterns as needed (`hack/test/sonar_pg_m04_*` …) +- [ ] Closes all keys in the four clusters above · REQ-SONAR-PG-M04…M07 in OpenSpec + +**Touches:** `hack/check-docs.sh`, `hack/check-api-docs.sh`, `hack/test/e8_s01_*`, +`hack/test/e8_s02_*`, `hack/test/e8_s03_*`, `hack/test/sonar_pg_02_*`, +`hack/test/sonar_pg_04_ci_*`, `hack/test/sonar_pg_04_e2e_*`, `hack/test/sonar_pg_m0{4,5,6,7}_*` +**Depends on:** none (rebase onto tip with E8 scripts) · **Parallel-safe-with:** E5-S18, S19, S21 +**Not in scope:** rewriting check-docs counting logic; Dockerfile; Go tests; coverage CI + +### E5-S21 — Dockerfile: quote expansions (P1) +**As a** release engineer **I want** Dockerfile `RUN`/`ADD`/`ENV` expansions quoted per docker:S6570 +**so that** 19 MAJOR Maintainability issues clear without breaking the terraform/plugin install layer. + +**Sonar keys / rule** +- `cluster/images/provider-gridscale/Dockerfile:27–39` — docker:S6570 ×19 + keys `AZ-GeyaXbiMwwowBPDZ4` … `AZ-GeyaXbiMwwowBPDaK` + +**Acceptance criteria** +- Given lines that expand `${PLUGIN_DIR}`, `${TERRAFORM_*}`, `${TARGETOS}`, `${TARGETARCH}`, + `${USER_ID}`, zip paths, etc. in the Setup Terraform block (≈27–39), when fixed, then each + flagged expansion is properly quoted per S6570. +- Given the E5-S17 contract, when this lane finishes, then local artefacts still use **`COPY`** + (not `ADD`) for `bin/…/provider` and `terraformrc.hcl`; remote URL `ADD` may remain. +- **Edge:** given a build with the quoted Dockerfile, when `docker build` / CI image build runs, + then terraform + provider plugin still land under `${PLUGIN_DIR}` and the non-root `USER` still + applies (no path-split regressions from quoting). + +**Done when** +- [ ] Meta test (extend `sonar_pg_08_*` or add `sonar_pg_m08_dockerfile_quoting_test.sh`) asserts + quoted forms for the flagged lines · **Level:** M +- [ ] Existing `bash hack/test/sonar_pg_08_dockerfile_copy_test.sh` still green +- [ ] Closes the 19× S6570 keys · REQ-SONAR-PG-M08 in OpenSpec + +**Touches:** `cluster/images/provider-gridscale/Dockerfile`, `hack/test/sonar_pg_m08_*` +(and/or `hack/test/sonar_pg_08_*` if extended in-lane) +**Depends on:** none · **Parallel-safe-with:** E5-S18, S19, S20 +**Not in scope:** base-image bumps; TERRAFORM_VERSION pin changes; coverage CI + +### First slice recommendation + +Start **E5-S18 (L-REG)** first — two CRITICAL fixes, tiny diff, zero product behaviour risk — then +fan out **L-CFGTEST + L-SHELL + L-DOCKER** in parallel under `/agent-loop-local`. If only one +implementer: S18 → S19 (remaining CRITICAL complexity) → S20 → S21. diff --git a/agent-context/INBOX.md b/agent-context/INBOX.md index a1780d8..1400f4d 100644 --- a/agent-context/INBOX.md +++ b/agent-context/INBOX.md @@ -12,28 +12,32 @@ remove here. This repo's INBOX is independent — never coordinate other repos f ## Decisions +### 🟡 DECIDED (awaiting approval) — D-026 no v0.3.1 for Batch 11 alone + + Context: Batch 11 on main after v0.3.0; operator said release if appropriate. + Options: A cut v0.3.1 · B skip until user-facing/upstream-TF (chose) · C never. + Chose: B — CI/Sonar/test hygiene only; install stays v0.3.0. + Revert: `git tag v0.3.1 && git push origin v0.3.1` + `gh workflow run publish-provider-package.yml -f version=v0.3.1`. + +### ✅ Resolved — Batch 11 landed via PR #41 (rebase-admin) + + Protect-main GH013 cleared by operator-authorized `gh pr create` + + `gh pr merge --rebase --admin` → `origin/main` @ `f9f4628` (E5-S18…S21 + coverage CI). + + _None open._ --- ## Operator tasks -1. **Revoke the old classic PAT** that was briefly stored as `GHCR_PAT` (local PAT in - `.envrc` only — do **not** put it in Actions secrets). -2. **Nudge/track upstream PRs** #509/#510/#511 if they stall; on merge, re-vendor - (`TERRAFORM_PROVIDER_VERSION`) and drop the local U-1/LB-1 overrides. -3. **Publish v0.3.0** — decided **D-022 → A** (release now) via `/operator-inbox` 2026-07-29. - Release-prep landed in **PR #35** (README + `examples/install.yaml` → v0.3.0; CHANGELOG left at - `[Unreleased]` — the `[0.3.0]` section is auto-generated by the tag automation, see below). - **Operator-only tail** (per standing permissions): - - **Merge PR #35** (release-prep) into `main`. - - **Tag** `v0.3.0` on the merged `main` commit and push. `tag.yaml` workflow is broken (ref lacks - `on.workflow_call`) → tag **manually**: `git tag v0.3.0 && git push origin v0.3.0`. - The tag push fires the `Changelog` workflow's `release` job → **auto-opens a `changelog/v0.3.0` - PR** with the `[0.3.0]` notes; merge that too. (It does **not** trigger publish.) - - **Publish**: `gh workflow run publish-provider-package.yml -f version=v0.3.0`. (Builds from the - tag — dispatch it right after tagging; it does **not** depend on the changelog PR being merged.) - - **Verify**: `cosign verify ghcr.io/platformrelay/provider-gridscale:v0.3.0 --certificate-oidc-issuer https://token.actions.githubusercontent.com --certificate-identity-regexp 'https://github.com/PlatformRelay/provider-gridscale/.*'`. +1. ~~**Revoke the old classic PAT**~~ — **DISMISSED 2026-08-05** (operator: "forget that"). +2. ~~**Nudge/track upstream PRs** #509/#510/#511~~ — **WAIT 2026-08-05** (operator: no need; wait for upstream). +3. ~~**Publish v0.3.0**~~ — **DONE** (already live 2026-07-31). Operator reconfirmed **A** 2026-08-05; + tag/release/publish-provider-package + GHCR digest already present — no re-cut. + Release: https://github.com/PlatformRelay/provider-gridscale/releases/tag/v0.3.0 +4. ~~**SonarCloud Autoscan**~~ — **DONE 2026-08-05** (operator disabled Automatic Analysis in Sonar UI). + Confirmed: Coverage run 30956241401 — `coverage` + `sonarcloud` both **success**. ### Non-blocking / optional @@ -46,6 +50,10 @@ _None open._ ## ✅ Resolved this session (recorded, no further call) +- **2026-08-05 open-questions G1–G4** — G1 dismiss PAT revoke · G2 Autoscan disabled · G3 publish + already complete (v0.3.0) · G4 wait on upstream TF PRs. + + - **E8 data-sources epic** — 4 stories landed 2026-07-28 (PRs #30–#33). 34 managed resources. BackupList + PublicNetwork observe-only controllers active at provider startup. - **v0.3.0 release** — decided **D-022 → A** (release now, 2026-07-29). Release-prep done; publish diff --git a/agent-context/coordination/OPERATOR-BOARD.md b/agent-context/coordination/OPERATOR-BOARD.md index 1c56ce5..ba45d3f 100644 --- a/agent-context/coordination/OPERATOR-BOARD.md +++ b/agent-context/coordination/OPERATOR-BOARD.md @@ -1,7 +1,21 @@ # OPERATOR-BOARD — provider-gridscale (lanes + locks) -Coordinator: session wrap **2026-07-21** (v0.2.1 security patch + docs). Base: `origin/main` @ `c53fc82`. -**No lanes In-flight.** **v0.2.1 published & signed** — see [`SESSION-HANDOFF.md`](SESSION-HANDOFF.md). +Coordinator: **agent-loop-local** cleanup — 2026-08-05. +Base: `origin/main` @ `f9f4628` (PR #41). **No lanes In-flight.** Backlog exhausted for implementable work. + +### Batch 11 — SonarCloud MAINT/RELIABILITY + coverage (2026-08-04) — ✅ Integrated + +Landed via [PR #41](https://github.com/PlatformRelay/provider-gridscale/pull/41) (`gh pr merge --rebase --admin`). Tip: `f9f4628`. + +| Lane | Story | State | +| --- | --- | --- | +| L-REG | **E5-S18** empty `init()` nested comments | ✅ Integrated (PR #41) | +| L-CFGTEST | **E5-S19** config test complexity + named types | ✅ Integrated (PR #41) | +| L-SHELL | **E5-S20** shell hygiene | ✅ Integrated (PR #41) | +| L-DOCKER | **E5-S21** Dockerfile quoting | ✅ Integrated (PR #41) | +| L-COV | **coverage CI** Sonar analysis wiring | ✅ Integrated (PR #41) | + +**D-024:** Autoscan disabled (operator 2026-08-05); Coverage+sonarcloud green on run 30956241401. ### Batch 10 — vuln + README/docs + logo candidates (2026-07-21) — landed on `main` @@ -111,6 +125,20 @@ IDs renumbered per **D-014** (gap E2-S06/07/08 → **E7-S01/02/03**, research E2 > **Session wrap complete** — prefer Marketplace/install tag **v0.1.1**. > Primary CI green on `d8433f5`; Docs Sync green on `d275897`. See SESSION-HANDOFF. +### Batch 11 — SonarCloud MAINT/RELIABILITY + coverage (2026-08-04) — ✅ Integrated on origin/main + +Landed via [PR #41](https://github.com/PlatformRelay/provider-gridscale/pull/41) (`gh pr merge --rebase --admin`). Tip: `origin/main` @ `f9f4628`. + +| Lane | Story | Owns (disjoint paths) | Auto-merge? | State | +| --- | --- | --- | --- | --- | +| L-REG | **E5-S18** empty `init()` nested comments (go:S1186 ×2) | register.go + sonar_pg_m01 + OpenSpec scaffold | yes | ✅ Integrated (PR #41 → `f9f4628`) | +| L-CFGTEST | **E5-S19** test complexity + named types | config/*_test.go + pg-m02/m03 | yes | ✅ Integrated (PR #41) | +| L-SHELL | **E5-S20** shell hygiene | hack/check-*.sh + e8/sonar_pg meta + pg-m04…m07 | yes | ✅ Integrated (PR #41) | +| L-DOCKER | **E5-S21** Dockerfile quoting | Dockerfile + sonar_pg_m08 | yes | ✅ Integrated (PR #41) | +| L-COV | **coverage CI** Sonar analysis wiring | coverage.yml, sonarcloud.yaml, sonar-project.properties, sonar_pg_09 | yes | ✅ Integrated (PR #41) | + +**D-024:** Autoscan disabled; Coverage+sonarcloud green (run 30956241401). + ## Deferred / blocked (not board lanes) - **E2-S04/S05 uptest** — **D-012 → B** (manual smoke only; do not wire CI lab creds). diff --git a/agent-context/coordination/SESSION-HANDOFF.md b/agent-context/coordination/SESSION-HANDOFF.md index 1561016..b80065a 100644 --- a/agent-context/coordination/SESSION-HANDOFF.md +++ b/agent-context/coordination/SESSION-HANDOFF.md @@ -1,44 +1,24 @@ # SESSION-HANDOFF — provider-gridscale -**Session wrap:** 2026-07-21 (vuln patch + README/docs → **v0.2.1**); **BRAND-2** Actioned 2026-07-25 -(keep press Bildmarke; candidates removed). -**HEAD:** `origin/main` @ `f250bdd` (E8 data-sources epic merged — PRs #30–#33). -**Latest package tag:** **`v0.2.2`** — published to GHCR + Upbound, **keyless-cosign signed**, -extensions verified, SBOM attested -([run 29868142402](https://github.com/PlatformRelay/provider-gridscale/actions/runs/29868142402)). -Prefer this over v0.2.0. Shipping Marketplace icon: **press Bildmarke** (BRAND-2 decided — keep). -**Merge model:** PRs into `main`; solo via `gh pr merge --rebase --admin`. - -E8 data-sources epic complete (PRs #30–#33): BackupList + PublicNetwork observe-only controllers + 38 YAML examples. +**Session:** agent-loop-local cleanup — 2026-08-05 +**HEAD:** `origin/main` @ (see tip after coordination PR) +**Latest package tag:** **`v0.3.0`** — published (GHCR + Upbound, signed). Prefer this install tag. ## Done this session | Item | Result | | --- | --- | -| GO-2026-5970 | `golang.org/x/text` → v0.39.0; `make vuln` / Govulncheck green | -| Scorecard Token-Permissions | Top-level `permissions:` on backport / tag / publish | -| README badges | Kollect-style native Actions badges + Scorecard/codecov/Marketplace/GHCR | -| Docs | `docs/README.md`, assurance Scorecard drivers | -| BRAND-2 | Keep press Bildmarke; `docs/assets/branding/candidates/` removed — see `decisions.md` | -| PR #15 | Merged rebase-admin → `main` | -| Release | Annotated tag + GitHub Release **v0.2.1**; publish **all green** | +| Orient | No open PRs; main CI green @ `f9f4628`; Batch 11 already Integrated | +| Coordination sync | INBOX G1–G4 + board/handoff/BACKLOG mark E5-S18…S21 landed; commit e8 OpenSpec | +| Cleanup | Stale E8 worktrees + local branches; delete superseded `worktree-release-v0.3.0-prep` | +| Release | **Skipped** — Batch 11 is CI/Sonar/test/Dockerfile hygiene; no user-facing package bump. Next cut when upstream TF merges or a real product change lands | ## Open for the operator -1. Revoke old classic PAT (local `.envrc` only). -2. Upstream TF PRs #509/#510/#511. +_None blocking._ Optional: wait on upstream TF #509/#510/#511; Scorecard polish (D-016). ## Do not -- Re-sign v0.1.1 / strip extensions. - Hand-edit generated trees or `config/schema.json`. - Bump `TERRAFORM_VERSION` past 1.5.7. -- Reopen BRAND-2 logo choice unless trademark / endorsement posture changes. - -## Release v0.2.2 (2026-07-25) - -- Tag + GitHub Release: https://github.com/PlatformRelay/provider-gridscale/releases/tag/v0.2.2 -- Publish (signed + SBOM + Upbound mirror): https://github.com/PlatformRelay/provider-gridscale/actions/runs/30152112594 — **success** -- Prefer install tag **`v0.2.2`** over v0.2.1. -- Sonar SECURITY open: 0. Deferred: maintainability (generated zz_*, nested-if in meta tests), E5-S12 empty-datasource fail-fast. - +- Force-push `main`. diff --git a/agent-context/decisions.md b/agent-context/decisions.md index 9be26a4..6b30f40 100644 --- a/agent-context/decisions.md +++ b/agent-context/decisions.md @@ -1,3 +1,20 @@ + +## 2026-08-05 · Operator Q&A — G1–G4 (open-questions) + +- **G1 PAT revoke:** Dismiss ("forget that"). Counterpoint: residual risk if token still valid; operator accepted. +- **G2 Sonar Autoscan:** Operator disabled Automatic Analysis in SonarCloud UI. Confirmed: Coverage run 30956241401 — `coverage` + `sonarcloud` both success. +- **G3 Publish v0.3.0:** A — authorize publish. **Already complete** (2026-07-31): tag, GitHub Release, publish-provider-package success, GHCR digest `sha256:5b274548…`. No re-tag. +- **G4 Upstream TF #509/#510/#511:** Wait — do not nudge. Revisit when upstream merges. + +## 2026-08-05 · D-026 — No v0.3.1 for Batch 11 alone + +**Date:** 2026-08-05 · **Status:** DECIDED (agent-loop-local cleanup) +**Context:** Batch 11 (E5-S18…S21 + coverage CI) is on `main` after `v0.3.0`. Operator authorized release "if appropriate." +**Options:** A) Cut v0.3.1 now · B) Skip until user-facing / upstream-TF change (chosen) · C) Hold forever on Batch 11. +**Decision:** **B** — Batch 11 is CI/Sonar/test/Dockerfile-quoting hygiene; Unreleased changelog is thin; install tag stays **v0.3.0**. +**Counterpoints:** Dockerfile quoting ships in the image build path — a pedantic patch release is defensible; deferred to keep GHCR/Marketplace noise low until a product change or upstream re-vendor. +**Revert:** cut `v0.3.1` from current `main` + publish-provider-package. + # decisions.md — provider-gridscale (append-only) Format per entry: **Decision** · date · context · choice · agent counterpoints (kept even when @@ -550,3 +567,28 @@ this fires the auto changelog PR (merge it) but **not** publish; (3) dispatch `publish-provider-package.yml -f version=v0.3.0` — it builds from the tag and does **not** depend on the changelog PR being merged first; (4) `cosign verify` the published digest. +## D-023 — Empty `register.go` `init()` fix style (E5-S18) → A (nested comments) + +**Date:** 2026-08-04 · **Status:** DECIDED (agent-loop-local; awaiting operator ratification) +**Context:** Sonar go:S1186 CRITICAL on `apis/{cluster,namespaced}/v1alpha1/register.go` +(`func init() {}`). Types already register via `SchemeBuilder` / generated `zz_*.go`. Wiring +`AddToScheme` in these `init()`s risks double-registration. Epic-writer labeled this **D-022**, but +**D-022** on `main` is already “Cut and release v0.3.0” — renumbered to **D-023**. +**Options:** A) Nested comment inside each empty `init()` (Sonar-accepted for S1186) · B) Delete +empty `init()`s · C) Call `SchemeBuilder.AddToScheme` from `init()` (double-register risk). +**Decision:** **A** — nested comments; no behaviour change. Operator invoked agent-loop-local with +explicit “choose A”. +**Counterpoints:** Deleting `init()` (B) is cleaner if the scaffold allows, but A is the +lowest-blast-radius Sonar-documented fix and matches the story AC. C was rejected. +**Revert:** remove the nested comments (or delete the `init`s if B is preferred later). + +## D-025 — e2e PG-04 meta allows workflow read-only floor (E5-S20) + +**Date:** 2026-08-04 · **Status:** DECIDED (agent-loop-local) +**Context:** `hack/test/sonar_pg_04_e2e_*` still forbade workflow-level `contents: read`, but +`main` @ a311484 intentionally added a workflow read-only floor. Meta test was already red on +`main` before L-SHELL. +**Decision:** Update e2e meta to allow `contents: read` floor; forbid workflow write/admin; +keep job-local `contents: read` checks. Log as decide-and-log (no stop). +**Revert:** restore prior “must be removed” assertion (would stay red until e2e.yaml changes). + diff --git a/openspec/changes/e8-data-sources/design.md b/openspec/changes/e8-data-sources/design.md new file mode 100644 index 0000000..c24bb3e --- /dev/null +++ b/openspec/changes/e8-data-sources/design.md @@ -0,0 +1,213 @@ +# Design — e8-data-sources (Epic E8) + +Architecture decisions behind the three-bucket approach. See `proposal.md` for scope and +`specs/datasources/spec.md` for the traceable REQs. + +--- + +## 1. Why Upjet schema injection fails + +Upjet v2's `NewProvider` (in `pkg/config/provider.go`) calls `schema.Provider.Schema()` on the +Terraform plugin binary and extracts only `ResourceSchemas`. `DataSourceSchemas` is never read, +never stored, and never passed to code generators. There is no `include-list` for data sources and +no CRD emitter for them. + +If an operator manually injects a data-source schema entry into `resource_schemas` (e.g., via the +config override mechanism), the following happens: + +1. Upjet generates a CRD for it — the CRD installs and validates correctly. +2. The generated controller calls `provider.Read()` on the Terraform plugin binary using the + data-source schema key. +3. The Terraform provider binary (`terraform-provider-gridscale`) has no `ResourcesMap` entry for + any data source — only for managed resources. The plugin returns a "no such resource type" error. +4. Every `Observe` call fails permanently. The CRD is installed but the controller is non-functional. + +**Verdict: do not inject.** The illusion of functionality is worse than the absence of coverage. + +--- + +## 2. Why managementPolicies: [Observe] covers 19 of 21 + +Crossplane's management policy feature (`--enable-management-policies=true`, defaulting `true` in +`cmd/provider/main.go`) allows any existing managed resource to operate in observe-only mode: + +```yaml +spec: + managementPolicies: ["Observe"] + providerConfigRef: + name: default +``` + +Combined with `crossplane.io/external-name: ` pointing at an existing gridscale resource, the +controller syncs external state into `status.atProvider` without ever calling `Create`, `Update`, or +`Delete` on the external API. + +For 19 of the 21 Terraform data sources, there is a 1:1 Terraform resource equivalent +(`gridscale_storage` data source ↔ `gridscale_storage` resource, etc.). The Crossplane MR for the +resource already exists and is already wired. Observe-Only uses the existing MR's controller — no +new controller, no new CRD, no new Go code. + +**Constraint:** the caller must supply the UUID as `crossplane.io/external-name`. Observe-Only does +not perform name→UUID resolution. This is documented in `docs/observe-only.md`. + +--- + +## 3. Why BackupList and PublicNetwork need hand-authored controllers + +`backup_list` and `public_network` are the two Terraform data sources with no resource twin: + +- `backup_list` queries the storage-backup subresource at `GET /objects/storages/{uuid}/backups`. + There is no `gridscale_backup` managed resource — backups are not directly manageable objects. +- `public_network` discovers the account's public network via `GET /objects/networks` filtered by + `network_type == "public"`. There is a `gridscale_network` MR, but the public network is a + singleton not owned by the user and not represented as a Crossplane `Network` object. + +Neither can be covered by the Observe-Only pattern because there is no resource twin to point +`crossplane.io/external-name` at. Both require a custom read-only Crossplane managed resource with a +dedicated controller that calls the gridscale REST API directly. + +### Controller pattern (both BackupList and PublicNetwork) + +``` +ExternalConnecter.Connect(ctx, mg) + ↓ calls o.SetupFn(ctx, kubeClient, mg) + ↓ returns terraform.Setup with Configuration map["uuid"]/"token"/"api_url" + ↓ constructs GridscaleClient{UserID: cfg["uuid"], Token: cfg["token"], APIURL: cfg["api_url"]} + +ExternalClient.Observe(ctx, mg) + ↓ GridscaleClient.Get(ctx, path, &response) + ↓ populates mg.Status.AtProvider + ↓ returns managed.ExternalObservation{ResourceExists: true, ResourceUpToDate: true} + +ExternalClient.Create/Update/Delete(ctx, mg) + ↓ returns errors.New("BackupList/PublicNetwork is observe-only; Create/Update/Delete not supported") +``` + +The `o.SetupFn` is `clients.TerraformSetupBuilder(...)` (already in `internal/clients/gridscale.go`) +— it extracts credentials from the `ProviderConfig` secret, validates `uuid` and `token` are +non-empty, and returns them in `terraform.Setup.Configuration`. The custom connector reads those same +keys: `setup.Configuration["uuid"]`, `setup.Configuration["token"]`, +`setup.Configuration["api_url"]`. + +This credential extraction pattern is identical to the Upjet-generated controllers and reuses the +existing tested path in `internal/clients/gridscale.go`. + +--- + +## 4. GridscaleClient — shared HTTP client + +**File:** `internal/clients/gridscale_http.go` + +```go +type GridscaleClient struct { + UserID string + Token string + APIURL string + http *http.Client +} + +func (c *GridscaleClient) Get(ctx context.Context, path string, out any) error +``` + +Sends `GET {APIURL}/{path}` with headers `X-Auth-UserID` and `X-Auth-Token`. Returns decoded JSON +into `out` on 200; wraps HTTP error status codes as non-nil `error` on 4xx/5xx. + +TDD: `internal/clients/gridscale_http_test.go` uses `httptest.NewServer` — written before the +implementation. Tests assert correct header propagation, JSON decode, and 4xx/5xx error paths. + +Default `APIURL`: `https://api.gridscale.io` (same as the upstream Terraform provider default). + +--- + +## 5. controller-gen usage and the generation boundary + +New hand-authored types in `apis/cluster/storage/v1alpha1/` (BackupList) and +`apis/cluster/gridscale/v1alpha1/` (PublicNetwork) carry controller-gen marker comments: + +```go +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:resource:scope=Cluster +``` + +Running `controller-gen object:headerFile=hack/boilerplate.go.txt paths="./apis/..."` regenerates: +- `apis/cluster/storage/v1alpha1/zz_generated.deepcopy.go` (storage package — distinct from gridscale) +- `apis/cluster/gridscale/v1alpha1/zz_generated.deepcopy.go` (gridscale package — distinct from storage) +- `apis/namespaced/storage/v1alpha1/zz_generated.deepcopy.go` (namespaced storage) +- `apis/namespaced/gridscale/v1alpha1/zz_generated.deepcopy.go` (namespaced gridscale) + +Running `controller-gen crd paths="./apis/..."` produces CRD YAML in `package/crds/`: +- `storage.gridscale.platformrelay.io_backuplists.yaml` +- `storage.gridscale.m.platformrelay.io_backuplists.yaml` +- `gridscale.gridscale.platformrelay.io_publicnetworks.yaml` +- `gridscale.gridscale.m.platformrelay.io_publicnetworks.yaml` + +These `zz_generated.deepcopy.go` files are generator output — they must not be hand-edited. They +are distinct per package (storage vs. gridscale) so S02 and S03 have no deepcopy collision. + +Note: `make generate` (Upjet codegen) regenerates the **Upjet-controlled** zz files in existing +packages. The new BackupList/PublicNetwork packages are new packages — controller-gen is run +separately (not as part of `make generate`) and only touches the new type files. The stories must +call out this distinction so implementors do not accidentally trigger Upjet codegen. + +--- + +## 6. Non-generated naming convention + +Custom controllers and their setup functions use non-`zz_` names: + +| File | Purpose | +|------|---------| +| `internal/controller/cluster/setup_custom.go` | `SetupCustom(mgr, o)` — registers BackupList + PublicNetwork cluster controllers | +| `internal/controller/namespaced/setup_custom.go` | `SetupCustom(mgr, o)` — registers namespaced variants | +| `apis/cluster/storage/v1alpha1/backuplist_types.go` | BackupList type definitions (hand-authored) | +| `apis/cluster/storage/v1alpha1/backuplist_register.go` | scheme registration | +| `apis/cluster/gridscale/v1alpha1/publicnetwork_types.go` | PublicNetwork type definitions (hand-authored) | +| `apis/cluster/gridscale/v1alpha1/publicnetwork_register.go` | scheme registration | +| `internal/controller/cluster/storage/backuplist/backuplist.go` | controller + reconciler | +| `internal/controller/cluster/gridscale/publicnetwork/publicnetwork.go` | controller + reconciler | + +Namespaced variants mirror the cluster paths under `apis/namespaced/` and +`internal/controller/namespaced/`. The `zz_setup.go` files in each controller package remain +Upjet-generated and untouched. + +New types register into their package's existing `SchemeBuilder` — they inherit scheme registration +because they live in already-registered packages: + +```go +// In backuplist_register.go — registers into storage/v1alpha1's existing SchemeBuilder +func init() { + SchemeBuilder.Register(&BackupList{}, &BackupListList{}) +} +``` + +--- + +## Architecture diagram + +```mermaid +flowchart LR + subgraph E8-S01 ["E8-S01 (independent)"] + OY["38 observe-only YAMLs\nexamples-generated/{cluster,namespaced}/observe/"] + OD["docs/observe-only.md"] + end + + subgraph E8-S02 ["E8-S02"] + HC["internal/clients/gridscale_http.go\nGridscaleClient.Get()"] + BL["BackupList CRD + controller\napis/cluster/storage/v1alpha1/\ninternal/controller/cluster/storage/backuplist/"] + end + + subgraph E8-S03 ["E8-S03 (depends on S02 for shared client)"] + PN["PublicNetwork CRD + controller\napis/cluster/gridscale/v1alpha1/\ninternal/controller/cluster/gridscale/publicnetwork/"] + end + + subgraph E8-S04 ["E8-S04 (depends on S02+S03)"] + SC["setup_custom.go (cluster + namespaced)"] + MA["cmd/provider/main.go"] + RM["README.md resource matrix"] + end + + E8-S02 --> E8-S03 + E8-S02 --> E8-S04 + E8-S03 --> E8-S04 +``` diff --git a/openspec/changes/e8-data-sources/proposal.md b/openspec/changes/e8-data-sources/proposal.md new file mode 100644 index 0000000..6436f92 --- /dev/null +++ b/openspec/changes/e8-data-sources/proposal.md @@ -0,0 +1,144 @@ +# Change: e8-data-sources — Data Sources (Epic E8, active) + +## Why + +Decision **D-015** (2026-07-16) rescoped E8 to "document omission + track upstream; no codegen" +because Upjet v2's `NewProvider` never reads `DataSourceSchemas` — only `ResourceSchemas`. That +made all 21 gridscale Terraform data sources ungenerable via the Upjet pipeline. + +D-015 is now **superseded**. Upjet's limitation still stands, but the correct response is not to +wait for upstream — it is to deliver Crossplane-idiomatic coverage using two complementary +strategies: + +1. **19 of 21** data sources have an existing managed-resource twin. Crossplane's built-in + `managementPolicies: ["Observe"]` mechanism turns any existing MR into a read-only observer: + set the policy, annotate `crossplane.io/external-name` with the target UUID, and the controller + syncs external state to `status.atProvider` without creating or deleting anything. Zero new code + and zero new CRDs are needed. + +2. **2 of 21** data sources (`backup_list`, `public_network`) have no resource twin and cannot be + covered by Observe-Only. They need purpose-built, hand-authored Crossplane managed resource + implementations with read-only controllers. + +Together these deliver all 21 data source use-cases with appropriate Crossplane idioms. + +## Supersedes / decisions consumed + +- **D-015** — rescoped; this change revives and completes E8 properly. +- **D-005** — test taxonomy (U/E/M/D) carried forward unchanged. + +## Scope (what E8 builds) + +### Bucket 1 — observe-only examples (E8-S01) + +19 data sources with a resource twin — covered by `managementPolicies: ["Observe"]` pattern. + +**Deliverable:** 38 YAML files (19 cluster-scoped + 19 namespace-scoped) under +`examples-generated/cluster/observe/` and `examples-generated/namespaced/observe/`, plus +`docs/observe-only.md` documenting the pattern. + +Resources and their groups: + +| Kind | Group (cluster) | Group (namespaced) | +|------|-----------------|-------------------| +| Backupschedule | `gridscale.gridscale.platformrelay.io/v1alpha1` | same, namespaced | +| Firewall | gridscale | gridscale | +| IPv4 | gridscale | gridscale | +| IPv6 | gridscale | gridscale | +| Isoimage | gridscale | gridscale | +| K8S | gridscale | gridscale | +| Loadbalancer | gridscale | gridscale | +| Network | gridscale | gridscale | +| Paas | gridscale | gridscale | +| Server | gridscale | gridscale | +| Snapshot | gridscale | gridscale | +| Snapshotschedule | gridscale | gridscale | +| Sshkey | gridscale | gridscale | +| Storage | gridscale | gridscale | +| Template | gridscale | gridscale | +| Application | `marketplace.gridscale.platformrelay.io/v1alpha1` | same, namespaced | +| StorageAccesskey | `object.gridscale.platformrelay.io/v1alpha1` | same, namespaced | +| Securityzone | `paas.gridscale.platformrelay.io/v1alpha1` | same, namespaced | +| Certificate | `ssl.gridscale.platformrelay.io/v1alpha1` | same, namespaced | + +**Important implementation note:** Kind names use flattened casing as generated by Upjet — +`Isoimage` (not `IsoImage`), `K8S` (not `K8s`), `Paas` (not `PaaS`), `Snapshotschedule` (not +`SnapshotSchedule`), `Backupschedule` (not `BackupSchedule`). Read `names.kind` from each CRD in +`package/crds/` — never hand-type Kind values. + +### Bucket 2a — BackupList custom CRD + controller (E8-S02) + +- **Kind:** `BackupList` — Group: `storage.gridscale.platformrelay.io`, Version: `v1alpha1` +- Both cluster-scoped (using `github.com/crossplane/crossplane-runtime/v2/apis/common/v1` + `ResourceSpec`) and namespace-scoped (using + `github.com/crossplane/crossplane-runtime/v2/apis/common/v2` `ManagedResourceSpec`) variants. +- Input: `spec.forProvider.storageUUID` (string, required) +- Output: `status.atProvider.storageBackups` — list of `{objectUuid, name, capacity float64, + createTime}` +- API: `GET /objects/storages/{storageUUID}/backups` +- TDD: `internal/clients/gridscale_http_test.go` written first; implements + `internal/clients/gridscale_http.go` with a `GridscaleClient.Get(ctx, path, out) error` method. + This shared client is also used by PublicNetwork (E8-S03 depends on S02 for this). +- controller-gen v0.20.1 regenerates `zz_generated.deepcopy.go` (storage package) and produces CRD + YAML in `package/crds/`. + +### Bucket 2b — PublicNetwork custom CRD + controller (E8-S03) + +- **Kind:** `PublicNetwork` — Group: `gridscale.gridscale.platformrelay.io`, Version: `v1alpha1` +- Both cluster-scoped and namespace-scoped variants. +- Input: none (singleton — always the account's public network, discovered by filter) +- Output: `status.atProvider.{name, status, networkType, locationUUID, locationName, + locationCountry, locationIATA, l2Security bool, deleteBlock bool, labels []string, createTime, + changeTime}` +- API: `GET /objects/networks`, filter `network_type == "public"` +- Reuses `GridscaleClient` from E8-S02. **Depends on E8-S02.** +- controller-gen regenerates `zz_generated.deepcopy.go` (gridscale package) and produces CRD YAML. + +### Bucket 3 — Wire + README (E8-S04) + +- `internal/controller/cluster/setup_custom.go` and `internal/controller/namespaced/setup_custom.go` + (hand-authored, no `zz_` prefix, never touched by `make generate`). +- `cmd/provider/main.go` calls the new `SetupCustom`/`SetupCustomGated` functions. +- `README.md` resource matrix updated with BackupList and PublicNetwork rows. +- `go build ./...` and `go vet ./...` pass on the assembled tree. +- **Depends on E8-S02 and E8-S03.** + +## Non-goals + +- **Upjet data-source codegen** — Upjet v2 has no `DataSourceSchemas` pipeline (D-015). E8 does not + fork Upjet, patch its codegen, or schema-inject data sources. +- **Schema injection into existing CRDs** — would produce CRDs that compile but fail at runtime + (no `ResourcesMap` backing in the TF provider binary). Explicitly rejected. +- **Observe-Only controllers for BackupList/PublicNetwork** — these have no resource twin and no + UUID to target; the Observe mechanism requires a pre-existing MR with a known external-name. +- **Name-to-UUID lookup / discovery for Bucket 1** — the caller must supply the target UUID as + `crossplane.io/external-name`. UUID discovery is out of scope for this change. +- **Upstream upjet feature request** — queued as a separate operator task per D-015 disposition. + This change does not block on it. +- **Hand-editing any `zz_*.go` file** — all zz files remain generator-owned. New deepcopy files for + BackupList/PublicNetwork packages are regenerated via controller-gen, not hand-written. + +## Counterpoints considered + +**"Schema injection: add the data source schemas to resource_schemas."** +Rejected. The Terraform provider binary has no `ResourcesMap` entry for data sources. The injected +CRD compiles and is installed, but the controller calls `provider.Read()` against a resource key +that does not exist — runtime failure. Upjet's `ExternalConnecter` would return provider errors on +every Observe call. The CRD would be misleadingly registered while silently non-functional. + +**"Observe-only is enough — just document it and ship 19/21."** +Partially accepted for Bucket 1 (19 resources with twins — observe-only is the correct idiom). +Rejected for full E8 scope: `backup_list` and `public_network` are in active operator use-cases +(backup enumeration for restore workflows; network topology discovery). The two unique data sources +have no twin, so Observe-Only cannot cover them. Hand-authored controllers are the right answer at +~300 lines of total Go. + +**"Wait for upstream upjet DataSourceSchemas support."** +Rejected as the primary strategy. Upstream has no open issue or PR for this feature. Hand-authored +coverage of the 2 unique data sources is stable, testable, and does not block on upstream timelines. + +## Links + +- D-015 original rescoping: `agent-context/decisions.md` §D-015 +- Upstream upjet — no `DataSourceSchemas` reference: `github.com/crossplane/upjet` `pkg/config/provider.go` +- Controller-gen v0.20.1 docs: https://github.com/kubernetes-sigs/controller-tools/tree/v0.20.1 diff --git a/openspec/changes/e8-data-sources/specs/datasources/spec.md b/openspec/changes/e8-data-sources/specs/datasources/spec.md new file mode 100644 index 0000000..e28e93f --- /dev/null +++ b/openspec/changes/e8-data-sources/specs/datasources/spec.md @@ -0,0 +1,374 @@ +# Spec — E8 Data Sources (e8-data-sources) + +Epic: **E8** · Change slug: `e8-data-sources` +Stories: E8-S01 (observe-only), E8-S02 (BackupList), E8-S03 (PublicNetwork), E8-S04 (wire+README) + +Every REQ maps to exactly one **Level** (U/E/M/D), one **Test** (artifact path that must exist +and pass after implementation), and one **Verify** (exact command). Implementation must add the +failing **Test** artifact before implementing (TDD rule per GUIDELINES §2). + +--- + +## REQ-E8-S01-01 (DS-01) — 38 observe-only YAML files exist under examples-generated/ + +**Level:** M · **Priority:** must · **Story:** E8-S01 + +**Given** E8-S01 has landed, +**when** `find examples-generated/cluster/observe -name '*.yaml' | wc -l` and +`find examples-generated/namespaced/observe -name '*.yaml' | wc -l` are each run, +**then** both return 19 (total: 38 files — 19 per scope, one per resource with a TF data-source twin). + +**Edge:** Given a YAML file for `Isoimage` or `K8S` (flattened-cased Kinds), +when `kubectl apply --dry-run=client` is run against a cluster with the provider CRDs installed, +then the manifest is accepted with no unknown-kind or unknown-field errors. +(This validates that Kind/apiVersion were read from `package/crds/` — not hand-typed.) + +**Test:** `hack/test/e8_s01_observe_yaml_count_test.sh` + +**Verify:** `bash hack/test/e8_s01_observe_yaml_count_test.sh` + +--- + +## REQ-E8-S01-02 (DS-02) — docs/observe-only.md exists and documents the pattern + +**Level:** M · **Priority:** must · **Story:** E8-S01 + +**Given** E8-S01 has landed, +**when** `docs/observe-only.md` is read, +**then** the file exists, contains the string `managementPolicies`, contains the string +`crossplane.io/external-name`, and explains the UUID requirement in at least one sentence. + +**Edge:** Given a user who does not know the external-name requirement, when they read the doc, +then at least one example YAML snippet with a placeholder UUID is shown. + +**Test:** `hack/test/e8_s01_observe_docs_test.sh` + +**Verify:** `bash hack/test/e8_s01_observe_docs_test.sh` + +--- + +## REQ-E8-S01-03 (DS-03) — uptest can apply an observe-only Sshkey manifest (creds-gated) + +**Level:** E · **Priority:** should · **Story:** E8-S01 + +**Given** a gridscale lab project with an existing Sshkey (UUID known and set as `crossplane.io/external-name`), +**when** `uptest run examples-generated/cluster/observe/gridscale/v1alpha1/sshkey.yaml` is executed +against a live cluster with the provider installed and `ProviderConfig` credentials configured, +**then** the manifest applies cleanly, the MR condition reaches `Synced: True / Ready: True`, and +`status.atProvider.name` is populated with the remote resource's name. + +**Edge:** Given a non-existent UUID in `crossplane.io/external-name`, when the provider observes it, +then the MR condition is set to `Synced: False` with a descriptive message — no panic, no crash, +no CUD operation attempted. + +**Test:** `examples-generated/cluster/observe/gridscale/v1alpha1/sshkey.yaml` + +**Verify:** `uptest run examples-generated/cluster/observe/gridscale/v1alpha1/sshkey.yaml` +(requires lab credentials; gated behind `/test-examples` trigger + nightly — per D-012 creds contract) + +--- + +## REQ-E8-S02-01 (DS-04) — GridscaleClient.Get sends correct headers and decodes JSON + +**Level:** U · **Priority:** must · **Story:** E8-S02 + +**Given** a `GridscaleClient{UserID: "test-uuid", Token: "test-token", APIURL: server.URL}` and an +`httptest.NewServer` returning HTTP 200 with a JSON body `{"key":"value"}`, +**when** `client.Get(ctx, "objects/test", &out)` is called, +**then** the server receives `X-Auth-UserID: test-uuid` and `X-Auth-Token: test-token` headers, +`out` is populated with the decoded JSON, and the error is `nil`. + +**Edge:** Given the mock server returns HTTP 403, +**when** `client.Get` is called, +**then** a non-nil error wrapping "403" (or the status text) is returned and `out` is unchanged. + +**Test:** `internal/clients/gridscale_http_test.go` + +**Verify:** `go test ./internal/clients/... -run TestGridscaleClient -v` + +--- + +## REQ-E8-S02-02 (DS-05) — BackupList CRD YAML exists in package/crds/ + +**Level:** M · **Priority:** must · **Story:** E8-S02 + +**Given** E8-S02 has landed and controller-gen has been run, +**when** `ls package/crds/` is executed, +**then** both `storage.gridscale.platformrelay.io_backuplists.yaml` and +`storage.gridscale.m.platformrelay.io_backuplists.yaml` are present. + +**Edge:** Given the CRD YAML is opened, when `spec.group` is read, then it equals +`storage.gridscale.platformrelay.io` (cluster) or `storage.gridscale.m.platformrelay.io` +(namespaced) respectively — not the gridscale group. + +**Test:** `hack/test/e8_s02_backuplist_crd_exists_test.sh` + +**Verify:** `bash hack/test/e8_s02_backuplist_crd_exists_test.sh` + +--- + +## REQ-E8-S02-03 (DS-07) — BackupList connector.Connect returns ExternalClient with credentials + +**Level:** U · **Priority:** must · **Story:** E8-S02 + +**Given** a mock `SetupFn` that returns +`terraform.Setup{Configuration: map[string]any{"uuid": "u1", "token": "t1", "api_url": "http://fake"}}`, +**when** `connector.Connect(ctx, &BackupList{...})` is called, +**then** the returned `ExternalClient` holds a `GridscaleClient` with `UserID=="u1"`, +`Token=="t1"`, and `APIURL=="http://fake"`, and no error is returned. + +**Edge:** Given `SetupFn` returns a non-nil error, +**when** `Connect` is called, +**then** `Connect` wraps and propagates the error and returns a nil client. + +**Test:** `internal/controller/cluster/storage/backuplist/backuplist_test.go` + +**Verify:** `go test ./internal/controller/cluster/storage/backuplist/... -run TestConnect -v` + +--- + +## REQ-E8-S02-04 (DS-08) — BackupList Observe populates status.atProvider.storageBackups + +**Level:** U · **Priority:** must · **Story:** E8-S02 + +**Given** a `GridscaleClient` backed by an `httptest.NewServer` returning a JSON array +`[{"object_uuid": "bu1", "name": "b1", "capacity": 10.0, "create_time": "..."}]`, +**when** `ExternalClient.Observe(ctx, backupList)` is called, +**then** `backupList.Status.AtProvider.StorageBackups` has length 1, the first entry has +`ObjectUUID == "bu1"` and `Name == "b1"`, and `observation.ResourceExists == true`. + +**Edge:** Given the server returns an empty array `[]`, +**when** `Observe` is called, +**then** `StorageBackups` has length 0 and `ResourceExists` is `true` — an empty backup list +means the storage exists with no backups, not that the storage is absent. + +**Test:** `internal/controller/cluster/storage/backuplist/backuplist_test.go` + +**Verify:** `go test ./internal/controller/cluster/storage/backuplist/... -run TestObserve -v` + +--- + +## REQ-E8-S02-05 (DS-09) — BackupList Create/Update/Delete return observe-only error + +**Level:** U · **Priority:** must · **Story:** E8-S02 + +**Given** an instantiated `ExternalClient` for a `BackupList`, +**when** `Create(ctx, mg)`, `Update(ctx, mg)`, or `Delete(ctx, mg)` is called, +**then** each returns a non-nil `error` containing the string "observe-only" (or equivalent), +and no HTTP request is made to the gridscale API. + +**Edge:** Given `Delete` is called (e.g., resource being finalized), when the observe-only error +is returned, then `errors.Is`-wrapping of a sentinel `errObserveOnly` error allows the reconciler +to surface it without infinite retry. (Use a package-level `var errObserveOnly = errors.New("...")`, +not a type assertion.) + +**Test:** `internal/controller/cluster/storage/backuplist/backuplist_test.go` + +**Verify:** `go test ./internal/controller/cluster/storage/backuplist/... -run TestObserveOnly -v` + +--- + +## REQ-E8-S03-01 (DS-06) — PublicNetwork CRD YAML exists in package/crds/ + +**Level:** M · **Priority:** must · **Story:** E8-S03 + +**Given** E8-S03 has landed and controller-gen has been run, +**when** `ls package/crds/` is executed, +**then** both `gridscale.gridscale.platformrelay.io_publicnetworks.yaml` and +`gridscale.gridscale.m.platformrelay.io_publicnetworks.yaml` are present. + +**Edge:** Given the CRD YAML is opened, when `spec.group` is read, then it equals +`gridscale.gridscale.platformrelay.io` (cluster) or `gridscale.gridscale.m.platformrelay.io` +(namespaced) — not the storage group. + +**Test:** `hack/test/e8_s03_publicnetwork_crd_exists_test.sh` + +**Verify:** `bash hack/test/e8_s03_publicnetwork_crd_exists_test.sh` + +--- + +## REQ-E8-S03-02 (DS-10) — PublicNetwork connector.Connect returns ExternalClient + +**Level:** U · **Priority:** must · **Story:** E8-S03 + +**Given** a mock `SetupFn` returning +`terraform.Setup{Configuration: map[string]any{"uuid":"u2","token":"t2"}}`, +**when** `connector.Connect(ctx, &PublicNetwork{...})` is called, +**then** the returned `ExternalClient` holds a `GridscaleClient` with `UserID=="u2"`, +`Token=="t2"`, and no error is returned. + +**Edge:** Given `SetupFn` returns an error, +**when** `Connect` is called, +**then** the error is propagated and the returned client is nil. + +**Test:** `internal/controller/cluster/gridscale/publicnetwork/publicnetwork_test.go` + +**Verify:** `go test ./internal/controller/cluster/gridscale/publicnetwork/... -run TestConnect -v` + +--- + +## REQ-E8-S03-03 (DS-11) — PublicNetwork Observe finds public network from GET /objects/networks + +**Level:** U · **Priority:** must · **Story:** E8-S03 + +**Given** a `GridscaleClient` backed by an `httptest.NewServer` returning a JSON network list that +includes one object with `network_type: "public"` and `name: "Public Network"`, +**when** `ExternalClient.Observe(ctx, publicNetwork)` is called, +**then** `publicNetwork.Status.AtProvider.Name == "Public Network"`, +`publicNetwork.Status.AtProvider.NetworkType == "public"`, +and `observation.ResourceExists == true`. + +**Edge:** Given the response contains only `network_type: "private"` entries, +**when** `Observe` is called, **then** `ResourceExists == false` and the error is `nil`. + +**Test:** `internal/controller/cluster/gridscale/publicnetwork/publicnetwork_test.go` + +**Verify:** `go test ./internal/controller/cluster/gridscale/publicnetwork/... -run TestObserve -v` + +--- + +## REQ-E8-S03-04 (DS-12) — PublicNetwork Observe sets ResourceExists=false when no public network + +**Level:** U · **Priority:** must · **Story:** E8-S03 + +**Given** a mock server returning an empty network list (`{"networks": {}}`), +**when** `Observe` is called, +**then** `observation.ResourceExists == false` and the error is `nil`. +(An absent public network is a valid empty observation, not an error condition.) + +**Test:** `internal/controller/cluster/gridscale/publicnetwork/publicnetwork_test.go` + +**Verify:** `go test ./internal/controller/cluster/gridscale/publicnetwork/... -run TestObserveNotFound -v` + +--- + +## REQ-E8-S03-05 (DS-13) — PublicNetwork Create/Update/Delete return observe-only error + +**Level:** U · **Priority:** must · **Story:** E8-S03 + +**Given** an instantiated `ExternalClient` for a `PublicNetwork`, +**when** `Create(ctx, mg)`, `Update(ctx, mg)`, or `Delete(ctx, mg)` is called, +**then** each returns a non-nil `error` containing "observe-only" and no HTTP request is made. + +**Edge:** Same sentinel-error pattern as DS-09 — use a package-level `var errObserveOnly` that +allows `errors.Is` matching by callers; no type assertion required. + +**Test:** `internal/controller/cluster/gridscale/publicnetwork/publicnetwork_test.go` + +**Verify:** `go test ./internal/controller/cluster/gridscale/publicnetwork/... -run TestObserveOnly -v` + +--- + +## REQ-E8-S04-01 (DS-14) — go build ./... passes with all new types + controllers wired + +**Level:** M · **Priority:** must · **Story:** E8-S04 + +**Given** E8-S02, E8-S03, and E8-S04 have all landed, +**when** `go build ./...` is run from the repo root, +**then** the exit code is 0 and no compilation errors are emitted. + +**Edge:** Given `setup_custom.go` imports a non-existent controller package (e.g., a typo in the +import path), when `go build` is run, then it fails with an import error — this is the compile-time +guard that prevents registration drift from the actual controller locations. + +**Test:** `hack/test/e8_s04_go_build_test.sh` + +**Verify:** `bash hack/test/e8_s04_go_build_test.sh` + +--- + +## REQ-E8-S04-02 (DS-15) — go vet ./... passes + +**Level:** M · **Priority:** must · **Story:** E8-S04 + +**Given** E8-S02, E8-S03, and E8-S04 have landed, +**when** `go vet ./...` is run, +**then** the exit code is 0. + +**Test:** `hack/test/e8_s04_go_vet_test.sh` + +**Verify:** `bash hack/test/e8_s04_go_vet_test.sh` + +--- + +## REQ-E8-S04-03 (DS-16) — BackupList and PublicNetwork registered in cluster setup_custom.go + +**Level:** M · **Priority:** must · **Story:** E8-S04 + +**Given** E8-S04 has landed, +**when** `grep -r "BackupList\|PublicNetwork" internal/controller/cluster/setup_custom.go` is run, +**then** the command exits 0 and both names appear in the output. + +**Edge:** Given `setup_custom.go` exists but the `SetupCustom` function signature does not match +`func(ctrl.Manager, controller.Options) error`, when `go build ./...` runs, then it fails at +compile time — no runtime-only registration bugs possible. + +**Test:** `hack/test/e8_s04_setup_custom_cluster_test.sh` + +**Verify:** `bash hack/test/e8_s04_setup_custom_cluster_test.sh` + +--- + +## REQ-E8-S04-04 (DS-17) — BackupList and PublicNetwork registered in namespaced setup_custom.go + +**Level:** M · **Priority:** must · **Story:** E8-S04 + +**Given** E8-S04 has landed, +**when** `grep -r "BackupList\|PublicNetwork" internal/controller/namespaced/setup_custom.go` is run, +**then** the command exits 0 and both names appear in the output. + +**Edge:** Given only the cluster variant is registered and the namespaced file is missing, when +`go build ./...` runs, the build still passes — the meta test explicitly checks the namespaced +file content to catch this omission before deployment. + +**Test:** `hack/test/e8_s04_setup_custom_namespaced_test.sh` + +**Verify:** `bash hack/test/e8_s04_setup_custom_namespaced_test.sh` + +--- + +## REQ-E8-S04-05 (DS-18) — README resource matrix includes BackupList and PublicNetwork + +**Level:** D · **Priority:** should · **Story:** E8-S04 + +**Note:** README is hand-maintained public surface. This REQ is categorised D (doc) because the +matrix row is authored alongside the implementation, not generated from source. (It is not a D in +the strict "generated docs in sync with source" sense — no `make docs` regeneration covers it. +A code-review step is required to validate group accuracy.) + +**Given** E8-S04 has landed, +**when** `grep -E "BackupList|PublicNetwork" README.md` is run, +**then** both names appear at least once (the resource matrix has been updated). + +**Edge:** Given the matrix row is added but the API group is incorrect, when a maintainer reviews, +then the discrepancy is caught by review — the meta test asserts name presence only, not group +accuracy. + +**Test:** `hack/test/e8_s04_readme_matrix_test.sh` + +**Verify:** `bash hack/test/e8_s04_readme_matrix_test.sh` + +--- + +## Summary table + +| REQ | ID | Level | Story | Test artifact | +|-----|-----|-------|-------|--------------| +| 38 observe-only YAMLs exist (19 cluster + 19 namespaced) | DS-01 | M | E8-S01 | `hack/test/e8_s01_observe_yaml_count_test.sh` | +| docs/observe-only.md exists with pattern | DS-02 | M | E8-S01 | `hack/test/e8_s01_observe_docs_test.sh` | +| uptest applies observe-only Sshkey (creds-gated) | DS-03 | E | E8-S01 | `examples-generated/cluster/observe/gridscale/v1alpha1/sshkey.yaml` | +| GridscaleClient headers + JSON decode | DS-04 | U | E8-S02 | `internal/clients/gridscale_http_test.go` | +| BackupList CRD YAML in package/crds/ | DS-05 | M | E8-S02 | `hack/test/e8_s02_backuplist_crd_exists_test.sh` | +| PublicNetwork CRD YAML in package/crds/ | DS-06 | M | E8-S03 | `hack/test/e8_s03_publicnetwork_crd_exists_test.sh` | +| BackupList Connect returns client+creds | DS-07 | U | E8-S02 | `internal/controller/cluster/storage/backuplist/backuplist_test.go` | +| BackupList Observe populates status | DS-08 | U | E8-S02 | `internal/controller/cluster/storage/backuplist/backuplist_test.go` | +| BackupList Create/Update/Delete → error | DS-09 | U | E8-S02 | `internal/controller/cluster/storage/backuplist/backuplist_test.go` | +| PublicNetwork Connect returns client | DS-10 | U | E8-S03 | `internal/controller/cluster/gridscale/publicnetwork/publicnetwork_test.go` | +| PublicNetwork Observe finds public net | DS-11 | U | E8-S03 | `internal/controller/cluster/gridscale/publicnetwork/publicnetwork_test.go` | +| PublicNetwork Observe → false when absent | DS-12 | U | E8-S03 | `internal/controller/cluster/gridscale/publicnetwork/publicnetwork_test.go` | +| PublicNetwork Create/Update/Delete → error | DS-13 | U | E8-S03 | `internal/controller/cluster/gridscale/publicnetwork/publicnetwork_test.go` | +| go build ./... passes | DS-14 | M | E8-S04 | `hack/test/e8_s04_go_build_test.sh` | +| go vet ./... passes | DS-15 | M | E8-S04 | `hack/test/e8_s04_go_vet_test.sh` | +| BackupList+PublicNetwork in cluster setup | DS-16 | M | E8-S04 | `hack/test/e8_s04_setup_custom_cluster_test.sh` | +| BackupList+PublicNetwork in namespaced setup | DS-17 | M | E8-S04 | `hack/test/e8_s04_setup_custom_namespaced_test.sh` | +| README matrix includes both Kinds | DS-18 | D | E8-S04 | `hack/test/e8_s04_readme_matrix_test.sh` | diff --git a/openspec/changes/e8-data-sources/tasks.md b/openspec/changes/e8-data-sources/tasks.md new file mode 100644 index 0000000..cfe2885 --- /dev/null +++ b/openspec/changes/e8-data-sources/tasks.md @@ -0,0 +1,125 @@ +# Tasks — e8-data-sources (provider-gridscale) + +Backlog stories: **E8-S01…E8-S04** (`agent-context/BACKLOG.md`). +Claim one story per worktree. S01 is independently mergeable — land it first. + +## Story/REQ/file-lock table + +| Story | REQs | File locks | Meta-test locks | Status | +|-------|------|------------|-----------------|--------| +| E8-S01 | DS-01, DS-02, DS-03 | `examples-generated/cluster/observe/`, `examples-generated/namespaced/observe/`, `docs/observe-only.md` | `hack/test/e8_s01_*` | ⬜ | +| E8-S02 | DS-04, DS-05, DS-07, DS-08, DS-09 | `internal/clients/gridscale_http_test.go`, `internal/clients/gridscale_http.go`, `apis/cluster/storage/v1alpha1/backuplist_*.go`, `apis/namespaced/storage/v1alpha1/backuplist_*.go`, `internal/controller/cluster/storage/backuplist/`, `internal/controller/namespaced/storage/backuplist/`, `package/crds/*backuplist*.yaml` | `hack/test/e8_s02_*` | ⬜ | +| E8-S03 | DS-06, DS-10, DS-11, DS-12, DS-13 | `apis/cluster/gridscale/v1alpha1/publicnetwork_*.go`, `apis/namespaced/gridscale/v1alpha1/publicnetwork_*.go`, `internal/controller/cluster/gridscale/publicnetwork/`, `internal/controller/namespaced/gridscale/publicnetwork/`, `package/crds/*publicnetwork*.yaml` | `hack/test/e8_s03_*` | ⬜ | +| E8-S04 | DS-14, DS-15, DS-16, DS-17, DS-18 | `internal/controller/cluster/setup_custom.go`, `internal/controller/namespaced/setup_custom.go`, `cmd/provider/main.go`, `README.md` | `hack/test/e8_s04_*` | ⬜ | + +## Parallelism + +``` +E8-S01 (independent — no Go, no CRDs, immediately mergeable) +E8-S02 (independent of S01; owns gridscale_http.go — S03 depends on S02) +E8-S03 (depends on S02 for GridscaleClient) +E8-S04 (depends on S02+S03) +``` + +S01 can land on `main` before S02/S03/S04 start. S02 and S03 **share** +`internal/clients/gridscale_http.go` — the shared client belongs to S02's file lock; +S03 reads it as a dependency, does not modify it. + +## Task sequence per story + +### E8-S01 — Observe-only YAML examples + +1. Read `names.kind` and `names.group` from each of the 19 CRDs in `package/crds/` — do not + hand-type Kind values; Upjet uses flattened casing (`Isoimage`, `K8S`, `Paas`, `Snapshotschedule`, + `StorageAccesskey`, `Securityzone`, etc.). +2. For each of the 19 resources, write a cluster-scoped YAML to + `examples-generated/cluster/observe//.yaml` with: + - `managementPolicies: ["Observe"]` + - `crossplane.io/external-name: ` annotation + - `meta.upbound.io/example-id: /v1alpha1/-observe` +3. Write the matching namespace-scoped YAML to + `examples-generated/namespaced/observe//.yaml`. +4. Write `docs/observe-only.md` explaining the pattern and the UUID requirement. +5. Verify: `find examples-generated/cluster/observe -name '*.yaml' | wc -l` → 19; + `find examples-generated/namespaced/observe -name '*.yaml' | wc -l` → 19. +6. Gate: `make reviewable` green (no generated-file drift, no lint failures). + +### E8-S02 — BackupList CRD + controller + +1. **Write failing test first:** create `internal/clients/gridscale_http_test.go` asserting: + - Correct `X-Auth-UserID` and `X-Auth-Token` headers on `GET` request. + - JSON decode into `out` on HTTP 200. + - Non-nil `error` wrapping the status code on HTTP 4xx/5xx. + Run `go test ./internal/clients/... -run TestGridscaleClient` — must fail (file doesn't exist). +2. Implement `internal/clients/gridscale_http.go` — `GridscaleClient` struct + `Get()` method. +3. Run `go test ./internal/clients/...` — must pass. +4. Write `apis/cluster/storage/v1alpha1/backuplist_types.go` with controller-gen markers + (`+kubebuilder:object:root=true`, `+kubebuilder:subresource:status`, + `+kubebuilder:resource:scope=Cluster`). Embed `v1.ResourceSpec` from + `github.com/crossplane/crossplane-runtime/v2/apis/common/v1`. +5. Write `apis/cluster/storage/v1alpha1/backuplist_register.go` registering into the package's + existing `SchemeBuilder`. +6. Mirror steps 4–5 for namespaced variant under `apis/namespaced/storage/v1alpha1/`. +7. Run `controller-gen object:headerFile=hack/boilerplate.go.txt paths="./apis/cluster/storage/..."` + to regenerate `apis/cluster/storage/v1alpha1/zz_generated.deepcopy.go`; repeat for namespaced. +8. Run `controller-gen crd paths="./apis/..."` to emit + `package/crds/storage.gridscale.platformrelay.io_backuplists.yaml` and + `package/crds/storage.gridscale.m.platformrelay.io_backuplists.yaml`. +9. Write `internal/controller/cluster/storage/backuplist/backuplist.go` — `Setup()`, + `ExternalConnecter.Connect()` (calls `o.SetupFn` → reads `setup.Configuration["uuid"/"token"/"api_url"]`), + `ExternalClient.Observe()` (calls `GridscaleClient.Get(ctx, "objects/storages/{uuid}/backups", &resp)` + and populates `status.atProvider.storageBackups`), + `Create/Update/Delete()` return observe-only error. +10. Mirror step 9 for namespaced variant. +11. Gate: `go test ./internal/clients/...` green; `go build ./...` passes; `make reviewable` clean. + +### E8-S03 — PublicNetwork CRD + controller + +**Depends on E8-S02 (GridscaleClient is available).** + +1. **Write failing test first:** add test to `internal/controller/cluster/gridscale/publicnetwork/` + (or a new `publicnetwork_test.go`) asserting that `Observe` with a mocked response containing + `network_type: "public"` sets `ResourceExists: true`; and that an all-private response sets + `ResourceExists: false`. +2. Write `apis/cluster/gridscale/v1alpha1/publicnetwork_types.go` with controller-gen markers. + Embed `v1.ResourceSpec`. `Status.AtProvider` contains: `name`, `status`, `networkType`, + `locationUUID`, `locationName`, `locationCountry`, `locationIATA`, `l2Security bool`, + `deleteBlock bool`, `labels []string`, `createTime`, `changeTime`. +3. Write `apis/cluster/gridscale/v1alpha1/publicnetwork_register.go`. +4. Mirror for namespaced variant. +5. Run controller-gen to regenerate deepcopy for gridscale package and emit CRD YAML: + `gridscale.gridscale.platformrelay.io_publicnetworks.yaml` and + `gridscale.gridscale.m.platformrelay.io_publicnetworks.yaml` in `package/crds/`. +6. Implement `internal/controller/cluster/gridscale/publicnetwork/publicnetwork.go` — + `Observe()` calls `GET /objects/networks`, filters for `network_type == "public"`, + sets `ResourceExists: false` when no match found, `ResourceExists: true` + populates + `status.atProvider` when matched. +7. Mirror step 6 for namespaced variant. +8. Gate: failing test from step 1 now green; `go build ./...` passes; `make reviewable` clean. + +### E8-S04 — Wire everything + README + +**Depends on E8-S02 and E8-S03.** + +1. Write `internal/controller/cluster/setup_custom.go` — `SetupCustom(mgr ctrl.Manager, o controller.Options) error` + calling `backuplist.Setup` and `publicnetwork.Setup`. No `zz_` prefix; not generated. +2. Write `internal/controller/namespaced/setup_custom.go` — mirrors cluster setup. +3. Edit `cmd/provider/main.go` to call `controllerCluster.SetupCustom` and + `controllerNamespaced.SetupCustom` alongside the existing `Setup` calls. Use the same + `customresourcesgate` / `SetupCustomGated` guard pattern already present in main.go so the + provider starts cleanly if a CRD is not yet installed. +4. Add BackupList and PublicNetwork rows to the README resource matrix. +5. Gate: `go build ./...` passes; `go vet ./...` passes; + `grep -r "BackupList\|PublicNetwork" internal/controller/cluster/setup_custom.go` non-empty; + `grep -r "BackupList\|PublicNetwork" internal/controller/namespaced/setup_custom.go` non-empty; + `make reviewable` clean. + +## Gate commands (all stories) + +```bash +go test ./internal/clients/... # DS-04 (S02) +go build ./... # DS-14 (S04) +go vet ./... # DS-15 (S04) +make reviewable # generation boundary guard (all stories) +make test # unit + contract suite (all stories) +```