From e12ad08708ce4732dd55efd2a590a10d010fc1b9 Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Sun, 16 Aug 2026 11:36:34 +0200 Subject: [PATCH 1/7] :memo: docs(changelog): regenerate CHANGELOG.md from git-cliff Pre-existing drift from already-merged docs/ADR commits (df77035 and predecessors); unrelated to this lane's EX-S05 work but required for `task check`'s changelog-verify to pass before committing. --- CHANGELOG.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 448771d..fa4b15c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -112,7 +112,7 @@ repository still gets a decision, never by following the link; no release carrie - :memo: docs(adr): add ADR-0011 Amendment 4 — Rego/OPA rule-7 capability boundary - :memo: docs(adr): add ADR-0015 host-side credential resolver amendment - :memo: docs(adr): retcon ADR narrative to read as planned phases -- :memo: docs(changelog): regenerate CHANGELOG.md for PR #59's operator-ruling commits +- :memo: docs(changelog): regenerate CHANGELOG.md from git-cliff ### Features - :sparkles: feat(docs): gate example pack and format claims against dogfood @@ -127,9 +127,6 @@ repository still gets a decision, never by following the link; no release carrie - :bug: fix(docs-gates): fail on unmapped format tokens and de-confound the tests-dir mutation (EX-S01 review) - :bug: fix(examples): make privilege-tier's proving fixture load-bearing -### Refactoring -- :recycle: refactor(examples): discover dogfood packs instead of a hardcoded 3-name loop - ### Testing - :white_check_mark: test(release): anchor the D-120 note check on its header sentence, not the bare token - :white_check_mark: test(release): key the merge-skip proof on commit shape, not subject prefix From 2fa9310ed9a48e6156cc9ca15225f0c267314c8b Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Sun, 16 Aug 2026 11:38:03 +0200 Subject: [PATCH 2/7] =?UTF-8?q?:sparkles:=20feat(examples):=20EX-S05=20HCL?= =?UTF-8?q?=20honesty=20=E2=80=94=20govern=20.tf,=20pin=20the=20measured?= =?UTF-8?q?=20opaque=20decision?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit infra-vars' class now matches envs/**/*.tf as well as *.tfvars, and a new vars/tf-opaque case (a generic resource/module block, invented names only) proves what happens: REVIEW with zero findings, matching adoptertest's undecidable-changeset guard. Measured, not assumed (REQ-EX-S05-02): the mechanism is not "parseHCL rejects the block" as the spec anticipated. internal/change/diff.go's baseProducerFor only routes .tfvars to the HCL parser; .tf falls through to the YAML producer default, which cannot parse HCL syntax at all — block or bare literal alike. A scratch-copy discriminating test (tfvars-shaped literal content under a .tf extension) confirms this: still opaque. So every .tf file is opaque -> REVIEW today regardless of content; only .tfvars gets structured diffing. Routing .tf to the HCL producer is unbuilt engine work, out of this story's internal/change scope — documented as a known limitation in config.yaml, the fixture files, expect.yaml, and examples/README.md, not glossed over. The existing S04 tfvars cases (nested-map-change et al.) still PASS with APPROVE, which is unreachable through the opaque path — positive evidence the adapter still diffs tfvars literals structurally (REQ-EX-S05-03). examples/README.md's Input formats line now claims tf; hack/docs/ example_format_inventory_test.sh's hardcoded real-tree assertion and its GOOD_README-derived polarity fixtures (cue/toml/hcl mutations, which anchored on the literal "tfvars." substring) are updated to match — REQ-EX-S05-04. Non-vacuity (REQ-EX-S05-05), verified in scratch copies: - deleting the tf-opaque case's .tf fixture content (base/head files) makes `assent test` hard-error (exit 2, "found 0" files under base/) - deleting the class's *.tf glob (leaving the explanatory comment, reworded to avoid a literal *.tf token that would have vacuously kept the comment-grep inventory check green) reddens hack/docs/example_format_inventory_test.sh --- examples/README.md | 8 ++++++- examples/packs/infra-vars/.assent/config.yaml | 12 +++++++++- .../vars/tf-opaque/base/envs/prod/backend.tf | 14 +++++++++++ .../.assent/tests/vars/tf-opaque/expect.yaml | 19 +++++++++++++++ .../.assent/tests/vars/tf-opaque/facts.yaml | 7 ++++++ .../vars/tf-opaque/head/envs/prod/backend.tf | 14 +++++++++++ hack/docs/example_format_inventory_test.sh | 23 ++++++++++--------- 7 files changed, 84 insertions(+), 13 deletions(-) create mode 100644 examples/packs/infra-vars/.assent/tests/vars/tf-opaque/base/envs/prod/backend.tf create mode 100644 examples/packs/infra-vars/.assent/tests/vars/tf-opaque/expect.yaml create mode 100644 examples/packs/infra-vars/.assent/tests/vars/tf-opaque/facts.yaml create mode 100644 examples/packs/infra-vars/.assent/tests/vars/tf-opaque/head/envs/prod/backend.tf diff --git a/examples/README.md b/examples/README.md index 6b45abf..de4f32f 100644 --- a/examples/README.md +++ b/examples/README.md @@ -8,7 +8,7 @@ don't run are lies, so the runnable ones are executed by the gates, not just rea - [`packs/`](packs/) — complete adopter policy trees (`topic-registry`, `service-catalog`, `infra-vars`). Each is a repo root: `assent lint ` is clean and `assent test ` passes, both under `task check`. Start here. - Input formats: yaml, json, tfvars. + Input formats: yaml, json, tfvars, tf. - [`policies/declarative/`](policies/declarative/) — standalone envelope rules with `assert` predicates - [`policies/rego/`](policies/rego/) — the tier-2 escape hatch for the same archetype. @@ -23,6 +23,12 @@ don't run are lies, so the runnable ones are executed by the gates, not just rea - [`render/`](render/) — committed finding fixtures for `assent render` - [`contracts/`](contracts/) — frozen contract fixtures (D-016 strict, named-consumer compat) +`.tf` is governed (never silently un-reviewed change) but does not yet structurally +diff at all — measured, not assumed: the differ only routes the `.tfvars` extension +to the HCL parser, so a `.tf` file's content, blocks or bare literals alike, is +opaque and falls back to REVIEW, never a partial parse (ADR-0003) — see the +`infra-vars` pack's `tf-opaque` case. Only `.tfvars` gets structured diffing today. + The authored surfaces here are the **frozen** `assent.dev/v1alpha1` schemas under `schemas/`, not drafts; the compatibility promises attached to them are in [`API_STABILITY.md`](../API_STABILITY.md). diff --git a/examples/packs/infra-vars/.assent/config.yaml b/examples/packs/infra-vars/.assent/config.yaml index 72a4c10..c215f40 100644 --- a/examples/packs/infra-vars/.assent/config.yaml +++ b/examples/packs/infra-vars/.assent/config.yaml @@ -15,7 +15,17 @@ environments: match: { paths: ["envs/**"] } classes: - name: infra-vars - match: { paths: ["envs/**/*.tfvars"] } + # Terraform config files under envs/ are governed too (EX-S05), so they are never + # silently un-reviewed change — but MEASURED (not assumed): the value-tree + # producer dispatch (internal/change/diff.go's baseProducerFor) only routes the + # tfvars extension to the HCL parser; every other extension, including this one, + # defaults to the YAML producer, which cannot parse HCL syntax at all (block or + # bare literal) and hits the opaque-change fallback above for a YAML-shaped + # reason, not an HCL one. Every Terraform config file is opaque -> REVIEW today, + # regardless of content — see the vars/tf-opaque case. Routing this extension to + # the HCL producer (so a literal-only Terraform config file could structurally + # diff, matching tfvars) is future engine work, not yet built. + match: { paths: ["envs/**/*.tfvars", "envs/**/*.tf"] } providers: author: type: builtin/gitlab-groups diff --git a/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/base/envs/prod/backend.tf b/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/base/envs/prod/backend.tf new file mode 100644 index 0000000..dd460f9 --- /dev/null +++ b/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/base/envs/prod/backend.tf @@ -0,0 +1,14 @@ +# HCL honesty fixture (EX-S05): a .tf resource/module BLOCK. Measured, not assumed +# (see expect.yaml): the .tf extension is not routed to the HCL parser at all +# today (only .tfvars is) — this file goes opaque via the YAML-producer default +# failing to parse HCL syntax, never a silent partial diff either way. Names are +# invented/generic (D-002): no real provider or company. +resource "example_compute_instance" "orders_api" { + instance_type = "standard-4" + replica_count = 3 +} + +module "networking" { + source = "./modules/networking" + cidr_block = "10.0.0.0/16" +} diff --git a/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/expect.yaml b/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/expect.yaml new file mode 100644 index 0000000..cef3f99 --- /dev/null +++ b/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/expect.yaml @@ -0,0 +1,19 @@ +# MEASURED (EX-S05, REQ-EX-S05-02), not assumed. The mechanism is NOT "parseHCL +# rejects the resource/module block" — internal/change/diff.go's baseProducerFor +# only routes the .tfvars extension to the HCL parser; .tf falls through to the +# YAML producer default, which cannot parse HCL syntax at all (this file's content +# fails as YAML: "document root is not a mapping"). So EVERY .tf file is opaque +# today regardless of content (verified: a literal-only, tfvars-shaped .tf body +# with no blocks is opaque too) — confirmed by a scratch-copy discriminating test, +# not by reading the code. The resulting ChangeSet is Opaque either way, so this +# case still demonstrates the honesty goal (a governed Terraform file that cannot +# be silently approved), just not via the block-shape mechanism the story +# originally assumed. +# +# The adoptertest harness's undecidable guard (Evaluate in +# internal/adoptertest/adoptertest.go) short-circuits an opaque/empty ChangeSet to +# a bare REVIEW with NO findings — it does not synthesize the aggregate.changeset +# finding the real `assent run` CLI path attaches via undecidableReview +# (cmd/assent/run.go). Confirmed byte-stable under `assent test --update`. +decision: REVIEW +findings: [] diff --git a/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/facts.yaml b/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/facts.yaml new file mode 100644 index 0000000..6016035 --- /dev/null +++ b/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/facts.yaml @@ -0,0 +1,7 @@ +author: + login: alice + groups: [orders-team] +band: + memory_mb: { min: 512, max: 4096 } + min_replicas: { min: 1, max: 8 } + max_replicas: { min: 1, max: 24 } diff --git a/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/head/envs/prod/backend.tf b/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/head/envs/prod/backend.tf new file mode 100644 index 0000000..7cca01c --- /dev/null +++ b/examples/packs/infra-vars/.assent/tests/vars/tf-opaque/head/envs/prod/backend.tf @@ -0,0 +1,14 @@ +# HCL honesty fixture (EX-S05): a .tf resource/module BLOCK. Measured, not assumed +# (see expect.yaml): the .tf extension is not routed to the HCL parser at all +# today (only .tfvars is) — this file goes opaque via the YAML-producer default +# failing to parse HCL syntax, never a silent partial diff either way. Names are +# invented/generic (D-002): no real provider or company. +resource "example_compute_instance" "orders_api" { + instance_type = "standard-8" + replica_count = 4 +} + +module "networking" { + source = "./modules/networking" + cidr_block = "10.0.0.0/16" +} diff --git a/hack/docs/example_format_inventory_test.sh b/hack/docs/example_format_inventory_test.sh index cc28d3c..2abdf77 100755 --- a/hack/docs/example_format_inventory_test.sh +++ b/hack/docs/example_format_inventory_test.sh @@ -5,9 +5,10 @@ # pack's governed format from class match.paths extensions (.yaml / .json / # .tfvars / .tf), and asserts examples/README.md names exactly those packs and # claims exactly those formats. A pack directory without .assent/tests/ is an -# incomplete tree (hard error). Claiming cue / kafka-acl / HCL before a fixture -# exists must go red; omitting a real pack from the README must go red the -# other way. +# incomplete tree (hard error). Claiming cue / kafka-acl / an unmapped token +# (e.g. "hcl", which pack_formats never emits — the real, mapped token is "tf", +# landed by EX-S05) must go red; omitting a real pack from the README must go +# red the other way. # # WIRED: `task docs-gates` runs this script (REQ-EX-S01-05). Deleting that # invocation reddens the wiring pin here and in truthlag_pins_test.sh. @@ -164,7 +165,7 @@ cat >"$GOOD_README" <<'EOF' # Examples - [`packs/`](packs/) — complete adopter policy trees (`topic-registry`, `service-catalog`, `infra-vars`). -Input formats: yaml, json, tfvars. +Input formats: yaml, json, tfvars, tf. EOF if inventory_ok "$ROOT" "$GOOD_README" >"$WORK/good.out" 2>"$WORK/good.err"; then @@ -190,7 +191,7 @@ else fi CUE="$WORK/readme.cue.md" -sed 's/tfvars\./tfvars, cue./' "$GOOD_README" >"$CUE" +sed 's/tf\./tf, cue./' "$GOOD_README" >"$CUE" if inventory_ok "$ROOT" "$CUE" >"$WORK/cue.out" 2>"$WORK/cue.err"; then fail "claiming cue stayed green (REQ-EX-S01-04 vacuous)" else @@ -198,7 +199,7 @@ else fi TOML="$WORK/readme.toml.md" -sed 's/tfvars\./tfvars, toml./' "$GOOD_README" >"$TOML" +sed 's/tf\./tf, toml./' "$GOOD_README" >"$TOML" if inventory_ok "$ROOT" "$TOML" >"$WORK/toml.out" 2>"$WORK/toml.err"; then fail "claiming toml stayed green (unrecognised token silently dropped — REQ-EX-S01-04 fail-open)" else @@ -206,11 +207,11 @@ else fi HCL="$WORK/readme.hcl.md" -sed 's/tfvars\./tfvars, hcl./' "$GOOD_README" >"$HCL" +sed 's/tf\./tf, hcl./' "$GOOD_README" >"$HCL" if inventory_ok "$ROOT" "$HCL" >"$WORK/hcl.out" 2>"$WORK/hcl.err"; then - fail "claiming hcl before a .tf fixture stayed green" + fail "claiming an unrecognised hcl token stayed green" else - pass "claiming hcl / .tf before S05 reddens" + pass "REQ-EX-S01-04: claiming an unmapped hcl token reddens (S05 landed: tf is now a real, mapped token)" fi # Incomplete tree: a fourth pack dir with .assent/ but no tests. The README @@ -243,8 +244,8 @@ if inventory_ok "$ROOT" "$ROOT/examples/README.md" >"$WORK/real.out" 2>"$WORK/re report="$(tr '\n' ' ' <"$WORK/real.out")" echo "$report" if echo "$report" | grep -q 'PACKS: infra-vars service-catalog topic-registry' \ - && echo "$report" | grep -q 'FORMATS: json tfvars yaml'; then - pass "REQ-EX-S01-01: real tree reports the three packs and yaml/json/tfvars" + && echo "$report" | grep -q 'FORMATS: json tf tfvars yaml'; then + pass "REQ-EX-S01-01: real tree reports the three packs and yaml/json/tf/tfvars" else fail "REQ-EX-S01-01: unexpected report: $report" fi From 14167b85c561760fde98988fcaaadeb46747a813 Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Sun, 16 Aug 2026 11:52:06 +0200 Subject: [PATCH 3/7] :white_check_mark: test(examples): close REQ-EX-S05-05 non-vacuity gap for infra-vars .tf governance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Independent review (F1, P1) found that `assent test`/`--coverage` cannot distinguish "the vars/tf-opaque .tf fixture is governed by the infra-vars class" from "it isn't governed at all": deleting envs/**/*.tf from config.yaml's match.paths, while leaving the fixture in place, produces byte-identical PASS vars/tf-opaque (REVIEW), exit 0. Root cause traced past adoptertest's undecidable-guard collapse (opaque and empty ChangeSets both fall through to a bare REVIEW/no-findings result) to a deeper, deliberate architectural fact: internal/catalogue's Input type omits Config entirely (D-017 B10 — "a rule's classes come from the binding graph ... so Config is deliberately absent"), and no other engine code path (selectBindingForTest, selectBinding, classify.go) reads Config.Classes either. `assent test` was never wired to consult class match.paths at all, for any pack — this is not specific to infra-vars or to .tf. Given that, making `assent test` itself discriminate would mean adding class-routing to the test/catalogue dispatch, a cross-pack engine change well outside this story's examples-only scope. Instead: a companion test (TestInfraVarsTFFixtureIsGovernedByItsClass) asserts, against the real config.yaml and the SAME internal/glob matcher the routing classifier and coverage loop use, that the tf-opaque fixture's path is actually covered by the infra-vars class's match.paths — and a mutation-control test proves that assertion is capable of failing. Verified empirically: deleting the .tf glob from config.yaml reddens this new test (and the existing inventory script); restoring it goes green again; `assent test`/`--coverage` remain unchanged either way, exactly as diagnosed. --- .../examples_infravars_tf_governance_test.go | 90 +++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 cmd/assent/examples_infravars_tf_governance_test.go diff --git a/cmd/assent/examples_infravars_tf_governance_test.go b/cmd/assent/examples_infravars_tf_governance_test.go new file mode 100644 index 0000000..21c7b5d --- /dev/null +++ b/cmd/assent/examples_infravars_tf_governance_test.go @@ -0,0 +1,90 @@ +package main + +import ( + "os" + "testing" + + "github.com/PlatformRelay/assent/internal/core/policy" + "github.com/PlatformRelay/assent/internal/glob" +) + +// examples_infravars_tf_governance_test.go closes the REQ-EX-S05-05 non-vacuity +// gap an independent review found in lane/ex-s05: `assent test`/`--coverage` +// never consult Config.Classes at all (catalogue.Input deliberately omits Config +// per D-017 B10 — "a rule's classes come from the binding graph (binding.class), +// which carries the class NAME directly"; see internal/catalogue/catalogue.go's +// Input doc comment). `assent test`'s selectBindingForTest picks ONE binding for +// the whole pack regardless of the case's File, so deleting `envs/**/*.tf` from +// infra-vars' class match.paths while leaving the vars/tf-opaque fixture in +// place produces byte-identical `assent test`/`--coverage` output (PASS +// vars/tf-opaque (REVIEW), exit 0) — the opaque-changeset collapse in +// adoptertest.Evaluate's undecidable guard cannot distinguish "governed change +// hit the opaque fallback" from "nothing here was governed at all", and no +// other engine code path reads Config.Classes to tell them apart either. +// +// This test proves governance directly against the SAME matcher the engine's +// routing/coverage code uses (internal/glob.Match — shared by +// internal/core/classify and internal/core/aggregate, imported here as a +// read-only consumer, not reimplemented), over the REAL config.yaml on disk — +// so it reddens the instant `envs/**/*.tf` is removed from the infra-vars +// class, exactly the mutation `assent test` itself cannot see. +func TestInfraVarsTFFixtureIsGovernedByItsClass(t *testing.T) { + const ( + configPath = "../../examples/packs/infra-vars/.assent/config.yaml" + className = "infra-vars" + // The vars/tf-opaque case's changed-file path (repo-relative, matching + // how change.Diff and the real matcher both see it). + tfFixturePath = "envs/prod/backend.tf" + ) + + raw, err := os.ReadFile(configPath) //nolint:gosec // fixed in-repo path relative to cmd/assent. + if err != nil { + t.Fatalf("read %s: %v", configPath, err) + } + cfg, err := policy.LoadConfig(raw) + if err != nil { + t.Fatalf("LoadConfig %s: %v", configPath, err) + } + + var class *policy.NamedMatch + for i := range cfg.Classes { + if cfg.Classes[i].Name == className { + class = &cfg.Classes[i] + break + } + } + if class == nil { + t.Fatalf("config.yaml declares no class named %q", className) + } + + matched := false + for _, pattern := range class.Match.Paths { + if glob.Match(pattern, tfFixturePath) { + matched = true + break + } + } + if !matched { + t.Fatalf("class %q's match.paths %v does not cover %q — the vars/tf-opaque"+ + " fixture would be ungoverned even though assent test/--coverage cannot"+ + " see that (REQ-EX-S05-05); restore the envs/**/*.tf glob", className, class.Match.Paths, tfFixturePath) + } +} + +// TestInfraVarsTFGovernanceAssertionCanFail is the mutation control (matching +// this repo's house style, e.g. hack/lint/depguard_test.sh): it proves the +// assertion above is not vacuously true by re-running the exact same glob.Match +// check against an in-memory class whose match.paths never mention .tf — the +// same shape the tf-opaque case would be left in if `envs/**/*.tf` were deleted +// from the real config.yaml. It must fail to match. +func TestInfraVarsTFGovernanceAssertionCanFail(t *testing.T) { + withoutTF := policy.NamedMatch{ + Name: "infra-vars", + Match: policy.PathMatch{Paths: []string{"envs/**/*.tfvars"}}, + } + if glob.Match(withoutTF.Match.Paths[0], "envs/prod/backend.tf") { + t.Fatalf("mutation control did not redden: %q unexpectedly matched %q — the"+ + " assertion in TestInfraVarsTFFixtureIsGovernedByItsClass would be vacuous", + withoutTF.Match.Paths[0], "envs/prod/backend.tf") + } +} From 7e3d00c595febe88082136697133b28bd2c2265c Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Sun, 16 Aug 2026 11:52:17 +0200 Subject: [PATCH 4/7] :memo: docs(changelog): regenerate CHANGELOG.md for the EX-S05 lane Picks up the feat(examples) and test(examples) subjects from this lane's two prior commits. --- CHANGELOG.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fa4b15c..1f61023 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -113,23 +113,24 @@ repository still gets a decision, never by following the link; no release carrie - :memo: docs(adr): add ADR-0015 host-side credential resolver amendment - :memo: docs(adr): retcon ADR narrative to read as planned phases - :memo: docs(changelog): regenerate CHANGELOG.md from git-cliff +- :memo: docs(changelog): regenerate CHANGELOG.md for the EX-S05 lane ### Features - :sparkles: feat(docs): gate example pack and format claims against dogfood - :sparkles: feat(examples): thicken service-catalog with nested sla/runtime objects - :sparkles: feat(examples): thicken topic-registry nested YAML and nested-pointer rules - :sparkles: feat(examples): thicken infra-vars nested tfvars maps (EX-S04) -- :sparkles: feat(examples): close REF-EX C1-C4 in topic-registry/service-catalog +- :sparkles: feat(examples): EX-S05 HCL honesty — govern .tf, pin the measured opaque decision ### Fixes - :bug: fix(ci): pin ci-audit-test in the AUD-S18 check-stage list - :bug: specs(p5-ex): make S10 schema freeze and S07 fence non-vacuous - :bug: fix(docs-gates): fail on unmapped format tokens and de-confound the tests-dir mutation (EX-S01 review) -- :bug: fix(examples): make privilege-tier's proving fixture load-bearing ### Testing - :white_check_mark: test(release): anchor the D-120 note check on its header sentence, not the bare token - :white_check_mark: test(release): key the merge-skip proof on commit shape, not subject prefix +- :white_check_mark: test(examples): close REQ-EX-S05-05 non-vacuity gap for infra-vars .tf governance ## [0.2.0] - 2026-08-09 ### Chores From 20b24cb395bb04469f132c0702d3489428d4a546 Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Sun, 16 Aug 2026 12:07:43 +0200 Subject: [PATCH 5/7] :white_check_mark: test(examples): close REQ-EX-S05-05's other disjunct (fixture deletion) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-2 independent review (F1, P1): the round-1 fix closed only the class-path-extension-deleted mutation. The OTHER disjunct REQ-EX-S05-05 names — deleting the vars/tf-opaque case's base/+head/ .tf fixture files outright, leaving config.yaml's *.tf glob intact — stayed invisible: `assent test --coverage` doesn't notice a case that silently vanished from discovery (--coverage is rule-polarity-level, not case-count-level), the inventory script derives FORMATS from config.yaml's declared glob extensions rather than checking any file exists, and the round-1 glob-match test only checked a hardcoded path STRING against the class pattern — it never touched the filesystem, so it can't tell the fixture is gone. TestInfraVarsTFFixtureFilesExist os.Stats the real base/ and head/ .tf files directly (plus a not-a-directory and not-empty check, the latter guarding against F2's "emptied to zero bytes" variant from round 1), with a mutation control proving the assertion can fail. Verified empirically, both disjuncts: - fixture directory deleted (reproducing the reviewer's exact repro) -> `assent test --coverage` on a scratch copy: exit 0, fully green, case silently absent (confirms the finding); the same mutation on the real tree -> TestInfraVarsTFFixtureFilesExist FAILs; restored -> green again. - class-path *.tf glob deleted (round-1's mutation, re-checked for regression) -> TestInfraVarsTFFixtureIsGovernedByItsClass and the inventory script both still FAIL as before; restored -> both green. F2 (P2): amends REQ-EX-S05-05's Test:/Verify: fields in openspec/specs/p5-ex-complex-examples/spec.md to name the actual three-part non-vacuity proof (inventory + this Go test, not --coverage alone) and record why --coverage can't do it (D-017 B10 — catalogue.Input omits Config, so no assent test/--coverage code path ever consults Config.Classes/match.paths). --- CHANGELOG.md | 1 + .../examples_infravars_tf_governance_test.go | 78 +++++++++++++++++-- openspec/specs/p5-ex-complex-examples/spec.md | 17 +++- 3 files changed, 87 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1f61023..116e6eb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -131,6 +131,7 @@ repository still gets a decision, never by following the link; no release carrie - :white_check_mark: test(release): anchor the D-120 note check on its header sentence, not the bare token - :white_check_mark: test(release): key the merge-skip proof on commit shape, not subject prefix - :white_check_mark: test(examples): close REQ-EX-S05-05 non-vacuity gap for infra-vars .tf governance +- :white_check_mark: test(examples): close REQ-EX-S05-05's other disjunct (fixture deletion) ## [0.2.0] - 2026-08-09 ### Chores diff --git a/cmd/assent/examples_infravars_tf_governance_test.go b/cmd/assent/examples_infravars_tf_governance_test.go index 21c7b5d..05f2c16 100644 --- a/cmd/assent/examples_infravars_tf_governance_test.go +++ b/cmd/assent/examples_infravars_tf_governance_test.go @@ -9,7 +9,7 @@ import ( ) // examples_infravars_tf_governance_test.go closes the REQ-EX-S05-05 non-vacuity -// gap an independent review found in lane/ex-s05: `assent test`/`--coverage` +// gap independent review found in lane/ex-s05: `assent test`/`--coverage` // never consult Config.Classes at all (catalogue.Input deliberately omits Config // per D-017 B10 — "a rule's classes come from the binding graph (binding.class), // which carries the class NAME directly"; see internal/catalogue/catalogue.go's @@ -22,12 +22,24 @@ import ( // hit the opaque fallback" from "nothing here was governed at all", and no // other engine code path reads Config.Classes to tell them apart either. // -// This test proves governance directly against the SAME matcher the engine's -// routing/coverage code uses (internal/glob.Match — shared by -// internal/core/classify and internal/core/aggregate, imported here as a -// read-only consumer, not reimplemented), over the REAL config.yaml on disk — -// so it reddens the instant `envs/**/*.tf` is removed from the infra-vars -// class, exactly the mutation `assent test` itself cannot see. +// REQ-EX-S05-05 names TWO mutations that must redden ("the .tf fixture OR +// class-path extension is deleted"). Round 1 closed only the class-path-glob +// disjunct (TestInfraVarsTFFixtureIsGovernedByItsClass, below). Round 2 (this +// addition, TestInfraVarsTFFixtureFilesExist) closes the other one: deleting +// the tf-opaque case's base/+head/ .tf files outright (leaving config.yaml's +// glob intact) is ALSO invisible to `--coverage` (rule-level, not case-count +// aware — a vanished no-findings case is simply absent from the run) and to +// hack/docs/example_format_inventory_test.sh (derives FORMATS from config.yaml's +// declared glob extensions, never checks a file exists on disk). The first +// round's glob-match assertion doesn't catch this either — it only checks a +// hardcoded path STRING against the class pattern, never os.Stats the fixture. +// +// Both tests prove governance/existence directly against the real filesystem +// and the SAME matcher the engine's routing/coverage code uses (internal/glob.Match +// — shared by internal/core/classify and internal/core/aggregate, imported here +// as a read-only consumer, not reimplemented) — so together they redden on +// EITHER of REQ-EX-S05-05's named mutations, exactly what `assent test`/ +// `--coverage`/inventory cannot see. func TestInfraVarsTFFixtureIsGovernedByItsClass(t *testing.T) { const ( configPath = "../../examples/packs/infra-vars/.assent/config.yaml" @@ -88,3 +100,55 @@ func TestInfraVarsTFGovernanceAssertionCanFail(t *testing.T) { withoutTF.Match.Paths[0], "envs/prod/backend.tf") } } + +// tfOpaqueCaseFiles are the vars/tf-opaque case's base/head fixture files +// (repo-relative to cmd/assent), the two files REQ-EX-S05-05's "the .tf +// fixture ... is deleted" disjunct is about. +var tfOpaqueCaseFiles = []string{ + "../../examples/packs/infra-vars/.assent/tests/vars/tf-opaque/base/envs/prod/backend.tf", + "../../examples/packs/infra-vars/.assent/tests/vars/tf-opaque/head/envs/prod/backend.tf", +} + +// TestInfraVarsTFFixtureFilesExist closes REQ-EX-S05-05's OTHER disjunct +// (round 2 of independent review): deleting the tf-opaque case's base/+head/ +// .tf files outright, while leaving config.yaml's *.tf glob intact, is +// invisible to `assent test --coverage` (the case simply vanishes from the +// run — --coverage counts rule polarity, not case count) and to +// hack/docs/example_format_inventory_test.sh (FORMATS comes from config.yaml's +// declared extensions, never from checking a file exists). Neither this file's +// class-match test proves the fixture is still there: it only checks a +// hardcoded path string against a glob pattern, never touching the filesystem. +// This test os.Stats the real files directly, so it reddens the instant either +// one is deleted or replaced by a directory/non-regular file. +func TestInfraVarsTFFixtureFilesExist(t *testing.T) { + for _, p := range tfOpaqueCaseFiles { + info, err := os.Stat(p) + if err != nil { + t.Fatalf("vars/tf-opaque fixture file missing: %s: %v — REQ-EX-S05-05's"+ + " honesty case can be silently deleted with config.yaml's *.tf glob"+ + " left intact, and assent test/--coverage/inventory would not notice", p, err) + } + if info.IsDir() { + t.Fatalf("vars/tf-opaque fixture path is a directory, not a file: %s", p) + } + if info.Size() == 0 { + t.Fatalf("vars/tf-opaque fixture file is empty: %s — an empty base/head pair"+ + " also collapses to the same undecidable REVIEW/no-findings result as a"+ + " real resource/module block (F2, round-1 review), so a zeroed-out"+ + " fixture would silently stop proving anything", p) + } + } +} + +// TestInfraVarsTFFixtureFilesExistAssertionCanFail is the mutation control for +// the test above: os.Stat over a path that does not exist must error, proving +// the assertion is capable of failing (not vacuously true because os.Stat +// never errors in this environment). +func TestInfraVarsTFFixtureFilesExistAssertionCanFail(t *testing.T) { + missing := "../../examples/packs/infra-vars/.assent/tests/vars/tf-opaque/base/envs/prod/does-not-exist.tf" + if _, err := os.Stat(missing); err == nil { + t.Fatalf("mutation control did not redden: os.Stat unexpectedly succeeded for"+ + " a path that should not exist: %s — TestInfraVarsTFFixtureFilesExist would"+ + " be vacuous", missing) + } +} diff --git a/openspec/specs/p5-ex-complex-examples/spec.md b/openspec/specs/p5-ex-complex-examples/spec.md index 8a556ee..7c53978 100644 --- a/openspec/specs/p5-ex-complex-examples/spec.md +++ b/openspec/specs/p5-ex-complex-examples/spec.md @@ -454,8 +454,21 @@ Requirements: - Level: L1 - **REQ-EX-S05-05** — Given the `.tf` fixture or class-path extension is deleted, when `--coverage` or inventory runs, then the gate is red (non-vacuity of the honesty case). - - Test: pack `--coverage` + inventory - - Verify: `./bin/assent test --coverage examples/packs/infra-vars` + Amendment (round-2 review): `--coverage` is architecturally blind to both mutations — + `catalogue.Input` deliberately omits `Config` (D-017 B10), so no `assent test`/ + `--coverage` code path ever consults `Config.Classes`/`match.paths`, and `--coverage` + itself is rule-polarity-level, not case-count-level, so a deleted case simply vanishes + from the run without reddening anything. Non-vacuity is proven by inventory (the + class-path-extension disjunct: `hack/docs/example_format_inventory_test.sh` derives + its `FORMATS` list from `config.yaml`'s declared glob extensions) plus a dedicated Go + test (the fixture-deleted disjunct: `cmd/assent/examples_infravars_tf_governance_test.go`'s + `TestInfraVarsTFFixtureIsGovernedByItsClass` — `os.Stat`s the fixture files and + `glob.Match`s the class's declared paths against the fixture's path, the same matcher + `internal/core/classify`/`internal/core/aggregate` use) — never by `--coverage` alone. + - Test: pack `--coverage` + inventory + `go test ./cmd/assent/... -run TestInfraVarsTF` + - Verify: `./bin/assent test --coverage examples/packs/infra-vars`; + `bash hack/docs/example_format_inventory_test.sh`; + `go test ./cmd/assent/... -run TestInfraVarsTF -v` - Level: L1 --- From fca566a92f35fc437c71af673adeff5891c29732 Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Sun, 16 Aug 2026 12:09:54 +0200 Subject: [PATCH 6/7] :memo: docs(openspec): correct test attribution in the REQ-EX-S05-05 amendment The previous amendment attributed BOTH the glob.Match class-path check and the os.Stat fixture-existence check to TestInfraVarsTFFixtureIsGovernedByItsClass (the round-1 test name). That test only does the glob.Match half; the os.Stat half is TestInfraVarsTFFixtureFilesExist (round 2). Names each test against its actual disjunct instead of collapsing both into one. --- CHANGELOG.md | 1 + openspec/specs/p5-ex-complex-examples/spec.md | 10 ++++++---- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 116e6eb..be02cfe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -114,6 +114,7 @@ repository still gets a decision, never by following the link; no release carrie - :memo: docs(adr): retcon ADR narrative to read as planned phases - :memo: docs(changelog): regenerate CHANGELOG.md from git-cliff - :memo: docs(changelog): regenerate CHANGELOG.md for the EX-S05 lane +- :memo: docs(openspec): correct test attribution in the REQ-EX-S05-05 amendment ### Features - :sparkles: feat(docs): gate example pack and format claims against dogfood diff --git a/openspec/specs/p5-ex-complex-examples/spec.md b/openspec/specs/p5-ex-complex-examples/spec.md index 7c53978..53d9dc6 100644 --- a/openspec/specs/p5-ex-complex-examples/spec.md +++ b/openspec/specs/p5-ex-complex-examples/spec.md @@ -460,11 +460,13 @@ Requirements: itself is rule-polarity-level, not case-count-level, so a deleted case simply vanishes from the run without reddening anything. Non-vacuity is proven by inventory (the class-path-extension disjunct: `hack/docs/example_format_inventory_test.sh` derives - its `FORMATS` list from `config.yaml`'s declared glob extensions) plus a dedicated Go - test (the fixture-deleted disjunct: `cmd/assent/examples_infravars_tf_governance_test.go`'s - `TestInfraVarsTFFixtureIsGovernedByItsClass` — `os.Stat`s the fixture files and + its `FORMATS` list from `config.yaml`'s declared glob extensions) plus two dedicated + Go tests in `cmd/assent/examples_infravars_tf_governance_test.go` — one per disjunct: + `TestInfraVarsTFFixtureIsGovernedByItsClass` (class-path-extension disjunct) `glob.Match`s the class's declared paths against the fixture's path, the same matcher - `internal/core/classify`/`internal/core/aggregate` use) — never by `--coverage` alone. + `internal/core/classify`/`internal/core/aggregate` use; `TestInfraVarsTFFixtureFilesExist` + (fixture-deleted disjunct) `os.Stat`s the fixture's base/head files directly — never by + `--coverage` alone. - Test: pack `--coverage` + inventory + `go test ./cmd/assent/... -run TestInfraVarsTF` - Verify: `./bin/assent test --coverage examples/packs/infra-vars`; `bash hack/docs/example_format_inventory_test.sh`; From 8d86a9738aefabc9541b665349d8c7b9dbc308ac Mon Sep 17 00:00:00 2001 From: Konrad Heimel Date: Sun, 16 Aug 2026 12:19:48 +0200 Subject: [PATCH 7/7] :memo: docs(changelog): regenerate CHANGELOG.md after rebase onto origin/main --- CHANGELOG.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index be02cfe..f96ebd7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -112,21 +112,28 @@ repository still gets a decision, never by following the link; no release carrie - :memo: docs(adr): add ADR-0011 Amendment 4 — Rego/OPA rule-7 capability boundary - :memo: docs(adr): add ADR-0015 host-side credential resolver amendment - :memo: docs(adr): retcon ADR narrative to read as planned phases +- :memo: docs(changelog): regenerate CHANGELOG.md for PR #59's operator-ruling commits - :memo: docs(changelog): regenerate CHANGELOG.md from git-cliff - :memo: docs(changelog): regenerate CHANGELOG.md for the EX-S05 lane - :memo: docs(openspec): correct test attribution in the REQ-EX-S05-05 amendment +- :memo: docs(changelog): regenerate CHANGELOG.md after rebase onto origin/main ### Features - :sparkles: feat(docs): gate example pack and format claims against dogfood - :sparkles: feat(examples): thicken service-catalog with nested sla/runtime objects - :sparkles: feat(examples): thicken topic-registry nested YAML and nested-pointer rules - :sparkles: feat(examples): thicken infra-vars nested tfvars maps (EX-S04) +- :sparkles: feat(examples): close REF-EX C1-C4 in topic-registry/service-catalog - :sparkles: feat(examples): EX-S05 HCL honesty — govern .tf, pin the measured opaque decision ### Fixes - :bug: fix(ci): pin ci-audit-test in the AUD-S18 check-stage list - :bug: specs(p5-ex): make S10 schema freeze and S07 fence non-vacuous - :bug: fix(docs-gates): fail on unmapped format tokens and de-confound the tests-dir mutation (EX-S01 review) +- :bug: fix(examples): make privilege-tier's proving fixture load-bearing + +### Refactoring +- :recycle: refactor(examples): discover dogfood packs instead of a hardcoded 3-name loop ### Testing - :white_check_mark: test(release): anchor the D-120 note check on its header sentence, not the bare token