Add product validation control plane

CHANGELOG.md

@@ -4,6 +4,30 @@ All notable changes to Tabellio are recorded here.
 
 ## Unreleased
 
+## 0.4.0 - 2026-07-16
+
+### Added
+
+- Product-validity manifests that bind acceptance outcomes, invariants, forbidden outcomes, risk, and required validator types to an exact candidate commit.
+- Typed schema, semantic, workflow, visual, operational, and security evidence with bounded metrics, artifact digests, and cost telemetry.
+- Fail-closed `passed`, `failed`, and `blocked` decisions with threshold and cost-budget enforcement.
+- `tabellio-validate gate` for CI enforcement without external JSON parsing.
+- `tabellio-validator` for repository-declared shell-free adapter commands, metric extraction, and common evidence emission.
+
+### Changed
+
+- Product-changing work now requires evidence beyond command exit status when the manifest declares typed validators.
+- Missing required evidence, missing metrics, command errors, timeouts, and unknown required cost telemetry block readiness.
+- Existing v0.1 manifests and v0.1/v0.2 result readers remain supported during migration.
+
+### Release Gates
+
+- `tabellio-preflight --profile release`
+- `npm run check`
+- Fallow whole-repository and changed-code scans
+- `npm pack --dry-run --json`
+- Exact merged-head Tabellio validation
+
 ## 0.3.0 - 2026-07-15
 
 ### Added

README.md

@@ -24,7 +24,7 @@ Tabellio attaches a structured evidence packet to a pull request. The packet is
 | Git | Repository, base branch, head branch, commit SHA, and PR metadata |
 | Runtime | Human, CI, or agent runtime that produced the change |
 | Diff | Changed files |
-| Validation | Commands run and check results |
+| Validation | Exact-head commands plus structural, semantic, workflow, visual, operational, and security results |
 | Approvals | Required, granted, denied, or skipped approvals |
 | Side effects | Deployment, migration, infra, billing, secret, provider, and destructive-action policy |
 | Artifacts | Evidence files generated by the run |
@@ -58,7 +58,7 @@ AI-assisted pull requests should not depend on reviewer trust alone. Tabellio gi
 | Layer | Tooling | Role |
 | --- | --- | --- |
 | Runtime | [Node.js 20+](https://nodejs.org/) | Runs the local writer and validators |
-| Validation | `tabellio-validate` on any trusted worker | Runs an exact committed command manifest and stores results on a Git ref |
+| Validation | `tabellio-validate` on any trusted worker | Runs exact committed command or typed product validators and stores results on a Git ref |
 | Evidence contract | [JSON Schema](https://json-schema.org/) | Validates the evidence envelope and external-action policy |
 | Code storage | [GitHub](https://github.com/) | Stores code refs and tags; provides a thin pull-request shell |
 | Stacked review | [git-spice](https://abhinav.github.io/git-spice/) | Stack engine for small dependent GitHub pull requests |
@@ -103,6 +103,12 @@ npm run tabellio:platform:check
 node scripts/tabellio-validate.mjs run --repo . --commit HEAD --manifest tabellio.validation.json
 ```
 
+Use `gate` in CI. It persists the same exact-head result but exits non-zero unless the final decision is `passed`:
+
+```bash
+tabellio-validate gate --repo . --repo-id owner/repository --base main --commit HEAD --manifest tabellio.validation.json
+```
+
 Keep `origin` limited to ordinary code branches and tags. Configure a separate private GitHub repository under another remote name before publishing control refs. The transport refuses to target `origin`.
 
 Before agent or release work, run:
@@ -162,6 +168,7 @@ node scripts/tabellio-run.mjs promote --run-id run-42 --repo .
 ```
 
 See [Agent run lifecycle](docs/agent-run-lifecycle.md) for state and failure behavior.
+See [Product validation](docs/product-validation.md) for acceptance contracts, typed evidence, cost gates, and repository adapters.
 
 Package scripts:
 

docs/codex-review.md

@@ -36,3 +36,5 @@ Codex review is not:
 - deployment approval
 - substitute for human ownership
 - proof that agent output is secure
+
+When the repository commits a product-validation manifest, review begins only after the exact pull-request head has a `passed` v0.3 decision. A `blocked` decision means the evidence is incomplete or untrustworthy; it is not a product pass and should not be converted into reviewer guesswork.

docs/operations-hardening.md

@@ -11,9 +11,10 @@ Tabellio keeps code and control state in standard Git objects, but production sa
 
 ## Bounded Work
 
-- Validation manifests allow at most 50 commands, 100 arguments per command, and one-hour command timeouts.
+- Validation manifests allow at most 50 commands or validators, 100 arguments per entry, and one-hour command timeouts.
 - Timed-out commands receive `SIGTERM`, then `SIGKILL` after one second. Fail-fast suites mark remaining commands skipped.
 - Full stdout and stderr are hashed; only the newest 16 KiB of each stream is retained.
+- Typed evidence files are limited to 1 MiB, 100 metrics, and 50 immutable artifact references. Required missing cost telemetry blocks validation instead of being treated as zero spend.
 - Agent reviews allow at most 1,000 findings. Review cycles bound feedback, fixes, check statuses, titles, bodies, summaries, event details, and retained event history.
 - Remote control-ref reads and atomic pushes use a 15-minute timeout. Local atomic ref updates use a 30-second timeout.
 

docs/product-validation.md

@@ -0,0 +1,84 @@
+# Product Validation
+
+Tabellio product validation answers a stronger question than “did tests exit zero?”: did the exact committed candidate satisfy its declared user outcomes while staying inside semantic, visual, operational, cost, and safety boundaries?
+
+## Contracts
+
+`tabellio-validation/v0.2` binds three layers in one committed manifest:
+
+1. `acceptance` records the source request, risk, required outcomes, invariants, forbidden outcomes, and required validator types.
+2. `validators` run shell-free argument arrays in the exact-head detached worktree.
+3. `policy` applies metric thresholds and cost requirements to a bounded evidence report.
+
+Validator types are `static`, `schema`, `semantic`, `workflow`, `visual`, `operational`, and `security`. Static validators can use command exit status alone. Every other type must emit `tabellio-validator-evidence/v0.1` at its declared relative path.
+
+## Evidence
+
+An evidence report contains:
+
+- `status`: the adapter’s passed, failed, or blocked observation;
+- numeric metrics with explicit units;
+- cost telemetry state, USD cost, model calls, and tool calls;
+- immutable artifact metadata for screenshots, traces, reports, or source snapshots.
+
+The runner validates the evidence shape and validator identity, embeds the bounded report in the durable result, and evaluates policy itself. Adapter-reported `passed` cannot override a failed metric threshold or exceeded cost cap.
+
+## Decision Semantics
+
+| Decision | Meaning |
+| --- | --- |
+| `passed` | Every required validator completed with trustworthy evidence inside all thresholds. |
+| `failed` | Trustworthy evidence proves a required product or budget threshold was missed. |
+| `blocked` | Required proof is missing or untrustworthy because of invalid evidence, missing metrics or cost telemetry, command errors, or timeouts. |
+
+Fail closed. Never translate `blocked` into `passed`, and never treat missing cost telemetry as zero.
+
+## Repository Adapters
+
+Keep product knowledge in each repository. The adapter runs the existing eval, browser, design, cost, or security harness and writes the common evidence contract.
+
+Use `tabellio-validator` for simple adapters. Commit `.tabellio/validators.json` with `tabellio-adapter/v0.1` profiles containing shell-free command arrays, evidence metrics, cost telemetry, and a concise summary. Metrics may map command pass/fail to numeric values or extract a numeric capture group from bounded command output.
+
+- AI systems: stable golden cases in pull requests; larger live suites on schedules or release gates.
+- Web applications: critical journeys, desktop/mobile screenshot comparisons, loaded-font assertions, overflow, and accessibility metrics.
+- Operational systems: projected and observed provider usage, worker counts, polling frequency, rate limits, latency, and spend.
+- Structured outputs: strict schema validation plus semantic oracles that catch plausible but wrong values.
+
+Use fixtures and isolated namespaces in pull requests. Live external mutation remains separately approval-gated.
+
+## Review Integration
+
+Review sync reads only the newest validation result stored under the exact pull-request head. `passed` becomes a successful local check; `failed` or `blocked` prevents merge readiness. A new head requires new validation.
+
+Run contract checks with:
+
+```bash
+node scripts/check-tabellio-validation.mjs --manifest examples/tabellio-validation/product-manifest.json
+node scripts/check-tabellio-validation.mjs --evidence examples/tabellio-validation/minimal-validator-evidence.json
+```
+
+Run the exact-head suite with:
+
+```bash
+node scripts/tabellio-validate.mjs run \
+  --repo . \
+  --repo-id example/project \
+  --base main \
+  --commit HEAD \
+  --manifest tabellio.validation.json
+```
+
+Use the fail-closed command in CI:
+
+```bash
+tabellio-validate gate \
+  --repo . \
+  --repo-id example/project \
+  --base main \
+  --commit HEAD \
+  --manifest tabellio.validation.json
+```
+
+`gate` still records the result. It exits non-zero when the decision is `failed` or `blocked`.
+
+Artifact upload and path-based automatic validator selection remain scheduler or repository-policy responsibilities. The first contract keeps evaluation portable without creating a second CI or deployment authority.

docs/releases/v0.4.0.md

@@ -0,0 +1,12 @@
+Tabellio v0.4.0 adds product-validity and cost-aware validation.
+
+## Highlights
+
+- Acceptance contracts declare user outcomes, invariants, forbidden outcomes, risk, and required evidence types.
+- Typed validators cover schema, semantics, workflows, visuals, operations, cost, and security.
+- Evidence carries numeric metrics, explicit units, cost telemetry, tool and model call counts, and immutable artifact metadata.
+- Policy evaluation belongs to the runner. Adapter-reported success cannot override a missed threshold or cost cap.
+- Missing or untrustworthy required proof is `blocked`, never an implied pass.
+- `tabellio-validate gate` records exact-head results and exits non-zero unless the decision is `passed`.
+
+Existing v0.1 validation manifests remain readable. Live provider calls, deployments, infrastructure changes, and publication remain separately approval-gated.

docs/validation-runner.md

@@ -24,6 +24,12 @@ Tabellio validation replaces the GitHub Actions runtime dependency with a small
 
 Commands are executable-plus-argument arrays. Tabellio never invokes a shell, interpolates command strings, or reads a newer working-tree manifest while validating an older commit.
 
+Manifest v0.1 remains supported. Product-facing repositories can use `tabellio-validation/v0.2`, which adds an immutable acceptance contract and typed `static`, `schema`, `semantic`, `workflow`, `visual`, `operational`, and `security` validators. Every required validator type must have a required validator. Non-static validators must write a bounded `tabellio-validator-evidence/v0.1` report.
+
+Validator policy can require numeric metric thresholds, cost telemetry, and a maximum validation cost. Missing or invalid evidence, missing required metrics, unavailable required cost telemetry, command errors, and timeouts produce `blocked`. Valid evidence that misses a product threshold or cost budget produces `failed`. Only complete required evidence inside every boundary produces `passed`.
+
+See [Product validation](product-validation.md) and [the example manifest](../examples/tabellio-validation/product-manifest.json).
+
 ## Run
 
 ```bash
@@ -47,7 +53,7 @@ The runner:
 7. Removes the worktree even after failure.
 8. Writes an integrity-protected result to `refs/tabellio/validations` with compare-and-swap retries.
 
-New results use `tabellio-validation-result/v0.2`. They require `checkpointRevision` so checkpoint proof remains bound to the pull-request head when the validated revision is a later squash-merge commit. Runtime readers continue to accept legacy v0.1 results.
+Command-manifest results use `tabellio-validation-result/v0.2`. Product-validation results use v0.3 and embed the acceptance digest, typed validator results, bounded evidence reports, total observed validation cost, and final policy decision. Both require `checkpointRevision` so checkpoint proof remains bound to the pull-request head when the validated revision is a later squash-merge commit. Runtime readers continue to accept legacy v0.1 and v0.2 results.
 
 Read the newest result for a commit:
 
@@ -64,3 +70,5 @@ Publishing `refs/tabellio/validations` to a remote is a separate approved Git wr
 ## Security Boundary
 
 Detached worktrees and isolated home directories protect host source and common credential paths; they are not a hostile-code sandbox. Validation commands can execute arbitrary repository code and use the network. Run untrusted contributions in a disposable VM, container, or sandbox with scoped credentials and network policy. Never place secrets in manifest arguments or print them: argv and bounded output tails are retained as durable evidence.
+
+Typed validation does not authorize live writes. Repository adapters should use fixtures, isolated namespaces, read-only credentials, and explicit cost caps. Artifact metadata must point to durable content-addressed storage when a reviewer needs the screenshot, trace, or report after the temporary worktree is removed.

docs/workflow-model.md

@@ -12,6 +12,8 @@ Tabellio turns an agentic coding run into a GitHub-bound context packet and, whe
 | Workspace | Isolated Git worktree assigned to one agent run |
 | Context packet | Integrity-protected binding between task, actor, exact commits, changed files, checkpoints, and merge preview |
 | Evidence envelope | JSON record of Git state, changed files, commands, checks, approvals, side-effect policy, and artifacts |
+| Acceptance contract | Committed outcomes, invariants, forbidden outcomes, risk, and required validator types |
+| Validator evidence | Bounded metrics, cost telemetry, and immutable artifact references emitted by one product validator |
 | External action policy | Default-deny policy for deploys, migrations, infra, DNS, billing, secrets, provider reads, and destructive actions |
 | Pull request | Human review surface for the diff plus evidence summary |
 | Stack | Ordered PR chain for larger changes |
@@ -34,7 +36,7 @@ task
   -> resolve immutable base commit
   -> isolated worktree and agent branch
   -> small commits and mandatory Entire checkpoint metadata
-  -> deterministic checks
+  -> deterministic and product-validity checks
   -> context packet
   -> read-only merge preview
   -> evidence envelope
@@ -60,7 +62,7 @@ Each PR should expose:
 
 The context packet is usable without a pull request. GitHub is the review and code-distribution surface; Tabellio's evidence remains independently verifiable.
 
-Evidence is not a claim that the work is correct. Evidence is the record reviewers inspect before trusting the work.
+Raw evidence is not automatically a correctness claim. A typed validation decision proves that the exact committed evidence met the committed thresholds. Human review still owns product judgment outside the declared contract.
 
 ## Stack Discipline
 
@@ -112,6 +114,6 @@ Future versions can add:
 - SLSA provenance export
 - in-toto link metadata
 - OpenTelemetry spans
-- model/tool eval suites
+- path-based validator policy selection
 - automated git-spice submission and merge orchestration for GitHub
 - GitHub merge queue metadata

examples/tabellio-validation/minimal-validator-evidence.json

@@ -0,0 +1,20 @@
+{
+  "schemaVersion": "tabellio-validator-evidence/v0.1",
+  "validatorId": "semantic-golden-set",
+  "status": "passed",
+  "summary": "Eight of ten required scenarios passed the strict semantic rubric.",
+  "metrics": [
+    {
+      "name": "strict_pass_rate",
+      "value": 0.8,
+      "unit": "ratio"
+    }
+  ],
+  "cost": {
+    "telemetry": "available",
+    "usd": 0.18,
+    "modelCalls": 10,
+    "toolCalls": 24
+  },
+  "artifacts": []
+}

examples/tabellio-validation/product-manifest.json

@@ -0,0 +1,78 @@
+{
+  "schemaVersion": "tabellio-validation/v0.2",
+  "id": "product-validity",
+  "failFast": true,
+  "requireEntireCheckpoint": true,
+  "acceptance": {
+    "id": "PLANE-101",
+    "source": "plane",
+    "risk": "high",
+    "outcomes": [
+      "The user receives a correct result for the requested workflow."
+    ],
+    "invariants": [
+      "Validation cannot mutate production resources."
+    ],
+    "forbiddenOutcomes": [
+      "Unknown cost telemetry is treated as zero cost."
+    ],
+    "requiredValidatorTypes": [
+      "semantic",
+      "operational"
+    ]
+  },
+  "validators": [
+    {
+      "id": "semantic-golden-set",
+      "type": "semantic",
+      "argv": [
+        "npm",
+        "run",
+        "eval:semantic:ci"
+      ],
+      "cwd": ".",
+      "timeoutMs": 600000,
+      "required": true,
+      "evidence": {
+        "path": ".artifacts/tabellio/semantic-evidence.json"
+      },
+      "policy": {
+        "metricThresholds": [
+          {
+            "metric": "strict_pass_rate",
+            "operator": "gte",
+            "value": 0.8
+          }
+        ],
+        "maxCostUsd": 1,
+        "requireCostTelemetry": true
+      }
+    },
+    {
+      "id": "runtime-cost-guard",
+      "type": "operational",
+      "argv": [
+        "npm",
+        "run",
+        "cost:check"
+      ],
+      "cwd": ".",
+      "timeoutMs": 120000,
+      "required": true,
+      "evidence": {
+        "path": ".artifacts/tabellio/cost-evidence.json"
+      },
+      "policy": {
+        "metricThresholds": [
+          {
+            "metric": "projected_monthly_cost_usd",
+            "operator": "lte",
+            "value": 20
+          }
+        ],
+        "maxCostUsd": 0.25,
+        "requireCostTelemetry": true
+      }
+    }
+  ]
+}

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@intelip/tabellio",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "description": "Tabellio: GitHub-native context and evidence for agentic development.",
   "type": "module",
   "license": "Apache-2.0",
@@ -16,6 +16,7 @@
     "tabellio-stack-operation": "scripts/tabellio-stack-operation.mjs",
     "tabellio-review": "scripts/tabellio-review.mjs",
     "tabellio-validate": "scripts/tabellio-validate.mjs",
+    "tabellio-validator": "scripts/tabellio-validator.mjs",
     "tabellio-control-ref": "scripts/tabellio-control-ref.mjs",
     "tabellio-ledger": "scripts/tabellio-ledger.mjs",
     "tabellio-preflight": "scripts/tabellio-preflight.mjs",
@@ -60,7 +61,7 @@
     "tabellio:release": "node scripts/tabellio-release.mjs",
     "tabellio:release:example:check": "node scripts/check-tabellio-release.mjs --intent examples/tabellio-release/minimal-intent.json",
     "tabellio:platform:check": "node scripts/check-tabellio-platform.mjs",
-    "tabellio:validate:example:check": "node scripts/check-tabellio-validation.mjs --manifest tabellio.validation.json && node scripts/check-tabellio-validation.mjs --result examples/tabellio-validation/minimal-result.json",
+    "tabellio:validate:example:check": "node scripts/check-tabellio-validation.mjs --manifest tabellio.validation.json && node scripts/check-tabellio-validation.mjs --manifest examples/tabellio-validation/product-manifest.json && node scripts/check-tabellio-validation.mjs --evidence examples/tabellio-validation/minimal-validator-evidence.json && node scripts/check-tabellio-validation.mjs --result examples/tabellio-validation/minimal-result.json",
     "tabellio:ledger": "node scripts/tabellio-ledger.mjs",
     "tabellio:ledger:check": "node scripts/check-tabellio-ledger.mjs --ledger tabellio-ledger.json",
     "tabellio:ledger:example:check": "node scripts/check-tabellio-ledger.mjs --ledger examples/tabellio-ledger/minimal-ledger.json",