Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/evidence-pipeline-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,34 @@ The contract is intentionally local and file-based:
- The Cloud IAM schemas preserve the event-time model: `eventTime` is normalized to `event_time`, optional `observedTime` is preserved as `observed_time`, and detection ordering is documented as `event_time`.
- These schemas describe reviewer evidence only. They do not claim production alert routing, case management, autonomous response, or final incident verdicts.

## Traced Example: `CCI-003`

This walks one committed object through the contract end to end, from artifact
to schema, as a worked reviewer example.

- Artifact: `demos/config-change-investigation-demo/artifacts/investigation_summary.json`
- Schema: `schemas/investigation_summary.schema.json`
- Object: `investigation_id: "CCI-003"` (`target_system: "vault-gateway"`, triggered by `cfg-004`)

Field-by-field check against the schema:

| Field | Value | Schema rule | Result |
| --- | --- | --- | --- |
| `investigation_id` | `"CCI-003"` | `pattern: ^CCI-[0-9]{3}$` | matches |
| `severity` | `"high"` | `enum: [low, medium, high, critical]` | matches |
| `evidence_counts.policy_denials` | `0` | integer, `minimum: 0` | matches |
| `evidence_counts.follow_on_events` | `0` | integer, `minimum: 0` | matches |

`CCI-003` satisfies every field in the schema. Its `evidence_counts` of `0`/`0`
is a valid, schema-conformant value, not a gap: it means bounded correlation
found no policy denials or follow-on events for `vault-gateway` inside the
configured window, while `severity: "high"` still reflects the risk of the
triggering change itself (see `evidence_counts` in
[`docs/vocabulary.md`](vocabulary.md#bounded-correlation)). This keeps the
artifact a bounded evidence summary rather than an incident verdict: it does
not claim `vault-gateway` was cleared, only that no additional evidence was
attached under the demo's bounded correlation rules.

## Verification

Run:
Expand Down
10 changes: 9 additions & 1 deletion docs/vocabulary.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,14 @@ families. It does not infer a shared root cause from unrelated telemetry, search
the whole dataset for plausible context, or promote nearby evidence into a final
incident verdict.

`evidence_counts` (as used in `investigation_summary.json`) reports how many
`policy_denials` and `follow_on_events` fell inside the bounded correlation
window and scope for that investigation, not how many exist in total. A count
of `0` means bounded correlation found no attached evidence for that
investigation, not that the triggering change was reviewed as low-risk or
cleared. Severity is set from the triggering change itself, so `evidence_counts`
of `0`/`0` can still appear alongside a `high` or `critical` `severity`.

## Time Semantics

Events, hits, signals, summaries, and reports should preserve the time semantics
Expand All @@ -80,4 +88,4 @@ but it does not add production platform claims. The repo still has:
- no alert routing
- no dashboard or case-management service
- no autonomous response
- no final incident verdicts
- no final incident verdicts
34 changes: 33 additions & 1 deletion tests/test_evidence_pipeline_schemas.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from __future__ import annotations

import json
import re
from pathlib import Path

from jsonschema import Draft202012Validator, FormatChecker
Expand Down Expand Up @@ -166,6 +167,37 @@ def test_evidence_pipeline_schemas_validate_committed_artifacts() -> None:
assert errors == [], f"{schema_path} failed for {artifact_path}\n{_error_summary(errors)}"


def test_cci_003_traces_to_investigation_summary_schema() -> None:
"""Reviewer trace for issue #76: CCI-003 -> investigation_summary schema."""
schema = _load_json("schemas/investigation_summary.schema.json")
Draft202012Validator.check_schema(schema)
# The top-level schema describes the array artifact; validate one traced
# record against the per-item schema so this reads as a single-object trace.
validator = Draft202012Validator(schema["items"], format_checker=FormatChecker())

artifact = _load_json(
"demos/config-change-investigation-demo/artifacts/investigation_summary.json"
)
records = {record["investigation_id"]: record for record in artifact}
assert "CCI-003" in records, "CCI-003 is missing from investigation_summary.json"

cci_003 = records["CCI-003"]
assert cci_003["target_system"] == "vault-gateway"
assert cci_003["triggering_change_id"] == "cfg-004"

errors = sorted(
validator.iter_errors(cci_003), key=lambda error: list(error.absolute_path)
)
assert errors == [], f"CCI-003 failed schema validation\n{_error_summary(errors)}"

assert re.fullmatch(r"CCI-[0-9]{3}", cci_003["investigation_id"])
assert cci_003["severity"] in {"low", "medium", "high", "critical"}
assert cci_003["evidence_counts"]["policy_denials"] >= 0
assert cci_003["evidence_counts"]["follow_on_events"] >= 0
assert isinstance(cci_003["evidence_counts"]["policy_denials"], int)
assert isinstance(cci_003["evidence_counts"]["follow_on_events"], int)


def test_evidence_pipeline_contract_docs_reference_schemas_and_artifacts() -> None:
contract_doc = (REPO_ROOT / "docs" / "evidence-pipeline-contract.md").read_text(
encoding="utf-8"
Expand Down Expand Up @@ -223,4 +255,4 @@ def test_schema_contracts_cover_all_five_demos_and_named_artifacts() -> None:

assert contracted_artifact_paths == REVIEWER_JSON_ARTIFACTS
for artifact_path in REVIEWER_JSON_ARTIFACTS:
assert (REPO_ROOT / artifact_path).is_file(), artifact_path
assert (REPO_ROOT / artifact_path).is_file(), artifact_path