diff --git a/ROADMAP.md b/ROADMAP.md index f3e2b77..540ff44 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -15,6 +15,18 @@ requirements-file dependency change through the generated report and confirm that local risk buckets remain policy-review evidence rather than package safety verdicts. +## v1.1: Input and Policy Semantics + +The v1.1 route is implementation-led: pin the real input support matrix, +version policy and report contracts, make policy decision evidence explicit, +then harden component identity canonicalization and same-input conflict +diagnostics. The detailed acceptance gates live in +[tools/sbom-diff-and-risk/docs/v1.1-input-and-policy-semantics.md](tools/sbom-diff-and-risk/docs/v1.1-input-and-policy-semantics.md). + +The tool remains in this monorepo. Repository extraction is deferred until +real third-party adoption creates an independent release or maintenance +boundary. + ## Parked Directions - Parser-boundary fixtures, only when the before/after input and normalized diff --git a/tools/sbom-diff-and-risk/README.md b/tools/sbom-diff-and-risk/README.md index ecf9bfb..552e898 100644 --- a/tools/sbom-diff-and-risk/README.md +++ b/tools/sbom-diff-and-risk/README.md @@ -42,6 +42,8 @@ For a consumer-facing GitHub Actions example, see [docs/github-actions-consumer-example.md](docs/github-actions-consumer-example.md). For regenerating checked-in local example outputs, see [docs/example-artifact-regeneration.md](docs/example-artifact-regeneration.md). +The v1.1 implementation sequence is fixed in +[docs/v1.1-input-and-policy-semantics.md](docs/v1.1-input-and-policy-semantics.md). 1. If you want to verify `sbom-diff-and-risk` itself, start with [docs/verification.md](docs/verification.md). @@ -93,11 +95,14 @@ When a `purl` includes a version, the tool keeps the full value in `Component.pu ## Supported Formats -- CycloneDX JSON -- SPDX JSON -- `requirements.txt` -- `pyproject.toml` via PEP 621 `[project]` metadata -- `pyproject.toml` dependency groups via PEP 735 `[dependency-groups]` with explicit selection +- CycloneDX JSON, top-level component subset +- SPDX JSON, top-level package subset +- `requirements.txt`, conservative PEP 508 subset +- `pyproject.toml`, PEP 621 arrays and explicitly selected PEP 735 groups + +See the test-backed [input format support matrix](docs/parser-boundaries.md) +for the exact fields and unsupported constructs. A recognized container format +does not imply full specification conformance. ## Risk Bucket Semantics @@ -219,7 +224,7 @@ The checked-in [examples/sample-summary.json](examples/sample-summary.json) arti For CI dashboard, job-summary, and local-threshold examples, see [docs/summary-json-ci-cookbook.md](docs/summary-json-ci-cookbook.md). `--policy-json PATH` writes only policy-related JSON sections from the full -report. It includes `policy_evaluation`, policy finding lists, `rule_catalog`, +report. It includes `policy_schema`, `policy_evaluation`, policy finding lists, `rule_catalog`, and `summary.policy` when policy evaluation is applied. For CI job-summary examples, see [docs/policy-decision-ci-cookbook.md](docs/policy-decision-ci-cookbook.md). diff --git a/tools/sbom-diff-and-risk/docs/parser-boundaries.md b/tools/sbom-diff-and-risk/docs/parser-boundaries.md index 2d289c0..a91f307 100644 --- a/tools/sbom-diff-and-risk/docs/parser-boundaries.md +++ b/tools/sbom-diff-and-risk/docs/parser-boundaries.md @@ -1,10 +1,45 @@ -# Parser boundaries +# Input format support matrix and parser boundaries `sbom-diff-and-risk` intentionally supports a conservative parser subset so local runs remain deterministic, auditable, and CI-friendly. The project does not try to emulate a package installer. When syntax would require resolver behavior, implicit includes, index lookups, or environment-specific side effects, the parser fails closed with an explicit error. -## requirements.txt +## Support matrix + +Every row below describes parser behavior implemented and covered by tests. A +recognized container format does not imply full specification conformance. + +| Input | CLI format | Implemented subset | Not claimed | +| --- | --- | --- | --- | +| CycloneDX JSON | `cyclonedx-json` | Top-level `components`; component name, version, purl, bom-ref, type, first usable license, supplier/author, and selected external-reference URLs | CycloneDX schema validation, dependency graphs, services, vulnerabilities, compositions, or recursive nested components | +| SPDX JSON | `spdx-json` | Top-level `packages`; package name, versionInfo, SPDXID, primary purpose, declared/concluded license, supplier/originator, purl external reference, and selected source URLs | SPDX schema validation, relationship graphs, files, snippets, annotations, or license analysis | +| requirements file | `requirements-txt` | The PEP 508 requirement subset listed below, with comments and deterministic line continuation | pip installation semantics, includes, constraints, index configuration, hashes, URLs, VCS references, archives, or local paths | +| `pyproject.toml` | `pyproject-toml` | PEP 621 dependency arrays and explicitly selected PEP 735 dependency groups, including local group includes | A general Python lockfile/parser, build backend interpretation, or Poetry/Hatch/PDM tool-specific tables | + +No XML SBOM, SPDX tag-value, YAML SBOM, package-lock, poetry.lock, uv.lock, +or other lockfile parser is currently registered. Such formats are unsupported, +not silently treated as one of the rows above. + +## CycloneDX JSON + +The parser requires a top-level JSON object with `bomFormat: CycloneDX`. It +reads only the top-level `components` array. Each component requires `name`; +all other normalized fields are optional. + +The parser does not currently constrain `specVersion` or validate the document +against a CycloneDX schema. Acceptance therefore means the implemented fields +were readable, not that the entire document is CycloneDX-conformant. + +## SPDX JSON + +The parser requires a top-level JSON object with a string `spdxVersion` and +reads only the top-level `packages` array. Each package requires `name`. + +The parser does not currently constrain the SPDX version or validate the +document against an SPDX schema. Relationships and file-level data do not +affect component identity or policy decisions. + +## Requirements files `requirements.txt` is treated as a narrow manifest format, not as "everything pip can do in a file". @@ -23,18 +58,19 @@ The project does not try to emulate a package installer. When syntax would requi When unsupported syntax appears, the parser raises `UnsupportedInputError` and the CLI returns exit code `2`. -## pyproject.toml +## `pyproject.toml` `pyproject.toml` support is also intentionally narrow: | Section | Status | Notes | | --- | --- | --- | -| `[project.dependencies]` | Supported | Parsed by default | -| `[project.optional-dependencies]` | Supported | Parsed by default and kept distinct from dependency groups | +| `[project]` `dependencies` | Supported | Parsed by default as a PEP 508 string array | +| `[project.optional-dependencies]` | Supported | Every declared optional group is parsed by default and kept distinct from dependency groups | | `[dependency-groups]` | Supported | Requires explicit `--pyproject-group ` selection | | `{ include-group = "name" }` inside dependency groups | Supported | Includes are resolved locally and deterministically | +| PEP 508 direct references in supported arrays | Supported | Recorded as local manifest evidence; no URL is fetched by the parser | | Missing requested dependency group | Explicit error | Reported as `InputSelectionError` | -| Poetry, Hatch, PDM, or other tool-specific dependency sections | Unsupported | Not parsed in v0.2 | +| Poetry, Hatch, PDM, or other tool-specific dependency sections | Unsupported | Not parsed | Dependency groups are not merged automatically with `[project.optional-dependencies]`. They solve different problems and are kept separate on purpose. diff --git a/tools/sbom-diff-and-risk/docs/policy-decision-explainability.md b/tools/sbom-diff-and-risk/docs/policy-decision-explainability.md index 47550ac..a9379cf 100644 --- a/tools/sbom-diff-and-risk/docs/policy-decision-explainability.md +++ b/tools/sbom-diff-and-risk/docs/policy-decision-explainability.md @@ -27,6 +27,10 @@ policy findings. ## Field contract +- `matched_rule_id`: Exact policy rule id that produced the decision. +- `exact_evidence`: Stable object containing the compared component key, + finding bucket, configured threshold, and observed value. Non-applicable + members remain `null` rather than disappearing. - `decision_reason`: Stable reason code for the policy decision. - `policy_rule`: Policy rule id that produced the decision. - `severity_source`: Source of the active severity, such as `block_on`, @@ -36,6 +40,11 @@ policy findings. decision, when applicable. - `observed_value`: Observed local value that was compared to the policy rule, when applicable. +- `confidence_level`: Evidence source level for this decision: + `policy_matched`, `provenance_recorded`, or `scorecard_recorded`. + +`confidence_level` describes which recorded evidence the decision used. It is +not a probability and does not express package safety. The full JSON report shape is documented in [report-schema.md](report-schema.md). Policy configuration fields and supported rules are documented in @@ -47,7 +56,15 @@ A policy finding with: ```json { + "matched_rule_id": "max_added_packages", "decision_reason": "added_package_count_exceeded_threshold", + "exact_evidence": { + "component_key": null, + "finding_bucket": null, + "matched_threshold": 0, + "observed_value": 1 + }, + "confidence_level": "policy_matched", "policy_rule": "max_added_packages", "severity_source": "block_on", "matched_threshold": 0, @@ -63,7 +80,15 @@ A policy finding with: ```json { + "matched_rule_id": "new_package", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/example-package", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, diff --git a/tools/sbom-diff-and-risk/docs/policy-schema.md b/tools/sbom-diff-and-risk/docs/policy-schema.md index fed0f4e..71f4100 100644 --- a/tools/sbom-diff-and-risk/docs/policy-schema.md +++ b/tools/sbom-diff-and-risk/docs/policy-schema.md @@ -1,20 +1,31 @@ # Policy schema -`sbom-diff-and-risk` supports YAML-only policy schemas in versions `1`, `2`, -and `3` for the local, provenance-aware, and optional Scorecard-aware policy -flows described here. +`sbom-diff-and-risk` identifies the serialized policy contract with: + +```yaml +policy_schema: sbom-diff-risk.policy.v1 +``` + +The separate integer `version` remains the policy capability level: `1` for +local rules, `2` for provenance-aware rules, and `3` for optional +Scorecard-aware rules. Keeping these concepts separate lets the serialized +contract evolve without relabeling existing rule capability levels. The schema is intentionally conservative and fail-closed: - unknown rule ids are rejected - unknown top-level keys are rejected +- unknown `policy_schema` values are rejected - invalid types are rejected +- policies that omit `policy_schema` remain readable as v1.0-compatible input +- normalized policy output always records `sbom-diff-risk.policy.v1` - version `1` remains the v0.2-compatible schema and existing v0.2 policies continue to work unchanged - version `2` adds provenance-aware gating for explicit PyPI enrichment evidence - version `3` adds optional Scorecard-aware gating for explicitly requested Scorecard enrichment ## Version 1 fields +- `policy_schema: sbom-diff-risk.policy.v1` - `version: 1` - `block_on: [rule_id, ...]` - `warn_on: [rule_id, ...]` @@ -113,6 +124,7 @@ see ## Version 1 example ```yaml +policy_schema: sbom-diff-risk.policy.v1 version: 1 block_on: - unknown_license @@ -130,6 +142,7 @@ ignore_rules: ## Version 2 example ```yaml +policy_schema: sbom-diff-risk.policy.v1 version: 2 block_on: - provenance_required @@ -147,6 +160,7 @@ allow_unattested_publishers: ## Version 3 example ```yaml +policy_schema: sbom-diff-risk.policy.v1 version: 3 warn_on: - scorecard_below_threshold diff --git a/tools/sbom-diff-and-risk/docs/policy-warning-reviewer-case.md b/tools/sbom-diff-and-risk/docs/policy-warning-reviewer-case.md index aade426..ea3669e 100644 --- a/tools/sbom-diff-and-risk/docs/policy-warning-reviewer-case.md +++ b/tools/sbom-diff-and-risk/docs/policy-warning-reviewer-case.md @@ -30,6 +30,7 @@ Reference inputs and policy: The minimal policy is intentionally small: ```yaml +policy_schema: sbom-diff-risk.policy.v1 version: 1 block_on: - unknown_license diff --git a/tools/sbom-diff-and-risk/docs/report-schema.md b/tools/sbom-diff-and-risk/docs/report-schema.md index f4f792d..86449a0 100644 --- a/tools/sbom-diff-and-risk/docs/report-schema.md +++ b/tools/sbom-diff-and-risk/docs/report-schema.md @@ -15,6 +15,7 @@ JSON reports currently use this top-level structure: | Field | Description | | --- | --- | +| `report_schema` | Stable report contract identifier; currently `sbom-diff-risk.report.v1`. | | `summary` | Compact run summary for deterministic machine consumption. | | `evidence_confidence` | Highest evidence-confidence level represented by this report. | | `components` | Added, removed, and changed component records. | @@ -53,6 +54,11 @@ These fields describe why a local policy rule produced a block, warning, or suppression. They are policy-decision metadata only; they are not dependency safety verdicts, CVE results, or proof that a package is safe or unsafe. +- `matched_rule_id`: Exact policy rule id that produced the decision. The + legacy `rule_id` and `policy_rule` fields remain available in report v1. +- `exact_evidence`: Structured local comparison evidence with + `component_key`, `finding_bucket`, `matched_threshold`, and + `observed_value`. Values remain `null` when that dimension does not apply. - `decision_reason`: Stable reason code for the policy decision, such as `risk_finding_matched_policy_rule`, `added_package_count_exceeded_threshold`, or @@ -66,6 +72,9 @@ safety verdicts, CVE results, or proof that a package is safe or unsafe. decision, when applicable. - `observed_value`: Observed local value that was compared to the policy rule, when applicable. +- `confidence_level`: Evidence level used for this decision. Local policy + matches use `policy_matched`; decisions based on recorded provenance or + Scorecard data use `provenance_recorded` or `scorecard_recorded`. Explanation fields appear only on policy finding objects. Risk findings in `risks` remain the analyzer's local heuristic findings and do not receive @@ -80,6 +89,7 @@ consumer snippets, see The `--policy-json PATH` CLI option writes a policy-only JSON sidecar using the same policy-related sections from the full JSON report: +- `policy_schema`, currently `sbom-diff-risk.policy.v1` - `policy_evaluation` - `blocking_findings` - `warning_findings` @@ -170,6 +180,8 @@ stable for tests and downstream consumers. ## Stability notes - JSON reports are intended for machine consumption. +- Consumers should select the contract using `report_schema` before relying on + required fields. - Golden samples lock important output shape for stable reviewer and CI expectations. - The schema is conservative and additive where possible. - Missing `summary.policy` means policy was not applied. diff --git a/tools/sbom-diff-and-risk/docs/v1.1-input-and-policy-semantics.md b/tools/sbom-diff-and-risk/docs/v1.1-input-and-policy-semantics.md new file mode 100644 index 0000000..5079241 --- /dev/null +++ b/tools/sbom-diff-and-risk/docs/v1.1-input-and-policy-semantics.md @@ -0,0 +1,90 @@ +# v1.1 technical route: Input and Policy Semantics + +v1.1 narrows ambiguity at the two boundaries that CI consumers depend on: +what an input parser actually supports, and why a policy decision was emitted. +It does not add another scientific-computing mini-project or move the tool out +of this monorepo. + +## Delivery status + +| Contract | Current v1.1 status | +| --- | --- | +| Real input support matrix | Implemented in [parser-boundaries.md](parser-boundaries.md) from the registered parsers | +| Policy schema identifier | Implemented as `sbom-diff-risk.policy.v1`; legacy policy files remain readable | +| Report schema identifier and compatibility tests | Implemented as `sbom-diff-risk.report.v1` across checked-in full-report fixtures | +| Per-decision rule, evidence, reason, and confidence | Implemented additively in report v1 policy finding objects | +| Component identity canonicalization | Next implementation slice; target semantics are fixed below | + +## Component identity target + +The canonical identity record will expose these dimensions separately: + +- `ecosystem`: trimmed and normalized to a registered ecosystem identifier. +- `package_name`: normalized with ecosystem-aware rules. PyPI names use PEP + 503 normalization; other ecosystems require explicit test-backed rules. +- `version`: trimmed but otherwise preserved as observed. The tool will not + infer semantic equivalence between unrelated version schemes. +- `purl`: parsed and normalized when present, while retaining the observed purl + in component evidence for auditability. +- `component_key`: versionless package identity used to align before and after + inputs. A version change remains a change, not an add plus remove. + +Identity authority remains `purl`, then `bom_ref`, then the normalized +`(ecosystem, package_name)` coordinate. A parseable purl is authoritative for +its ecosystem and package coordinate. Explicit metadata that disagrees with +that coordinate is a conflict, not an alternative identity. + +Within one input: + +- two records with the same key and identical normalized metadata fail closed + as `duplicate_component`; +- two records with the same key but different normalized metadata fail closed + as `conflicting_metadata`; +- conflicting ecosystem, package name, or version information between a purl + and explicit fields also fails closed as `conflicting_metadata`; +- metadata differences across the before and after inputs remain normal diff + evidence and do not become same-input conflicts. + +The next code slice should introduce a typed canonical identity object and +diagnostic error codes before changing report presentation. Cross-format tests +must cover CycloneDX to SPDX alignment, PyPI name normalization, versioned +purls, exact duplicates, and conflicting metadata. + +## Policy and decision contract + +`policy_schema` identifies the serialized policy family. The existing integer +`version` continues to select rule capabilities and remains compatible with +v1.0 policy files. + +Every emitted policy finding includes: + +- `matched_rule_id` for the exact matching rule; +- `exact_evidence` for the local values used in the comparison; +- `decision_reason` as a stable machine-readable reason code; +- `confidence_level` for the recorded evidence source. + +The legacy `rule_id`, `policy_rule`, `matched_threshold`, and `observed_value` +fields remain in report schema v1. Consumers can migrate to the grouped fields +without a flag day. + +## Compatibility gate + +Before v1.1 release: + +1. Every checked-in full JSON report must declare + `sbom-diff-risk.report.v1` and satisfy required-field/type tests. +2. Every canonical policy example must declare + `sbom-diff-risk.policy.v1`; legacy omission remains a tested compatibility + path. +3. Unknown policy schema identifiers must fail closed. +4. Policy decisions in local, provenance, and Scorecard fixtures must expose + rule, evidence, reason, and confidence fields. +5. Existing v1.0 fields remain readable for the lifetime of report schema v1. + +## Repository boundary + +`sbom-diff-and-risk` remains under `tools/` in this monorepo for v1.1. Naming +friction alone is not a migration trigger. A separate repository should be +considered only after real third-party adoption creates an independent release +cadence, issue stream, or packaging boundary. Production PyPI publishing is a +separate decision and is not implied by this route. diff --git a/tools/sbom-diff-and-risk/examples/policy-decisions/fail.json b/tools/sbom-diff-and-risk/examples/policy-decisions/fail.json index b4df6a0..a1c6d15 100644 --- a/tools/sbom-diff-and-risk/examples/policy-decisions/fail.json +++ b/tools/sbom-diff-and-risk/examples/policy-decisions/fail.json @@ -1,5 +1,6 @@ { "example_type": "policy-decision-example", + "policy_schema": "sbom-diff-risk.policy.v1", "review_decision": "fail", "decision_source": "summary.policy", "summary": { @@ -13,8 +14,16 @@ "policy_findings": [ { "rule_id": "max_added_packages", + "matched_rule_id": "max_added_packages", "level": "block", "decision_reason": "added_package_count_exceeded_threshold", + "exact_evidence": { + "component_key": null, + "finding_bucket": null, + "matched_threshold": 0, + "observed_value": 1 + }, + "confidence_level": "policy_matched", "policy_rule": "max_added_packages", "severity_source": "block_on", "matched_threshold": 0, diff --git a/tools/sbom-diff-and-risk/examples/policy-decisions/pass.json b/tools/sbom-diff-and-risk/examples/policy-decisions/pass.json index 64c8c84..853f749 100644 --- a/tools/sbom-diff-and-risk/examples/policy-decisions/pass.json +++ b/tools/sbom-diff-and-risk/examples/policy-decisions/pass.json @@ -1,5 +1,6 @@ { "example_type": "policy-decision-example", + "policy_schema": "sbom-diff-risk.policy.v1", "review_decision": "pass", "decision_source": "summary.policy", "summary": { diff --git a/tools/sbom-diff-and-risk/examples/policy-decisions/warn.json b/tools/sbom-diff-and-risk/examples/policy-decisions/warn.json index b963f47..6c44f75 100644 --- a/tools/sbom-diff-and-risk/examples/policy-decisions/warn.json +++ b/tools/sbom-diff-and-risk/examples/policy-decisions/warn.json @@ -1,5 +1,6 @@ { "example_type": "policy-decision-example", + "policy_schema": "sbom-diff-risk.policy.v1", "review_decision": "warn", "decision_source": "summary.policy", "summary": { @@ -13,12 +14,21 @@ "policy_findings": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, "observed_value": "new_package", + "component_key": "purl:pkg:pypi/urllib3", "component_name": "urllib3" } ], diff --git a/tools/sbom-diff-and-risk/examples/policy-minimal.yml b/tools/sbom-diff-and-risk/examples/policy-minimal.yml index 0a858dd..73fdbaa 100644 --- a/tools/sbom-diff-and-risk/examples/policy-minimal.yml +++ b/tools/sbom-diff-and-risk/examples/policy-minimal.yml @@ -1,3 +1,4 @@ +policy_schema: sbom-diff-risk.policy.v1 version: 1 block_on: - unknown_license diff --git a/tools/sbom-diff-and-risk/examples/policy-provenance-minimal.yml b/tools/sbom-diff-and-risk/examples/policy-provenance-minimal.yml index 00458ba..f917f3f 100644 --- a/tools/sbom-diff-and-risk/examples/policy-provenance-minimal.yml +++ b/tools/sbom-diff-and-risk/examples/policy-provenance-minimal.yml @@ -1,4 +1,5 @@ # Missing attestation remains a review signal, not proof of compromise. +policy_schema: sbom-diff-risk.policy.v1 version: 2 warn_on: - missing_attestation diff --git a/tools/sbom-diff-and-risk/examples/policy-provenance-strict.yml b/tools/sbom-diff-and-risk/examples/policy-provenance-strict.yml index 9bf25e6..8e9e4c1 100644 --- a/tools/sbom-diff-and-risk/examples/policy-provenance-strict.yml +++ b/tools/sbom-diff-and-risk/examples/policy-provenance-strict.yml @@ -1,4 +1,5 @@ # Explicit provenance requirements for enriched PyPI evidence. +policy_schema: sbom-diff-risk.policy.v1 version: 2 block_on: - provenance_required diff --git a/tools/sbom-diff-and-risk/examples/policy-scorecard-minimal.yml b/tools/sbom-diff-and-risk/examples/policy-scorecard-minimal.yml index 0eb6f91..67511ce 100644 --- a/tools/sbom-diff-and-risk/examples/policy-scorecard-minimal.yml +++ b/tools/sbom-diff-and-risk/examples/policy-scorecard-minimal.yml @@ -1,3 +1,4 @@ +policy_schema: sbom-diff-risk.policy.v1 version: 3 warn_on: - scorecard_below_threshold diff --git a/tools/sbom-diff-and-risk/examples/policy-strict.yml b/tools/sbom-diff-and-risk/examples/policy-strict.yml index 9ebe01c..443e97c 100644 --- a/tools/sbom-diff-and-risk/examples/policy-strict.yml +++ b/tools/sbom-diff-and-risk/examples/policy-strict.yml @@ -1,3 +1,4 @@ +policy_schema: sbom-diff-risk.policy.v1 version: 1 block_on: - unknown_license diff --git a/tools/sbom-diff-and-risk/examples/sample-policy-fail-report.json b/tools/sbom-diff-and-risk/examples/sample-policy-fail-report.json index a88a064..cd77878 100644 --- a/tools/sbom-diff-and-risk/examples/sample-policy-fail-report.json +++ b/tools/sbom-diff-and-risk/examples/sample-policy-fail-report.json @@ -1,4 +1,5 @@ { + "report_schema": "sbom-diff-risk.report.v1", "summary": { "added": 1, "removed": 0, @@ -312,6 +313,7 @@ "applied": true, "policy_path": "examples/policy-strict.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 1, "block_on": [ "unknown_license", @@ -335,9 +337,17 @@ "blocking_violations": [ { "rule_id": "max_added_packages", + "matched_rule_id": "max_added_packages", "level": "block", "message": "Added package count 1 exceeds max_added_packages=0.", "decision_reason": "added_package_count_exceeded_threshold", + "exact_evidence": { + "component_key": null, + "finding_bucket": null, + "matched_threshold": 0, + "observed_value": 1 + }, + "confidence_level": "policy_matched", "policy_rule": "max_added_packages", "severity_source": "block_on", "matched_threshold": 0, @@ -349,9 +359,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/requests", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -363,9 +381,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -379,9 +405,17 @@ "warning_violations": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "message": "Component was not present in the before input.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, @@ -404,9 +438,17 @@ "blocking_findings": [ { "rule_id": "max_added_packages", + "matched_rule_id": "max_added_packages", "level": "block", "message": "Added package count 1 exceeds max_added_packages=0.", "decision_reason": "added_package_count_exceeded_threshold", + "exact_evidence": { + "component_key": null, + "finding_bucket": null, + "matched_threshold": 0, + "observed_value": 1 + }, + "confidence_level": "policy_matched", "policy_rule": "max_added_packages", "severity_source": "block_on", "matched_threshold": 0, @@ -418,9 +460,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/requests", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -432,9 +482,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -448,9 +506,17 @@ "warning_findings": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "message": "Component was not present in the before input.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, @@ -613,6 +679,7 @@ "applied": true, "policy_path": "examples/policy-strict.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 1, "block_on": [ "unknown_license", @@ -636,9 +703,17 @@ "blocking_violations": [ { "rule_id": "max_added_packages", + "matched_rule_id": "max_added_packages", "level": "block", "message": "Added package count 1 exceeds max_added_packages=0.", "decision_reason": "added_package_count_exceeded_threshold", + "exact_evidence": { + "component_key": null, + "finding_bucket": null, + "matched_threshold": 0, + "observed_value": 1 + }, + "confidence_level": "policy_matched", "policy_rule": "max_added_packages", "severity_source": "block_on", "matched_threshold": 0, @@ -650,9 +725,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/requests", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -664,9 +747,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -680,9 +771,17 @@ "warning_violations": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "message": "Component was not present in the before input.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, diff --git a/tools/sbom-diff-and-risk/examples/sample-policy-warn-report.json b/tools/sbom-diff-and-risk/examples/sample-policy-warn-report.json index e790ee7..0dc2a55 100644 --- a/tools/sbom-diff-and-risk/examples/sample-policy-warn-report.json +++ b/tools/sbom-diff-and-risk/examples/sample-policy-warn-report.json @@ -1,4 +1,5 @@ { + "report_schema": "sbom-diff-risk.report.v1", "summary": { "added": 1, "removed": 0, @@ -312,6 +313,7 @@ "applied": true, "policy_path": "examples/policy-minimal.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 1, "block_on": [ "unknown_license" @@ -327,9 +329,17 @@ "warning_violations": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "message": "Component was not present in the before input.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, @@ -353,9 +363,17 @@ "warning_findings": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "message": "Component was not present in the before input.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, @@ -518,6 +536,7 @@ "applied": true, "policy_path": "examples/policy-minimal.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 1, "block_on": [ "unknown_license" @@ -533,9 +552,17 @@ "warning_violations": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "message": "Component was not present in the before input.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, diff --git a/tools/sbom-diff-and-risk/examples/sample-policy.json b/tools/sbom-diff-and-risk/examples/sample-policy.json index 66c6d67..15a2239 100644 --- a/tools/sbom-diff-and-risk/examples/sample-policy.json +++ b/tools/sbom-diff-and-risk/examples/sample-policy.json @@ -1,8 +1,10 @@ { + "policy_schema": "sbom-diff-risk.policy.v1", "policy_evaluation": { "applied": true, "policy_path": "examples/policy-strict.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 1, "block_on": [ "unknown_license", @@ -26,9 +28,17 @@ "blocking_violations": [ { "rule_id": "max_added_packages", + "matched_rule_id": "max_added_packages", "level": "block", "message": "Added package count 1 exceeds max_added_packages=0.", "decision_reason": "added_package_count_exceeded_threshold", + "exact_evidence": { + "component_key": null, + "finding_bucket": null, + "matched_threshold": 0, + "observed_value": 1 + }, + "confidence_level": "policy_matched", "policy_rule": "max_added_packages", "severity_source": "block_on", "matched_threshold": 0, @@ -40,9 +50,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/requests", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -54,9 +72,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -70,9 +96,17 @@ "warning_violations": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "message": "Component was not present in the before input.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, @@ -95,9 +129,17 @@ "blocking_findings": [ { "rule_id": "max_added_packages", + "matched_rule_id": "max_added_packages", "level": "block", "message": "Added package count 1 exceeds max_added_packages=0.", "decision_reason": "added_package_count_exceeded_threshold", + "exact_evidence": { + "component_key": null, + "finding_bucket": null, + "matched_threshold": 0, + "observed_value": 1 + }, + "confidence_level": "policy_matched", "policy_rule": "max_added_packages", "severity_source": "block_on", "matched_threshold": 0, @@ -109,9 +151,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/requests", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -123,9 +173,17 @@ }, { "rule_id": "stale_package", + "matched_rule_id": "stale_package", "level": "block", "message": "stale_package was not evaluated because enrichment mode is disabled.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "not_evaluated", + "matched_threshold": null, + "observed_value": "not_evaluated" + }, + "confidence_level": "policy_matched", "policy_rule": "stale_package", "severity_source": "block_on", "matched_threshold": null, @@ -139,9 +197,17 @@ "warning_findings": [ { "rule_id": "new_package", + "matched_rule_id": "new_package", "level": "warn", "message": "Component was not present in the before input.", "decision_reason": "risk_finding_matched_policy_rule", + "exact_evidence": { + "component_key": "purl:pkg:pypi/urllib3", + "finding_bucket": "new_package", + "matched_threshold": null, + "observed_value": "new_package" + }, + "confidence_level": "policy_matched", "policy_rule": "new_package", "severity_source": "warn_on", "matched_threshold": null, diff --git a/tools/sbom-diff-and-risk/examples/sample-provenance-report.json b/tools/sbom-diff-and-risk/examples/sample-provenance-report.json index 4ad94bb..26b613d 100644 --- a/tools/sbom-diff-and-risk/examples/sample-provenance-report.json +++ b/tools/sbom-diff-and-risk/examples/sample-provenance-report.json @@ -1,4 +1,5 @@ { + "report_schema": "sbom-diff-risk.report.v1", "summary": { "added": 2, "removed": 0, @@ -372,6 +373,7 @@ "applied": true, "policy_path": "examples/policy-provenance-strict.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 2, "block_on": [ "provenance_required", @@ -393,9 +395,17 @@ "blocking_violations": [ { "rule_id": "provenance_required", + "matched_rule_id": "provenance_required", "level": "block", "message": "Provenance is required for new package, but no attestations were published for this PyPI package.", "decision_reason": "required_provenance_not_satisfied", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": "attestation_missing" + }, + "confidence_level": "provenance_recorded", "policy_rule": "provenance_required", "severity_source": "block_on", "matched_threshold": null, @@ -407,9 +417,21 @@ }, { "rule_id": "unverified_provenance", + "matched_rule_id": "unverified_provenance", "level": "block", "message": "PyPI attestations were present, but publisher kinds manual upload did not match allow_provenance_publishers=github actions.", "decision_reason": "provenance_publisher_not_verified", + "exact_evidence": { + "component_key": "purl:pkg:pypi/legacy-lib", + "finding_bucket": null, + "matched_threshold": [ + "github actions" + ], + "observed_value": [ + "manual upload" + ] + }, + "confidence_level": "provenance_recorded", "policy_rule": "unverified_provenance", "severity_source": "block_on", "matched_threshold": [ @@ -427,9 +449,17 @@ "warning_violations": [ { "rule_id": "missing_attestation", + "matched_rule_id": "missing_attestation", "level": "warn", "message": "PyPI release metadata was fetched, but no attestations were published for this package release.", "decision_reason": "attestation_not_published", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": false + }, + "confidence_level": "provenance_recorded", "policy_rule": "missing_attestation", "severity_source": "warn_on", "matched_threshold": null, @@ -452,9 +482,17 @@ "blocking_findings": [ { "rule_id": "provenance_required", + "matched_rule_id": "provenance_required", "level": "block", "message": "Provenance is required for new package, but no attestations were published for this PyPI package.", "decision_reason": "required_provenance_not_satisfied", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": "attestation_missing" + }, + "confidence_level": "provenance_recorded", "policy_rule": "provenance_required", "severity_source": "block_on", "matched_threshold": null, @@ -466,9 +504,21 @@ }, { "rule_id": "unverified_provenance", + "matched_rule_id": "unverified_provenance", "level": "block", "message": "PyPI attestations were present, but publisher kinds manual upload did not match allow_provenance_publishers=github actions.", "decision_reason": "provenance_publisher_not_verified", + "exact_evidence": { + "component_key": "purl:pkg:pypi/legacy-lib", + "finding_bucket": null, + "matched_threshold": [ + "github actions" + ], + "observed_value": [ + "manual upload" + ] + }, + "confidence_level": "provenance_recorded", "policy_rule": "unverified_provenance", "severity_source": "block_on", "matched_threshold": [ @@ -486,9 +536,17 @@ "warning_findings": [ { "rule_id": "missing_attestation", + "matched_rule_id": "missing_attestation", "level": "warn", "message": "PyPI release metadata was fetched, but no attestations were published for this package release.", "decision_reason": "attestation_not_published", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": false + }, + "confidence_level": "provenance_recorded", "policy_rule": "missing_attestation", "severity_source": "warn_on", "matched_threshold": null, @@ -669,6 +727,7 @@ "applied": true, "policy_path": "examples/policy-provenance-strict.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 2, "block_on": [ "provenance_required", @@ -690,9 +749,17 @@ "blocking_violations": [ { "rule_id": "provenance_required", + "matched_rule_id": "provenance_required", "level": "block", "message": "Provenance is required for new package, but no attestations were published for this PyPI package.", "decision_reason": "required_provenance_not_satisfied", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": "attestation_missing" + }, + "confidence_level": "provenance_recorded", "policy_rule": "provenance_required", "severity_source": "block_on", "matched_threshold": null, @@ -704,9 +771,21 @@ }, { "rule_id": "unverified_provenance", + "matched_rule_id": "unverified_provenance", "level": "block", "message": "PyPI attestations were present, but publisher kinds manual upload did not match allow_provenance_publishers=github actions.", "decision_reason": "provenance_publisher_not_verified", + "exact_evidence": { + "component_key": "purl:pkg:pypi/legacy-lib", + "finding_bucket": null, + "matched_threshold": [ + "github actions" + ], + "observed_value": [ + "manual upload" + ] + }, + "confidence_level": "provenance_recorded", "policy_rule": "unverified_provenance", "severity_source": "block_on", "matched_threshold": [ @@ -724,9 +803,17 @@ "warning_violations": [ { "rule_id": "missing_attestation", + "matched_rule_id": "missing_attestation", "level": "warn", "message": "PyPI release metadata was fetched, but no attestations were published for this package release.", "decision_reason": "attestation_not_published", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": false + }, + "confidence_level": "provenance_recorded", "policy_rule": "missing_attestation", "severity_source": "warn_on", "matched_threshold": null, @@ -790,9 +877,17 @@ "blocking": [ { "rule_id": "provenance_required", + "matched_rule_id": "provenance_required", "level": "block", "message": "Provenance is required for new package, but no attestations were published for this PyPI package.", "decision_reason": "required_provenance_not_satisfied", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": "attestation_missing" + }, + "confidence_level": "provenance_recorded", "policy_rule": "provenance_required", "severity_source": "block_on", "matched_threshold": null, @@ -804,9 +899,21 @@ }, { "rule_id": "unverified_provenance", + "matched_rule_id": "unverified_provenance", "level": "block", "message": "PyPI attestations were present, but publisher kinds manual upload did not match allow_provenance_publishers=github actions.", "decision_reason": "provenance_publisher_not_verified", + "exact_evidence": { + "component_key": "purl:pkg:pypi/legacy-lib", + "finding_bucket": null, + "matched_threshold": [ + "github actions" + ], + "observed_value": [ + "manual upload" + ] + }, + "confidence_level": "provenance_recorded", "policy_rule": "unverified_provenance", "severity_source": "block_on", "matched_threshold": [ @@ -824,9 +931,17 @@ "warning": [ { "rule_id": "missing_attestation", + "matched_rule_id": "missing_attestation", "level": "warn", "message": "PyPI release metadata was fetched, but no attestations were published for this package release.", "decision_reason": "attestation_not_published", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": false + }, + "confidence_level": "provenance_recorded", "policy_rule": "missing_attestation", "severity_source": "warn_on", "matched_threshold": null, @@ -857,9 +972,17 @@ "blocking": [ { "rule_id": "provenance_required", + "matched_rule_id": "provenance_required", "level": "block", "message": "Provenance is required for new package, but no attestations were published for this PyPI package.", "decision_reason": "required_provenance_not_satisfied", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": "attestation_missing" + }, + "confidence_level": "provenance_recorded", "policy_rule": "provenance_required", "severity_source": "block_on", "matched_threshold": null, @@ -871,9 +994,21 @@ }, { "rule_id": "unverified_provenance", + "matched_rule_id": "unverified_provenance", "level": "block", "message": "PyPI attestations were present, but publisher kinds manual upload did not match allow_provenance_publishers=github actions.", "decision_reason": "provenance_publisher_not_verified", + "exact_evidence": { + "component_key": "purl:pkg:pypi/legacy-lib", + "finding_bucket": null, + "matched_threshold": [ + "github actions" + ], + "observed_value": [ + "manual upload" + ] + }, + "confidence_level": "provenance_recorded", "policy_rule": "unverified_provenance", "severity_source": "block_on", "matched_threshold": [ @@ -891,9 +1026,17 @@ "warning": [ { "rule_id": "missing_attestation", + "matched_rule_id": "missing_attestation", "level": "warn", "message": "PyPI release metadata was fetched, but no attestations were published for this package release.", "decision_reason": "attestation_not_published", + "exact_evidence": { + "component_key": "purl:pkg:pypi/mystery-lib", + "finding_bucket": null, + "matched_threshold": null, + "observed_value": false + }, + "confidence_level": "provenance_recorded", "policy_rule": "missing_attestation", "severity_source": "warn_on", "matched_threshold": null, diff --git a/tools/sbom-diff-and-risk/examples/sample-report.json b/tools/sbom-diff-and-risk/examples/sample-report.json index 5f9caf6..6dd2971 100644 --- a/tools/sbom-diff-and-risk/examples/sample-report.json +++ b/tools/sbom-diff-and-risk/examples/sample-report.json @@ -1,4 +1,5 @@ { + "report_schema": "sbom-diff-risk.report.v1", "summary": { "added": 1, "removed": 0, diff --git a/tools/sbom-diff-and-risk/examples/sample-requirements-report.json b/tools/sbom-diff-and-risk/examples/sample-requirements-report.json index c0d4ca6..36b5b1c 100644 --- a/tools/sbom-diff-and-risk/examples/sample-requirements-report.json +++ b/tools/sbom-diff-and-risk/examples/sample-requirements-report.json @@ -1,4 +1,5 @@ { + "report_schema": "sbom-diff-risk.report.v1", "summary": { "added": 1, "removed": 0, diff --git a/tools/sbom-diff-and-risk/examples/sample-scorecard-report.json b/tools/sbom-diff-and-risk/examples/sample-scorecard-report.json index f70e4df..d65ab08 100644 --- a/tools/sbom-diff-and-risk/examples/sample-scorecard-report.json +++ b/tools/sbom-diff-and-risk/examples/sample-scorecard-report.json @@ -1,4 +1,5 @@ { + "report_schema": "sbom-diff-risk.report.v1", "summary": { "added": 2, "removed": 0, @@ -286,6 +287,7 @@ "applied": true, "policy_path": "examples/policy-scorecard-minimal.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 3, "block_on": [], "warn_on": [ @@ -304,9 +306,17 @@ "warning_violations": [ { "rule_id": "scorecard_below_threshold", + "matched_rule_id": "scorecard_below_threshold", "level": "warn", "message": "Scorecard score 6.0 is below minimum_scorecard_score=7.0 for repository github.com/psf/requests.", "decision_reason": "scorecard_score_below_threshold", + "exact_evidence": { + "component_key": "purl:pkg:pypi/requests", + "finding_bucket": null, + "matched_threshold": 7.0, + "observed_value": 6.0 + }, + "confidence_level": "scorecard_recorded", "policy_rule": "scorecard_below_threshold", "severity_source": "warn_on", "matched_threshold": 7.0, @@ -330,9 +340,17 @@ "warning_findings": [ { "rule_id": "scorecard_below_threshold", + "matched_rule_id": "scorecard_below_threshold", "level": "warn", "message": "Scorecard score 6.0 is below minimum_scorecard_score=7.0 for repository github.com/psf/requests.", "decision_reason": "scorecard_score_below_threshold", + "exact_evidence": { + "component_key": "purl:pkg:pypi/requests", + "finding_bucket": null, + "matched_threshold": 7.0, + "observed_value": 6.0 + }, + "confidence_level": "scorecard_recorded", "policy_rule": "scorecard_below_threshold", "severity_source": "warn_on", "matched_threshold": 7.0, @@ -535,6 +553,7 @@ "applied": true, "policy_path": "examples/policy-scorecard-minimal.yml", "effective_policy": { + "policy_schema": "sbom-diff-risk.policy.v1", "version": 3, "block_on": [], "warn_on": [ @@ -553,9 +572,17 @@ "warning_violations": [ { "rule_id": "scorecard_below_threshold", + "matched_rule_id": "scorecard_below_threshold", "level": "warn", "message": "Scorecard score 6.0 is below minimum_scorecard_score=7.0 for repository github.com/psf/requests.", "decision_reason": "scorecard_score_below_threshold", + "exact_evidence": { + "component_key": "purl:pkg:pypi/requests", + "finding_bucket": null, + "matched_threshold": 7.0, + "observed_value": 6.0 + }, + "confidence_level": "scorecard_recorded", "policy_rule": "scorecard_below_threshold", "severity_source": "warn_on", "matched_threshold": 7.0, diff --git a/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_evaluator.py b/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_evaluator.py index 9f5cd99..598d51c 100644 --- a/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_evaluator.py +++ b/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_evaluator.py @@ -4,7 +4,15 @@ from urllib.parse import urlparse from .diffing import component_key -from .models import Component, ComponentChange, ProvenanceStatus, RiskBucket, RiskFinding, ScorecardStatus +from .models import ( + Component, + ComponentChange, + EvidenceConfidence, + ProvenanceStatus, + RiskBucket, + RiskFinding, + ScorecardStatus, +) from .policy_models import PolicyConfig, PolicyEvaluation, PolicyLevel, PolicyViolation @@ -93,6 +101,11 @@ def evaluate_policy( observed_value="unavailable", component_key=component_key(component), component_name=component.name, + confidence_level=( + EvidenceConfidence.PROVENANCE_RECORDED.value + if component.provenance is not None + else EvidenceConfidence.POLICY_MATCHED.value + ), ), blocking_violations=blocking_violations, warning_violations=warning_violations, @@ -111,6 +124,7 @@ def evaluate_policy( observed_value=False, component_key=component_key(component), component_name=component.name, + confidence_level=EvidenceConfidence.PROVENANCE_RECORDED.value, ), blocking_violations=blocking_violations, warning_violations=warning_violations, @@ -131,6 +145,7 @@ def evaluate_policy( observed_value=list(assessment.publisher_kinds), component_key=component_key(component), component_name=component.name, + confidence_level=EvidenceConfidence.PROVENANCE_RECORDED.value, ), blocking_violations=blocking_violations, warning_violations=warning_violations, @@ -165,6 +180,11 @@ def evaluate_policy( observed_value=_provenance_observed_value(assessment), component_key=component_key(component), component_name=component.name, + confidence_level=( + EvidenceConfidence.PROVENANCE_RECORDED.value + if component.provenance is not None + else EvidenceConfidence.POLICY_MATCHED.value + ), ), blocking_violations=blocking_violations, warning_violations=warning_violations, @@ -243,6 +263,7 @@ def evaluate_policy( observed_value=scorecard_score, component_key=component_key(component), component_name=component.name, + confidence_level=EvidenceConfidence.SCORECARD_RECORDED.value, ), blocking_violations=blocking_violations, warning_violations=warning_violations, @@ -300,6 +321,7 @@ def _policy_violation( component_key: str | None = None, component_name: str | None = None, finding_bucket: str | None = None, + confidence_level: str = EvidenceConfidence.POLICY_MATCHED.value, ) -> PolicyViolation: return PolicyViolation( rule_id=rule_id, @@ -313,6 +335,7 @@ def _policy_violation( component_key=component_key, component_name=component_name, finding_bucket=finding_bucket, + confidence_level=confidence_level, ) @@ -340,6 +363,7 @@ def _record_violation( component_name=violation.component_name, finding_bucket=violation.finding_bucket, suppression_reason="ignored_by_policy", + confidence_level=violation.confidence_level, ) ) return 1 @@ -359,6 +383,7 @@ def _record_violation( component_key=violation.component_key, component_name=violation.component_name, finding_bucket=violation.finding_bucket, + confidence_level=violation.confidence_level, ), blocking_violations, warning_violations, diff --git a/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_models.py b/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_models.py index bc4a955..ababea0 100644 --- a/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_models.py +++ b/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_models.py @@ -4,6 +4,8 @@ from enum import StrEnum from typing import Any +from .schema_versions import POLICY_SCHEMA_V1 + class PolicyLevel(StrEnum): BLOCK = "block" @@ -42,6 +44,7 @@ class PolicyLevel(StrEnum): @dataclass(slots=True, frozen=True) class PolicyConfig: version: int + policy_schema: str = POLICY_SCHEMA_V1 block_on: tuple[str, ...] = () warn_on: tuple[str, ...] = () max_added_packages: int | None = None @@ -68,6 +71,7 @@ class PolicyViolation: component_name: str | None = None finding_bucket: str | None = None suppression_reason: str | None = None + confidence_level: str = "policy_matched" @dataclass(slots=True) diff --git a/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_parser.py b/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_parser.py index ff3751f..023f03d 100644 --- a/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_parser.py +++ b/tools/sbom-diff-and-risk/src/sbom_diff_risk/policy_parser.py @@ -14,8 +14,10 @@ V2_PROVENANCE_POLICY_RULE_IDS, V3_SCORECARD_POLICY_RULE_IDS, ) +from .schema_versions import POLICY_SCHEMA_V1 _V1_SUPPORTED_POLICY_KEYS = { + "policy_schema", "version", "block_on", "warn_on", @@ -55,6 +57,15 @@ def load_policy(path: Path) -> PolicyConfig: if unknown_keys: raise PolicyError(f"Invalid policy schema in {path}: unsupported keys: {', '.join(unknown_keys)}.") + policy_schema = payload.get("policy_schema", POLICY_SCHEMA_V1) + if not isinstance(policy_schema, str): + raise PolicyError(f"Invalid policy schema in {path}: policy_schema must be a string.") + if policy_schema != POLICY_SCHEMA_V1: + raise PolicyError( + f"Invalid policy schema in {path}: unsupported policy_schema {policy_schema!r}; " + f"expected {POLICY_SCHEMA_V1!r}." + ) + version = payload.get("version") if not isinstance(version, int): raise PolicyError(f"Invalid policy schema in {path}: version must be an integer.") @@ -132,6 +143,7 @@ def load_policy(path: Path) -> PolicyConfig: return normalize_policy( PolicyConfig( version=version, + policy_schema=policy_schema, block_on=block_on, warn_on=warn_on, max_added_packages=max_added_packages, @@ -166,6 +178,7 @@ def build_policy( seed = base_policy or PolicyConfig(version=1) merged = PolicyConfig( version=seed.version, + policy_schema=seed.policy_schema, block_on=_merge_strings(seed.block_on, cli_block_on), warn_on=_merge_strings(seed.warn_on, cli_warn_on), max_added_packages=seed.max_added_packages, @@ -187,6 +200,7 @@ def normalize_policy(policy: PolicyConfig) -> PolicyConfig: ignore_rules = tuple(dict.fromkeys(policy.ignore_rules)) return PolicyConfig( version=normalized_version, + policy_schema=policy.policy_schema, block_on=block_on, warn_on=warn_on, max_added_packages=policy.max_added_packages, diff --git a/tools/sbom-diff-and-risk/src/sbom_diff_risk/presentation.py b/tools/sbom-diff-and-risk/src/sbom_diff_risk/presentation.py index 7200391..dba261b 100644 --- a/tools/sbom-diff-and-risk/src/sbom_diff_risk/presentation.py +++ b/tools/sbom-diff-and-risk/src/sbom_diff_risk/presentation.py @@ -148,6 +148,7 @@ def policy_config_to_dict(policy: PolicyConfig | None) -> dict[str, Any] | None: if policy is None: return None payload = { + "policy_schema": policy.policy_schema, "version": policy.version, "block_on": list(policy.block_on), "warn_on": list(policy.warn_on), @@ -176,9 +177,17 @@ def policy_config_to_dict(policy: PolicyConfig | None) -> dict[str, Any] | None: def policy_violation_to_dict(violation: PolicyViolation) -> dict[str, Any]: return { "rule_id": violation.rule_id, + "matched_rule_id": violation.policy_rule or violation.rule_id, "level": violation.level.value if violation.level is not None else None, "message": violation.message, "decision_reason": violation.decision_reason, + "exact_evidence": { + "component_key": violation.component_key, + "finding_bucket": violation.finding_bucket, + "matched_threshold": violation.matched_threshold, + "observed_value": violation.observed_value, + }, + "confidence_level": violation.confidence_level, "policy_rule": violation.policy_rule, "severity_source": violation.severity_source, "matched_threshold": violation.matched_threshold, diff --git a/tools/sbom-diff-and-risk/src/sbom_diff_risk/report_json.py b/tools/sbom-diff-and-risk/src/sbom_diff_risk/report_json.py index 3f8994b..cbc8961 100644 --- a/tools/sbom-diff-and-risk/src/sbom_diff_risk/report_json.py +++ b/tools/sbom-diff-and-risk/src/sbom_diff_risk/report_json.py @@ -8,6 +8,7 @@ from .presentation import build_policy_report_sections, build_trust_signal_report_sections from .policy_models import PolicyEvaluation from .scorecard_enrichment import scorecard_evidence_to_dict +from .schema_versions import POLICY_SCHEMA_V1, REPORT_SCHEMA_V1 def render_report_json(report: CompareReport) -> str: @@ -15,6 +16,7 @@ def render_report_json(report: CompareReport) -> str: trust_signal_sections = build_trust_signal_report_sections(report) evidence_confidence = evidence_confidence_value(report) payload = { + "report_schema": REPORT_SCHEMA_V1, "summary": _summary_to_dict(report), "evidence_confidence": evidence_confidence, "components": { @@ -57,7 +59,9 @@ def render_summary_json(report: CompareReport) -> str: def render_policy_json(report: CompareReport) -> str: policy_sections = build_policy_report_sections(report.metadata.policy_evaluation) + effective_policy = report.metadata.policy_evaluation.effective_policy if report.metadata.policy_evaluation else None payload: dict[str, object] = { + "policy_schema": effective_policy.policy_schema if effective_policy is not None else POLICY_SCHEMA_V1, "policy_evaluation": policy_sections["policy_evaluation"], "blocking_findings": policy_sections["blocking_findings"], "warning_findings": policy_sections["warning_findings"], diff --git a/tools/sbom-diff-and-risk/src/sbom_diff_risk/schema_versions.py b/tools/sbom-diff-and-risk/src/sbom_diff_risk/schema_versions.py new file mode 100644 index 0000000..2c5c100 --- /dev/null +++ b/tools/sbom-diff-and-risk/src/sbom_diff_risk/schema_versions.py @@ -0,0 +1,5 @@ +from __future__ import annotations + + +POLICY_SCHEMA_V1 = "sbom-diff-risk.policy.v1" +REPORT_SCHEMA_V1 = "sbom-diff-risk.report.v1" diff --git a/tools/sbom-diff-and-risk/tests/test_cli_policy_json.py b/tools/sbom-diff-and-risk/tests/test_cli_policy_json.py index 1472e5b..96c3a31 100644 --- a/tools/sbom-diff-and-risk/tests/test_cli_policy_json.py +++ b/tools/sbom-diff-and-risk/tests/test_cli_policy_json.py @@ -4,6 +4,7 @@ from pathlib import Path from sbom_diff_risk import cli +from sbom_diff_risk.schema_versions import POLICY_SCHEMA_V1 def test_cli_policy_json_writes_policy_only_file(tmp_path: Path) -> None: @@ -27,6 +28,7 @@ def test_cli_policy_json_writes_policy_only_file(tmp_path: Path) -> None: payload = json.loads(policy_path.read_text(encoding="utf-8")) assert exit_code == 1 + assert payload["policy_schema"] == POLICY_SCHEMA_V1 assert payload["summary"]["policy"] == { "status": "fail", "blocking": 3, @@ -91,6 +93,7 @@ def test_cli_policy_json_without_policy_records_not_applied(tmp_path: Path) -> N payload = json.loads(policy_path.read_text(encoding="utf-8")) assert exit_code == 0 + assert payload["policy_schema"] == POLICY_SCHEMA_V1 assert payload["policy_evaluation"]["applied"] is False assert payload["policy_evaluation"]["exit_code"] == 0 assert "summary" not in payload @@ -122,7 +125,12 @@ def test_cli_policy_json_omitted_does_not_write_policy_file(tmp_path: Path) -> N def _policy_sidecar_from_full_report(report_payload: dict[str, object]) -> dict[str, object]: + evaluation = report_payload["policy_evaluation"] + assert isinstance(evaluation, dict) + effective_policy = evaluation["effective_policy"] + assert isinstance(effective_policy, dict) policy_payload = { + "policy_schema": effective_policy["policy_schema"], "policy_evaluation": report_payload["policy_evaluation"], "blocking_findings": report_payload["blocking_findings"], "warning_findings": report_payload["warning_findings"], diff --git a/tools/sbom-diff-and-risk/tests/test_policy.py b/tools/sbom-diff-and-risk/tests/test_policy.py index 5aa933f..0bb008e 100644 --- a/tools/sbom-diff-and-risk/tests/test_policy.py +++ b/tools/sbom-diff-and-risk/tests/test_policy.py @@ -9,6 +9,7 @@ from sbom_diff_risk.policy_evaluator import evaluate_policy from sbom_diff_risk.policy_models import PolicyConfig, PolicyLevel from sbom_diff_risk.policy_parser import build_policy, load_policy +from sbom_diff_risk.schema_versions import POLICY_SCHEMA_V1 def test_policy_parser_accepts_minimal_policy() -> None: @@ -16,6 +17,7 @@ def test_policy_parser_accepts_minimal_policy() -> None: policy = load_policy(policy_path) + assert policy.policy_schema == POLICY_SCHEMA_V1 assert policy.version == 1 assert policy.block_on == ("unknown_license",) assert policy.warn_on == ("new_package",) @@ -37,6 +39,28 @@ def test_policy_parser_rejects_unknown_key(tmp_path: Path) -> None: load_policy(path) +def test_policy_parser_accepts_legacy_policy_without_schema_identifier(tmp_path: Path) -> None: + path = tmp_path / "policy.yml" + path.write_text("version: 1\nwarn_on: [new_package]\n", encoding="utf-8") + + policy = load_policy(path) + + assert policy.policy_schema == POLICY_SCHEMA_V1 + + +@pytest.mark.parametrize("schema", ["sbom-diff-risk.policy.v2", 1]) +def test_policy_parser_rejects_unsupported_policy_schema(tmp_path: Path, schema: object) -> None: + path = tmp_path / "policy.yml" + if isinstance(schema, str): + rendered_schema = schema + else: + rendered_schema = str(schema) + path.write_text(f"policy_schema: {rendered_schema}\nversion: 1\n", encoding="utf-8") + + with pytest.raises(PolicyError, match="policy_schema"): + load_policy(path) + + def test_policy_parser_rejects_invalid_version(tmp_path: Path) -> None: path = tmp_path / "policy.yml" path.write_text("version: 4\n", encoding="utf-8") @@ -87,6 +111,7 @@ def test_policy_evaluator_blocks_on_finding_bucket() -> None: assert evaluation.blocking_violations[0].severity_source == "block_on" assert evaluation.blocking_violations[0].matched_threshold is None assert evaluation.blocking_violations[0].observed_value == "unknown_license" + assert evaluation.blocking_violations[0].confidence_level == "policy_matched" def test_policy_evaluator_warns_on_rule_when_configured() -> None: diff --git a/tools/sbom-diff-and-risk/tests/test_policy_decision_examples.py b/tools/sbom-diff-and-risk/tests/test_policy_decision_examples.py index 4e445f5..d981572 100644 --- a/tools/sbom-diff-and-risk/tests/test_policy_decision_examples.py +++ b/tools/sbom-diff-and-risk/tests/test_policy_decision_examples.py @@ -3,6 +3,8 @@ import json from pathlib import Path +from sbom_diff_risk.schema_versions import POLICY_SCHEMA_V1 + EXAMPLES_DIR = Path(__file__).resolve().parents[1] / "examples" / "policy-decisions" DECISION_FILES = { @@ -62,6 +64,18 @@ def test_pass_warn_fail_examples_mirror_runtime_policy_statuses() -> None: assert policy_summary["blocking"] > 0 assert any(finding["level"] == "block" for finding in payload["policy_findings"]) + assert payload["policy_schema"] == POLICY_SCHEMA_V1 + for finding in payload["policy_findings"]: + assert finding["matched_rule_id"] == finding["rule_id"] + assert set(finding["exact_evidence"]) == { + "component_key", + "finding_bucket", + "matched_threshold", + "observed_value", + } + assert finding["decision_reason"] + assert finding["confidence_level"] == "policy_matched" + def test_needs_review_is_consumer_interpretation_not_runtime_status() -> None: payload = _read_example("needs-review.json") diff --git a/tools/sbom-diff-and-risk/tests/test_report_schema_compatibility.py b/tools/sbom-diff-and-risk/tests/test_report_schema_compatibility.py new file mode 100644 index 0000000..c6c5b5f --- /dev/null +++ b/tools/sbom-diff-and-risk/tests/test_report_schema_compatibility.py @@ -0,0 +1,86 @@ +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from sbom_diff_risk.schema_versions import POLICY_SCHEMA_V1, REPORT_SCHEMA_V1 + + +_FULL_REPORT_FIXTURES = ( + "sample-report.json", + "sample-policy-warn-report.json", + "sample-policy-fail-report.json", + "sample-requirements-report.json", + "sample-provenance-report.json", + "sample-scorecard-report.json", +) + +_REQUIRED_REPORT_FIELDS = { + "report_schema", + "summary", + "evidence_confidence", + "components", + "risks", + "policy_evaluation", + "blocking_findings", + "warning_findings", + "suppressed_findings", + "rule_catalog", + "metadata", + "notes", +} + +_REQUIRED_POLICY_DECISION_FIELDS = { + "rule_id", + "matched_rule_id", + "decision_reason", + "exact_evidence", + "confidence_level", +} + + +@pytest.mark.parametrize("fixture_name", _FULL_REPORT_FIXTURES) +def test_full_report_fixtures_are_v1_compatible(fixture_name: str) -> None: + payload = _read_json_fixture(fixture_name) + + assert payload["report_schema"] == REPORT_SCHEMA_V1 + assert set(payload) >= _REQUIRED_REPORT_FIELDS + assert isinstance(payload["summary"], dict) + assert isinstance(payload["components"], dict) + assert isinstance(payload["risks"], list) + assert isinstance(payload["metadata"], dict) + assert isinstance(payload["notes"], list) + + for section in ("blocking_findings", "warning_findings", "suppressed_findings"): + findings = payload[section] + assert isinstance(findings, list) + for finding in findings: + assert set(finding) >= _REQUIRED_POLICY_DECISION_FIELDS + assert finding["matched_rule_id"] == finding["rule_id"] + assert set(finding["exact_evidence"]) == { + "component_key", + "finding_bucket", + "matched_threshold", + "observed_value", + } + assert finding["confidence_level"] in { + "policy_matched", + "provenance_recorded", + "scorecard_recorded", + } + + +def test_policy_sidecar_uses_v1_policy_schema() -> None: + payload = _read_json_fixture("sample-policy.json") + + assert payload["policy_schema"] == POLICY_SCHEMA_V1 + assert payload["policy_evaluation"]["effective_policy"]["policy_schema"] == POLICY_SCHEMA_V1 + + +def _read_json_fixture(name: str) -> dict[str, object]: + path = Path(__file__).resolve().parents[1] / "examples" / name + payload = json.loads(path.read_text(encoding="utf-8")) + assert isinstance(payload, dict) + return payload diff --git a/tools/sbom-diff-and-risk/tests/test_reports.py b/tools/sbom-diff-and-risk/tests/test_reports.py index f35ca14..757e5da 100644 --- a/tools/sbom-diff-and-risk/tests/test_reports.py +++ b/tools/sbom-diff-and-risk/tests/test_reports.py @@ -22,6 +22,7 @@ from sbom_diff_risk.report_json import render_policy_json, render_report_json, render_summary_json from sbom_diff_risk.report_md import render_report_markdown from sbom_diff_risk.risk import evaluate_risks, summarize_risks +from sbom_diff_risk.schema_versions import POLICY_SCHEMA_V1, REPORT_SCHEMA_V1 def test_report_json_matches_cyclonedx_golden_pass() -> None: @@ -124,6 +125,7 @@ def test_report_json_keeps_legacy_sections() -> None: payload = json.loads(render_report_json(report)) assert set(payload) >= { + "report_schema", "summary", "evidence_confidence", "components", @@ -141,6 +143,7 @@ def test_report_json_keeps_legacy_sections() -> None: "metadata", "notes", } + assert payload["report_schema"] == REPORT_SCHEMA_V1 assert payload["metadata"]["policy_evaluation"] == payload["policy_evaluation"] assert payload["metadata"]["enrichment"] == payload["enrichment_metadata"] assert payload["metadata"]["evidence_confidence"] == payload["evidence_confidence"] @@ -222,7 +225,15 @@ def test_report_json_policy_findings_include_explanation_fields() -> None: finding = payload["warning_findings"][0] assert finding["rule_id"] == "new_package" + assert finding["matched_rule_id"] == "new_package" assert finding["decision_reason"] == "risk_finding_matched_policy_rule" + assert finding["exact_evidence"] == { + "component_key": finding["component_key"], + "finding_bucket": "new_package", + "matched_threshold": None, + "observed_value": "new_package", + } + assert finding["confidence_level"] == "policy_matched" assert finding["policy_rule"] == "new_package" assert finding["severity_source"] == "warn_on" assert finding["matched_threshold"] is None @@ -419,7 +430,12 @@ def _read_example(name: str) -> str: def _policy_sidecar_from_full_report(report_payload: dict[str, object]) -> dict[str, object]: + evaluation = report_payload["policy_evaluation"] + assert isinstance(evaluation, dict) + effective_policy = evaluation["effective_policy"] + assert isinstance(effective_policy, dict) policy_payload = { + "policy_schema": effective_policy["policy_schema"], "policy_evaluation": report_payload["policy_evaluation"], "blocking_findings": report_payload["blocking_findings"], "warning_findings": report_payload["warning_findings"],