diff --git a/docs/vendor-neutrality.md b/docs/vendor-neutrality.md index c4cff8e09..135c0a919 100644 --- a/docs/vendor-neutrality.md +++ b/docs/vendor-neutrality.md @@ -594,7 +594,7 @@ Organization scope (declared, orthogonal to vendor): ASF = 14, agnostic = 56. | `preflight-audit` | analytics | any | ✅ agnostic | | `privacy-llm` | privacy | any | ✅ agnostic | | `probe-templates` | sandbox | any | ✅ agnostic | -| `sandbox-lint` | sandbox | Claude Code, Kiro, OpenCode | ✅ portable | +| `sandbox-lint` | sandbox | Claude Code, Codex, Cursor, Gemini CLI, Kiro, OpenCode | ✅ portable | | `security-tracker-stats-dashboard` | analytics | any | ✅ agnostic | | `skill-and-tool-validator` | framework-dev | any | ✅ agnostic | | `skill-evals` | framework-dev | any | ✅ agnostic | @@ -609,9 +609,9 @@ Organization scope (declared, orthogonal to vendor): ASF = 14, agnostic = 56. Harness → substrate tools it supports: - **Claude Code** (3): `agent-guard`, `sandbox-lint`, `spec-loop` -- **Codex** (1): `spec-loop` -- **Cursor** (1): `spec-loop` -- **Gemini CLI** (1): `spec-loop` +- **Codex** (2): `sandbox-lint`, `spec-loop` +- **Cursor** (2): `sandbox-lint`, `spec-loop` +- **Gemini CLI** (2): `sandbox-lint`, `spec-loop` - **Kiro** (3): `agent-guard`, `sandbox-lint`, `spec-loop` - **OpenCode** (3): `agent-guard`, `sandbox-lint`, `spec-loop` - **any harness** (19): `agent-isolation`, `dashboard-generator`, `dev`, `egress-gateway`, `permission-audit`, `pilot-report-validator`, `pr-management-stats`, `preflight-audit`, `privacy-llm`, `probe-templates`, `security-tracker-stats-dashboard`, `skill-and-tool-validator`, `skill-evals`, `skill-reconciler-diff`, `spec-inventory`, `spec-status-index`, `spec-validator`, `symlink-lint`, `vendor-neutrality-score` diff --git a/tools/sandbox-lint/README.md b/tools/sandbox-lint/README.md index f7d0add65..25fc91854 100644 --- a/tools/sandbox-lint/README.md +++ b/tools/sandbox-lint/README.md @@ -9,6 +9,7 @@ - [Prerequisites](#prerequisites) - [What it checks](#what-it-checks) - [How to use](#how-to-use) + - [Harness-neutral posture check (any runtime)](#harness-neutral-posture-check-any-runtime) - [CI wiring](#ci-wiring) - [Updating the baseline](#updating-the-baseline) - [Residual risk](#residual-risk) @@ -22,7 +23,7 @@ **Capability:** substrate:sandbox -**Harness:** Claude Code, OpenCode, Kiro +**Harness:** Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Kiro Lints `.claude/settings.json` against the shipped baseline at `tools/sandbox-lint/expected.json`, and against the security @@ -66,6 +67,22 @@ not be a blanket `.*`. uv run --project tools/sandbox-lint sandbox-lint --kiro .kiro/agents/.json ``` +**Any other harness (Codex, Cursor, Gemini CLI, …).** `--any-harness` +validates the harness-neutral OS-level security posture: checks that the +two enforcement components shared across all runtimes — +`tools/agent-isolation/agent-iso.sh` (layer 0, clean-env credential strip) +and `tools/agent-guard/` dispatch core (layer 3, harness-neutral `--exec` +command gating) — are present in the framework tree. Runtimes without a +dedicated `settings.json` or `opencode.json` get the same enforcement +posture through these two OS-level layers rather than through a +harness-specific config file. + +```bash +uv run --project tools/sandbox-lint sandbox-lint --any-harness +# or with an explicit framework root: +uv run --project tools/sandbox-lint sandbox-lint --any-harness /path/to/magpie +``` + ## Prerequisites - **Runtime:** Python 3.11+ run via `uv` (stdlib only, no third-party deps). @@ -118,6 +135,52 @@ uv run --directory tools/sandbox-lint --group dev sandbox-lint \ Exit code is `0` on a clean pass, `1` on any invariant violation or baseline drift. +## Harness-neutral posture check (any runtime) + +For runtimes that do not expose a per-harness sandbox configuration file +(Codex, Cursor, Gemini CLI, Kiro, and any other agent runtime not listed +under `--settings` or `--opencode`), the security posture is enforced at +the OS level by two harness-agnostic components: + +Layer numbers follow the +[RFC-AI-0002](https://magpie.apache.org/docs/rfcs/rfc-ai-0002/) four-layer model +(Layer 0 clean-env, Layer 1 filesystem sandbox, Layer 2 tool permissions, Layer +3 forced confirmation). This presence check covers the two layers that need no +per-harness `settings.json`: + +| Layer | Component | Purpose | +|---|---|---| +| Layer 0 (clean-env) | `tools/agent-isolation/agent-iso.sh` | Strips credential-shaped env vars (`GH_TOKEN`, `AWS_*`, `ANTHROPIC_API_KEY`, …) before exec, so even a fully sandboxed session cannot exfiltrate secrets via the environment. Made harness-agnostic (`agent-iso `) by `harness-posture-portability`. | +| Layer 3 (forced confirmation / action guard) | `tools/agent-guard/` dispatch core | The harness-neutral equivalent of Claude Code's `permissions.ask`. Inspects every shell command before execution and denies the ones that break a hard framework rule (`git push`, wrong commit trailer, premature `--ready-for-review`, …). The `--exec` path (added by `harness-guard-exec-mode`) lets any harness or shell wrapper enforce these rules without a harness-specific hook adapter. | + +`sandbox-lint --any-harness [FRAMEWORK_ROOT]` checks that both components +are present. `FRAMEWORK_ROOT` is auto-detected by walking up from CWD: the +nearest ancestor that either contains `tools/agent-isolation/` directly (the +framework tree) or ships the installed snapshot at +`.apache-magpie/tools/agent-isolation/` (an adopter repo). + +```bash +# From an adopter repo root — the .apache-magpie/ snapshot is auto-detected: +uv run --project tools/sandbox-lint sandbox-lint --any-harness + +# From the Magpie framework tree itself: +uv run --project tools/sandbox-lint sandbox-lint --any-harness + +# Or point at an explicit framework root: +uv run --project tools/sandbox-lint sandbox-lint --any-harness /path/to/magpie +``` + +Exit code is `0` when both components are present, `1` when any are +missing. A `0` confirms the enforcement components are **present**, not that a +harness is actively wired to them (see Scope below). The output identifies which +layer is missing and points to the relevant setup documentation. + +**Scope.** This check validates the *presence* of the enforcement components, +not their live runtime behaviour. To probe whether the OS sandbox is +actually active in a running agent session (SSH-agent reachability, +localhost port binding, docker/podman socket), use +[`setup-isolated-setup-doctor`](../../skills/setup-isolated-setup-doctor/SKILL.md). + ## CI wiring The lint runs in two places: diff --git a/tools/sandbox-lint/src/sandbox_lint/__init__.py b/tools/sandbox-lint/src/sandbox_lint/__init__.py index 3a3ab3b88..88dc20dc8 100644 --- a/tools/sandbox-lint/src/sandbox_lint/__init__.py +++ b/tools/sandbox-lint/src/sandbox_lint/__init__.py @@ -294,13 +294,49 @@ def _lint_kiro(config_path: Path) -> int: return 1 +def _lint_any_harness(framework_root: Path | None) -> int: + """Validate the harness-neutral OS-level security posture. + + Checks that the two harness-agnostic enforcement components — + ``tools/agent-isolation/`` (layer 0, clean-env wrapper) and + ``tools/agent-guard/`` (layer 3, action guard) — are present in the + framework tree. This is the posture check for runtimes that do not have a + settings.json or opencode.json (Codex, Cursor, Gemini CLI, …). + """ + from sandbox_lint.posture import PostureViolation, check_posture_violations, find_framework_root + + root = framework_root if framework_root is not None else find_framework_root() + violations: list[PostureViolation] = check_posture_violations(root) + if not violations: + # Presence check only: this proves the enforcement components exist in + # the tree, NOT that a harness is wired to use them (see the module + # docstring and README § Scope). Word the success line so it cannot be + # mistaken for a live-enforcement guarantee. + print( + "sandbox-lint: OK (harness-neutral enforcement components present at " + f"{root}; run setup-isolated-setup-doctor to verify live enforcement)" + ) + return 0 + print( + f"sandbox-lint: harness-neutral posture violations at {root} —\n" + " For runtimes without a settings.json (Codex, Cursor, Gemini CLI, …)\n" + " the security posture is enforced by these OS-level components:", + file=sys.stderr, + ) + for v in violations: + print(f" - [{v.layer}] {v.detail}", file=sys.stderr) + return 1 + + def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser( prog="sandbox-lint", description=( "Lint .claude/settings.json against the shipped baseline and the " - "security invariants from docs/security/threat-model.md (M.29); or, " - "with --opencode, lint an opencode.json permission policy." + "security invariants from docs/security/threat-model.md (M.29); " + "or lint an OpenCode opencode.json permission policy (--opencode); " + "or validate the harness-neutral OS-level posture for runtimes " + "without a dedicated config file (--any-harness)." ), ) parser.add_argument( @@ -315,7 +351,10 @@ def main(argv: list[str] | None = None) -> int: default=DEFAULT_EXPECTED, help=f"Path to the canonical baseline (default: {DEFAULT_EXPECTED})", ) - parser.add_argument( + # --opencode and --any-harness select alternate lint modes and are mutually + # exclusive; passing both is a usage error rather than one silently winning. + mode = parser.add_mutually_exclusive_group() + mode.add_argument( "--opencode", type=Path, default=None, @@ -326,7 +365,7 @@ def main(argv: list[str] | None = None) -> int: "config (invariants only — no baseline diff)." ), ) - parser.add_argument( + mode.add_argument( "--kiro", type=Path, default=None, @@ -337,6 +376,21 @@ def main(argv: list[str] | None = None) -> int: "Code sandbox config (invariants only — no baseline diff)." ), ) + mode.add_argument( + "--any-harness", + nargs="?", + const=True, # True when flag given without a path → auto-detect framework root + default=None, # None when flag not given at all + metavar="FRAMEWORK_ROOT", + help=( + "Validate the harness-neutral security posture for runtimes without " + "a dedicated settings file (Codex, Cursor, Gemini CLI, …). " + "Checks that the OS-level enforcement components (agent-isolation " + "layer-0 clean-env wrapper and agent-guard layer-3 action guard) " + "are present. Optionally supply the framework root directory; " + "default: auto-detected by walking up from the current directory." + ), + ) args = parser.parse_args(argv) if args.kiro is not None: @@ -344,6 +398,11 @@ def main(argv: list[str] | None = None) -> int: if args.opencode is not None: return _lint_opencode(args.opencode) + if args.any_harness is not None: + # True → auto-detect; string path → convert to Path + explicit_root = None if args.any_harness is True else Path(args.any_harness) + return _lint_any_harness(explicit_root) + settings = _load_json(args.settings) expected = _load_json(args.expected) diff --git a/tools/sandbox-lint/src/sandbox_lint/posture.py b/tools/sandbox-lint/src/sandbox_lint/posture.py new file mode 100644 index 000000000..a6ff4ead1 --- /dev/null +++ b/tools/sandbox-lint/src/sandbox_lint/posture.py @@ -0,0 +1,148 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Harness-neutral security-posture check for sandbox-lint. + +For agentic runtimes that do not have a dedicated sandbox configuration file +(Codex, Cursor, Gemini CLI, Kiro, and others), the security posture comes from +OS-level enforcement components shared across all harnesses. This module +validates that those components are present in the framework tree so the +security posture holds regardless of which harness drives the session. + +The check is statically conservative: it verifies artifact presence, not live +runtime behaviour. For runtime diagnostics — SSH-agent reachability, localhost +port binding, docker/podman socket — use ``setup-isolated-setup-doctor``. + +Layer numbers below follow the RFC-AI-0002 four-layer model (Layer 0 clean-env, +Layer 1 filesystem sandbox, Layer 2 tool permissions, Layer 3 forced +confirmation). The two harness-neutral components this check validates cover the +two layers that do not require a per-harness ``settings.json``: + +* **Layer 0 — clean-env wrapper** (``agent-iso `` / ``agent-iso.sh``): + strips credential-shaped environment variables before exec. Provided by + ``tools/agent-isolation/``. Harness-agnostic since + ``harness-posture-portability``. +* **Layer 3 — forced confirmation / action guard** (``agent-guard --exec + ``): the harness-neutral equivalent of Claude Code's ``permissions.ask`` + (RFC-AI-0002 Layer 3). It inspects every shell command before execution and + denies ones that break a hard framework rule (unauthorised ``git push``, wrong + commit-trailer format, …). The ``--exec`` path lets any harness or shell + wrapper enforce guard rules without a harness-specific hook adapter. Provided + by ``tools/agent-guard/``. Harness-neutral since ``harness-guard-exec-mode``. + +Layers 1 (filesystem sandbox) and 2 (tool permissions) remain harness- or +OS-specific and are out of scope for this presence check. + +Runtimes that expose a pre-tool hook API (Claude Code, OpenCode) can also wire +Layer 3 as an in-process hook; for other runtimes the ``--exec`` wrapper path +is the enforcement mechanism. Both paths call the same ``dispatch()`` core. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path + + +@dataclass(frozen=True) +class PostureViolation: + """A single harness-neutral posture violation.""" + + layer: str # "layer-0" or "layer-3" + detail: str # human-readable description of what is missing and how to fix it + + +def check_posture_violations(framework_root: Path) -> list[PostureViolation]: + """Return posture violations; empty list means the harness-neutral posture is in place. + + ``framework_root`` is the directory that contains ``tools/agent-isolation/`` + and ``tools/agent-guard/``. In a self-adoption setup this is the Magpie repo + root; in an adopter repo it is ``.apache-magpie/`` (the installed snapshot). + """ + violations: list[PostureViolation] = [] + + # --- Layer 0: clean-env wrapper ------------------------------------------- + # The harness-agnostic ``agent-iso `` entry point added by + # ``harness-posture-portability`` is the canonical generic path. The artifact + # to probe is the underlying script, which is the same for every harness. + agent_iso = framework_root / "tools" / "agent-isolation" / "agent-iso.sh" + if not agent_iso.is_file(): + violations.append( + PostureViolation( + layer="layer-0", + detail=( + f"clean-env wrapper not found at {agent_iso}. " + "Without it credential-shaped environment variables " + "(ANTHROPIC_API_KEY, GH_TOKEN, AWS_*, …) leak into the agent " + "session even when an OS-level sandbox is in place. " + "Install the framework snapshot or point --any-harness at " + "the correct framework root. " + "See tools/agent-isolation/README.md" + ), + ) + ) + + # --- Layer 3: action guard (harness-neutral --exec path) ------------------ + # The dispatch core is stdlib-only and is invoked directly as + # python3 …/agent_guard/__init__.py --exec + # for any harness that can wrap shell commands through an executable. + # The --exec path was added by harness-guard-exec-mode. + agent_guard = framework_root / "tools" / "agent-guard" / "src" / "agent_guard" / "__init__.py" + if not agent_guard.is_file(): + violations.append( + PostureViolation( + layer="layer-3", + detail=( + f"action guard dispatch core not found at {agent_guard}. " + "Without it, dangerous shell commands (git push, gh pr create, …) " + "run ungated. Wire the guard via an in-process hook (Claude Code, " + "OpenCode) or the harness-neutral --exec wrapper path " + "(any other runtime). " + "See tools/agent-guard/README.md § Harness-neutral path" + ), + ) + ) + + return violations + + +def find_framework_root(start: Path | None = None) -> Path: + """Walk up the directory tree to locate the framework root. + + At each ancestor this checks two layouts, in order: + + 1. **Self-adoption / framework tree** — the directory directly contains + ``tools/agent-isolation/`` (the Magpie repo itself). Returns that + directory. + 2. **Adopter repo** — the directory contains ``.apache-magpie/`` with the + installed snapshot (``.apache-magpie/tools/agent-isolation/``). Returns + the ``.apache-magpie`` snapshot dir, which is the ``framework_root`` + :func:`check_posture_violations` expects. Without this, a bare + ``--any-harness`` from an adopter repo root would miss the snapshot and + falsely report both layers missing. + + Falls back to *start* (default: ``Path.cwd()``) so the violation messages + are still useful even when the search fails. + """ + root = start if start is not None else Path.cwd() + for candidate in [root, *root.parents]: + if (candidate / "tools" / "agent-isolation").is_dir(): + return candidate + snapshot = candidate / ".apache-magpie" + if (snapshot / "tools" / "agent-isolation").is_dir(): + return snapshot + return root # caller's checks will report the missing artifacts diff --git a/tools/sandbox-lint/tests/test_posture.py b/tools/sandbox-lint/tests/test_posture.py new file mode 100644 index 000000000..01737c3c3 --- /dev/null +++ b/tools/sandbox-lint/tests/test_posture.py @@ -0,0 +1,185 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Tests for the harness-neutral posture check (sandbox_lint.posture).""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from sandbox_lint import main +from sandbox_lint.posture import PostureViolation, check_posture_violations, find_framework_root + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +REPO_ROOT = Path(__file__).resolve().parents[3] + + +def _plant_agent_iso(root: Path) -> Path: + p = root / "tools" / "agent-isolation" / "agent-iso.sh" + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text("#!/bin/bash\n# stub\n") + return p + + +def _plant_agent_guard(root: Path) -> Path: + p = root / "tools" / "agent-guard" / "src" / "agent_guard" / "__init__.py" + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text("# stub dispatch core\ndef dispatch(cmd, cwd): return None\n") + return p + + +# --------------------------------------------------------------------------- +# check_posture_violations +# --------------------------------------------------------------------------- + + +def test_both_components_present_no_violations(tmp_path: Path) -> None: + _plant_agent_iso(tmp_path) + _plant_agent_guard(tmp_path) + assert check_posture_violations(tmp_path) == [] + + +def test_missing_agent_iso_yields_layer0_violation(tmp_path: Path) -> None: + _plant_agent_guard(tmp_path) + violations = check_posture_violations(tmp_path) + assert len(violations) == 1 + assert violations[0].layer == "layer-0" + assert "clean-env" in violations[0].detail + assert "agent-iso" in violations[0].detail + + +def test_missing_agent_guard_yields_layer3_violation(tmp_path: Path) -> None: + _plant_agent_iso(tmp_path) + violations = check_posture_violations(tmp_path) + assert len(violations) == 1 + assert violations[0].layer == "layer-3" + assert "action guard" in violations[0].detail + assert "agent-guard" in violations[0].detail + + +def test_both_missing_yields_two_violations(tmp_path: Path) -> None: + violations = check_posture_violations(tmp_path) + assert len(violations) == 2 + layers = {v.layer for v in violations} + assert layers == {"layer-0", "layer-3"} + + +def test_violation_is_frozen_dataclass() -> None: + v = PostureViolation(layer="layer-0", detail="something missing") + assert v.layer == "layer-0" + with pytest.raises(Exception): + v.layer = "mutated" # type: ignore[misc] + + +# --------------------------------------------------------------------------- +# find_framework_root +# --------------------------------------------------------------------------- + + +def test_find_framework_root_locates_repo_root(tmp_path: Path) -> None: + # Plant a tools/agent-isolation dir so the search terminates at tmp_path. + (tmp_path / "tools" / "agent-isolation").mkdir(parents=True) + # Search from a child directory. + child = tmp_path / "a" / "b" / "c" + child.mkdir(parents=True) + assert find_framework_root(child) == tmp_path + + +def test_find_framework_root_falls_back_to_start(tmp_path: Path) -> None: + # No tools/agent-isolation anywhere — should return the start path. + result = find_framework_root(tmp_path) + assert result == tmp_path + + +def test_find_framework_root_detects_adopter_snapshot(tmp_path: Path) -> None: + # Adopter layout: the snapshot lives under .apache-magpie/. A bare + # --any-harness from the repo root must resolve to the snapshot dir, not + # fall back to the repo root (which would falsely report both layers + # missing). + snapshot = tmp_path / ".apache-magpie" + (snapshot / "tools" / "agent-isolation").mkdir(parents=True) + assert find_framework_root(tmp_path) == snapshot + + +def test_cli_any_harness_auto_detects_adopter_snapshot(tmp_path: Path) -> None: + """Bare --any-harness from an adopter repo root passes when the snapshot is complete.""" + import os + + snapshot = tmp_path / ".apache-magpie" + _plant_agent_iso(snapshot) + _plant_agent_guard(snapshot) + old_cwd = os.getcwd() + try: + os.chdir(tmp_path) + assert main(["--any-harness"]) == 0 + finally: + os.chdir(old_cwd) + + +def test_cli_opencode_and_any_harness_mutually_exclusive(tmp_path: Path) -> None: + """Passing both --opencode and --any-harness is a usage error (SystemExit).""" + with pytest.raises(SystemExit): + main(["--opencode", str(tmp_path / "opencode.json"), "--any-harness"]) + + +# --------------------------------------------------------------------------- +# CLI integration (--any-harness) +# --------------------------------------------------------------------------- + + +def test_cli_any_harness_ok_on_repo_root() -> None: + """--any-harness on the actual Magpie repo returns 0 (both components present).""" + rc = main(["--any-harness", str(REPO_ROOT)]) + assert rc == 0 + + +def test_cli_any_harness_fails_on_empty_dir(tmp_path: Path) -> None: + """--any-harness on an empty directory returns 1 (components missing).""" + rc = main(["--any-harness", str(tmp_path)]) + assert rc == 1 + + +def test_cli_any_harness_auto_detects_root() -> None: + """--any-harness with no path auto-detects the framework root from CWD.""" + import os + + old_cwd = os.getcwd() + try: + os.chdir(REPO_ROOT) + rc = main(["--any-harness"]) + assert rc == 0 + finally: + os.chdir(old_cwd) + + +def test_cli_any_harness_partial_setup_fails(tmp_path: Path) -> None: + """--any-harness with only one component still fails.""" + _plant_agent_iso(tmp_path) + rc = main(["--any-harness", str(tmp_path)]) + assert rc == 1 + + +def test_cli_any_harness_full_setup_passes(tmp_path: Path) -> None: + _plant_agent_iso(tmp_path) + _plant_agent_guard(tmp_path) + rc = main(["--any-harness", str(tmp_path)]) + assert rc == 0