Skip to content

feat: verification — inline guardrails + async judge (Phase 5) #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MANVENDRA-github merged 1 commit into from
Jun 27, 2026
Merged

feat: verification — inline guardrails + async judge (Phase 5) #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,21 @@ CACHE_SIMILARITY_THRESHOLD=0.92
CACHE_TTL_SECONDS=3600
CACHE_MAX_ENTRIES=1000

# ── Verification: guardrails + judge ─────────────────────────────────
# Inline deterministic guardrails (JSON/schema/policy/PII) on non-streaming responses.
VERIFY_ENABLED=false
# When a guardrail finds a violation: flag-only (false) or block with 422 (true).
# A guardrail *execution error* always fails closed (blocks) regardless of this.
GUARDRAILS_BLOCK=false
# Async local-Ollama LLM judge (1–5 score + reason), attached to the trace out-of-band.
JUDGE_ENABLED=false
# Fraction of (non-cached) responses to judge, 0–1.
JUDGE_SAMPLE_RATE=0.1
# (Guardrail blocklist / PII categories live in sentinel.config.json under "guardrails".)

# ── Local models via Ollama (keyless — no quota, no rate limit) ───────
# OpenAI-compatible base URL of YOUR Ollama (used for local completions and the
# semantic-cache embeddings via EMBED_MODEL; the judge reuses it in a later phase).
# OpenAI-compatible base URL of YOUR Ollama (used for local completions, the
# semantic-cache embeddings via EMBED_MODEL, and the verification judge via JUDGE_MODEL).
OLLAMA_BASE_URL=http://localhost:11434/v1
JUDGE_MODEL=qwen2.5:7b
EMBED_MODEL=nomic-embed-text
Expand Down
12 changes: 11 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A self-hostable **verifying LLM gateway** — a drop-in, OpenAI-compatible proxy that **routes** (cheapest capable model + fallback), **semantically caches**, and **verifies** (deterministic guardrails inline + a local Ollama judge) every LLM call, with full OpenTelemetry tracing. Unlike after-the-fact observability tools, it can flag or block a bad response _before it returns_.

> 🚧 **Early development.** Product spec in [`PRP_SPEC.md`](./PRP_SPEC.md), phased build in [`ROADMAP.md`](./ROADMAP.md), contributor/agent guidance in [`CLAUDE.md`](./CLAUDE.md). Currently at **Phase 4 — routing, fallback & rate-limit survival** (inline verification lands in later phases).
> 🚧 **Early development.** Product spec in [`PRP_SPEC.md`](./PRP_SPEC.md), phased build in [`ROADMAP.md`](./ROADMAP.md), contributor/agent guidance in [`CLAUDE.md`](./CLAUDE.md). Currently at **Phase 5 — verification: guardrails + judge** (a quality dashboard lands next).

## What works today (Phase 1)

Expand Down Expand Up @@ -115,6 +115,16 @@ Sentinel survives flaky providers and free-tier rate limits instead of passing t

Every routed request records which provider/model actually served it, whether a fallback was used, and the retry count — queryable via `GET /traces?fallbackUsed=true` (and `?routedProvider=`). Streaming requests fail over up to the first chunk; once the SSE response is committed, a mid-stream error is surfaced as an inline event.

## Verification

Sentinel can check a response's quality **in the request path**, not just after the fact. Enable with `VERIFY_ENABLED=true` and/or `JUDGE_ENABLED=true`; both are off by default.

- **Inline guardrails (deterministic, fail-closed).** On non-streaming responses, Sentinel checks JSON validity and schema match (when the request asked for JSON), plus a policy/PII engine — emails, Luhn-checked cards, SSNs, phone numbers, IPs, API-key-like tokens, a configurable content blocklist, and refusal detection (configured under `guardrails` in `sentinel.config.json`). A violation is **flagged** by default (recorded, response still returned); set `GUARDRAILS_BLOCK=true` to return **422** instead. A guardrail that errors always **fails closed** (blocks). Traces store violation **category codes only** (e.g. `pii.email`) — never the matched value.
- **Async LLM judge (sampled).** A fraction (`JUDGE_SAMPLE_RATE`) of responses are scored 1–5 with a short reason by a local Ollama model (`JUDGE_MODEL`), **out of band** — it never adds latency to the response. The response under review is wrapped as untrusted data so it can't talk the judge into a pass; a judge failure is recorded as "unscored", never a pass.
- **Regression tracking.** Each request carries a model-independent **prompt fingerprint**, so `GET /regression` groups judge scores by `(prompt, model)` — compare the groups sharing a fingerprint to see how one prompt's quality differs across models or versions.

Verdicts are queryable: `GET /traces?guardrailStatus=block`, `?judgeScoreMax=2`, `?promptFingerprint=…`. Inline guardrails apply to non-streaming responses; streamed responses are judged from their buffered output after they complete.

## Development

```bash
Expand Down
21 changes: 11 additions & 10 deletions SECURITY_REVIEW_LOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ Each item: assume an attacker is actively trying it. Tick only when a test prove

### 2. Prompt-injection & judge integrity

- [ ] Prompt/response content treated as untrusted data, never as instructions to Sentinel.
- [ ] The LLM-judge prompt isolates the content under review (delimited/structured) so a response can't talk the judge into "pass"; judge failure ⇒ "unscored", never "passed".
- [ ] Guardrails cannot be disabled by request content.
- [x] Prompt/response content treated as untrusted data, never as instructions to Sentinel.
- [x] The LLM-judge prompt isolates the content under review (delimited/structured) so a response can't talk the judge into "pass"; judge failure ⇒ "unscored", never "passed".
- [x] Guardrails cannot be disabled by request content.

### 3. AuthN / AuthZ / tenant isolation

Expand Down Expand Up @@ -64,12 +64,13 @@ Each item: assume an attacker is actively trying it. Tick only when a test prove

## Review log

| ID | Date | Area (1–8) | Change / PR | Risk | Severity | Status | Mitigation / notes |
| ------ | ---------- | ---------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| — | 2026-06-27 | — | Initial context docs (no code) | n/a | n/a | n/a | Baseline. |
| SR-001 | 2026-06-27 | 1, 3, 4 | Phase 1 pass-through proxy | Unauthenticated access; key/secret leakage; SSRF via request-controlled URLs | high | mitigated | Bearer Sentinel-key auth required — 401 on missing/invalid (tested). Provider keys read from env via `apiKeyEnv`, never hard-coded, never returned to clients. Provider base URLs come only from the config file, never request input. `authorization`/`x-api-key` redacted in request logs via pino `redact` (configured; an explicit log-assertion test is still TODO — box 1.2 left unticked). |
| SR-002 | 2026-06-27 | 3, 6 | Phase 2 tracing & persistence | Unauthorized trace access; secrets/PII in stored traces | high | mitigated | `GET /traces` gated by a separate `SENTINEL_ADMIN_KEY` — 401 without it (tested), distinct from client keys. Traces are metadata-only (model, provider, tokens, latency, status) — no prompt/response bodies persisted. API keys stored as a SHA-256 hash, never raw. Per-key trace scoping deferred (admin-only for now). |
| SR-003 | 2026-06-27 | 5 | Phase 3 semantic cache | Cross-tenant cache leakage; wrong-answer collisions | high | mitigated | Cache entries are bucketed per-tenant by API-key hash — no cross-tenant hits (tested). The bucket also keys on model/temperature/max_tokens/stream/embed-model, so requests that change the answer never collide; semantic matching happens only within a bucket above a conservative threshold (0.92, configurable). Cache fails open (embed errors → miss). |
| SR-004 | 2026-06-27 | 7 | Phase 4 routing & fallback | Self-inflicted DoS: unbounded retries/fallback amplify load; a slow or hostile provider stalls the event loop; throttle/router as a new untrusted-input path | med | mitigated | Each attempt is bounded by a per-attempt `AbortController` timeout (`REQUEST_TIMEOUT_MS`) so a slow provider can't hang the loop (tested). Retries are capped (`MAX_RETRIES`) and fallback walks a finite, **config-defined** candidate chain — request input never adds providers/URLs (no new SSRF surface). A per-provider token bucket paces outbound calls to each provider's `rpm`, protecting upstreams and avoiding self-induced 429 storms. Terminal 4xx errors fail fast without amplification. Inbound per-client rate-limiting (box 7.1) is still deferred. |
| ID | Date | Area (1–8) | Change / PR | Risk | Severity | Status | Mitigation / notes |
| ------ | ---------- | ---------- | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------- | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| — | 2026-06-27 | — | Initial context docs (no code) | n/a | n/a | n/a | Baseline. |
| SR-001 | 2026-06-27 | 1, 3, 4 | Phase 1 pass-through proxy | Unauthenticated access; key/secret leakage; SSRF via request-controlled URLs | high | mitigated | Bearer Sentinel-key auth required — 401 on missing/invalid (tested). Provider keys read from env via `apiKeyEnv`, never hard-coded, never returned to clients. Provider base URLs come only from the config file, never request input. `authorization`/`x-api-key` redacted in request logs via pino `redact` (configured; an explicit log-assertion test is still TODO — box 1.2 left unticked). |
| SR-002 | 2026-06-27 | 3, 6 | Phase 2 tracing & persistence | Unauthorized trace access; secrets/PII in stored traces | high | mitigated | `GET /traces` gated by a separate `SENTINEL_ADMIN_KEY` — 401 without it (tested), distinct from client keys. Traces are metadata-only (model, provider, tokens, latency, status) — no prompt/response bodies persisted. API keys stored as a SHA-256 hash, never raw. Per-key trace scoping deferred (admin-only for now). |
| SR-003 | 2026-06-27 | 5 | Phase 3 semantic cache | Cross-tenant cache leakage; wrong-answer collisions | high | mitigated | Cache entries are bucketed per-tenant by API-key hash — no cross-tenant hits (tested). The bucket also keys on model/temperature/max_tokens/stream/embed-model, so requests that change the answer never collide; semantic matching happens only within a bucket above a conservative threshold (0.92, configurable). Cache fails open (embed errors → miss). |
| SR-004 | 2026-06-27 | 7 | Phase 4 routing & fallback | Self-inflicted DoS: unbounded retries/fallback amplify load; a slow or hostile provider stalls the event loop; throttle/router as a new untrusted-input path | med | mitigated | Each attempt is bounded by a per-attempt `AbortController` timeout (`REQUEST_TIMEOUT_MS`) so a slow provider can't hang the loop (tested). Retries are capped (`MAX_RETRIES`) and fallback walks a finite, **config-defined** candidate chain — request input never adds providers/URLs (no new SSRF surface). A per-provider token bucket paces outbound calls to each provider's `rpm`, protecting upstreams and avoiding self-induced 429 storms. Terminal 4xx errors fail fast without amplification. Inbound per-client rate-limiting (box 7.1) is still deferred. |
| SR-005 | 2026-06-27 | 2 | Phase 5 verification (guardrails + judge) | Prompt-injection via response content talking the judge into a "pass"; PII/secrets leaking into traces via verdicts; request content disabling guardrails | high | mitigated | Boxes 2.1–2.3 ticked (tested). Guardrails/judge treat prompt+response as **data**, never instructions — only regex/JSON inspection, no eval. The judge prompt wraps the response behind explicit delimiters and labels it untrusted DATA; the verdict is parsed solely from the judge's own JSON output, so an injected `{"score":5}` in the response can't set the score (tested); a judge failure ⇒ "unscored" (null), **never** a pass. Guardrail policy comes only from env/config file, never the request body. Traces persist violation **category codes** (`pii.email`) — never the matched PII/secret value. Inline guardrails fail **closed**: a check that throws blocks. |

> Add a row per high-risk change. Status ∈ {open, mitigated, accepted}. Severity ∈ {low, med, high, critical}.
Loading
Loading