Multi-backend image QA with a stable inference contract and release-style gates (GO / REVIEW / NO_GO) in the full pipeline.
This repository is the public slice: versioned docs, a normative JSON contract, and a stdlib-only mock roundtrip you can run in minutes. Batch CLI, evaluation, and CI are part of the full Agentic Testing Framework codebase, which is not published in this repository.
| You are | A backend or platform engineer wiring third-party vision / LLM inference into a batch or agentic QA pipeline |
| The pain | Every provider returns a different JSON shape; automation starts parsing msg, CI drifts, release logic forks per vendor |
| What we publish here | A small, stable DTO—decision, code, msg, optional backend—documented in docs/inference-contract.md |
| Prove your stack | Run the mock server + client (no GPU, no API keys required): |
cd examples/mock_api_roundtrip
python3 mock_server.py # terminal 1
python3 run_client.py # terminal 2 → expect decision + code in JSON| Next step | Link |
|---|---|
| ~3 min walkthrough | docs/integrator-guide.md |
| Field semantics (normative) | docs/inference-contract.md |
| Why the contract exists (article) | docs/articles/2026-05-23-inference-contract-engineering.md |
| CI contract smoke (article + workflow) | docs/articles/2026-06-13-gating-merges-on-inference-contract-ci.md · bash examples/mock_api_roundtrip/ci_smoke.sh |
| Mock → production checklist (article) | docs/articles/2026-06-20-mock-to-production-backend-checklist.md |
Scope: Inference here answers what the model thinks about the asset (
Optimal,Blurry, …). Release gates (GO/REVIEW/NO_GO) live in the evaluation layer of the full framework (not duplicated in this minimal example).
Full framework: Batch CLI, Streamlit demo, full test suite, and CI belong to the complete Agentic Testing Framework pipeline, outside this public slice. This repo stays small and reproducible on purpose.
Naming: All articles and docs here use Agentic Testing Framework as the product name.
| Path | Purpose |
|---|---|
docs/ |
Integrator notes, inference contract, and versioned articles |
examples/mock_api_roundtrip/ |
Mock HTTP server, client, and CI contract smoke (ci_smoke.sh) |
cd examples/mock_api_roundtrip
python3 mock_server.py
# other terminal:
python3 run_client.pySee examples/mock_api_roundtrip/README.md for troubleshooting and optional MOCK_INFER_API_KEY.
| Resource | Location |
|---|---|
| Full framework (batch CLI, eval, CI) | Not in this repo — see public slice vs. full framework |
| Docs index | docs/README.md |
| Integrator guide | docs/integrator-guide.md |
| Inference contract | docs/inference-contract.md |
| Technical articles | docs/articles/ |
| Mock API example + CI smoke | examples/mock_api_roundtrip/ |
| Article — Gating merges on the inference contract | docs/articles/2026-06-13-gating-merges-on-inference-contract-ci.md |
This project is licensed under the MIT License — see LICENSE.