Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@
/.vscode/
*.local.php
.DS_Store
/build/
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@ cases: post-deploy smoke checks, scheduled monitoring, CI gates.
- [`docs/claude/roadmap.md`](docs/claude/roadmap.md) — planned work, known limitations.
- [`README.md`](README.md) — user-facing docs (install, CLI, DSL cheat-sheet, JSON schema).

## Status (as of 2026-04-26)
## Status (as of 2026-08-03)

- Version 0.1 released: package skeleton + full source + tests + CI + docs + examples + bundled dev-server.
- All checks green: PHPStan max (0 errors), PHP-CS-Fixer (clean), PHPUnit (81 tests, 210 assertions), examples (26/26 against the bundled dev-server).
- Recent additions: `expectHtmlElement()` (DOM-based HTML assertion), `defaultRetries()` alias, `ResolvableAssertion` interface so assertion args can resolve `{KEY}` variables at runtime (`RedirectAssertion` is the first user), and `head()` / `options()` DSL methods (HEAD uses `CURLOPT_NOBODY`, dev server maps HEAD→GET and answers OPTIONS with `Allow`). See `docs/claude/architecture.md` and `roadmap.md`.
- Latest tag: v0.4.0; v0.5.0 prepared (HTML report + payload snapshots + test source locations).
- All checks green: PHPStan max (0 errors), PHP-CS-Fixer (clean), PHPUnit (105 tests, 290 assertions), examples (26/26 against the bundled dev-server).
- Recent additions: `HtmlReporter` (`--output-html=FILE`) — self-contained HTML report with filters, group tallies, failure details, request/response payloads (+ cURL), per-test source `file:line` and session chain context (template: `src/Reporting/templates/report.html`); JSON schema v3 (`request`/`response`/`source` blocks, `meta.base_path`, `meta.payloads`; `--no-payloads`, `--payload-limit=`), `expectHtmlElement()` (DOM-based HTML assertion), `defaultRetries()` alias, `ResolvableAssertion` interface so assertion args can resolve `{KEY}` variables at runtime (`RedirectAssertion` is the first user), and `head()` / `options()` DSL methods (HEAD uses `CURLOPT_NOBODY`, dev server maps HEAD→GET and answers OPTIONS with `Allow`). See `docs/claude/architecture.md` and `roadmap.md`.

## ⚠️ Keep this documentation up to date

Expand Down
98 changes: 88 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Fast, simple and extensible HTTP smoke-testing toolkit for PHP 8.4+.

A fluent DSL for declaring HTTP smoke tests, parallel execution, capture chains
between requests, cookie-shared session flows, pluggable variable sources
(`.env`, JSON, OS env, your own), pluggable reporters (console, JSON, Markdown,
(`.env`, JSON, OS env, your own), pluggable reporters (console, JSON, Markdown, HTML,
GitHub Actions step summary, your own), and a small DI container so any service
can be swapped without forking the package.

Expand All @@ -28,8 +28,9 @@ vendor/bin/http-smoke dev
`{@hash}` in subsequent URLs / bodies / headers.
- **Variable substitution** — `{ENV_VAR}` placeholders resolved from `.env`,
`smokeHttp.json`, OS env, CLI overrides — or any custom `VariableSourceInterface`.
- **Pluggable reporters** — console, JSON (canonical artefact), Markdown, GitHub
step summary; add your own by implementing `ReporterInterface`.
- **Pluggable reporters** — console, JSON (canonical artefact), Markdown, standalone
HTML (browsable, with filters and chain context), GitHub step summary; add your
own by implementing `ReporterInterface`.
- **Pluggable HTTP client** — default is curl_multi; implement
`HttpClientInterface` to swap in a mock or alternative backend.
- **Retry** — per-test or per-group retry on any failure (useful for
Expand Down Expand Up @@ -115,6 +116,9 @@ http-smoke <environment> [options]
--group=NAME Only run group(s); supports wildcards (api.*)
--output=FILE Write Markdown report
--output-json=FILE Write canonical JSON report
--output-html=FILE Write standalone HTML report
--no-payloads Omit request/response payloads
--payload-limit=BYTES Truncate each payload body (default: 16384)
--no-console Suppress console output
--no-github-summary Skip GITHUB_STEP_SUMMARY
--verbose, -v Show full request/response detail
Expand Down Expand Up @@ -232,10 +236,11 @@ use Stromcom\HttpSmoke\Container\Container;
use Stromcom\HttpSmoke\Variable\Source\ArraySource;

return static function (SmokeConfig $config): void {
$config->configDir = __DIR__ . '/tests/SmokeHttp';
$config->concurrency = 10;
$config->jsonOutputPath = __DIR__ . '/build/smoke.json';
$config->configDir = __DIR__ . '/tests/SmokeHttp';
$config->concurrency = 10;
$config->jsonOutputPath = __DIR__ . '/build/smoke.json';
$config->markdownOutputPath = __DIR__ . '/build/smoke.md';
$config->htmlOutputPath = __DIR__ . '/build/smoke.html';

// Plug in additional variable sources
$config->extraVariableSources[] = new ArraySource([
Expand Down Expand Up @@ -282,17 +287,19 @@ container via `$config->configureContainer`.

## JSON report schema

The JSON report is the canonical machine-readable artefact. Markdown and GitHub
step-summary outputs are derived from it.
The JSON report is the canonical machine-readable artefact. Markdown, HTML and
GitHub step-summary outputs are derived from it.

```json
{
"meta": {
"schema_version": 2,
"schema_version": 3,
"environment": "prod",
"generated_at": "2026-04-26T12:00:00+00:00",
"duration_s": 4.521,
"concurrency": 10,
"base_path": "/var/www/app",
"payloads": true,
"summary": { "total": 15, "passed": 13, "failed": 1, "skipped": 1, "success": false }
},
"groups": [
Expand All @@ -310,9 +317,33 @@ step-summary outputs are derived from it.
"attempts": 1,
"total_duration_ms": 234,
"session": { "label": "user lifecycle" },
"source": {
"file": "tests/SmokeHttp/api/users.php",
"absolute_path": "/var/www/app/tests/SmokeHttp/api/users.php",
"line": 24
},
"failures": ["Expected status 201, got 500"],
"skip_reason": null,
"chain_context": [/* preceding session steps */]
"chain_context": [/* preceding session steps */],
"request": {
"method": "POST",
"url": "https://example.com/api/users/",
"query": null,
"headers": { "Authorization": "«redacted»", "Content-Type": "application/json" },
"body_encoding": "json",
"body": { "text": "{\"name\":\"Ada\"}", "size_bytes": 14, "truncated": false, "binary": false },
"timeout_s": 10,
"cookie_jar": true,
"insecure_tls": false,
"user_agent": "StromcomSmokeTest/1.0"
},
"response": {
"status_code": 500,
"headers": { "content-type": "application/json" },
"content_type": "application/json",
"body": { "text": "{\"error\":\"boom\"}", "size_bytes": 16, "truncated": false, "binary": false },
"transport_error": null
}
}
]
}
Expand All @@ -322,6 +353,53 @@ step-summary outputs are derived from it.

---

## HTML report

```bash
vendor/bin/http-smoke staging --output-html=build/smoke.html
```

One self-contained file — no CDN, no assets, no server. Open it in a browser or
publish it as a CI artefact. It embeds the canonical JSON and renders:

- summary cards + pass/fail/skip ratio bar, environment / duration / concurrency meta,
- groups with per-group tallies; failed tests are expanded by default,
- per-test detail: method, URL, HTTP code, duration, attempts (total time incl. retries),
- all failure messages verbatim, skip reasons,
- **payloads** — the request exactly as sent (headers incl. the implicit
`Content-Type`, query parameters, body in its wire encoding) and the response
(headers, pretty-printed body), plus a ready-to-run **cURL** reproduction,
- **source** — the definition file and line of each test
(`tests/SmokeHttp/api/users.php:24`), as plain selectable text,
- **chain context** for session tests — every preceding step of the session
(method, status, label, URL, duration) with the failing step highlighted,
- **filters**: full-text search (label / URL / failure message / payload / path),
status (all / failed / passed / skipped), group, definition file, retried-only,
sessions-only, plus expand/collapse all and “show payloads”.

Dark mode follows the OS setting. The layout logic lives in
`src/Reporting/templates/report.html`; `HtmlReporter` only injects the JSON.

### Payloads and secrets

Payloads land in both the JSON and the HTML report, so treat those artefacts as
sensitive. Sensitive headers (`Authorization`, `Cookie`, `Set-Cookie`,
`X-Api-Key`, …) are replaced with `«redacted»`, bodies are truncated at 16 kB and
binary bodies are reported as size only.

```bash
vendor/bin/http-smoke prod --output-html=report.html --no-payloads
vendor/bin/http-smoke prod --output-html=report.html --payload-limit=65536
```

```php
$config->includePayloads = false; // drop request/response blocks entirely
$config->payloadMaxBytes = 65536; // per-body cap
$config->redactedHeaders = ['authorization', 'x-tenant-secret'];
```

---

## Examples

See [`examples/`](./examples) for ready-to-run test suites and a sample
Expand Down
11 changes: 7 additions & 4 deletions docs/claude/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,20 @@ CLI input → Console → Config → Discovery → Definition
Result → Report → Reporters
(Console / Json / Markdown / GitHub)
(Console / Json / Markdown / Html / GitHub)
```

## Domain folders

| Folder | Purpose |
|---|---|
| `Definition/` | Fluent DSL: `Suite`, `GroupBuilder`, `RequestBuilder`, immutable `TestCase`, `GroupConfig`. Builds the test plan; nothing here knows how requests are sent. |
| `Definition/` | Fluent DSL: `Suite`, `GroupBuilder`, `RequestBuilder`, immutable `TestCase`, `GroupConfig`, `SourceLocation` (definition file+line of each test, captured via `debug_backtrace` in `GroupBuilder::startRequest()`). Builds the test plan; nothing here knows how requests are sent. |
| `Assertion/` | `AssertionInterface` + concrete impls (Status, Json, JsonPath, JsonHasKeys, BodyContains, HeaderContains, Redirect, HtmlElement, Callback). Each `evaluate(Response)` returns `null` (pass) or a failure message. `ResolvableAssertion` is an opt-in sub-interface for assertions whose args contain `{KEY}` placeholders — `CaseTranslator::resolveAssertions()` rebuilds them with the runtime `VariableResolver` before `evaluate()` is called. |
| `Capture/` | `CaptureInterface` (JsonPath, Header) + `CaptureStore` (runtime `{@name}` substitution). |
| `Variable/` | `VariableResolver` + `VariableSourceInterface` (Array, EnvFile, JsonFile, Getenv). Layered, last-added-wins. Throws `VariableNotFoundException` for unresolved `{KEY}`. |
| `Http/` | `HttpClientInterface` + immutable `Request`/`Response` VO + `Curl\CurlMultiClient` (default; parallel via `curl_multi_*`, single via `curl_exec`, cookie jar support). |
| `Execution/` | `Runner` (orchestrator), `Result`, `Report`, `CaseTranslator` (translates `TestCase` → `Request`, applying variables + captures). |
| `Reporting/` | `ReporterInterface` (`onStart`/`onResult`/`onEnd`) + `Console`, `Json`, `Markdown`, `GithubSummary`, `Null`. JSON is canonical; Markdown + GitHub summary derive from it. |
| `Execution/` | `Runner` (orchestrator), `Result` (carries the `TestCase`, the `Response` **and the `Request` actually sent**), `Report`, `CaseTranslator` (translates `TestCase` → `Request`, applying variables + captures). |
| `Reporting/` | `ReporterInterface` (`onStart`/`onResult`/`onEnd`) + `Console`, `Json`, `Markdown`, `Html`, `GithubSummary`, `Null`. JSON is canonical; Markdown, HTML + GitHub summary derive from it. `templates/report.html` is the HTML shell (inline CSS/JS, no external assets). `Support\PayloadFormatter` snapshots request/response payloads (header redaction, body truncation, binary detection). |
| `Discovery/` | `ConfigDiscovery` — recursive `*.php` walk + filename filter. Each definition file returns `Closure(Suite): void`. |
| `Config/` | `SmokeConfig` (root config DTO) + `SmokeConfigLoader` (loads `smoke.config.php`). |
| `Container/` | Lightweight PSR-11 container + `ServiceFactory::build()` wires everything. `getTyped(class)` for type-narrowed retrieval. |
Expand All @@ -72,6 +72,9 @@ via `smoke.config.php` (`extraVariableSources[]`, `extraReporters[]`,
- **Circuit breaker**: each `GroupConfig::$maxFailures` — once exceeded within a group, remaining cases in that group are skipped with reason "Circuit breaker: …".
- **HTTP methods**: DSL supports `get` / `post` / `put` / `patch` / `delete` / `head` / `options`. `head()` and `options()` always send no body. In `CurlMultiClient`, `HEAD` is dispatched via `CURLOPT_NOBODY` (otherwise curl waits for a body that never arrives); `OPTIONS` uses `CURLOPT_CUSTOMREQUEST`. Body assertions on `HEAD` will simply see an empty response body — this is intentional, not a bug.
- **JSON report schema** is versioned (`JsonReporter::SCHEMA_VERSION`). Markdown + GitHub-summary reporters consume the JSON via `MarkdownReporter::build($data)` — clean separation, easy to derive other formats.
- **Payload snapshots** (schema v3). `Request` owns the wire encoding (`encodedBody()`, `effectiveHeaders()`, `bodyEncoding()`) — `CurlMultiClient` uses the same methods, so what a report shows is exactly what was sent. `Runner` threads the sent `Request` into `Result`; `JsonReporter` emits `request`/`response` blocks via `PayloadFormatter`. Defaults: sensitive headers → `«redacted»`, bodies truncated at 16 kB, non-UTF-8 bodies reported as size only. Toggles: `--no-payloads` / `--payload-limit=BYTES`, `$config->includePayloads`, `$config->payloadMaxBytes`, `$config->redactedHeaders`. Markdown + GitHub-summary reporters build their JSON with `includePayloads: false` (they don't render payloads).
- **Test source location** (schema v3). `SourceLocation::capture()` walks the backtrace to the first frame outside `src/Definition`, so each `TestCase` knows its definition file+line. `meta.base_path` (= project root, set by `RunCommand`) makes the paths relative for display. Surfaced in JSON (`tests[].source`, including `absolute_path` for tooling), HTML (badge, "Defined in" block as plain text — deliberately no editor deep-link, file filter) and as the dim `at file:line` line under console failures + the Markdown failure details.
- **HTML report** (`HtmlReporter`, `--output-html=FILE`) is a single self-contained file: `src/Reporting/templates/report.html` with `__SMOKE_TITLE__` / `__SMOKE_DATA__` placeholders, the canonical JSON embedded in a `<script type="application/json">` block (escaped with `JSON_HEX_TAG|JSON_HEX_AMP`, so a `</script>` in a label can't break out), and vanilla JS rendering it client-side: summary cards, per-group tallies, expandable test rows (details rendered lazily on first open), failure lists, session/retry/source badges, request+response payload panes with pretty-printed JSON and a cURL reproduction, chain-context timeline, plus live filters (search across labels/URLs/failures/payloads/paths, status, group, definition file, retried-only, sessions-only). Rendering lives in the template's JS — PHP only injects data, so the same template can later back a standalone JSON viewer.

## What NOT to assume

Expand Down
19 changes: 12 additions & 7 deletions docs/claude/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,18 @@

## Planned work

### Next big task: JSON report viewer
The `JsonReporter` produces a canonical, schema-versioned JSON report. The
package needs a dedicated viewer — currently console + markdown are good for
single CI runs, but there's no way to browse historical runs, drill into failed
session chains visually, or compare environments. Open shape: standalone static
HTML+JS, PHP server, or CLI TUI — no commitment yet. (Memory:
`project_future_json_viewer.md`.)
### JSON report viewer — first half shipped
`HtmlReporter` (`--output-html=FILE`) now renders a single run as a
self-contained HTML page: filters, per-group tallies, expandable tests, failure
messages, request/response payloads (+ cURL reproduction), the definition
file:line of each test, and session chain context. It reads the canonical JSON,
so the same template's JS can back a standalone viewer later.

Still open (the original "viewer" ask, memory `project_future_json_viewer.md`):
- browsing **historical** runs (load N JSON files, trend over time),
- comparing environments side by side,
- a drop-in viewer that takes an arbitrary `report.json` (file picker /
drag & drop) instead of being generated by a run.

## Known limitations

Expand Down
3 changes: 3 additions & 0 deletions docs/claude/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ Useful options when iterating:
--no-github-summary # skip writing $GITHUB_STEP_SUMMARY (useful locally)
--output-json=build/report.json # canonical JSON artifact
--output=build/report.md # Markdown report
--output-html=build/report.html # standalone HTML report (open in a browser)
--no-payloads # omit request/response payloads from JSON + HTML
--payload-limit=65536 # per-body cap for payloads (default 16384)
--var=KEY=VALUE # one-off variable override
```

Expand Down
12 changes: 12 additions & 0 deletions src/Config/SmokeConfig.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
use Closure;
use Stromcom\HttpSmoke\Container\Container;
use Stromcom\HttpSmoke\Reporting\ReporterInterface;
use Stromcom\HttpSmoke\Reporting\Support\PayloadFormatter;
use Stromcom\HttpSmoke\Variable\VariableSourceInterface;

final class SmokeConfig
Expand All @@ -33,6 +34,17 @@ final class SmokeConfig

public ?string $markdownOutputPath = null;

public ?string $htmlOutputPath = null;

public ?string $projectRoot = null;

public bool $includePayloads = true;

public int $payloadMaxBytes = PayloadFormatter::DEFAULT_MAX_BYTES;

/** @var list<string> */
public array $redactedHeaders = PayloadFormatter::DEFAULT_REDACTED_HEADERS;

public bool $githubSummary = true;

public bool $consoleReporter = true;
Expand Down
9 changes: 9 additions & 0 deletions src/Console/InputParser.php
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,16 @@ public function parse(array $argv): ParsedInput
$input->noConsole = true;
} elseif ($arg === '--no-github-summary') {
$input->noGithubSummary = true;
} elseif ($arg === '--no-payloads') {
$input->noPayloads = true;
} elseif (str_starts_with($arg, '--payload-limit=')) {
$input->payloadMaxBytes = (int) substr($arg, 16);
} elseif (str_starts_with($arg, '--concurrency=')) {
$input->concurrency = (int) substr($arg, 14);
} elseif (str_starts_with($arg, '--output-json=')) {
$input->jsonOutputPath = substr($arg, 14);
} elseif (str_starts_with($arg, '--output-html=')) {
$input->htmlOutputPath = substr($arg, 14);
} elseif (str_starts_with($arg, '--output=')) {
$input->markdownOutputPath = substr($arg, 9);
} elseif (str_starts_with($arg, '--base-url=')) {
Expand Down Expand Up @@ -80,6 +86,9 @@ public function helpText(string $binary = 'http-smoke'): string
--group=NAME Run only the specified group (supports wildcards: api.*)
--output=FILE Write Markdown report to file
--output-json=FILE Write canonical JSON report to file
--output-html=FILE Write standalone HTML report to file
--no-payloads Omit request/response payloads from JSON + HTML reports
--payload-limit=BYTES Truncate each payload body (default: 16384)
--no-console Suppress console reporter
--no-github-summary Skip writing to GITHUB_STEP_SUMMARY
--verbose, -v Show full request/response details
Expand Down
Loading
Loading