HTML report, payload snapshots and test source locations - #1
Merged
Conversation
HtmlReporter (--output-html=FILE) renders one run as a single self-contained HTML file: summary cards, per-group tallies, expandable tests, failure details, request/response payloads with a cURL reproduction, session chain context and live filters. The canonical JSON is embedded and rendered client-side by the template's JS, so the same template can later back a standalone JSON viewer. JSON schema bumped to v3: - tests[].request / tests[].response — payloads as actually sent/received, produced by Reporting\Support\PayloadFormatter (sensitive headers redacted, bodies truncated at 16 kB, binary bodies reported as size only) - tests[].source — definition file + line of each test - meta.base_path, meta.payloads Request now owns the wire encoding (encodedBody(), effectiveHeaders(), bodyEncoding()) and CurlMultiClient uses those methods, so reports cannot drift from what was sent. Runner threads the sent Request into Result. Definition\SourceLocation captures the file+line of each get()/post()/... call, surfaced in the HTML report (badge, "Defined in" block, file filter), under console failures (at file:line) and in the Markdown failure details. New CLI flags: --output-html=FILE, --no-payloads, --payload-limit=BYTES New config: htmlOutputPath, includePayloads, payloadMaxBytes, redactedHeaders, projectRoot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes since v0.4.0
HTML report —
--output-html=FILEOne self-contained file: no CDN, no assets, no server. Open it in a browser or
publish it as a CI artefact. It embeds the canonical JSON and renders it
client-side:
all failure messages verbatim, skip reasons,
(method, status, label, URL, duration) with the failing step highlighted,
status, group, definition file, retried-only, sessions-only, plus
expand/collapse all and “show payloads”.
Dark mode follows the OS setting. Test details render lazily on first open, so
large suites stay fast. The layout lives in
src/Reporting/templates/report.html—HtmlReporteronly injects the JSON, sothe same template can later back a standalone viewer for any
report.json.Payload snapshots — what actually went over the wire
Each test now carries the request as sent and the response as received:
headers (including the implicit
Content-Type), query parameters, body in itswire encoding, plus a ready-to-run cURL reproduction in the HTML report.
JSON bodies are pretty-printed for display.
Requestowns the encoding (encodedBody(),effectiveHeaders(),bodyEncoding()) andCurlMultiClientuses those same methods — a report cannotdrift from what was really sent.
Secrets:
Authorization,Proxy-Authorization,Cookie,Set-Cookie,X-Api-Key,X-Auth-Token,X-Csrf-Tokenare replaced with«redacted»,bodies are truncated at 16 kB (original size reported) and non-UTF-8 bodies are
reported as size only. Both the JSON and HTML artefacts now contain request and
response bodies — treat them accordingly in CI.
Markdown and GitHub step-summary reports are unaffected — they build their JSON
with payloads off.
Test source locations — find the failing test instantly
Every test records the definition file and line of its
get()/post()/ …call (
Definition\SourceLocation, captured from the backtrace at build time):…/api/users.php:24badge in the test row, a “Defined in” blockin the detail, and a filter by definition file,
at tests/SmokeHttp/api/users.php:24line under each failure,`file:line`in the failure details,tests[].sourcewithfile(relative tometa.base_path),absolute_pathandline.JSON report schema v3
JsonReporter::SCHEMA_VERSIONis now3:meta.base_path— project root, used to shorten source paths,meta.payloads— whether payloads are included in this artefact,tests[].source— definition file + line,tests[].request/tests[].response— payload snapshots (nullwhenpayloads are off, for skipped tests, or when no request was sent).
Consumers of v2 keep working for every previously existing field — v3 only adds
keys.
New CLI flags
New config options (
smoke.config.php)