Skip to content

Commit 5438466

Browse files
ganglerigangleri
committed
wip
1 parent 6469007 commit 5438466

44 files changed

Lines changed: 2073 additions & 7381 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 113 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,37 @@
11
# httpsuite
22

3-
Run `.http` files as API tests, for local development and CI. A single binary
4-
with no runtime dependencies beyond the Go standard library and two uRadical
5-
modules ([httpparser](../httpparser) for parsing, [webbrowser](../webbrowser)
6-
for the forthcoming `--ui`).
3+
Run `.http` files as API tests, for local development and CI.
74

8-
httpsuite executes the requests in your `.http` files, checks any `# @expect`
9-
assertions you attach, and reports pass/fail with the right output for where
5+
httpsuite executes the requests in your `.http` files, runs any JetBrains HTTP
6+
Client **pre-request and response-handler scripts** attached to them, checks any
7+
`# @expect` assertions, and reports pass/fail with the right output for where
108
it's running — coloured and aligned on a terminal, plain and diffable in CI.
119

10+
It ships as a **single static binary** with **no CGO and no JVM**. The JavaScript
11+
for scripts is executed by the pure-Go [goja](https://github.com/dop251/goja)
12+
engine, so the same binary runs unchanged on macOS, Windows, and any Linux
13+
distro — including Alpine/musl and `FROM scratch`/distroless containers. That
14+
footprint is the main reason to reach for httpsuite over JetBrains' own `ijhttp`
15+
CLI, which requires a JDK.
16+
1217
## Install
1318

1419
```sh
1520
go install github.com/uradical/httpsuite@latest
1621
```
1722

18-
Requires Go 1.21 or later.
23+
Requires Go 1.26 or later (see `go.mod`). Released binaries are static and carry
24+
no toolchain requirement.
1925

2026
## Usage
2127

2228
```
23-
httpsuite [--var key=value]... [--ui] [path]
29+
httpsuite [--var key=value]... [--ui] [--version] [path]
2430
```
2531

2632
- `path` is optional and defaults to the current directory.
2733
- `--var key=value` overrides a `{{placeholder}}`. Repeatable.
34+
- `--version` prints version information and exits.
2835
- `--ui` is reserved for a results UI and currently prints a notice and exits.
2936

3037
```sh
@@ -34,8 +41,8 @@ httpsuite ./api/users.http # run a single file
3441
httpsuite --var token=abc ./api # override {{token}}
3542
```
3643

37-
Exit code is **0** when everything passes, **1** when any request or assertion
38-
fails, and **2** on a usage or setup error.
44+
Exit code is **0** when everything passes, **1** when any request, assertion, or
45+
script test fails, and **2** on a usage or setup error.
3946

4047
## Discovery
4148

@@ -78,15 +85,85 @@ execution time. Resolution order, from lowest to highest precedence:
7885
1. File-level `@key = value` declarations inside the `.http` file
7986
2. OS environment variables
8087
3. `--var key=value` flags
88+
4. Values set by scripts via `client.global.set(...)` (see below)
8189

8290
A placeholder that is still unresolved at execution time fails that request.
8391

92+
## Scripts
93+
94+
httpsuite runs the JetBrains HTTP Client scripting subset. Scripts are plain
95+
JavaScript (ES2015+; goja provides the language). Two kinds are supported:
96+
97+
- **Pre-request** scripts run before the request is sent, introduced with `<`:
98+
99+
```http
100+
< {% request.variables.set("nonce", Date.now().toString()) %}
101+
POST {{base}}/orders
102+
103+
### or from an external file, resolved relative to the .http file
104+
< ./scripts/sign.js
105+
GET {{base}}/secure
106+
```
107+
108+
- **Response-handler** scripts run after the response arrives, introduced with
109+
`>`. They typically register tests:
110+
111+
```http
112+
GET {{base}}/users/1
113+
114+
> {%
115+
client.test("status is 200", () => {
116+
client.assert(response.status === 200, "got " + response.status);
117+
});
118+
client.global.set("userName", response.body.name); // reuse in later requests
119+
%}
120+
121+
### or from an external file
122+
GET {{base}}/users/2
123+
> ./scripts/check-user.js
124+
```
125+
126+
Script body statements run first; then every `client.test(...)` block runs in
127+
registration order (matching JetBrains). A failing `client.assert` marks that
128+
one test failed and the remaining tests still run.
129+
130+
### Available API
131+
132+
| Object | Highlights |
133+
|------------|------------|
134+
| `client` | `test(name, fn)`, `assert(cond, msg)`, `log(...)`, `exit()`, and `client.global` (`set`/`get`/`isEmpty`/`clear`/`clearAll`, plus `global.headers.set`/`clear` to inject headers into later requests). `client.global` persists across every request in a run. |
135+
| `response` | `status`, `contentType.{mimeType,charset}`, `headers.valueOf(name)` / `valuesOf(name)`, `cookies()` / `cookiesByName(name)`, and `body` — a parsed object for JSON, a DOM `Document` for XML/HTML, or a raw string otherwise (based on `Content-Type`). |
136+
| `request` | Pre-request: `method`, `url`/`body` (`getRaw`/`tryGetSubstituted`), `headers`, `environment.get(name)`, `variables.get`/`set`. Response: `method`, `url()`, `body()`, `headers`. |
137+
| `crypto`† | `crypto.hmac.{sha256,sha384,sha512,sha3}`, and a synchronous `crypto.subtle` (`digest`, `generateKey`, `importKey`, `exportKey`, `sign`, `verify`) for RSA-PSS / ECDSA / HMAC. |
138+
| `jwt`† | `sign(payload, secret, {algorithm})`, `verify(token, secret)`, `decode(token)` — HS/RS/PS/ES 256/384/512. |
139+
| helpers | `jsonPath(body, "$.a.b[0]")`, `xpath(doc, "//tag")`, `console.log`, `btoa`/`atob`, `URLSearchParams`, `structuredClone`, `string2byteArray`, `DOMParser`/`XMLSerializer`. |
140+
141+
The `client`, `response`, `request`, and DOM/`jsonPath`/base64 surfaces are
142+
**verified against the real JetBrains HTTP Client CLI** (`ijhttp`) — see
143+
[`conformance/`](conformance), which diffs both tools' JUnit output test-by-test
144+
and reports CONFORMANT.
145+
146+
† `crypto` and `jwt` are **httpsuite extensions**: they are not defined in stock
147+
`ijhttp`, so a script that relies on them will run in httpsuite but not in
148+
GoLand/ijhttp. (`xpath`, `URLSearchParams`, `structuredClone`, and
149+
`string2byteArray` are provided by httpsuite but not yet verified against
150+
ijhttp.)
151+
152+
A request **fails** if there is a network error or timeout, any `# @expect`
153+
assertion fails, any `client.test` has a failing `client.assert`, or a
154+
pre-request or response script throws an uncaught error. A pre-request error
155+
skips the HTTP call.
156+
157+
**Not supported:** `async`/`await`/`Promise` resolution (scripts must be
158+
synchronous), ES module `import`/`export`, and streaming response handlers.
159+
Shell execution (`exec`/`execSync`/`spawn`) is deliberately blocked.
160+
84161
## Assertions (`# @expect`)
85162

86-
Attach checks to a request with `# @expect` comments. All assertions on a
87-
request are evaluated — evaluation never stops at the first failure — and a
88-
request fails if **any** assertion fails. A request with no assertions passes as
89-
long as it returns any HTTP response.
163+
For simple checks you don't need a script — attach `# @expect` comments to a
164+
request. All assertions on a request are evaluated (evaluation never stops at
165+
the first failure) and a request fails if **any** fails. A request with no
166+
assertions and no failing tests passes as long as it returns any HTTP response.
90167

91168
```http
92169
# @name createUser
@@ -154,23 +231,26 @@ Date format tokens: `YYYY MM DD` (date) and `HH mm ss` (time).
154231
# @expect duration < 500 # milliseconds elapsed for the request
155232
```
156233

157-
> JetBrains `> {% ... %}` response-handler scripts are not executed. When one is
158-
> found, httpsuite prints a warning and continues — use `# @expect` instead.
159-
160234
## Output
161235

162236
On a terminal, results are coloured (green pass, red fail, bold summary) with
163-
`✓`/`✗` marks on assertion lines. With no TTY (CI), the same layout is printed
164-
without ANSI codes and with `PASS`/`FAIL` words instead of glyphs:
237+
`✓`/`✗` marks on assertion and test lines, `⚠` for script logs. With no TTY
238+
(CI), the same layout is printed without ANSI codes and with `PASS`/`FAIL`/`LOG`
239+
words instead of glyphs:
165240

166241
```
167-
FAIL POST https://api.example.com/users 422 89ms
168-
PASS status 2xx
169-
FAIL status == 201 expected 201, got 422
170-
FAIL body.id exists field not present in response
171-
PASS body.name == "Alan"
172-
173-
1 requests 0 passed 1 failed 89ms
242+
PASS POST https://api.example.com/login 200 89ms
243+
PASS Status is 200
244+
PASS Token extracted
245+
FAIL GET https://api.example.com/profile 401 12ms
246+
PASS Request executed
247+
FAIL Profile returned expected status 200, got 401
248+
LOG checking auth header value "Bearer undefined"
249+
250+
SCRIPT ERROR GET https://api.example.com/orders
251+
TypeError: Cannot read property 'id' of undefined
252+
253+
3 requests 1 passed 2 failed 190ms
174254
```
175255

176256
## Development
@@ -179,3 +259,10 @@ FAIL POST https://api.example.com/users 422 89ms
179259
go test ./... # unit tests
180260
go vet ./...
181261
```
262+
263+
Building a release binary is CGO-free and cross-compiles to every supported
264+
target:
265+
266+
```sh
267+
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags "-s -w" .
268+
```

‎conformance/README.md‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
# httpsuite ↔ ijhttp conformance harness
2+
3+
This harness answers one question empirically: **do httpsuite and the JetBrains
4+
HTTP Client CLI (`ijhttp`) produce the same pass/fail results for the same
5+
`.http` files?**
6+
7+
It runs every `.http` file under [`tests/`](tests) through both tools, converts
8+
each tool's output to JUnit XML, and diffs the two reports **by test name**. A
9+
green run means httpsuite matches JetBrains on that suite.
10+
11+
## Verified results
12+
13+
Run against real `ijhttp` (IntelliJ HTTP Client CLI, GraalJS engine, JDK 25):
14+
15+
- **`tests/` — CONFORMANT.** All tests agree between httpsuite and ijhttp across:
16+
- status codes, JSON body access, cross-request `client.global` state
17+
- `response.headers.valueOf`/`valuesOf`, `response.contentType`
18+
- ES language features (regex lookahead, named capture groups, per-iteration
19+
`let`, destructuring, base64 `btoa`/`atob`)
20+
- **pre-request scripts** (`< {% %}`) and `request.variables.set`
21+
- `response.cookies()` / `cookiesByName()`
22+
- **XML `response.body` as a DOM** (`getElementsByTagName`, …)
23+
- the `request` object inside a response handler (`request.method`, `body()`)
24+
- the `jsonPath(body, "$.path")` helper
25+
- **`extensions/` — httpsuite-only.** `crypto.*` and `jwt` are **not** defined in
26+
stock ijhttp (`ReferenceError: crypto is not defined`), verified against the
27+
GraalJS engine. These are additive httpsuite features, kept out of the parity
28+
verdict in [`extensions/`](extensions).
29+
30+
Bottom line: across the JetBrains HTTP Client scripting surface — including the
31+
DOM and `jsonPath`, which turned out to be shared, not httpsuite-specific —
32+
httpsuite is a faithful match. It adds `crypto`/`jwt` on top.
33+
34+
> ijhttp emits per-request lifecycle testcases (`Response`, `Response Handler`,
35+
> `Pre-request Handler`) that have no httpsuite equivalent; the harness filters
36+
> these so only real `client.test` assertions are compared.
37+
38+
## How it works
39+
40+
1. Builds `httpsuite` and the bundled example server, and starts the server on
41+
`127.0.0.1:8799` (the `@base` hard-coded in the test files).
42+
2. For each `tests/*.http`:
43+
- runs `httpsuite --report <xml> <file>` and parses the JUnit report;
44+
- runs `ijhttp <file> --report` and parses its JUnit report;
45+
- compares the two by test name.
46+
3. Prints per-test agreement and a final verdict, exiting non-zero on any
47+
divergence.
48+
49+
Both tools emit JUnit XML natively, so the comparison is structured, not
50+
text-scraping: httpsuite grew a `--report` flag for exactly this, matching
51+
`ijhttp`'s `--report`.
52+
53+
## Running
54+
55+
```sh
56+
# from the httpsuite module root
57+
go run ./conformance
58+
```
59+
60+
Without `ijhttp` on `PATH`, the harness runs the httpsuite side only and prints
61+
its results, then notes that the comparison was skipped (exit 0). Once `ijhttp`
62+
is available it performs the full side-by-side diff.
63+
64+
Flags:
65+
66+
| Flag | Default | Purpose |
67+
|---------------|------------------------|---------|
68+
| `--dir` | `conformance/tests` | directory of `.http` files |
69+
| `--ijhttp` | `ijhttp` on `PATH` | path to the ijhttp binary or a wrapper; empty skips the comparison |
70+
| `--addr` | `127.0.0.1:8799` | example-server address (must match `@base` in the files) |
71+
| `--httpsuite` | *(build from source)* | path to a prebuilt httpsuite binary |
72+
| `--keep` | `false` | keep the temp work dir (reports, binaries) for inspection |
73+
74+
## Getting `ijhttp`
75+
76+
`ijhttp` requires a JDK (25+). Two common ways to provide it:
77+
78+
**Homebrew (macOS/Linux)** — simplest; runs natively and can reach the local
79+
server directly:
80+
81+
```sh
82+
brew install ijhttp
83+
go run ./conformance # ijhttp is auto-detected on PATH
84+
```
85+
86+
**Docker** — via JetBrains' published image. Wrap it so it presents the
87+
`ijhttp <file> --report` interface the harness expects, then:
88+
89+
```sh
90+
go run ./conformance --ijhttp ./ci/ijhttp-docker.sh
91+
```
92+
93+
Note that in a container the host server at `127.0.0.1:8799` may not be
94+
reachable without host networking (`--network host` on Linux, or
95+
`host.docker.internal` on macOS/Windows — which would also require the test
96+
files' `@base` to point there). The Homebrew path avoids this entirely.
97+
98+
## Interpreting the output
99+
100+
- `✓ name httpsuite=PASS ijhttp=PASS` — the tools agree. Good.
101+
- `✗ name ... <-- MISMATCH` — same test, different verdict. This is the
102+
signal that matters: a likely **httpsuite bug** (or a genuine behavioural
103+
difference worth investigating).
104+
- `! name ... <-- HTTPSUITE ONLY / IJHTTP ONLY` — a test produced by one tool
105+
and not the other. Usually a **naming/coverage** difference (e.g. httpsuite
106+
exposes an API JetBrains doesn't, or a script errored in one engine so its
107+
tests never registered), not necessarily a bug — but worth a look.
108+
109+
## Writing conformance files
110+
111+
Keep the test files to the **JetBrains HTTP Client API that both tools support**
112+
so a green run is meaningful:
113+
114+
- Register checks with `client.test(name, fn)` + `client.assert(...)` — **not**
115+
httpsuite's `# @expect` (which ijhttp doesn't understand).
116+
- Use `@base = http://127.0.0.1:8799` and `{{base}}` — in-file variables work in
117+
both tools, and the hard-coded port matches the example server.
118+
- Prefer deterministic assertions (known response fields, known crypto vectors).
119+
120+
The current suite covers status codes and JSON body access, cross-request
121+
`client.global` state, response headers/content type, a spread of ES language
122+
features (regex lookahead, named groups, per-iteration `let`, destructuring,
123+
base64), and HMAC test vectors. Add files as coverage grows; each is compared
124+
independently.
125+
126+
## CI
127+
128+
The harness exits non-zero on any divergence (or run failure), so it drops
129+
straight into a pipeline step once `ijhttp` is provided:
130+
131+
```sh
132+
go run ./conformance --ijhttp "$(command -v ijhttp)"
133+
```

0 commit comments

Comments
 (0)