-
Notifications
You must be signed in to change notification settings - Fork 1
Add proof-of-work challenge primitives #5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
69fa80c
25a73dc
58f2a8b
3217bcc
39cb473
b63d6c6
ea45d86
0dc6eac
6ba8a73
875328e
048b12c
0b6d5d4
ee52772
9a032f6
8ee4b04
cb61d0b
4375832
037db6e
49e228c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| # WAF Challenge — JS reference | ||
|
|
||
| Version-locked JavaScript companions to the PHP `Utopia\WAF\Challenge` | ||
| primitives. These are the **single source of truth** for the challenge | ||
| solver used by two consumers: | ||
|
|
||
| - the **edge browser interstitial** (`Utopia\WAF\Challenge\Interstitial` inlines | ||
| this algorithm), and | ||
| - the **web + node SDK retry interceptors**. | ||
|
|
||
| Keeping them here, next to the primitives, means the solver and the | ||
| `DIFFICULTY_DEFAULT_*` constants change together. | ||
|
|
||
| ## Files | ||
|
|
||
| | File | Purpose | | ||
| | --- | --- | | ||
| | `solver.js` | Dependency-free pure-JS SHA-256 + challenge solver. `solve()` (chunked/yielding, for browsers) and `solveSync()` (blocking, for node). Mirrors the PHP `Verifier` leading-zero-bit rule exactly. | | ||
| | `interceptor.js` | Reference SDK retry interceptor: single-flight solve, token caching with a skew-safe deadline, one re-solve on a stale-token 403, never loops. | | ||
|
|
||
| ## Contract | ||
|
|
||
| A client finds the smallest non-negative integer `solution` such that | ||
|
|
||
| ``` | ||
| sha256(nonce + '.' + solution) has >= difficulty leading zero bits | ||
| ``` | ||
|
|
||
| then `POST`s `{ nonce, solution }` to `/v1/waf/challenge` (API) or | ||
| `/__waf/challenge` (edge site) to obtain a clearance token/cookie. | ||
|
|
||
| ## Why pure-JS (not `crypto.subtle`) | ||
|
|
||
| `crypto.subtle.digest` is async and its per-call overhead makes a | ||
| digest-in-a-loop 10–100× slower than a tight synchronous JS implementation for | ||
| this access pattern. The browser solver therefore uses the inline function and | ||
| yields between chunks (`requestAnimationFrame`) to keep the tab responsive. | ||
|
|
||
| ## Verified interop | ||
|
|
||
| `solver.js` is checked against the SHA-256 known-answer vector for `"abc"` and, | ||
| end-to-end, a nonce minted by the PHP `Issuer` is solved here and accepted by the | ||
| PHP `Verifier` — so browser/SDK solutions are valid server-side. | ||
|
|
||
| ## SDK integration (phase 3) | ||
|
|
||
| The generated `appwrite`/`node-appwrite` SDKs wrap their transport with | ||
| `createInterceptor({ endpoint, projectId, doFetch })` and route all requests | ||
| through `interceptor.request(url, init)`. `doFetch` performs one raw request and | ||
| returns `{ status, headers, json }`. Web solves via `solve()`; node via | ||
| `solveSync()`. Ship web first (it exercises both the browser solver and the CORS | ||
| header exposure), then node. | ||
|
|
||
| ## Benchmark gate (phase 1) | ||
|
|
||
| Before tagging `0.1.0`, run `solver.js` on the reference devices (mid-range | ||
| Android, ~3-gen-old iPhone/Safari, low-end laptop). Acceptance: **p95 solve ≤ 4 s** | ||
| on the mid-range Android at `DIFFICULTY_DEFAULT_BROWSER`. Challenge solve time is | ||
| exponentially distributed (p95 ≈ 3× median), so target median ≤ ~1.3 s. Set the | ||
| final `DIFFICULTY_DEFAULT_BROWSER` to the largest value that passes. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| // Reference WAF-challenge retry interceptor for the Appwrite SDKs (web + node). | ||
| // | ||
| // This is the canonical behaviour the generated SDKs wrap around their transport | ||
| // (fetch on web, https/undici on node). It is intentionally transport-agnostic: | ||
| // pass a `doFetch` that performs one request and returns { status, headers, json }. | ||
| // | ||
| // Behaviour (HLD §2.4): | ||
| // 1. Trigger only on a response whose error type is 'waf_challenge_required'. | ||
| // 2. Read the challenge parameters from the CORS-exposed X-Appwrite-WAF-* headers. | ||
| // 3. SINGLE-FLIGHT: N concurrent requests that all 403 share ONE solve, keyed by | ||
| // (endpoint, projectId, nonce-audience); all retry with the minted token. | ||
| // 4. Solve with solver.js, POST /v1/waf/challenge, cache { token, deadline }. | ||
| // deadline = now + expiresIn - 30s (skew margin; prefer expiresIn over the | ||
| // absolute Expires so a wrong local clock cannot shorten/extend the window). | ||
| // 5. Attach X-Appwrite-WAF-Token on subsequent requests while now < deadline. | ||
| // 6. On a challenge 403 DESPITE a cached token (expiry race / IP change): drop | ||
| // the cache, re-solve ONCE, then surface the error. Never loop. | ||
|
|
||
| 'use strict'; | ||
|
|
||
| const { solve, solveSync } = require('./solver.js'); | ||
|
|
||
| const CHALLENGE_ERROR = 'waf_challenge_required'; | ||
| const DEADLINE_SKEW_SECONDS = 30; | ||
|
|
||
| // Solve in a browser (yielding) when available, else synchronously (node). | ||
| function solveChallenge(nonce, difficulty) { | ||
| return (typeof window !== 'undefined') | ||
| ? solve(nonce, difficulty) | ||
| : Promise.resolve(solveSync(nonce, difficulty)); | ||
| } | ||
|
|
||
| function isChallenge(res) { | ||
| if (!res || res.status !== 403) return false; | ||
| const body = res.json || {}; | ||
| return body.type === CHALLENGE_ERROR; | ||
| } | ||
|
|
||
| function readChallenge(res) { | ||
| const h = res.headers || {}; | ||
| const get = (k) => h[k] || h[k.toLowerCase()] || ''; | ||
| return { | ||
| nonce: get('X-Appwrite-WAF-Nonce'), | ||
| difficulty: parseInt(get('X-Appwrite-WAF-Difficulty') || '0', 10) || 0, | ||
| expiresIn: parseInt(get('X-Appwrite-WAF-Expires-In') || '0', 10) || 0, | ||
| }; | ||
| } | ||
|
Comment on lines
+27
to
+47
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
|
||
| /** | ||
| * Create an interceptor bound to one project/endpoint. | ||
| * | ||
| * @param {object} opts | ||
| * @param {string} opts.endpoint e.g. https://cloud.appwrite.io/v1 | ||
| * @param {string} opts.projectId | ||
| * @param {(url:string, init:object)=>Promise<{status:number,headers:object,json:any}>} opts.doFetch | ||
| * @param {()=>number} [opts.now] injectable clock (seconds), for tests | ||
| */ | ||
| function createInterceptor(opts) { | ||
| const now = opts.now || (() => Math.floor(Date.now() / 1000)); | ||
| const key = opts.endpoint + '|' + opts.projectId; // single-flight + cache key | ||
| const cache = new Map(); // key -> { token, deadline } | ||
|
Comment on lines
+60
to
+61
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The spec comment on line 9 says the single-flight/cache key is keyed by |
||
| const inflight = new Map(); // key -> Promise<token> (single-flight) | ||
|
|
||
| async function mintToken(res) { | ||
| // Coalesce concurrent solves for the same audience. | ||
| if (inflight.has(key)) return inflight.get(key); | ||
|
|
||
| const p = (async () => { | ||
| const { nonce, difficulty, expiresIn } = readChallenge(res); | ||
| const solution = await solveChallenge(nonce, difficulty); | ||
| // Solve UNAUTHENTICATED: send only the project header — never the API key | ||
| // or session. Solving a challenge is not a scoped operation, and a | ||
| // narrowly-scoped server key would otherwise be rejected by the scope | ||
| // check. The server route accepts guests here (scope 'public'/'global'). | ||
| const solveRes = await opts.doFetch(opts.endpoint + '/waf/challenge', { | ||
| method: 'POST', | ||
| headers: { 'content-type': 'application/json', 'x-appwrite-project': opts.projectId }, | ||
| body: JSON.stringify({ nonce, solution }), | ||
| }); | ||
| if (solveRes.status < 200 || solveRes.status >= 300) { | ||
| throw new Error('waf challenge solve failed: ' + solveRes.status); | ||
| } | ||
| const token = (solveRes.json || {}).token; | ||
| const ttl = (solveRes.json || {}).expiresIn || expiresIn; | ||
| cache.set(key, { token, deadline: now() + ttl - DEADLINE_SKEW_SECONDS }); | ||
| return token; | ||
| })().finally(() => inflight.delete(key)); | ||
|
|
||
| inflight.set(key, p); | ||
| return p; | ||
| } | ||
|
|
||
| function attachToken(init) { | ||
| const cached = cache.get(key); | ||
| if (cached && now() < cached.deadline) { | ||
| init = init || {}; | ||
| init.headers = Object.assign({}, init.headers, { 'X-Appwrite-WAF-Token': cached.token }); | ||
| } | ||
| return init; | ||
| } | ||
|
|
||
| // Wrap a single request with challenge handling. `alreadyRetried` guards the | ||
| // "re-solve once on a stale-token 403" path so we can never loop. | ||
| async function request(url, init, alreadyRetried) { | ||
| const res = await opts.doFetch(url, attachToken(init)); | ||
| if (!isChallenge(res)) return res; | ||
|
|
||
| if (alreadyRetried) return res; // solved-and-still-challenged: surface it | ||
| if (cache.has(key)) cache.delete(key); // stale token — drop and re-solve once | ||
|
|
||
| await mintToken(res); | ||
| return request(url, init, true); | ||
| } | ||
|
|
||
| return { request, _cache: cache }; | ||
| } | ||
|
|
||
| module.exports = { createInterceptor, isChallenge, readChallenge, solveChallenge }; | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| { | ||
| "_comment": "Generated by reference/train-ml-engine.js — logistic regression over the bot-detection signal vector. Paste into MlEngine::DEFAULT_COEFFICIENTS / DEFAULT_INTERCEPT.", | ||
| "features": [ | ||
| "ipReputation", | ||
| "asnReputation", | ||
| "tlsMismatch", | ||
| "missingHeaders", | ||
| "headless", | ||
| "automationFlags", | ||
| "behavioralRisk" | ||
| ], | ||
| "intercept": -6.52634, | ||
| "coefficients": { | ||
| "ipReputation": 3.562063, | ||
| "asnReputation": 4.67617, | ||
| "tlsMismatch": 1.375458, | ||
| "missingHeaders": 1.896716, | ||
| "headless": 4.576935, | ||
| "automationFlags": 2.71498, | ||
| "behavioralRisk": 4.826937 | ||
| }, | ||
| "metrics": { | ||
| "samples": 20000, | ||
| "accuracy": 1, | ||
| "precision": 1, | ||
| "recall": 1, | ||
| "falsePositiveRate": 0 | ||
| }, | ||
| "canonicalCases": { | ||
| "clean human": { | ||
| "p": 0.0015, | ||
| "tier": "allow" | ||
| }, | ||
| "one missing header": { | ||
| "p": 0.0027, | ||
| "tier": "allow" | ||
| }, | ||
| "legit VPN user": { | ||
| "p": 0.011, | ||
| "tier": "allow" | ||
| }, | ||
| "headless bot (no interaction)": { | ||
| "p": 0.9907, | ||
| "tier": "deny" | ||
| }, | ||
| "curl/python (tls mismatch)": { | ||
| "p": 0.9945, | ||
| "tier": "deny" | ||
| }, | ||
| "datacenter abuser": { | ||
| "p": 0.6299, | ||
| "tier": "interactive" | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
premtsd-code/captcha: dev-mainis a personal GitHub repository (not theutopia-phporg) registered as a VCS source withno-api: true. Based on the test imports (Context,Signer,Ip,ClearancefromUtopia\WAF\Challenge\*) and the fact that none of those classes appear in the currentsrc/Challenge/tree, this package provides the entire HMAC signing/verification foundation. Pinning the core cryptographic primitives to a mutable personal fork means any push topremtsd-code/captcha:mainsilently replaces the WAF's secret-handling and token-validation code for all downstream consumers. Additionally,dev-mainwithminimum-stability: stablestill set will causecomposer installto fail outright unless consumers add a stability flag. The dependency should be moved to the officialutopia-phporganisation, tagged as a stable release, and removed from a personal VCS override.Prompt To Fix With AI