Fix/challenge - #217
Conversation
📝 WalkthroughWalkthroughAdds Anubis challenge detection and solving with portable SHA-256 proof-of-work support. Integrates cookie handling and retries into page fetching. Adds typed request errors, public exports, Yarn updates, and comprehensive tests. ChangesAnubis integration
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant fetchPage
participant passChallenge
participant AnubisEndpoint
fetchPage->>passChallenge: Detect challenge and pass request data
passChallenge->>AnubisEndpoint: Submit proof-of-work or metarefresh exchange
AnubisEndpoint-->>passChallenge: Return authentication cookie or redirect
passChallenge-->>fetchPage: Return challenge result
fetchPage->>AnubisEndpoint: Retry page request with authentication
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
package.jsonOops! Something went wrong! :( ESLint: 10.6.0 TypeError [ERR_IMPORT_ATTRIBUTE_MISSING]: Module "file:///.eslintrc.json?mtime=1785785496802" needs an import attribute of "type: json" src/anubis/challenge.tsOops! Something went wrong! :( ESLint: 10.6.0 TypeError [ERR_IMPORT_ATTRIBUTE_MISSING]: Module "file:///.eslintrc.json?mtime=1785785496802" needs an import attribute of "type: json" src/anubis/client.tsOops! Something went wrong! :( ESLint: 10.6.0 TypeError [ERR_IMPORT_ATTRIBUTE_MISSING]: Module "file:///.eslintrc.json?mtime=1785785496802" needs an import attribute of "type: json"
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #217 +/- ##
==========================================
- Coverage 98.71% 98.46% -0.26%
==========================================
Files 34 39 +5
Lines 781 1044 +263
Branches 202 244 +42
==========================================
+ Hits 771 1028 +257
- Misses 10 16 +6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (2)
src/anubis/sha256.ts (1)
23-25: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winHoist the
TextEncoderinstance out ofsha256.
sha256is called once per nonce in the proof-of-work loop. Difficulty 4 needs roughly 65k calls, and difficulty 5 far more. Each call allocates a newTextEncoder. Create one module-level encoder and reuse it.♻️ Proposed refactor
const HEX = '0123456789abcdef'; + +// Reused across calls: the PoW loop hashes tens of thousands of strings. +const encoder = new TextEncoder(); /** SHA-256 digest of `text` (UTF-8) as 32 raw bytes. */ export const sha256 = (text: string): Uint8Array => { - const bytes = new TextEncoder().encode(text); + const bytes = encoder.encode(text); const length = bytes.length;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/anubis/sha256.ts` around lines 23 - 25, Hoist the TextEncoder allocation out of sha256 by creating a module-level encoder, then reuse that instance when encoding text inside sha256. Keep the hashing behavior unchanged while avoiding per-call encoder creation in the proof-of-work loop.src/anubis/challenge.ts (1)
145-170: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winRestrict the metarefresh exchange URL to the challenge origin.
directive.urlis read out of the response body or theRefreshheader.new URL(directive.url, url)accepts an absolute URL to any host. The exchange request then sendsrequestHeadersand thetecharo.lol-anubis-cookie-verificationcookie to that host. Compare the resolved origin with the origin ofurland refuse a cross-origin target.🛡️ Proposed guard
let passUrl: URL; if (directive) { passUrl = new URL(directive.url, url); + // The directive is page-supplied; never hand the verification cookie to + // another origin. + if (passUrl.origin !== new URL(url).origin) { + return null; + } } else {Also applies to: 232-247
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/anubis/challenge.ts` around lines 145 - 170, Update metarefreshPassUrl to validate directive.url after resolving it against url: compare the resulting URL’s origin with the origin of url and return null for cross-origin targets before waiting or returning the exchange URL. Preserve same-origin directive handling and the existing fallback URL construction.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.yarnrc.yml:
- Around line 7-8: Update the npmMinimalAgeGate setting in .yarnrc.yml to use
the default or another non-zero delay, and add a brief comment explaining the
rationale if configuring it explicitly; do not leave it set to 0.
In `@package.json`:
- Line 129: Update the packageManager entry to reference a published
`@yarnpkg/cli-dist` release instead of the unavailable yarn@4.18.0 value, ensuring
Corepack can enable it during immutable installs.
In `@src/anubis/challenge.ts`:
- Around line 59-71: Validate the parsed result in parseChallenge before
returning it, requiring rules.algorithm and challenge.id/challenge.randomData to
be strings and rules.difficulty to be a number; return null for missing or
invalid fields. In src/anubis/proof-of-work.ts lines 33-41, ensure the solver
rejects difficulty values that are not non-negative integers before beginning
the search.
- Around line 84-88: Update hidesSetCookie and the passChallenge state handling
to distinguish runtimes that cannot expose Set-Cookie from responses where Node
explicitly exposes no cookies. Only set platformCookieJar when the runtime lacks
getSetCookie; preserve the credential-less fetch behavior when getSetCookie
exists but returns an empty array, including the later fetchPage flow.
In `@src/anubis/client.ts`:
- Around line 52-55: Update the client state around reset() and pass() so each
reset advances a generation counter, and each in-flight pass() captures its
generation before awaiting. Ignore stale results when the exchange resolves,
preventing older requests from restoring cookie or platformCookieJar after
reset(), while preserving normal current-generation behavior.
---
Nitpick comments:
In `@src/anubis/challenge.ts`:
- Around line 145-170: Update metarefreshPassUrl to validate directive.url after
resolving it against url: compare the resulting URL’s origin with the origin of
url and return null for cross-origin targets before waiting or returning the
exchange URL. Preserve same-origin directive handling and the existing fallback
URL construction.
In `@src/anubis/sha256.ts`:
- Around line 23-25: Hoist the TextEncoder allocation out of sha256 by creating
a module-level encoder, then reuse that instance when encoding text inside
sha256. Keep the hashing behavior unchanged while avoiding per-call encoder
creation in the proof-of-work loop.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: e2e05dbb-8dbe-4b7d-95af-91e7fdeafb15
⛔ Files ignored due to path filters (1)
yarn.lockis excluded by!**/yarn.lock,!**/*.lock
📒 Files selected for processing (12)
.yarnrc.ymlpackage.jsonsrc/anubis/challenge.tssrc/anubis/client.tssrc/anubis/index.tssrc/anubis/proof-of-work.tssrc/anubis/sha256.tssrc/errors.tssrc/fetchers/index.tssrc/index.tstests/anubis.test.tstests/fetchers.test.ts
|
|
||
| npmMinimalAgeGate: 0 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
Yarn npmMinimalAgeGate configuration default value
💡 Result:
The current default value for the Yarn npmMinimalAgeGate configuration is 1d (one day) [1][2]. This setting, introduced in Yarn 4.12, restricts the installation of packages to those published to the npm registry at least the specified duration ago, serving as a security measure to mitigate the risk of installing newly published, potentially malicious packages [1][2]. While the default value has undergone adjustments—including a period where it was set to 3 days (4320m) [3]—it is currently established as 1d [1][2]. Users can override this behavior by setting npmMinimalAgeGate to 0 in their .yarnrc.yml file, or by using the --no-time-gate flag during specific installation commands [1][3][2]. Additionally, the npmPreapprovedPackages setting allows for specific exceptions to this rule [1][2].
Citations:
- 1: yarnpkg/berry@b11eb6d
- 2: https://yarnpkg.com/features/security
- 3: feat: set default npmMinimalAgeGate to 4320 minutes (3 days) yarnpkg/berry#7092
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Repository files matching .yarnrc.yml or yarnrc:"
fd -a '(^|/)yarnrc(\.yml|\.yaml)?$' . | sed 's#^\./##' || true
if [ -f .yarnrc.yml ]; then
echo
echo ".yarnrc.yml excerpt:"
nl -ba .yarnrc.yml | sed -n '1,40p'
fi
echo
echo "Package manager declarations:"
if [ -f package.json ]; then
node -e "const p=require('./package.json'); console.log(JSON.stringify({packageManager:p.packageManager,engines:p.engines},null,2))"
fi
echo
echo "Search for npmMinimalAgeGate / no-time-gate:"
rg -n "npmMinimalAgeGate|no-time-gate|time-gate|Minimal.*Age|AgeGate" . || trueRepository: bartholomej/node-csfd-api
Length of output: 275
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Repository files matching yarnrc:"
find . -type f \( -name '.yarnrc.yml' -o -name '.yarnrc.yaml' -o -name 'yarnrc.yml' -o -name 'yarnrc.yaml' \) -print
if [ -f .yarnrc.yml ]; then
echo
echo ".yarnrc.yml excerpt:"
i=1
while IFS= read -r line || [ -n "$line" ]; do
printf '%5d %s\n' "$i" "$line"
[ "$i" -ge 80 ] && break
i=$((i+1))
done < .yarnrc.yml
fi
echo
echo "Package manager declarations:"
if [ -f package.json ]; then
node -e "const p=require('./package.json'); console.log(JSON.stringify({packageManager:p.packageManager,engines:p.engines},null,2))"
fi
echo
echo "Search for npmMinimalAgeGate / no-time-gate:"
grep -RInE 'npmMinimalAgeGate|no-time-gate|time-gate|Minimal.*Age|AgeGate' . || trueRepository: bartholomej/node-csfd-api
Length of output: 584
Keep npmMinimalAgeGate above 0.
This project pins Yarn 4.18.0, where the default npm age-gate delay is active. Setting npmMinimalAgeGate: 0 removes that protection for every install path. Use the default or set a non-zero value with a short comment explaining the rationale.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.yarnrc.yml around lines 7 - 8, Update the npmMinimalAgeGate setting in
.yarnrc.yml to use the default or another non-zero delay, and add a brief
comment explaining the rationale if configuring it explicitly; do not leave it
set to 0.
| }, | ||
| "sideEffects": false, | ||
| "packageManager": "yarn@4.14.1" | ||
| "packageManager": "yarn@4.18.0" |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Confirm the published Yarn version and check CI/workflow pins.
curl -s https://registry.npmjs.org/@yarnpkg/cli-dist | jq -r '.versions | keys | map(select(startswith("4."))) | .[-8:]'
fd -H -t f -e yml -e yaml . .github --exec sh -c 'echo "== $1"; rg -n "yarn|corepack|node-version" "$1" || true' _ {}
rg -n '"__metadata"|version:' yarn.lock | head -20Repository: bartholomej/node-csfd-api
Length of output: 2033
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== packageManager =="
python3 - <<'PY'
import json
from pathlib import Path
p = json.loads(Path("package.json").read_text())
print(p.get("packageManager"))
PY
echo "== yarn.lock metadata/header candidates =="
rg -n '^[#] yarn.lock|^__metadata|^[^#].*:| yarn_version:' yarn.lock | head -40
echo "== lock first 30 lines =="
sed -n '1,35p' yarn.lockRepository: bartholomej/node-csfd-api
Length of output: 1926
🌐 Web query:
Yarn 4.18.0 @yarnpkg/cli-dist npm versions
💡 Result:
As of August 3, 2026, the version 4.18.0 of @yarnpkg/cli was published on GitHub on July 29, 2026 [1][2]. However, the corresponding npm package @yarnpkg/cli-dist does not currently list 4.18.0 as an available version [3][4]. The latest version published to the @yarnpkg/cli-dist npm registry is 4.17.0 [3][4]. While GitHub repositories and npm distribution packages for Yarn Berry (v4+) are linked, there can sometimes be a delay or specific build process requirements for the @yarnpkg/cli-dist package to be updated on npm after a release is tagged on GitHub [1]. Users are advised to check the official npm registry page for updates [3].
Citations:
- 1: https://github.com/yarnpkg/berry/releases/tag/%40yarnpkg/cli/4.18.0
- 2: https://newreleases.io/project/github/yarnpkg/berry/release/@yarnpkg%2Fcli%2F4.18.0
- 3: https://www.npmjs.com/package/@yarnpkg/cli-dist
- 4: https://npmx.dev/package/@yarnpkg/cli-dist
Set packageManager to a published @yarnpkg/cli-dist version.
yarn@4.18.0 is not listed on npm, so Corepack will fail to enable this version during yarn install --immutable. Pin packageManager to an available @yarnpkg/cli-dist release or publish/update the artifact before use.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@package.json` at line 129, Update the packageManager entry to reference a
published `@yarnpkg/cli-dist` release instead of the unavailable yarn@4.18.0
value, ensuring Corepack can enable it during immutable installs.
| const parseChallenge = (html: string): ParsedChallenge | null => { | ||
| const match = html.match( | ||
| /<script id="anubis_challenge" type="application\/json">([\s\S]*?)<\/script>/ | ||
| ); | ||
| if (!match) { | ||
| return null; | ||
| } | ||
| try { | ||
| return JSON.parse(match[1].trim()) as ParsedChallenge; | ||
| } catch { | ||
| return null; | ||
| } | ||
| }; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Server-supplied challenge fields reach the solver without validation. parseChallenge casts arbitrary JSON to ParsedChallenge, so rules.algorithm, rules.difficulty, challenge.id, and challenge.randomData can be missing or of the wrong type. A missing rules throws a TypeError at src/anubis/challenge.ts line 218. A NaN or negative difficulty makes the solver return nonce 0 with a digest that satisfies nothing.
src/anubis/challenge.ts#L59-L71: check the four fields afterJSON.parseand returnnullwhen any field has the wrong type.src/anubis/proof-of-work.ts#L33-L41: reject adifficultythat is not a non-negative integer before the search starts.
📍 Affects 2 files
src/anubis/challenge.ts#L59-L71(this comment)src/anubis/proof-of-work.ts#L33-L41
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/anubis/challenge.ts` around lines 59 - 71, Validate the parsed result in
parseChallenge before returning it, requiring rules.algorithm and
challenge.id/challenge.randomData to be strings and rules.difficulty to be a
number; return null for missing or invalid fields. In
src/anubis/proof-of-work.ts lines 33-41, ensure the solver rejects difficulty
values that are not non-negative integers before beginning the search.
| // Set-Cookie is a forbidden response header outside Node, so seeing none on a | ||
| // response that certainly carried them means the runtime (browser, React | ||
| // Native) is hiding them and keeping the cookies in its own jar instead. | ||
| const hidesSetCookie = (headers: Headers): boolean => | ||
| typeof headers.getSetCookie !== 'function' || headers.getSetCookie().length === 0; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
hidesSetCookie conflates a missing cookie with a hidden cookie.
On Node, getSetCookie exists. If an interstitial arrives without any Set-Cookie header, getSetCookie() returns an empty array and this function reports true. passChallenge then sets platformCookieJar = true, and the client keeps that state. Every later fetchPage call sends credentials: 'include', which removes the credential-less default described in src/fetchers/index.ts lines 119-122. Separate the capability check from the cookie-presence check.
♻️ Proposed change
-const hidesSetCookie = (headers: Headers): boolean =>
- typeof headers.getSetCookie !== 'function' || headers.getSetCookie().length === 0;
+// A runtime without `getSetCookie` cannot expose Set-Cookie at all. A runtime
+// that has it but returned nothing simply got a response without cookies.
+const hidesSetCookie = (headers: Headers): boolean =>
+ typeof headers.getSetCookie !== 'function';Also applies to: 191-209
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/anubis/challenge.ts` around lines 84 - 88, Update hidesSetCookie and the
passChallenge state handling to distinguish runtimes that cannot expose
Set-Cookie from responses where Node explicitly exposes no cookies. Only set
platformCookieJar when the runtime lacks getSetCookie; preserve the
credential-less fetch behavior when getSetCookie exists but returns an empty
array, including the later fetchPage flow.
| reset: () => { | ||
| cookie = null; | ||
| pending = null; | ||
| }, |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
reset() does not stop an in-flight exchange from restoring the cookie.
reset() clears cookie and pending, but the promise created by an earlier pass() call keeps running. When it resolves, lines 76-77 write cookie and platformCookieJar again. A caller that resets after a block then keeps using the cookie it asked to discard. Track a generation counter and ignore results from an older generation.
♻️ Proposed change
let cookie: string | null = null;
let platformCookieJar = false;
let pending: Promise<ChallengeResult | null> | null = null;
+ let generation = 0;
return {
isChallenge: isAnubisChallenge,
getCookie: () => cookie,
setCookie: (value) => {
cookie = value;
},
reset: () => {
cookie = null;
pending = null;
+ generation++;
},
usesPlatformCookieJar: () => platformCookieJar,
pass: async (html, headers, url, requestHeaders) => {
+ const startedAt = generation;
if (!pending) { const result = await pending;
if (!result) {
return false;
}
+ if (startedAt !== generation) {
+ return false;
+ }
cookie = result.cookie;Also applies to: 58-79
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/anubis/client.ts` around lines 52 - 55, Update the client state around
reset() and pass() so each reset advances a generation counter, and each
in-flight pass() captures its generation before awaiting. Ignore stale results
when the exchange resolves, preventing older requests from restoring cookie or
platformCookieJar after reset(), while preserving normal current-generation
behavior.
2 participants
Description
Resolve challenge
Type of change
Summary by CodeRabbit
New Features
Bug Fixes
Chores