Skip to content

feat: retry rate-limited requests and share pagination logic#218

Merged
pchuri merged 2 commits into
mainfrom
feat/rate-limit-retry
Jul 19, 2026
Merged

feat: retry rate-limited requests and share pagination logic#218
pchuri merged 2 commits into
mainfrom
feat/rate-limit-retry

Conversation

@pchuri

@pchuri pchuri commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Summary

Confluence Cloud rate-limits aggressively, and bulk operations (copy --recursive, getAll* listings) previously hard-failed on the first 429/503, potentially leaving a partially-copied tree. This PR adds transparent retry handling and deduplicates the pagination scaffolding that was copy-pasted across four methods.

Rate-limit retry

  • Retry throttled/unavailable responses in the shared axios response interceptor, so every command benefits (including api and recursive copy).
  • 429 responses are retried for all methods (the server rejected the request before processing). 503 responses are retried only for read methods (GET/HEAD/OPTIONS), since a proxy can return 503 after the backend already applied a write — replaying a POST/PUT could duplicate content or mask a successful update with a version conflict.
  • Honor the Retry-After header (both seconds and HTTP-date formats), falling back to exponential backoff (1s → 2s → 4s), max 3 attempts, 60s delay cap.
  • Stream bodies (attachment uploads) are excluded — a consumed stream cannot be replayed.

Shared paginator

  • New collectPaginated(fetchPage, { maxResults, pageSize, start }) helper replaces the four duplicated while (hasNext) loops in getSpaces, getAllComments, getAllAttachments, and getAllProperties.
  • Behavior-preserving, with one improvement: the final page now requests only the remaining number of items instead of a full page (the approach getSpaces already used). The existing maxResults: 0 = unlimited quirk is preserved.

Test plan

  • 8 new tests: 429 retry success (read and write), 503 retry success for reads, 503 write NOT retried, gives up after maxRetries (attempt count asserted), 400 not retried, stream body not retried, and retryDelayMs unit coverage (Retry-After seconds/date/backoff/cap).
  • Existing pagination tests (multi-page getSpaces, getAllProperties accumulation) pass unchanged, confirming parity.
  • Full suite: 828 passed, lint clean.

- Retry 429/503 responses in the axios response interceptor,
  honoring Retry-After (seconds or HTTP-date) with exponential
  backoff fallback (max 3 attempts, 60s delay cap)
- Skip retries for stream bodies, which cannot be replayed
- Extract shared collectPaginated helper and use it in getSpaces,
  getAllComments, getAllAttachments, and getAllProperties
- Request only the remaining number of items on the final page
  instead of a full page
@pchuri pchuri self-assigned this Jul 19, 2026
A 503 can be returned by a proxy after the backend already applied a
write, so replaying POST/PUT could create duplicate content or mask a
successful update with a version conflict. 429 responses are rejected
before processing and remain retryable for all methods.
@pchuri

pchuri commented Jul 19, 2026

Copy link
Copy Markdown
Owner Author

Retry policy rationale: why 429 retries all methods but 503 only reads

Two review rounds flagged replay safety of the retry logic, so recording the reasoning here.

503 → retried only for GET/HEAD/OPTIONS. A 503 commonly comes from a gateway/proxy after the backend timed out, so the write may have already been applied. Replaying a POST could duplicate a page/comment, and replaying a versioned PUT that actually succeeded would fail with a misleading version conflict. This was fixed in 9a422e0.

429 → retried for all methods (including writes), intentionally. Rate limiters reject requests before processing — that is their semantic purpose (RFC 6585), and Atlassian's rate-limiting docs explicitly state that 429'd requests were not processed and are safe to retry. Gating 429 retries to reads would also defeat the primary motivation of this PR: copy --recursive fans out POSTs and is exactly what Confluence Cloud throttles with 429. The "proxy returns 429 after committing the write" scenario would require a rate limiter sitting behind the backend, which is not a realistic deployment.

If a real environment surfaces post-commit 429s, the right follow-up would be an idempotency guard, not disabling write retries.

@pchuri
pchuri merged commit 281b9a6 into main Jul 19, 2026
6 checks passed
github-actions Bot pushed a commit that referenced this pull request Jul 19, 2026
# [2.18.0](v2.17.1...v2.18.0) (2026-07-19)

### Features

* retry rate-limited requests and share pagination logic ([#218](#218)) ([281b9a6](281b9a6))
@github-actions

Copy link
Copy Markdown

🎉 This PR is included in version 2.18.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant