fix(lotw): retry transient 5xx from LoTW + browser User-Agent#226
Merged
Conversation
LoTW's front end intermittently returns 503 Service Unavailable when it's throttling or under load, and Vercel's serverless egress IPs get throttled far more readily than a self-hosted box on a stable IP. That's why Wavelog — whose LoTW request logic is otherwise equivalent and also has no retry — doesn't hit this, but our cron does. A single 503 was failing the entire upload/download leg with no retry. Add fetchLotwWithRetry() in src/lib/lotw.ts: retries 5xx and network errors up to 3 times with exponential backoff (2s/4s/8s) and returns any 2xx untouched, since LoTW encodes auth and other failures inside a 200 body rather than the status line. Swap the Nextlog/1.0.0 User-Agent for a browser UA (some WAF front ends 503 non-browser clients). Applied to every LoTW HTTP call: the cron upload/download (lotw-sync.ts), the manual per-contact upload/download routes, credential validation, and the CRL status check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
Pull request overview
Improves LoTW (Logbook of The World) reliability in Nextlog by adding a shared retry wrapper for transient failures and standardizing LoTW requests to use a browser-like User-Agent, addressing intermittent 5xx responses (notably 503) seen from cloud/serverless egress.
Changes:
- Added
fetchLotwWithRetry()andLOTW_USER_AGENTinsrc/lib/lotw.tsto retry transient 5xx and network errors with backoff. - Updated scheduled LoTW sync upload/download to use the retry wrapper and browser
User-Agent. - Updated single-contact upload/download API routes to use the retry wrapper and browser
User-Agent.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/lib/lotw.ts | Introduces retry/backoff helper and browser UA constant; applies retry to credential validation + CRL checks. |
| src/lib/lotw-sync.ts | Uses retry helper + UA for cron/scheduled LoTW upload and download operations. |
| src/app/api/lotw/upload-contact/route.ts | Uses retry helper + UA for manual single-contact LoTW upload. |
| src/app/api/lotw/download-contact/route.ts | Uses retry helper + UA for manual single-contact LoTW download/confirmation. |
- Derive backoff from the attempt number (base·2^(N-1)) instead of a fixed 3-element array whose last slot was never reached with 3 attempts — removes the dead config / off-by-one ambiguity. - Cancel the discarded response body before retrying a 5xx so undici can reuse the connection rather than leaking sockets across retries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The unified sync cron was failing LoTW download with
LoTW download failed: 503 OK. The503is transient on ARRL's side — thelotwreport.adi/ upload front end returns503 Service Unavailablewhen it's throttling or under load. A direct probe of the endpoint with our exact params returns a clean200, so the request itself is fine; the 503 tracks the caller.Why Wavelog works and we didn't
I compared our LoTW logic against Wavelog's
application/controllers/Lotw.php. They're essentially identical — same endpoint, same query params, and Wavelog has no retry and no 503 handling either (it just reports "unexpected HTTP status 503" and stops). The difference isn't the code, it's the network origin:So the fix is to be more resilient than Wavelog, precisely because our egress is worse.
Changes
fetchLotwWithRetry()(new, insrc/lib/lotw.ts): retries5xxand network errors up to 3 attempts with exponential backoff (2s / 4s / 8s). Any2xxis returned untouched — LoTW encodes auth and other failures inside a200body, so those still flow through existing detection. After retries are exhausted the final response is returned, so the existing!okfailure path and log rows are preserved.User-AgentreplacesNextlog/1.0.0on every LoTW call — some WAF front ends503non-browser clients.src/lib/lotw-sync.ts)validateLoTWCredentials+checkLotwCertCrlVerification
npm run typecheck— cleannpm run lint— cleannpm run build— succeedsFollow-up if 503s persist
If LoTW still
503s after this ships, it points to ARRL hard-throttling Vercel's IP range — no client tweak fixes that. The durable answer is a stable egress IP (a fixed-IP proxy, or Vercel dedicated egress), recreating Wavelog's "one steady IP" advantage.🤖 Generated with Claude Code