fix(qrz): rewrite auto-sync onto the API-key path, delete phantom status system#219
Merged
Conversation
…tus system QRZ auto-sync could never work. It authenticated with username/password against logbook.qrz.com, which only accepts pre-issued logbook API keys, and it recorded results in qrz_sync_status/qrz_sync_date/qrz_logbook_id/ qrz_sync_error columns that exist in no migration or schema — every status UPDATE threw, and backgroundAutoSync swallowed the error. Uploads silently did nothing. - New src/lib/qrz-sync-service.ts: single home for QRZ upload/download, used by auto-sync, the manual per-contact and bulk routes, and (next PR) the sync cron. All paths use the station qrz_api_key and record state in the ADIF-style qrz_qsl_sent/qrz_qsl_rcvd columns the UI reads. 'M' (modified) rows re-upload with OPTION=REPLACE; failures mark 'R' and are retried on every sweep. - Auto-sync now runs inside next/server after() — a bare fire-and-forget promise is frozen on Vercel when the response returns. - Contact edits flag already-uploaded copies 'Y' -> 'M' (flagForReupload) instead of writing the nonexistent qrz_sync_status column, which threw on every edit of core QSO fields. - QRZ downloads keep an incremental cursor in stations.qrz_last_qsl_rcvd_date (previously never written) via MODSINCE. - Delete uploadQSOToQRZ (username/password handshake), updateQrzSyncStatus, dead Contact.matchQSO, dead generateAdifForLoTW, and the phantom fields on ContactData; migration 0004 drops the orphaned qrz_sync_status_enum type. - Rename the misleading contacts query param qrz_sync_status=not_synced to qrz_filter=not_sent (it always filtered by qrz_qsl_sent). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This was referenced Jul 13, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors QRZ Logbook auto-sync to use station-scoped QRZ Logbook API keys (instead of the legacy username/password flow), consolidating upload/download logic into a dedicated service and removing the previous “phantom status” tracking that referenced nonexistent schema columns.
Changes:
- Introduces
src/lib/qrz-sync-service.tsas the single implementation for QRZ upload sweeps and confirmation downloads, persisting state via the existingqrz_qsl_*fields. - Reworks auto-sync to run via
next/serverafter()(instead of fire-and-forget promises) and flags already-uploaded QSOs for re-upload using the'M'marker. - Removes legacy QRZ upload code paths and drops the orphaned
qrz_sync_status_enumtype via migration0004.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/models/Station.ts | Adds QRZ download cursor field and a helper to advance it after successful fetches. |
| src/models/Contact.ts | Removes phantom QRZ sync fields; updates pending-upload query; adds reupload flagging method. |
| src/lib/qrz.ts | Removes legacy username/password QRZ logbook upload function. |
| src/lib/qrz-sync-service.ts | New centralized QRZ sync service used by routes and auto-sync. |
| src/lib/qrz-auto-sync.ts | Auto-sync rewritten to use station API key + centralized service with logging. |
| src/lib/lotw.ts | Removes dead LoTW ADIF generation helper. |
| src/app/api/contacts/route.ts | Renames QRZ filter param; runs auto-sync via after() on create. |
| src/app/api/contacts/qrz-sync/route.ts | Bulk QRZ sync route now delegates to the sync service and caches stations. |
| src/app/api/contacts/qrz-download/route.ts | QRZ download route now delegates confirmation logic to the sync service. |
| src/app/api/contacts/[id]/route.ts | Flags re-upload on core edits and runs auto-sync via after(). |
| src/app/api/contacts/[id]/qrz-sync/route.ts | Per-contact QRZ sync now uses station API key + sync service. |
| src/app/admin/page.tsx | Updates admin fetch to use the renamed qrz_filter=not_sent query param. |
| drizzle/migrations/meta/0004_snapshot.json | Adds migration snapshot metadata for the new migration state. |
| drizzle/migrations/meta/_journal.json | Registers migration 0004_drop_orphaned_qrz_sync_status_enum. |
| drizzle/migrations/0004_drop_orphaned_qrz_sync_status_enum.sql | Drops the orphaned qrz_sync_status_enum type. |
- Retry failed re-uploads with OPTION=REPLACE: infer a prior ship from qrz_qsl_sent_date so 'R' retries of modified QSOs don't get deduped as inserts and silently never update. - Push the station filter into SQL: findQrzSentNotConfirmed now takes stationId instead of loading all stations' rows and filtering in memory. - Flag re-upload on actual core-field changes: compare payload values against the stored contact instead of truthiness, so clearing a field triggers sync and no-op PUTs don't false-flag 'Y' rows to 'M'. - Reword qrz-auto-sync header: pre-flight skips only log, they don't mark the contact 'R'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
patrickrb
added a commit
that referenced
this pull request
Jul 13, 2026
* fix(qrz): rewrite auto-sync onto the API-key path, delete phantom status system
QRZ auto-sync could never work. It authenticated with username/password
against logbook.qrz.com, which only accepts pre-issued logbook API keys, and
it recorded results in qrz_sync_status/qrz_sync_date/qrz_logbook_id/
qrz_sync_error columns that exist in no migration or schema — every status
UPDATE threw, and backgroundAutoSync swallowed the error. Uploads silently
did nothing.
- New src/lib/qrz-sync-service.ts: single home for QRZ upload/download,
used by auto-sync, the manual per-contact and bulk routes, and (next PR)
the sync cron. All paths use the station qrz_api_key and record state in
the ADIF-style qrz_qsl_sent/qrz_qsl_rcvd columns the UI reads. 'M'
(modified) rows re-upload with OPTION=REPLACE; failures mark 'R' and are
retried on every sweep.
- Auto-sync now runs inside next/server after() — a bare fire-and-forget
promise is frozen on Vercel when the response returns.
- Contact edits flag already-uploaded copies 'Y' -> 'M' (flagForReupload)
instead of writing the nonexistent qrz_sync_status column, which threw on
every edit of core QSO fields.
- QRZ downloads keep an incremental cursor in
stations.qrz_last_qsl_rcvd_date (previously never written) via MODSINCE.
- Delete uploadQSOToQRZ (username/password handshake), updateQrzSyncStatus,
dead Contact.matchQSO, dead generateAdifForLoTW, and the phantom fields on
ContactData; migration 0004 drops the orphaned qrz_sync_status_enum type.
- Rename the misleading contacts query param qrz_sync_status=not_synced to
qrz_filter=not_sent (it always filtered by qrz_qsl_sent).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(security): require CRON_SECRET on cron + LoTW sync endpoints
The cron routes trusted any request whose host/user-agent/x-vercel-id header
mentioned vercel, which disabled the CRON_SECRET check on every *.vercel.app
deployment. Separately, /api/lotw/{upload,download} skipped auth entirely on
a spoofable X-Cron-Job: true header, letting anonymous callers trigger
uploads/downloads for any station_id.
- New src/lib/cron-auth.ts: strict Bearer CRON_SECRET check, no fallbacks.
- Cron routes require the Bearer token unconditionally and fail closed (500)
when CRON_SECRET is unset. Vercel attaches the header automatically when
the env var exists.
- X-Cron-Job is now only a mode discriminator: cron mode additionally
requires the valid Bearer token; the cron routes' internal fetches send it.
- README: "Scheduled sync" section for Vercel + self-hosted crontab setup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(sync): unified hourly sync cron for QRZ + LoTW
Replace the two daily LoTW-only cron routes with a single /api/cron/sync
orchestrator that runs the full cycle per station, in order: QRZ upload
sweep, LoTW upload, QRZ confirmation download, LoTW confirmation download.
Uploads run before downloads so fresh QSOs can confirm in the same cycle,
and QRZ/LoTW legs are serialized so the cross-service 'M' re-upload flags
never race. QRZ finally gets automation — previously upload was on-save or
manual only and confirmation download was manual only.
- QRZ legs iterate active stations with an API key whose owner enabled
qrz_auto_sync, via uploadPendingForStation (250 QSOs/station/run cap,
'R' failures retried every sweep) and downloadConfirmationsForStation
(incremental MODSINCE cursor).
- LoTW legs keep the existing per-station selection, <1h dedupe, and
internal fetch to /api/lotw/{upload,download} with the CRON_SECRET Bearer
header, preserving lotw_*_logs bookkeeping.
- vercel.json: single hourly cron (Vercel Pro), maxDuration 300 for the new
route; old cron routes deleted; README updated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(qrz): address Copilot review feedback on #219
- Retry failed re-uploads with OPTION=REPLACE: infer a prior ship from
qrz_qsl_sent_date so 'R' retries of modified QSOs don't get deduped
as inserts and silently never update.
- Push the station filter into SQL: findQrzSentNotConfirmed now takes
stationId instead of loading all stations' rows and filtering in memory.
- Flag re-upload on actual core-field changes: compare payload values
against the stored contact instead of truthiness, so clearing a field
triggers sync and no-op PUTs don't false-flag 'Y' rows to 'M'.
- Reword qrz-auto-sync header: pre-flight skips only log, they don't
mark the contact 'R'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cron): authenticate before env validation in unified sync route
The detailed missing-env-var response was reachable by unauthenticated
callers, letting them probe deployment configuration. Fail closed on a
missing CRON_SECRET (generic message), enforce the Bearer token, then
run the remaining env checks.
Addresses Copilot review feedback on #221.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
patrickrb
added a commit
that referenced
this pull request
Jul 13, 2026
…ings (#222) * fix(qrz): rewrite auto-sync onto the API-key path, delete phantom status system QRZ auto-sync could never work. It authenticated with username/password against logbook.qrz.com, which only accepts pre-issued logbook API keys, and it recorded results in qrz_sync_status/qrz_sync_date/qrz_logbook_id/ qrz_sync_error columns that exist in no migration or schema — every status UPDATE threw, and backgroundAutoSync swallowed the error. Uploads silently did nothing. - New src/lib/qrz-sync-service.ts: single home for QRZ upload/download, used by auto-sync, the manual per-contact and bulk routes, and (next PR) the sync cron. All paths use the station qrz_api_key and record state in the ADIF-style qrz_qsl_sent/qrz_qsl_rcvd columns the UI reads. 'M' (modified) rows re-upload with OPTION=REPLACE; failures mark 'R' and are retried on every sweep. - Auto-sync now runs inside next/server after() — a bare fire-and-forget promise is frozen on Vercel when the response returns. - Contact edits flag already-uploaded copies 'Y' -> 'M' (flagForReupload) instead of writing the nonexistent qrz_sync_status column, which threw on every edit of core QSO fields. - QRZ downloads keep an incremental cursor in stations.qrz_last_qsl_rcvd_date (previously never written) via MODSINCE. - Delete uploadQSOToQRZ (username/password handshake), updateQrzSyncStatus, dead Contact.matchQSO, dead generateAdifForLoTW, and the phantom fields on ContactData; migration 0004 drops the orphaned qrz_sync_status_enum type. - Rename the misleading contacts query param qrz_sync_status=not_synced to qrz_filter=not_sent (it always filtered by qrz_qsl_sent). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): require CRON_SECRET on cron + LoTW sync endpoints The cron routes trusted any request whose host/user-agent/x-vercel-id header mentioned vercel, which disabled the CRON_SECRET check on every *.vercel.app deployment. Separately, /api/lotw/{upload,download} skipped auth entirely on a spoofable X-Cron-Job: true header, letting anonymous callers trigger uploads/downloads for any station_id. - New src/lib/cron-auth.ts: strict Bearer CRON_SECRET check, no fallbacks. - Cron routes require the Bearer token unconditionally and fail closed (500) when CRON_SECRET is unset. Vercel attaches the header automatically when the env var exists. - X-Cron-Job is now only a mode discriminator: cron mode additionally requires the valid Bearer token; the cron routes' internal fetches send it. - README: "Scheduled sync" section for Vercel + self-hosted crontab setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sync): unified hourly sync cron for QRZ + LoTW Replace the two daily LoTW-only cron routes with a single /api/cron/sync orchestrator that runs the full cycle per station, in order: QRZ upload sweep, LoTW upload, QRZ confirmation download, LoTW confirmation download. Uploads run before downloads so fresh QSOs can confirm in the same cycle, and QRZ/LoTW legs are serialized so the cross-service 'M' re-upload flags never race. QRZ finally gets automation — previously upload was on-save or manual only and confirmation download was manual only. - QRZ legs iterate active stations with an API key whose owner enabled qrz_auto_sync, via uploadPendingForStation (250 QSOs/station/run cap, 'R' failures retried every sweep) and downloadConfirmationsForStation (incremental MODSINCE cursor). - LoTW legs keep the existing per-station selection, <1h dedupe, and internal fetch to /api/lotw/{upload,download} with the CRON_SECRET Bearer header, preserving lotw_*_logs bookkeeping. - vercel.json: single hourly cron (Vercel Pro), maxDuration 300 for the new route; old cron routes deleted; README updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sync): sync activity log, /sync page, truthful indicators + settings Sync failures were invisible: QRZ had no history at all, auto-sync failures vanished, and the settings UI implied username/password powered logbook sync. This adds the visibility layer: - New sync_logs table (migration 0005) + SyncLog model. QRZ upload sweeps, downloads, manual bulk syncs, and failed on-save auto-syncs all write rows (trigger manual/auto/cron); no-op cron sweeps are not logged so the feed stays readable. LoTW keeps its dedicated log tables. - GET /api/sync/logs: one reverse-chronological feed merging sync_logs with normalized lotw_upload_logs / lotw_download_logs rows. - New /sync page ("Sync Activity") listing every run with service, direction, station, trigger, status, and full error message; linked from the navbar More menu. - QRZSyncIndicator + LotwSyncIndicator learn the 'M' (modified - pending re-upload, amber) and 'I' (excluded, muted) states so flagged rows no longer render as "upload failed"/"not uploaded". - Settings truthfulness: profile QRZ section relabeled "XML Lookup Credentials" with a note that logbook sync uses per-station API keys, and an amber warning when auto-sync is on but no station has a key; station form API-key fields relabeled "QRZ.com Logbook API Key (required for logbook sync)". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(qrz): address Copilot review feedback on #219 - Retry failed re-uploads with OPTION=REPLACE: infer a prior ship from qrz_qsl_sent_date so 'R' retries of modified QSOs don't get deduped as inserts and silently never update. - Push the station filter into SQL: findQrzSentNotConfirmed now takes stationId instead of loading all stations' rows and filtering in memory. - Flag re-upload on actual core-field changes: compare payload values against the stored contact instead of truthiness, so clearing a field triggers sync and no-op PUTs don't false-flag 'Y' rows to 'M'. - Reword qrz-auto-sync header: pre-flight skips only log, they don't mark the contact 'R'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cron): authenticate before env validation in unified sync route The detailed missing-env-var response was reachable by unauthenticated callers, letting them probe deployment configuration. Fail closed on a missing CRON_SECRET (generic message), enforce the Bearer token, then run the remaining env checks. Addresses Copilot review feedback on #221. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sync): strip BOM from drizzle journal, guard NaN limit/offset - Remove the UTF-8 BOM (and CRLF endings) from drizzle/migrations/meta/_journal.json so strict JSON parsers and the Drizzle migrator can read it. - /api/sync/logs now falls back to defaults on non-numeric or negative limit/offset instead of passing NaN into SQL and returning a 500. Addresses Copilot review feedback on #222. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
"Uploads to QRZ silently do nothing." The auto-sync path was doubly broken and perfectly silent about it:
uploadQSOToQRZdid a username/password handshake againstlogbook.qrz.com/api, expecting aKEY=session token back. The QRZ Logbook API only accepts pre-issued logbook API keys (stations.qrz_api_key); username/password belongs to the XML lookup API on a different host. The handshake never yields a key, so every upload failed.Contact.updateQrzSyncStatusintoqrz_sync_status/qrz_sync_date/qrz_logbook_id/qrz_sync_error— columns that exist in no migration and no schema (only an orphaned enum type was ever created). Every status UPDATE threw. The same phantom write ran on every contact edit (PUT /api/contacts/[id]).backgroundAutoSyncended in.catch(() => {}), and on Vercel the bare fire-and-forget promise is frozen when the response returns, so the sync often never even ran.QRZSyncIndicator) readsqrz_qsl_sent/rcvd— a different system than auto-sync wrote, so even a hypothetical success would have shown nothing.Fix
src/lib/qrz-sync-service.ts— single home for QRZ sync, shared by on-save auto-sync, the per-contact route, the bulk route, and the upcoming sync cron:syncContactToQrz— station API key viauploadQSOToQRZWithApiKey, records intoqrz_qsl_*(the columns the UI reads).'M'(modified) rows re-upload withOPTION=REPLACE; failures mark'R'and are retried on every sweep; duplicates mark sent+confirmed.uploadPendingForStation— station-scoped sweep overfindQrzNotSent(nowNULL/'N'/'R'/'M', excluding'I').downloadConfirmationsForStation— extracted from the download route, now with an incrementalMODSINCEcursor instations.qrz_last_qsl_rcvd_date(column existed but was never used).after()fromnext/serverand logs every failure; contact edits flag shipped copies'Y' → 'M'via newContact.flagForReupload(QRZ and LoTW).uploadQSOToQRZ,Contact.updateQrzSyncStatus, deadContact.matchQSO, deadgenerateAdifForLoTW, the phantomContactDatafields. Migration0004drops the orphanedqrz_sync_status_enumtype./api/contacts?qrz_sync_status=not_syncedparam toqrz_filter=not_sent(admin page updated in the same PR).Response shapes of
/api/contacts/[id]/qrz-sync,/api/contacts/qrz-sync, and/api/contacts/qrz-downloadare preserved for the search/admin pages.Notes for reviewers
qrz_username/qrz_passwordremain — they are legitimately used for XML callsign lookups. Auto-sync is still gated onusers.qrz_auto_sync.qrz_api_keynow get a visiblelogger.warnon auto-sync instead of nothing; PR 5 in this series adds the sync-activity UI.Verification
npm run lint/npm run typecheck/npm run buildclean.🤖 Generated with Claude Code