Skip to content

fix(qrz): rewrite auto-sync onto the API-key path, delete phantom status system#219

Merged
patrickrb merged 2 commits into
mainfrom
fix/qrz-auto-sync-api-key
Jul 13, 2026
Merged

fix(qrz): rewrite auto-sync onto the API-key path, delete phantom status system#219
patrickrb merged 2 commits into
mainfrom
fix/qrz-auto-sync-api-key

Conversation

@patrickrb

Copy link
Copy Markdown
Owner

Problem

"Uploads to QRZ silently do nothing." The auto-sync path was doubly broken and perfectly silent about it:

  1. Wrong API. uploadQSOToQRZ did a username/password handshake against logbook.qrz.com/api, expecting a KEY= session token back. The QRZ Logbook API only accepts pre-issued logbook API keys (stations.qrz_api_key); username/password belongs to the XML lookup API on a different host. The handshake never yields a key, so every upload failed.
  2. Phantom columns. Results were recorded via Contact.updateQrzSyncStatus into qrz_sync_status / qrz_sync_date / qrz_logbook_id / qrz_sync_error — columns that exist in no migration and no schema (only an orphaned enum type was ever created). Every status UPDATE threw. The same phantom write ran on every contact edit (PUT /api/contacts/[id]).
  3. Swallowed errors. backgroundAutoSync ended in .catch(() => {}), and on Vercel the bare fire-and-forget promise is frozen when the response returns, so the sync often never even ran.
  4. The UI (QRZSyncIndicator) reads qrz_qsl_sent/rcvd — a different system than auto-sync wrote, so even a hypothetical success would have shown nothing.

Fix

  • New src/lib/qrz-sync-service.ts — single home for QRZ sync, shared by on-save auto-sync, the per-contact route, the bulk route, and the upcoming sync cron:
    • syncContactToQrz — station API key via uploadQSOToQRZWithApiKey, records into qrz_qsl_* (the columns the UI reads). 'M' (modified) rows re-upload with OPTION=REPLACE; failures mark 'R' and are retried on every sweep; duplicates mark sent+confirmed.
    • uploadPendingForStation — station-scoped sweep over findQrzNotSent (now NULL/'N'/'R'/'M', excluding 'I').
    • downloadConfirmationsForStation — extracted from the download route, now with an incremental MODSINCE cursor in stations.qrz_last_qsl_rcvd_date (column existed but was never used).
  • Auto-sync runs inside after() from next/server and logs every failure; contact edits flag shipped copies 'Y' → 'M' via new Contact.flagForReupload (QRZ and LoTW).
  • Deleted: uploadQSOToQRZ, Contact.updateQrzSyncStatus, dead Contact.matchQSO, dead generateAdifForLoTW, the phantom ContactData fields. Migration 0004 drops the orphaned qrz_sync_status_enum type.
  • Renamed the misleading /api/contacts?qrz_sync_status=not_synced param to qrz_filter=not_sent (admin page updated in the same PR).

Response shapes of /api/contacts/[id]/qrz-sync, /api/contacts/qrz-sync, and /api/contacts/qrz-download are preserved for the search/admin pages.

Notes for reviewers

  • User-level qrz_username/qrz_password remain — they are legitimately used for XML callsign lookups. Auto-sync is still gated on users.qrz_auto_sync.
  • Stations without a qrz_api_key now get a visible logger.warn on auto-sync instead of nothing; PR 5 in this series adds the sync-activity UI.

Verification

🤖 Generated with Claude Code

…tus system

QRZ auto-sync could never work. It authenticated with username/password
against logbook.qrz.com, which only accepts pre-issued logbook API keys, and
it recorded results in qrz_sync_status/qrz_sync_date/qrz_logbook_id/
qrz_sync_error columns that exist in no migration or schema — every status
UPDATE threw, and backgroundAutoSync swallowed the error. Uploads silently
did nothing.

- New src/lib/qrz-sync-service.ts: single home for QRZ upload/download,
  used by auto-sync, the manual per-contact and bulk routes, and (next PR)
  the sync cron. All paths use the station qrz_api_key and record state in
  the ADIF-style qrz_qsl_sent/qrz_qsl_rcvd columns the UI reads. 'M'
  (modified) rows re-upload with OPTION=REPLACE; failures mark 'R' and are
  retried on every sweep.
- Auto-sync now runs inside next/server after() — a bare fire-and-forget
  promise is frozen on Vercel when the response returns.
- Contact edits flag already-uploaded copies 'Y' -> 'M' (flagForReupload)
  instead of writing the nonexistent qrz_sync_status column, which threw on
  every edit of core QSO fields.
- QRZ downloads keep an incremental cursor in
  stations.qrz_last_qsl_rcvd_date (previously never written) via MODSINCE.
- Delete uploadQSOToQRZ (username/password handshake), updateQrzSyncStatus,
  dead Contact.matchQSO, dead generateAdifForLoTW, and the phantom fields on
  ContactData; migration 0004 drops the orphaned qrz_sync_status_enum type.
- Rename the misleading contacts query param qrz_sync_status=not_synced to
  qrz_filter=not_sent (it always filtered by qrz_qsl_sent).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
nodelog Ready Ready Preview, Comment Jul 13, 2026 5:14pm

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors QRZ Logbook auto-sync to use station-scoped QRZ Logbook API keys (instead of the legacy username/password flow), consolidating upload/download logic into a dedicated service and removing the previous “phantom status” tracking that referenced nonexistent schema columns.

Changes:

  • Introduces src/lib/qrz-sync-service.ts as the single implementation for QRZ upload sweeps and confirmation downloads, persisting state via the existing qrz_qsl_* fields.
  • Reworks auto-sync to run via next/server after() (instead of fire-and-forget promises) and flags already-uploaded QSOs for re-upload using the 'M' marker.
  • Removes legacy QRZ upload code paths and drops the orphaned qrz_sync_status_enum type via migration 0004.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/models/Station.ts Adds QRZ download cursor field and a helper to advance it after successful fetches.
src/models/Contact.ts Removes phantom QRZ sync fields; updates pending-upload query; adds reupload flagging method.
src/lib/qrz.ts Removes legacy username/password QRZ logbook upload function.
src/lib/qrz-sync-service.ts New centralized QRZ sync service used by routes and auto-sync.
src/lib/qrz-auto-sync.ts Auto-sync rewritten to use station API key + centralized service with logging.
src/lib/lotw.ts Removes dead LoTW ADIF generation helper.
src/app/api/contacts/route.ts Renames QRZ filter param; runs auto-sync via after() on create.
src/app/api/contacts/qrz-sync/route.ts Bulk QRZ sync route now delegates to the sync service and caches stations.
src/app/api/contacts/qrz-download/route.ts QRZ download route now delegates confirmation logic to the sync service.
src/app/api/contacts/[id]/route.ts Flags re-upload on core edits and runs auto-sync via after().
src/app/api/contacts/[id]/qrz-sync/route.ts Per-contact QRZ sync now uses station API key + sync service.
src/app/admin/page.tsx Updates admin fetch to use the renamed qrz_filter=not_sent query param.
drizzle/migrations/meta/0004_snapshot.json Adds migration snapshot metadata for the new migration state.
drizzle/migrations/meta/_journal.json Registers migration 0004_drop_orphaned_qrz_sync_status_enum.
drizzle/migrations/0004_drop_orphaned_qrz_sync_status_enum.sql Drops the orphaned qrz_sync_status_enum type.

Comment thread src/lib/qrz-sync-service.ts Outdated
Comment thread src/lib/qrz-sync-service.ts Outdated
Comment thread src/app/api/contacts/[id]/route.ts Outdated
Comment thread src/lib/qrz-auto-sync.ts Outdated
- Retry failed re-uploads with OPTION=REPLACE: infer a prior ship from
  qrz_qsl_sent_date so 'R' retries of modified QSOs don't get deduped
  as inserts and silently never update.
- Push the station filter into SQL: findQrzSentNotConfirmed now takes
  stationId instead of loading all stations' rows and filtering in memory.
- Flag re-upload on actual core-field changes: compare payload values
  against the stored contact instead of truthiness, so clearing a field
  triggers sync and no-op PUTs don't false-flag 'Y' rows to 'M'.
- Reword qrz-auto-sync header: pre-flight skips only log, they don't
  mark the contact 'R'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@patrickrb
patrickrb merged commit e5d148f into main Jul 13, 2026
7 checks passed
@patrickrb
patrickrb deleted the fix/qrz-auto-sync-api-key branch July 13, 2026 17:39
patrickrb added a commit that referenced this pull request Jul 13, 2026
* fix(qrz): rewrite auto-sync onto the API-key path, delete phantom status system

QRZ auto-sync could never work. It authenticated with username/password
against logbook.qrz.com, which only accepts pre-issued logbook API keys, and
it recorded results in qrz_sync_status/qrz_sync_date/qrz_logbook_id/
qrz_sync_error columns that exist in no migration or schema — every status
UPDATE threw, and backgroundAutoSync swallowed the error. Uploads silently
did nothing.

- New src/lib/qrz-sync-service.ts: single home for QRZ upload/download,
  used by auto-sync, the manual per-contact and bulk routes, and (next PR)
  the sync cron. All paths use the station qrz_api_key and record state in
  the ADIF-style qrz_qsl_sent/qrz_qsl_rcvd columns the UI reads. 'M'
  (modified) rows re-upload with OPTION=REPLACE; failures mark 'R' and are
  retried on every sweep.
- Auto-sync now runs inside next/server after() — a bare fire-and-forget
  promise is frozen on Vercel when the response returns.
- Contact edits flag already-uploaded copies 'Y' -> 'M' (flagForReupload)
  instead of writing the nonexistent qrz_sync_status column, which threw on
  every edit of core QSO fields.
- QRZ downloads keep an incremental cursor in
  stations.qrz_last_qsl_rcvd_date (previously never written) via MODSINCE.
- Delete uploadQSOToQRZ (username/password handshake), updateQrzSyncStatus,
  dead Contact.matchQSO, dead generateAdifForLoTW, and the phantom fields on
  ContactData; migration 0004 drops the orphaned qrz_sync_status_enum type.
- Rename the misleading contacts query param qrz_sync_status=not_synced to
  qrz_filter=not_sent (it always filtered by qrz_qsl_sent).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(security): require CRON_SECRET on cron + LoTW sync endpoints

The cron routes trusted any request whose host/user-agent/x-vercel-id header
mentioned vercel, which disabled the CRON_SECRET check on every *.vercel.app
deployment. Separately, /api/lotw/{upload,download} skipped auth entirely on
a spoofable X-Cron-Job: true header, letting anonymous callers trigger
uploads/downloads for any station_id.

- New src/lib/cron-auth.ts: strict Bearer CRON_SECRET check, no fallbacks.
- Cron routes require the Bearer token unconditionally and fail closed (500)
  when CRON_SECRET is unset. Vercel attaches the header automatically when
  the env var exists.
- X-Cron-Job is now only a mode discriminator: cron mode additionally
  requires the valid Bearer token; the cron routes' internal fetches send it.
- README: "Scheduled sync" section for Vercel + self-hosted crontab setup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(sync): unified hourly sync cron for QRZ + LoTW

Replace the two daily LoTW-only cron routes with a single /api/cron/sync
orchestrator that runs the full cycle per station, in order: QRZ upload
sweep, LoTW upload, QRZ confirmation download, LoTW confirmation download.
Uploads run before downloads so fresh QSOs can confirm in the same cycle,
and QRZ/LoTW legs are serialized so the cross-service 'M' re-upload flags
never race. QRZ finally gets automation — previously upload was on-save or
manual only and confirmation download was manual only.

- QRZ legs iterate active stations with an API key whose owner enabled
  qrz_auto_sync, via uploadPendingForStation (250 QSOs/station/run cap,
  'R' failures retried every sweep) and downloadConfirmationsForStation
  (incremental MODSINCE cursor).
- LoTW legs keep the existing per-station selection, <1h dedupe, and
  internal fetch to /api/lotw/{upload,download} with the CRON_SECRET Bearer
  header, preserving lotw_*_logs bookkeeping.
- vercel.json: single hourly cron (Vercel Pro), maxDuration 300 for the new
  route; old cron routes deleted; README updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(qrz): address Copilot review feedback on #219

- Retry failed re-uploads with OPTION=REPLACE: infer a prior ship from
  qrz_qsl_sent_date so 'R' retries of modified QSOs don't get deduped
  as inserts and silently never update.
- Push the station filter into SQL: findQrzSentNotConfirmed now takes
  stationId instead of loading all stations' rows and filtering in memory.
- Flag re-upload on actual core-field changes: compare payload values
  against the stored contact instead of truthiness, so clearing a field
  triggers sync and no-op PUTs don't false-flag 'Y' rows to 'M'.
- Reword qrz-auto-sync header: pre-flight skips only log, they don't
  mark the contact 'R'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cron): authenticate before env validation in unified sync route

The detailed missing-env-var response was reachable by unauthenticated
callers, letting them probe deployment configuration. Fail closed on a
missing CRON_SECRET (generic message), enforce the Bearer token, then
run the remaining env checks.

Addresses Copilot review feedback on #221.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
patrickrb added a commit that referenced this pull request Jul 13, 2026
…ings (#222)

* fix(qrz): rewrite auto-sync onto the API-key path, delete phantom status system

QRZ auto-sync could never work. It authenticated with username/password
against logbook.qrz.com, which only accepts pre-issued logbook API keys, and
it recorded results in qrz_sync_status/qrz_sync_date/qrz_logbook_id/
qrz_sync_error columns that exist in no migration or schema — every status
UPDATE threw, and backgroundAutoSync swallowed the error. Uploads silently
did nothing.

- New src/lib/qrz-sync-service.ts: single home for QRZ upload/download,
  used by auto-sync, the manual per-contact and bulk routes, and (next PR)
  the sync cron. All paths use the station qrz_api_key and record state in
  the ADIF-style qrz_qsl_sent/qrz_qsl_rcvd columns the UI reads. 'M'
  (modified) rows re-upload with OPTION=REPLACE; failures mark 'R' and are
  retried on every sweep.
- Auto-sync now runs inside next/server after() — a bare fire-and-forget
  promise is frozen on Vercel when the response returns.
- Contact edits flag already-uploaded copies 'Y' -> 'M' (flagForReupload)
  instead of writing the nonexistent qrz_sync_status column, which threw on
  every edit of core QSO fields.
- QRZ downloads keep an incremental cursor in
  stations.qrz_last_qsl_rcvd_date (previously never written) via MODSINCE.
- Delete uploadQSOToQRZ (username/password handshake), updateQrzSyncStatus,
  dead Contact.matchQSO, dead generateAdifForLoTW, and the phantom fields on
  ContactData; migration 0004 drops the orphaned qrz_sync_status_enum type.
- Rename the misleading contacts query param qrz_sync_status=not_synced to
  qrz_filter=not_sent (it always filtered by qrz_qsl_sent).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(security): require CRON_SECRET on cron + LoTW sync endpoints

The cron routes trusted any request whose host/user-agent/x-vercel-id header
mentioned vercel, which disabled the CRON_SECRET check on every *.vercel.app
deployment. Separately, /api/lotw/{upload,download} skipped auth entirely on
a spoofable X-Cron-Job: true header, letting anonymous callers trigger
uploads/downloads for any station_id.

- New src/lib/cron-auth.ts: strict Bearer CRON_SECRET check, no fallbacks.
- Cron routes require the Bearer token unconditionally and fail closed (500)
  when CRON_SECRET is unset. Vercel attaches the header automatically when
  the env var exists.
- X-Cron-Job is now only a mode discriminator: cron mode additionally
  requires the valid Bearer token; the cron routes' internal fetches send it.
- README: "Scheduled sync" section for Vercel + self-hosted crontab setup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(sync): unified hourly sync cron for QRZ + LoTW

Replace the two daily LoTW-only cron routes with a single /api/cron/sync
orchestrator that runs the full cycle per station, in order: QRZ upload
sweep, LoTW upload, QRZ confirmation download, LoTW confirmation download.
Uploads run before downloads so fresh QSOs can confirm in the same cycle,
and QRZ/LoTW legs are serialized so the cross-service 'M' re-upload flags
never race. QRZ finally gets automation — previously upload was on-save or
manual only and confirmation download was manual only.

- QRZ legs iterate active stations with an API key whose owner enabled
  qrz_auto_sync, via uploadPendingForStation (250 QSOs/station/run cap,
  'R' failures retried every sweep) and downloadConfirmationsForStation
  (incremental MODSINCE cursor).
- LoTW legs keep the existing per-station selection, <1h dedupe, and
  internal fetch to /api/lotw/{upload,download} with the CRON_SECRET Bearer
  header, preserving lotw_*_logs bookkeeping.
- vercel.json: single hourly cron (Vercel Pro), maxDuration 300 for the new
  route; old cron routes deleted; README updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(sync): sync activity log, /sync page, truthful indicators + settings

Sync failures were invisible: QRZ had no history at all, auto-sync failures
vanished, and the settings UI implied username/password powered logbook
sync. This adds the visibility layer:

- New sync_logs table (migration 0005) + SyncLog model. QRZ upload sweeps,
  downloads, manual bulk syncs, and failed on-save auto-syncs all write
  rows (trigger manual/auto/cron); no-op cron sweeps are not logged so the
  feed stays readable. LoTW keeps its dedicated log tables.
- GET /api/sync/logs: one reverse-chronological feed merging sync_logs with
  normalized lotw_upload_logs / lotw_download_logs rows.
- New /sync page ("Sync Activity") listing every run with service,
  direction, station, trigger, status, and full error message; linked from
  the navbar More menu.
- QRZSyncIndicator + LotwSyncIndicator learn the 'M' (modified - pending
  re-upload, amber) and 'I' (excluded, muted) states so flagged rows no
  longer render as "upload failed"/"not uploaded".
- Settings truthfulness: profile QRZ section relabeled "XML Lookup
  Credentials" with a note that logbook sync uses per-station API keys, and
  an amber warning when auto-sync is on but no station has a key; station
  form API-key fields relabeled "QRZ.com Logbook API Key (required for
  logbook sync)".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(qrz): address Copilot review feedback on #219

- Retry failed re-uploads with OPTION=REPLACE: infer a prior ship from
  qrz_qsl_sent_date so 'R' retries of modified QSOs don't get deduped
  as inserts and silently never update.
- Push the station filter into SQL: findQrzSentNotConfirmed now takes
  stationId instead of loading all stations' rows and filtering in memory.
- Flag re-upload on actual core-field changes: compare payload values
  against the stored contact instead of truthiness, so clearing a field
  triggers sync and no-op PUTs don't false-flag 'Y' rows to 'M'.
- Reword qrz-auto-sync header: pre-flight skips only log, they don't
  mark the contact 'R'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cron): authenticate before env validation in unified sync route

The detailed missing-env-var response was reachable by unauthenticated
callers, letting them probe deployment configuration. Fail closed on a
missing CRON_SECRET (generic message), enforce the Bearer token, then
run the remaining env checks.

Addresses Copilot review feedback on #221.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sync): strip BOM from drizzle journal, guard NaN limit/offset

- Remove the UTF-8 BOM (and CRLF endings) from
  drizzle/migrations/meta/_journal.json so strict JSON parsers and the
  Drizzle migrator can read it.
- /api/sync/logs now falls back to defaults on non-numeric or negative
  limit/offset instead of passing NaN into SQL and returning a 500.

Addresses Copilot review feedback on #222.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants