Skip to content

Defer media body downloads during refresh#29

Merged
tripledoublev merged 5 commits into
mainfrom
codex/defer-media-downloads-refresh
Jun 9, 2026
Merged

Defer media body downloads during refresh#29
tripledoublev merged 5 commits into
mainfrom
codex/defer-media-downloads-refresh

Conversation

@tripledoublev

@tripledoublev tripledoublev commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • make group refresh metadata-only after the remote collection is available, so one slow media body no longer blocks file-list discovery
  • keep refreshed_files for API compatibility while deferring body refreshes to explicit media endpoints
  • add a bounded media download helper for /media endpoints with per-peer and overall request budgets
  • report is_downloaded from the local group blob store instead of asking a remote peer
  • bump save to 0.2.6 and save-dweb-backend to v0.3.9, which pulls veilid-iroh-blobs v0.3.5
  • use the backend's budgeted peer-download API from save-dweb-backend v0.3.9 instead of keeping a duplicate media retry loop in this bridge
  • return an empty media list for empty repositories instead of surfacing the DHT root-hash error

Closes #23.

Verification

  • cargo check
  • cargo check --locked
  • cargo test test_upload_list_delete -- --nocapture
  • cargo test test_refresh_group_with_file -- --nocapture
  • cargo test test_refresh_member_joined_before_owner_uploads -- --nocapture
  • git diff --check

Review notes

  • Spawned review agents before opening the PR.
  • First review found is_downloaded was still using the remote-repo hash check and route lookup was outside the timeout; both were fixed.
  • Second review found writable repos with missing local blobs would skip explicit body recovery; fixed by removing the !repo.can_write() gate for file-body downloads.
  • Final review after the version bump and Bug: 500 Internal Server Error when fetching media from an empty repository #23 fix found no blocking issues; its changelog note was addressed before pushing.
  • Backend PR Add overall timeout for peer downloads save-dweb-backend#38 moved the overall download budget into Group::download_hash_from_peers_with_timeout; CI passed, it was merged, tagged, and released as v0.3.9.
  • Final save-rust review after deleting the local helper found no blocking issues.
  • Added regression coverage asserting first and repeated refreshes expose all_files without media bodies, keep refreshed_files empty, keep is_downloaded false until explicit media GET, then flip it true after download.

@tripledoublev

Copy link
Copy Markdown
Contributor Author

Follow-up on the download_file can_write review note: the dropped guard is intentional. In the backend, Group::has_hash checks the local blob store directly, so writable repos that still have the local body skip peer download. If the body is genuinely missing, the endpoint now attempts bounded peer recovery before falling through to Repo::get_file_stream, which only reads local blobs and does not perform a hidden unbounded download. The collection-hash guard remains read-only-only because writable repos own their collection metadata locally.

@tripledoublev tripledoublev merged commit ddbfccb into main Jun 9, 2026
1 check passed
@tripledoublev tripledoublev deleted the codex/defer-media-downloads-refresh branch June 9, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: 500 Internal Server Error when fetching media from an empty repository

1 participant