Skip to content

feat(upload): add multipart uploader with resumable chunked transfer#577

Open
thetapilla wants to merge 3 commits into
OpenListTeam:mainfrom
thetapilla:feat/multipart-upload
Open

feat(upload): add multipart uploader with resumable chunked transfer#577
thetapilla wants to merge 3 commits into
OpenListTeam:mainfrom
thetapilla:feat/multipart-upload

Conversation

@thetapilla

@thetapilla thetapilla commented Jul 5, 2026

Copy link
Copy Markdown

Summary / 摘要

为上传界面新增 Multipart 上传方式(配套后端分块上传 API),解决经 CDN 反代(如 Cloudflare 免费计划 100MB 请求体上限)时网页端无法上传大文件的问题。

用户可感知的变化:

  • 上传方式下拉框新增 Multipart 选项(由后端 multipart_enabled 公共设置门控,后端未启用或版本过旧时不显示,自动向后兼容)
  • 断点续传:上传中断(断网、刷新页面)后重新拖入同一文件,自动从已接收的分片之后继续;勾选"尝试秒传"时以文件哈希作为身份凭证
  • 失败重试:上传失败的行新增"重试"按钮,复用仍持有的 File 句柄从断点恢复,无需重新选择文件
  • 秒传:勾选"尝试秒传"时哈希随会话初始化上报,秒传命中后剩余分片立即停发
  • 选中 Multipart 时"添加为任务"复选框禁用——multipart 会话是同步流水线(进度可查、失败可断点重试),与任务队列"先收完再传"的语义互斥,详见后端 PR 的设计说明

实现要点:

  • 新增 src/pages/home/uploads/multipart.ts:3 并发升序分片调度;init 返回的已收区间直接驱动续传;秒传完成信号使剩余分片停发
  • 流控与容错:429/409 视为流控信号(独立于错误预算耐心重试);分片请求网络错误后先探询会话状态再决定重试或报错,避免把"会话已完成的提前响应"误报为失败
  • 单分片以下的小文件静默回退 Stream 直传
  • init 幂等(依托后端续传语义),瞬时网络故障自动重试而不判死整个上传
  • uploader 注册表新增条目并按公共设置门控;i18n 新增重试按钮与两个设置项的英文文案

兼容性说明:不改变既有 Stream / Form / HTTP Direct 上传方式的任何行为(对现有代码的修改均以"选中 Multipart"为条件门控);重试按钮对所有上传方式的失败行生效(非 multipart 方式点击重试等效于重新上传该文件)。

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。
  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了公开 API、配置、存储格式或迁移行为。
  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Related repository PRs / 关联仓库 PR:

Related Issues / 关联 Issue

Relates to OpenListTeam/OpenList#460

Testing / 测试

测试平台:macOS arm64,Node 26 / pnpm 11。

  • go test ./...
    不适用前端仓
  • Manual test / 手动测试:

自动化检查:

  • pnpm build 构建通过
  • pnpm lint(tsc):本 PR 未引入任何新类型错误(仓库现存 4 个与本 PR 无关的既有错误,干净分支上同样存在)
  • 提交经 husky pre-commit 的 prettier 自动格式化

浏览器手动验证(对接本地后端配套分支联调):

  • 覆盖已有文件 × 尝试秒传四种组合行为符合预期
  • 大文件上传中刷新页面后重拖同一文件,进度从断点起跳继续
  • 3×3 大文件并发上传全程无误报,进入后端写入阶段后进度正常收尾
  • 失败行重试按钮从断点恢复上传
  • 秒传命中场景(123 云盘真实账号):哈希计算完成后随即完成,无剩余分片发送
  • 后端关闭 multipart_enabled 后下拉框不再显示 Multipart 选项

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmtgo fmtprettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • ChatGPT
  • Codex
  • GitHub Copilot
  • Claude
  • Gemini
  • Other (please specify) / 其他(请注明):

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

- Add a chunked resumable uploader driving /api/fs/multipart with three concurrent ascending chunk requests
- Resume interrupted sessions transparently from init's received ranges and fall back to Stream for single-chunk files
- Treat flow control (429/409) and transient network errors as retryable, probing session status before declaring failure
- Stop sending remaining chunks once the server reports the session completed (rapid upload)
- Register the uploader gated by the multipart_enabled public setting
- Add a Retry button on failed rows that reuses the kept File handle, resuming multipart uploads from the breakpoint
- Disable "add as task" while the multipart uploader is selected, since sessions are synchronous pipelines
- Report progress and speed from a high-water mark: a chunk rejected by flow control or a network hiccup drops its in-flight bytes from the running sum until it is resent, which made the displayed speed go negative and the progress bar jump backwards
- Rejected chunks now show as a progress plateau with speed falling to zero
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant