refactor multipart upload to use concurrent producers-consumers - #3475
Closed
emma-sg wants to merge 21 commits into
Closed
refactor multipart upload to use concurrent producers-consumers#3475emma-sg wants to merge 21 commits into
emma-sg wants to merge 21 commits into
Conversation
emma-sg
force-pushed
the
issue-2814-multi-wacz-upload--multipart-stream-parallelism
branch
from
July 9, 2026 20:22
154cf46 to
67b64ab
Compare
tw4l
reviewed
Jul 9, 2026
emma-sg
force-pushed
the
issue-2814-multi-wacz-upload
branch
from
July 13, 2026 18:10
1bd6cda to
b28bba3
Compare
emma-sg
force-pushed
the
issue-2814-multi-wacz-upload--multipart-stream-parallelism
branch
from
July 13, 2026 20:47
009fb0c to
3e6757e
Compare
emma-sg
force-pushed
the
issue-2814-multi-wacz-upload
branch
from
July 15, 2026 22:11
2dd5b90 to
12a07d1
Compare
emma-sg
force-pushed
the
issue-2814-multi-wacz-upload--multipart-stream-parallelism
branch
from
July 15, 2026 22:12
562bca8 to
b849c5c
Compare
emma-sg
force-pushed
the
issue-2814-multi-wacz-upload
branch
from
July 16, 2026 21:02
12a07d1 to
3ccb276
Compare
- move `UploadFileReader` class definition from `uploads.py` to `models.py` - centralize `CHUNK_SIZE` constant in models.py - add type annotations to `FilePreparer`, `upload_thumbnail_stream`, and `StorageOps` methods
frontend for testing
test impact from just http2 buffer size fix
…ily) to" This reverts commit 6b3fece.
emma-sg
force-pushed
the
issue-2814-multi-wacz-upload--multipart-stream-parallelism
branch
from
July 16, 2026 21:03
9728381 to
a079713
Compare
removes the cgroup-based memory pressure backpressure mechanism and simplifies memory limiting to use a per-part overhead multiplier
Member
Author
|
Experiment - decided against this in favour of properly doing multi-part uploads directly to the storage bucket. |
emma-sg
deleted the
issue-2814-multi-wacz-upload--multipart-stream-parallelism
branch
July 23, 2026 22:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.