Skip to content

fix: fail fast on client send when payload exceeds MAX_TOTAL_CHUNKS#81

Merged
sanity merged 2 commits into
mainfrom
fix/chunk-failfast
Jul 1, 2026
Merged

fix: fail fast on client send when payload exceeds MAX_TOTAL_CHUNKS#81
sanity merged 2 commits into
mainfrom
fix/chunk-failfast

Conversation

@sanity

@sanity sanity commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Problem

The WebSocket client's send path chunks a large ClientRequest and sends every chunk without checking MAX_TOTAL_CHUNKS. The cap is 256 chunks x 256 KiB = 64 MiB (MAX_TOTAL_CHUNKS * CHUNK_SIZE).

When a serialized request exceeds that cap, the client streams the entire oversized payload to the node, which then rejects the very first chunk during reassembly (ReassemblyBuffer::receive_chunk returns TotalChunksTooLarge for total > MAX_TOTAL_CHUNKS). Every byte after the first chunk is wasted: a payload well above the 64 MiB cap is uploaded in full only to be discarded on arrival.

Both send paths had this gap:

  • regular.rs::process_request (native/tokio path used by fdev)
  • browser.rs::send (wasm path used by River)

Solution

Add a small, reusable, tested helper in streaming.rs:

pub fn ensure_chunkable(len: usize) -> Result<(), StreamError>

It computes the chunk count with the same math as chunk_bytes (len.div_ceil(CHUNK_SIZE).max(1)) so the client's total equals what the node validates on the wire, and returns StreamError::TotalChunksTooLarge { total, max } when it would exceed MAX_TOTAL_CHUNKS.

Both send paths call ensure_chunkable(msg.len()) before entering the chunking branch, so an oversized request fails fast with a clear error and no chunk is sent:

  • regular.rs: maps the error into Error::OtherError and returns it; the request handler surfaces it to the caller (the client's recv() gets the error).
  • browser.rs: surfaces it via the existing Error::ConnectionError(serde_json::Value) + error_handler pattern, matching how the file already reports send failures to the JS caller.

The guard mirrors the node's own reassembly cap — this is a client-side pre-check of an invariant the node already enforces.

Note on error propagation (regular.rs)

Returning Err from process_request breaks the request-handler loop, which tears down the WebApi connection after delivering the error to recv(). For an unsendable request that is acceptable behavior. Surfacing the error for just this request without tearing down the connection would require threading response_tx into process_request (a signature/flow change); it was not deemed worth the added complexity for a request that cannot be sent regardless. The caller still receives the clear TotalChunksTooLarge error either way.

Testing

Added three cheap, deterministic unit tests in streaming.rs (they pass only the length integer — no multi-MiB payload is allocated):

  • ensure_chunkable_rejects_oversized(MAX_TOTAL_CHUNKS + 1) * CHUNK_SIZE is rejected with TotalChunksTooLarge { total: 257, max: 256 }.
  • ensure_chunkable_accepts_at_limit — exactly MAX_TOTAL_CHUNKS * CHUNK_SIZE is accepted; one byte over is rejected.
  • ensure_chunkable_accepts_small — 0, 1 KiB, and CHUNK_THRESHOLD are accepted.

No dedicated regular.rs send-path test: triggering the guard end-to-end requires a >64 MiB serialized request, i.e. a >64 MiB allocation, which these tests deliberately avoid. The helper unit tests plus the existing node-side total_too_large_error reassembly test cover the behavior.

Local verification:

  • cargo fmt — clean
  • cargo clippy --all-targets --features net -- -D warnings — clean
  • cargo test --workspace --features contract,net,testing,trace (the CI test command) — all pass, incl. the 3 new tests
  • cargo build --target wasm32-unknown-unknown --features net --lib — compiles (CI's wasm jobs run on default features and don't compile the net-gated browser.rs, so browser.rs was verified locally against the wasm target)

Companion to freenet/freenet-core#4653

[AI-assisted - Claude]

sanity and others added 2 commits July 1, 2026 12:10
The WebSocket client's send path chunked a large ClientRequest and streamed
every chunk without checking MAX_TOTAL_CHUNKS. When a payload exceeds the cap
(256 chunks x 256 KiB = 64 MiB), the client streamed the ENTIRE oversized
payload to the node, which rejects the first chunk during reassembly
(ReassemblyBuffer::receive_chunk returns TotalChunksTooLarge). Every byte after
the first chunk was wasted bandwidth (a ~450 MiB payload would upload in full
before the node rejects it).

Add a reusable, tested ensure_chunkable(len) helper in streaming.rs that mirrors
the node's reassembly cap, and call it on both send paths (regular/native and
browser/wasm) before any chunk is sent, so an oversized request fails fast with
a clear TotalChunksTooLarge error and nothing is streamed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a call-site regression test (test_send_oversized_rejected_before_streaming)
that drives a real WebApi over a localhost socket with an oversized Put (state
one byte over the 64 MiB MAX_TOTAL_CHUNKS * CHUNK_SIZE cap) and asserts recv()
returns a "exceeds maximum" (TotalChunksTooLarge) error. Verified that removing
the ensure_chunkable(...) guard from process_request makes this test fail (the
client streams the payload and recv() instead returns a connection-reset error),
so the guard can't be silently deleted without CI catching it. The prior
ensure_chunkable unit tests only covered the helper, not the wiring.

Also document at the guard call site that returning Err tears down the WebApi
connection (request_handler break), which is intentional and acceptable for an
unsendable over-cap request: the error is still delivered to recv() first, and
the browser/wasm path surfaces it without teardown. We deliberately do not thread
response_tx through process_request for per-request error reporting.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sanity sanity merged commit 75e3a46 into main Jul 1, 2026
8 checks passed
@sanity sanity deleted the fix/chunk-failfast branch July 1, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant