Address data channel thundering herd due to concurrent sendFile(...) calls#2010
Address data channel thundering herd due to concurrent sendFile(...) calls#20101egoman wants to merge 4 commits into
Conversation
🦋 Changeset detectedLatest commit: 042e6d4 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
size-limit report 📦
|
e48e2b0 to
d4a824c
Compare
| if (this.isClosed) { | ||
| throw new UnexpectedConnectionState('engine closed'); | ||
| } | ||
| if (this.isBufferStatusLow(kind)) { |
There was a problem hiding this comment.
issue: this path here means ordering isn't guaranteed anymore.
it needs to live after acquiring the lock
| // updated bufferedAmount rather than the stale (still-low) value. The caller's continuation is | ||
| // scheduled when this promise settles; deferring the unlock to a later task lets that send run | ||
| // first. | ||
| setTimeout(unlock, 0); |
There was a problem hiding this comment.
how many packets will this affect worse case?
I'm not a huge fan of the setTimeout here as timeouts get heavily throttled under certain circumstances, e.g. in backgrounded tabs
|
Lukas and I have been going back and forth on this in the background - it's fairly tricky to get right and come up with a solution that generalizes both to data streams and data tracks. Closing this in favor of #2013, as this approach seems to generalize better. |
Fixes #1995
Problem
Sending
nfiles concurrently over a data stream can result in all of thesendFilecalls attempting to send their first packet via the reliable data channel at the same time. When this happens, multiple writers can think the buffer status is low at the same time and all calldc.send(...), exhausting the data channel buffer and causing the buffer to overflow (which causes data to get dropped).Pre-existing workaround
The reporter of the issue has had success with converting concurrent
sendFilecalls into sequential ones - soawait Promise.all([sendFile(...), sendFile(...), sendFile(...)])vsawait sendFile(...); await sendFile(...); await sendFile(...);. The latter works because there's only one data stream actively writing to the data channel at a time, so there's no contention.Solution
To address this, I've opted to sequentialize
waitForBufferStatusLow(...)resolutions when the the data channel is not "low". So what will now happen:await waitForBufferStatusLow(...)works as it did before as long as it is in "low status", and by default has no sequentializing semantics.await waitForBufferStatusLow(...)gets called and the data channel is no longer in low status, then theresolveis cached inwaitForBufferedStatusLowResolves. Once the data channel is no longer in "low status", instead of resolving all promises ASAP (ie, a "thundering herd"), instead resolve them in sequence and verify that the data channel buffer is still in "low status" between resolutions. This way events naturally resolve in the order they were called until the data channel buffer is full again, where they pause until the buffer is no longer full again and so on until the buffer exhausts.Demo
Code added to demo app to test this
Before, here was the behavior:
Screen.Recording.2026-07-13.at.5.08.04.PM.mov
Now, with this pull request:
Screen.Recording.2026-07-13.at.4.50.00.PM.mov
Note the decreased upload speed, because the bandwidth is being split between all file uploads.