fix(KNO-13857): reduce websocket reconnect load when connections can't recover#1031
Draft
kylemcd wants to merge 2 commits into
Draft
fix(KNO-13857): reduce websocket reconnect load when connections can't recover#1031kylemcd wants to merge 2 commits into
kylemcd wants to merge 2 commits into
Conversation
…t recover Escalate the Phoenix socket reconnect backoff toward a 10-minute cap (was a fixed 30s cap that Phoenix reset on every successful open), so a client that can never reconnect - e.g. after an API key rotation leaves stale credentials rejected on every upgrade - settles into a slow retry cadence instead of a tight loop. The escalation survives brief "connect then immediately drop" cycles and resets once a connection stays up for 30s. Also: - teardown() always disconnects the socket, even mid-reconnect, so a reauth that replaces the client can't leak a socket that retries forever. - Hidden background tabs stop retrying: a socket mid-reconnect when the page is hidden is disconnected, not just connected ones, and resumes on visible. - Reconnect promptly on the browser `online` event so recovery after a real network drop doesn't wait out the longer backoff.
🦋 Changeset detectedLatest commit: b657fe1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
1 task
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b6bb977. Configure here.
Addresses comment by @cursor: the `online` handler could revive a socket that was cleanly/deliberately disconnected. Gate it on whether Phoenix wants to reconnect (unclean drop or failed attempt), read from the close event's `wasClean`, instead of on whether the socket was ever active.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes KNO-13857 (follow-up to INC-408).
A customer rotated their public API key; their deployed clients kept the old one. Every websocket reconnect got a
403that could never succeed, so each client retried forever — fleet-wide that was a ~1000% spike that saturated the sockets pods. The SDK can't stop a bad key, but it shouldn't turn one into a firehose. Three changes inpackages/clientfix that:ApiClientso it survives the brief "open then drop" cycles that used to reset it. A dead connection settles to ~1 try / 5 min; the first ~6 attempts are unchanged, so real blips still recover fast.teardown()always disconnects (even mid-retry), so a token refresh can't leave an old socket retrying forever with dead credentials.Plus an
onlinelistener that reconnects immediately when the network returns, so the longer backoff doesn't strand real users.Result: a stuck tab drops from ~401 reconnects to ~29 (foreground) / ~6 (hidden) over the incident window — a ~14–46× fleet reduction that turns the incident's +1000% into +27–71%. (Applies to clients that upgrade and redeploy.)
Checklist
Covered in
test/api.test.tsandtest/pageVisibility.test.ts. Full client suite (582 tests),type:check,lint, andformat:checkpass.