Bugfix: Server-initiated room disconnect shutdown fix#205
Bugfix: Server-initiated room disconnect shutdown fix#205alan-george-lk wants to merge 15 commits into
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
xianshijing-lk
left a comment
There was a problem hiding this comment.
lgtm, assuming you address those comments.
| const bool has_room_state = connection_state_ != ConnectionState::Disconnected || listener_id_ != 0 || | ||
| room_handle_ || local_participant_ || !remote_participants_.empty(); | ||
| if (teardown_started_ || !has_room_state) { | ||
| return false; |
There was a problem hiding this comment.
should we return true if nothing to do or it is being torndown ?
There was a problem hiding this comment.
I'm open to changing this, right now our public API for disconnect() states:
@returns trueif the graceful disconnect succeeds;falseif the room was already disconnected (no-op) or the graceful disconnect fails.
Which this matches, false == noop. I get how that could be confusing, but I think the idea is true is when the disconnect actually happened
There was a problem hiding this comment.
That is fair. We can keep the current behavior to avoid breaking changes.
There was a problem hiding this comment.
FWIW we have a public connectionState so apps can check the state before trying to disconnect
…gfix-room-shutdown
Align the shared cleanup helper with LocalParticipant/FfiClient naming, keep disconnect() returning false for no-ops per its documented contract, and cover Room reuse after server-initiated shutdown. Co-authored-by: Cursor <cursoragent@cursor.com>
|
lgtm, thanks |
| } | ||
|
|
||
| // old_* state is destroyed here when going out of scope | ||
| (void)shutdown(false, DisconnectReason::Unknown, false); |
There was a problem hiding this comment.
is EOS not a DisconnectReason?
There was a problem hiding this comment.
TODO: Confirm the flow is this on server disconnect:
- Server disconnect event (do shutdown)
- EOS (don't need shutdown)
Verify that the EOS is double-calling shutdown right now as written (after server close)
3cbddb6 to
a2e6a9e
Compare
| bool should_notify = false; | ||
| { | ||
| const std::scoped_lock<std::mutex> guard(lock_); | ||
| already_disconnected = (connection_state_ == ConnectionState::Disconnected); | ||
| connection_state_ = ConnectionState::Disconnected; | ||
| } | ||
| if (already_disconnected) { | ||
| break; | ||
| } | ||
| DisconnectedEvent ev; | ||
| ev.reason = toDisconnectReason(re.disconnected().reason()); | ||
| if (delegate_snapshot) { | ||
| // Local shutdown marks the state before awaiting the FFI response | ||
| // and notifies the delegate itself. Suppress that duplicate while | ||
| // passing server-initiated disconnects through unchanged. | ||
| should_notify = connection_state_ != ConnectionState::Disconnected; | ||
| } | ||
| if (should_notify && delegate_snapshot) { | ||
| DisconnectedEvent ev; | ||
| ev.reason = toDisconnectReason(re.disconnected().reason()); | ||
| delegate_snapshot->onDisconnected(*this, ev); | ||
| } | ||
| break; |
There was a problem hiding this comment.
🔴 Server disconnect notification can fire twice when a user disconnect races with a server disconnect event
The disconnect notification is decided (should_notify = true at src/room.cpp:1202) without updating the room's connection state, so a concurrent user-initiated disconnect on another thread can also pass its own state check and fire the same notification, resulting in the delegate receiving two disconnect callbacks instead of one.
Impact: The application's disconnect handler runs twice with different reasons, which can cause double-cleanup, incorrect state transitions, or crashes in user code that assumes exactly-one semantics.
Race window between kDisconnected handler and user disconnect()
The old code at the base commit always set connection_state_ = Disconnected inside the kDisconnected handler under the lock. This created mutual exclusion: whichever path (FFI event or user disconnect()) set the state first would prevent the other from firing the delegate.
The new code at src/room.cpp:1196-1209 only reads the state but never writes it:
bool should_notify = false;
{
const std::scoped_lock<std::mutex> guard(lock_);
should_notify = connection_state_ != ConnectionState::Disconnected;
// ← state is NOT updated here
}
if (should_notify && delegate_snapshot) {
delegate_snapshot->onDisconnected(*this, ev);
}Race sequence:
- FFI thread:
kDisconnectedhandler acquires lock, readsConnected, setsshould_notify = true, releases lock. - User thread:
disconnect()→shutdown()acquires lock, also readsConnected(unchanged!), moves out all state, setsDisconnected, releases lock, callsdelegate->onDisconnected(ClientInitiated). - FFI thread: resumes and calls
delegate->onDisconnected(RoomDeleted)becauseshould_notifywas already captured astrue.
Result: delegate receives two onDisconnected calls. The integration test UserDisconnect explicitly asserts delegate.count.load() == 1, so this race can cause test failures and violates the documented contract.
Prompt for agents
In src/room.cpp, the kDisconnected event handler (around line 1196-1209) reads connection_state_ under the lock to decide whether to notify the delegate, but does not update the state. This creates a race window where a concurrent user-initiated disconnect() on another thread can also pass its state check and fire the delegate, resulting in two onDisconnected calls.
The fix should restore the old behavior of setting connection_state_ = ConnectionState::Disconnected inside the kDisconnected handler's locked section, matching what the base commit did. This ensures mutual exclusion: whichever path (the FFI kDisconnected event or the user's disconnect() call to shutdown()) sets the state first will prevent the other from firing the delegate.
Specifically, in the kDisconnected case block, after computing should_notify, add: connection_state_ = ConnectionState::Disconnected; inside the locked section. This way shutdown() on another thread will see the state as already Disconnected and return false (no-op), preventing the double notification.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
shutdown()helper, which by proxy cleans up event codeBefore fix, when server deleted the room:
Above observed on Mac, but crashed on Linux (user-reported). No longer after the fix.
Testing
lk --dev room deleteemitFfiEventinto standalone header helperffi_utils.hfor use across two tests nowTicket
BOT-464