fix(gc): release side-table locks before visiting prototype slots (collector self-deadlock)

[proggeramlug]

Unblocks every open PR: all runtime-scoped cargo-test runs since this morning's merge train (#6079, #6083, #6130, and the post-merge nextTick branch run) wedged at the 3-hour job timeout. This was misattributed to each PR in turn — the deadlock is on main.

Root cause

lldb on the parked test binary shows a single-thread self-deadlock in the copying minor collector:

visit_closure_static_prototype_slot_mut      ← holds closure-prototypes Mutex (dynamic_props.rs)
  → visit → CopyingNurseryCollector::move_young   (the prototype closure is young → moved)
    → gc_type_after_payload_move
      → closure_dynamic_props_owner_moved    ← locks the SAME Mutex → parks forever

std::sync::Mutex is non-reentrant, so the collector parks at 0% CPU inside test_geisterhand_callback_then_json_reviver_copied_minor_gc (a forced minor GC from within a JSON reviver), every other callback_scanners test queues behind the copying-nursery isolation lock, and libtest sits silent until the job timeout.

visit_object_static_prototype_slot_mut (#2820 Object.setPrototypeOf side table) has the identical bug — an ordinary-object move re-enters object_static_prototype_owner_moved on the same lock.

The hazardous shape dates to #2345 (May); one of this morning's runtime merges made the young-movable-prototype case fire deterministically. The fix is correct regardless of trigger.

Fix

The pattern the two sibling visitors in the same file already use (visit_closure_dynamic_prop_values_mut, scan_closure_dynamic_props_roots_mut): remove the entry → drop the lock → visit → re-insert, re-keying to the forwarded owner address in case the visit moved the owner itself (self-referential prototype).

Audited the remaining gc_type_after_payload_move hooks (overflow fields, Map/Set, exotic expando, error side tables): their visitors either collect slots before visiting (RefCell pattern) or use remove/merge — no other instance of visit-under-Mutex.

Verification

• cargo test -p perry-runtime --lib gc::tests::runtime_roots::callback_scanners — 26/26, 0.4s, twice (was: wedged indefinitely, reproduced locally at 0% CPU with the exact CI stack)
• Full cargo test -p perry-runtime --lib — 1162/1162 in 3.6s
• Deadlock reproduced and re-verified across runtime states at 04b8170, 042c4a9, and c78c20b — pre-dates today's runtime commits; not attributable to fix(runtime): Number("0x…") wider than u64 rounds to f64 instead of NaN #6123/perf(runtime): drain nextTick queue with VecDeque::pop_front (was O(n²)) #6126

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved prototype handling during garbage collection so prototype data is updated without holding internal locks during callbacks.
  • Fixed cases where moved or forwarded prototype entries could be re-associated incorrectly, helping prevent stale or inconsistent prototype state.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 53462eaf-70e1-45d7-9a26-b9df0fb5c9e2

📥 Commits

Reviewing files that changed from the base of the PR and between 7159c2a and ff1dca1.

📒 Files selected for processing (2)

• crates/perry-runtime/src/closure/dynamic_props.rs
• crates/perry-runtime/src/object/prototype_chain.rs

---

📝 Walkthrough

Walkthrough

Both visit_closure_static_prototype_slot_mut and visit_object_static_prototype_slot_mut are refactored from a locked in-place mutation pattern to a remove-unlock-visit-reinsert pattern, releasing the side-table mutex before invoking the visitor callback and reinserting entries under forwarded owner addresses when applicable.

Changes

Prototype Slot Lock Refactor

Layer / File(s)	Summary
Release map lock during visitor callback crates/perry-runtime/src/closure/dynamic_props.rs, crates/perry-runtime/src/object/prototype_chain.rs	Both prototype slot visitor functions now remove the entry from the side-table map under lock, release the lock, invoke the visitor callback on the removed prototype bits, then reinsert the entry keyed by the forwarded owner address if the visit relocated the object, or the original owner otherwise.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant VisitFn as visit_static_prototype_slot_mut
  participant Map as PrototypesMap
  participant Visitor as visit callback

  Caller->>VisitFn: request slot visit(owner)
  VisitFn->>Map: lock and remove(owner)
  Map-->>VisitFn: proto_bits
  VisitFn->>Visitor: visit(&mut proto_bits) (lock released)
  Visitor-->>VisitFn: possibly forwarded owner
  VisitFn->>Map: lock and reinsert(forwarded_or_original_owner, proto_bits)

Loading

Suggested labels: run-extended-tests

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main GC deadlock fix and the key change to release side-table locks before visiting prototype slots.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}