Fix multithreaded deadlock between the body cache and the compiler lock

[ChrisRackauckas-Claude]

Please ignore until reviewed by @ChrisRackauckas.

Fixes #141.

What changed and why

_lookup_body is called from the expansion of the generated_callfunc
@generated method, so it runs while the calling thread holds Julia's compiler
lock — and it took _cache_lock. Meanwhile _cache_body holds _cache_lock
while inserting into the cache Dict, and that insert can compile a
specialization (setindex! → rehash! → codegen) and block on the compiler
lock. Classic lock-order inversion; because _cache_lock is a SpinLock, the
hung threads sit at 100% CPU.

Reads are now lock-free. The cache is a stack of Dicts, newest first, none of
which is mutated once published; a reader takes the stack with a single atomic
load. Writers still serialize on _cache_lock and publish a replacement stack.
Copying the whole map per insert would be O(n²) for a package that generates
thousands of functions, so levels merge on the logarithmic method (a level merges
down once it reaches half the next level's size): sizes more than double going
down the stack, bounding both a reader's probes and an insert's copying at
O(log n).

The cache object stored in each initialized module changes from Dict() to an
internal RuntimeGeneratedFunctions._BodyCache. That name
(#_RuntimeGeneratedFunctions_cache) is not public API, but it is a shape change
for anything reaching into it.

Failing before / passing after

Reproducer from the issue, on Julia 1.10.11 (also reproduces on 1.13.0-rc3):

Before (master, --threads=8, timeout -s KILL 90; exit 137 = killed while hung):

--- run 1
[2179500] signal (15): Terminated
...
lock at ./locks-mt.jl:48
_lookup_body at src/RuntimeGeneratedFunctions.jl:306      <-- spins on _cache_lock
generated_callfunc_body at src/RuntimeGeneratedFunctions.jl:243
#s1#1 at src/RuntimeGeneratedFunctions.jl:372 [inlined]
jl_type_infer at src/gf.c:396                             <-- holds the compiler lock
...
_jl_mutex_wait at src/threading.c:857                     <-- waits for the compiler lock
jl_generate_fptr_impl at src/jitlayers.cpp:483
rehash! at ./dict.jl:194
setindex! at ./dict.jl:399
#6 at src/RuntimeGeneratedFunctions.jl:299                 <-- holds _cache_lock
_cache_body at src/RuntimeGeneratedFunctions.jl:274

After (same command, 5 consecutive runs):

OK
run 1 exit=0
OK
run 2 exit=0
OK
run 3 exit=0
OK
run 4 exit=0
OK
run 5 exit=0

The regression test discriminates

The existing threaded test ran in-process, and the suite runs single-threaded, so
it never exercised this. It now runs test/shared/threaded_rgf.jl in a
subprocess with --threads=8 and a 600 s watchdog.

With the fix reverted (git stash push src/RuntimeGeneratedFunctions.jl) and the
new test in place, include("test/core_tests.jl") on Julia 1.10:

Test Failed at test/core_tests.jl:148
  Expression: success(proc)

ERROR: LoadError: There was an error during testing

CI Core also runs on macOS and Windows; to check it discriminates on a
low-core-count runner, taskset -c 0,1 (2 cores) on the unfixed code hangs 3/3
at --threads=8 and 3/3 at --threads=2, and passes 3/3 at both with the fix.

Verification

Run locally on this branch:

$ GROUP=Core  julia +1.10 --project -e 'using Pkg; Pkg.test()'   -> tests passed
$ GROUP=QA    julia +1.10 --project -e 'using Pkg; Pkg.test()'   -> QA | 18  18  18.9s, tests passed
$ GROUP=Core  julia +1.12 --project -e 'using Pkg; Pkg.test()'   -> tests passed
$ GROUP=QA    julia +1.12 --project -e 'using Pkg; Pkg.test()'   -> QA | 20  20  25.0s, tests passed
$ GROUP=Core  julia +1.11 --project -e 'using Pkg; Pkg.test()'   -> tests passed
$ GROUP=QA    julia +1.11 --project -e 'using Pkg; Pkg.test()'   -> QA | 20  20  26.6s, tests passed
$ GROUP=Core  julia +rc   --project -e 'using Pkg; Pkg.test()'   -> tests passed   (1.13.0-rc3)
$ GROUP=QA    julia +rc   --project -e 'using Pkg; Pkg.test()'   -> QA | 20  20  20.8s, tests passed
$ julia --project=docs docs/make.jl                              -> builds clean
$ Runic.main(["--check", "--diff", "src", "test", "docs"])       -> clean
$ typos src test docs README.md                                  -> clean

Performance

The cost is on inserts. Cache-only microbenchmark (_cache_body on n distinct
ids, then _lookup_body on each), Julia 1.10:

n	insert (master)	insert (this PR)	lookup (master)	lookup (this PR)
10,000	0.019 s	0.041 s	0.005 s	0.005 s
50,000	0.056 s	0.202 s	0.035 s	0.021 s
100,000	0.085 s	0.492 s	0.068 s	0.062 s

Scaling is n log n, not n²: 10× the entries costs 12× the time. End-to-end
RuntimeGeneratedFunction construction (which is dominated by SHA-1 hashing the
expression, and in real use by codegen):

n	master	this PR
10,000	0.199 s	0.216 s
50,000	0.945 s	1.088 s

CI

All 16 checks green: Core on {ubuntu, macOS, Windows} × {lts, 1, pre}, QA, Downgrade,
Documentation, Runic, typos. My local runs were Linux-only, so CI is what covers the
new subprocess test's kill(proc, Base.SIGKILL) watchdog on macOS and Windows.

Not verified

• Downstream packages (ModelingToolkit, Symbolics, …) were not run against this.
• Non-x86 architectures. The atomics are :acquire/:release rather than
  relying on x86 store ordering, so this should be fine, but it was not tested.

Worth pushing back on

• Writers still use Threads.SpinLock. Now that readers never take it, a
  ReentrantLock would be safe and would stop other writers burning cores while
  one of them compiles under the lock. I left it alone to keep the diff to the
  deadlock, but it is a reasonable follow-up.
• Version bumped 0.5.24 → 0.5.25. The internal per-module cache object type
  changes, which is a patch-level change only because that name is not public
  API.

🤖 Generated with Claude Code

https://claude.ai/code/session_01B9uAJ7kQsHJtp7Yb2qdFVV