bench: round-1 investigation harness (Float64/DMRG2 parity, TDVP + thread-scaling suites, Rusty scripts, diagnostics)#450
Draft
lkdvos wants to merge 8 commits into
Draft
Conversation
ITensorMPS exposes only two-site dmrg, so the single-site suite-1 variant is dropped and both suites now compare MPSKit DMRG2 (trscheme = truncrank(chi)) against ITensor's default dmrg. Both sides now run Float64 (MPSKit's ComplexF64 default was a 2-4x BLAS handicap on this real-symmetric Hamiltonian). Sweep budget cut from the 30-sweep placeholder to 10, pending the chi = 256 pilot. Trajectory recorders on both sides additionally capture per-sweep GC seconds and allocated bytes (investigation diagnostics). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Suite 5: global quench from the Néel state on the spin-1 Heisenberg chain, two-site growth phase then single-site measure phase, mirrored on both libraries; per-step (Sz, chi, walltime) trajectories and wall-seconds per unit physical time. Krylov exponentiation tolerance matched at 1e-10 on both sides. Smoke gate: mid-chain Sz agrees across libraries to ~1e-13. Suite 7: suite-1 workload at one (N, chi) point, one process per (JULIA_NUM_THREADS x BLAS-threads) grid point, driven externally. Both run.jl entry points gain --suite=5,7 and --blas-threads=n (default 1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…iagnostic - benchmark/slurm/: one manually-submitted sbatch per (workload x library), single exclusive node, pinned-constraint placeholder, plus a README with the sanity-gate checklist. - plot_results.jl: cross-library trajectory overlays, sanity-gate + time-to-accuracy tables (E_ref = best energy across both libraries), TDVP observable-gate/throughput panels, thread-scaling speedup plot. - profile_sweep.jl: flat + tree profile of one DMRG2 sweep, with GC and allocation totals. - ANALYSIS.md: question-driven skeleton, to be filled only from real cluster results. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…me limits) The chi = 256, N = 100 pilot crossed the 1e-8 relative gate on sweep 1 and moved < 5e-9 absolute over 11 further sweeps, so suite 1 drops to 6 sweeps (suite 2 keeps 10: seed-growth needs ~6 sweeps to reach chi = 512 first). chi = 1024 leaves the default schedule — extrapolated single-threaded cost is days per point; extend once the chi <= 512 numbers justify it. Suite 7 moves to chi = 256 x 3 sweeps so the 7-point thread grid fits a day. sbatch limits sized accordingly (48 h DMRG, 24 h TDVP/threads); pilot also logged an odd/even sweep-cost alternation, queued in ANALYSIS.md Q5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the six per-workload sbatch scripts with one driver so every compared number comes from the same machine by construction: - phases 1-2 (DMRG, TDVP) run the two libraries concurrently, each single-threaded and taskset-pinned to one core on opposite sockets (2 of 96 cores active; no shared core, L3, or memory controller); - phase 3 (thread grid) runs one point at a time on the idle node; - inside every process, pin_cores() uses ThreadPinning.jl to pin Julia threads 1:1 within the affinity mask (pinthreads(:affinitymask)) and gives OpenBLAS workers disjoint mask cores when the pool is real. SMOKE=1 bash benchmark/slurm/run_all.sbatch rehearses the whole driver locally as plain bash; rehearsal passed on a 2-socket workstation with no pinning warnings, and disjoint 1:1 placement was asserted directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 🚀 New features to boost your workflow:
|
Member
|
This looks interesting. You had a discussion about this today? With Claude then? What is Rusty? Where can I see the results 😄 ? |
Member
Author
|
Accidentally opened a PR for this, still trying to run the benchmark on our cluster (rusty) |
Two failures from the first Rusty submissions: - job 6579930: batch shells have no interactive PATH, so 'taskset ... julia' died with 'No such file or directory'; - job 6579944: 'module load julia' is 1.11.2, which silently ignores the repo's Pkg [workspace] section, so --project=benchmark cannot see the root manifest and errors with a misleading 'Package ThreadPinning ... does not seem to be installed'. The script now resolves julia itself: $JULIA override, else the newest 1.12+ binary in the shared-home juliaup depot (~/.julia/juliaup, GPFS, identical on all nodes and the same install that resolved the manifests), else PATH — then prints the choice and fail-fasts on julia < 1.12. Also adds an ERR trap pointing at the per-process logs. Verified by rehearsing the full driver under env -i with no julia on PATH (the cluster batch-shell condition): all phases pass. Keeps the user's partition=gen edit; drops the module load. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'suite1_..._N100_<ts>.json' sorts lexically before 'suite1_..._N20_<ts>.json', so the first cluster analysis silently compared stale N=20 smoke files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
All sanity gates pass (energies <= 4.6e-9 relative at matched chi; TDVP Sz trajectories agree to ~1.5e-8). Headline findings: - TDVP (matched inner Krylov tol): MPSKit 2-4x faster at all chi. - Two-site DMRG: ITensor 1.5-1.6x faster dense, ~12x faster per U(1) sweep at chi=512 — attributed (REVIEW-flagged) to inner-eigensolver policy: ITensor caps local solves at krylovdim=3/maxiter=1 while MPSKit's dynamic tolerances drive krylovdim-30 solves toward 1e-14 under the run-past-convergence protocol; per-sweep costs and allocation volumes match that story. - U(1) grow-from-seed reaches the 1e-8 gate in ~2 s at any chi cap (workflow win, labelled as such). - Thread grid: MPSKit scales to 4.8x at (8,8) vs ITensor's 2.2x, from an 8x slower single-core baseline on this workload. pin_cores fix: read the affinity mask BEFORE pinthreads narrows the calling thread (JULIA_NUM_THREADS=1 previously saw a 1-core mask and bailed out of BLAS pinning — grid points (1,4)/(1,8) ran with floating BLAS workers), and re-pin Julia threads after openblas_pinthreads, whose pool includes the calling thread at nthreads=1. Verified locally for both regimes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Round 1 of the MPSKit-vs-ITensorMPS performance investigation, per the methodology decisions settled on 2026-07-05.
Targets the `bench/itensor-comparison` branch, not `main`.
What changed
Suites 1–2 (finite DMRG) reworked for exact parity
New suite 5 — TDVP throughput
New suite 7 — thread scaling
Rusty submission scripts (`benchmark/slurm/`)
Diagnostics
Verification
Not in this PR
Full-scale numbers (they come from Rusty via `benchmark/slurm/`, submitted manually), any `docs/src/benchmarks.md` content, SU(2)/TeNPy/cylinder suites.
🤖 Generated with Claude Code