diff --git a/.gitignore b/.gitignore index c08530bf9..cd3992068 100644 --- a/.gitignore +++ b/.gitignore @@ -266,4 +266,8 @@ docs/beginner/tutorials/html5/*.html petsc-custom/petsc petsc-custom/.petsc-version Untitled*.ipynb + +# Simulation output dumped at repo root (belongs in ~/+Simulations) Expt_Grains/ +*.passive_swarm.*.h5 +*.passive_swarm.*.xdmf diff --git a/CLAUDE.md b/CLAUDE.md index e75f70953..e3c85d21b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,5 +1,10 @@ # Underworld3 AI Assistant Context +> **MANDATORY**: Read `docs/developer/UW3_STYLE_CHARTER.md` before writing any code. +> It is the normative style contract for every session (human or AI), it is two pages, +> and it WINS over the surrounding code and over any other style document on conflict. +> Core clause: match the Charter, not the code next door — and flag deviations you find. + > **Note**: Human-readable developer documentation is in `docs/developer/` (Sphinx/MyST format). > For development history and completed migrations, see `docs/developer/ai-notes/historical-notes.md` diff --git a/docs/developer/UW3_STYLE_CHARTER.md b/docs/developer/UW3_STYLE_CHARTER.md new file mode 100644 index 000000000..65996e056 --- /dev/null +++ b/docs/developer/UW3_STYLE_CHARTER.md @@ -0,0 +1,160 @@ +# UW3 Style Charter + +**Status**: Normative. Every development session — human or AI — loads this document +and follows it. It is deliberately short; there is no excuse for not reading all of it. + +## 1. The Founding Rule + +Anyone — specifically, any working geodynamicist — must be able to read this code and +understand it. Over-complication and inconsistent patterns are the enemies of that +rule: a clever construction that saves ten lines but costs the reader a detour has +made the code worse. When in doubt, write the plain version. + +Don't forget this: Underworld3 is a Python API for building mathematical models, +particularly for geoscience. The founding rule applies as much to the Python scripts +and Jupyter notebooks as it does to the code we write under the hood. We write code +that makes models easy to read, and we always hide complex or irrelevant detail at +least one level below the notebooks and scripts. Inner workings (private +`_method_xxx()`) are never visible to the user. + +Consistent patterns: common choices are the defaults; uncommon choices are harder to +find. Documentation is great — the docstrings turn into the documentation and are +Jupyter-friendly. We do not shy away from equations in docstrings, and we do provide +good, simple examples. We don't overdo it, we don't force-feed our users with emoji, +and we don't pat ourselves on the back for our cleverness. We also don't care how +complicated something was to code or debug; we just report how it works. + +## 2. Match the Charter, Not the Surrounding Code + +**This is a load-bearing clause.** A month of drift means existing code is not a +reliable pattern to copy — drifted code copied becomes drift squared. You MUST follow +this Charter even where nearby code disagrees with it. When you find such a +disagreement, do not silently imitate it and do not silently "fix" it either: follow +the Charter in the code you write, and flag the existing deviation with a +`# TODO(BUG):` marker or a note to the maintainer. NEVER treat "the file next door +does it this way" as authority. + +## 3. Naming + +- Names state what a thing IS or DOES. No hedging prefixes — `maybe_`, `try_`, `do_` + are banned (this rule has been enforced twice already; it keeps coming back). +- Private helpers and attributes take a single underscore prefix; everything without + one is public API and must behave like it. +- Never name a variable `model`: use `constitutive_model` (material behaviour) or + `orchestration_model` / `uw.get_default_model()` (serialization system). + +```python +# BAD: def _maybe_install_snes_update(self): ... +# GOOD: def _attach_snes_update_dispatcher(self): ... +``` + +- Usually default to the mathematical naming - surface_flux_recovery is better than compute_topography. Geographic meshes might be the main exception. + +## 4. Comments + +- Comments state the physical or mathematical intent and the constraints the code + cannot show: why this term is in the weak form, why this barrier must precede that + collective, why a tolerance has this value. +- Never narrate mechanics (`# increment i`), never address the reviewer + (`# as requested`), never leave editorial musings (`# why is this here??`) — file + those as `# TODO(DESIGN):` or delete them. +- Delete commented-out code. Git remembers; the reader should not have to. Leave a TODO if there is reason to look up the deleted code for another time. +- Every intentional exception swallow states its sanctioned failure mode on the same + block. A bare `except Exception: pass` with no comment is a defect. +- Refer to methods by name, never by line number — line references rot within weeks. + +## 5. Structure + +- DRY after the 2nd occurrence: the moment you would paste a block a second time, + extract it instead. Copy-paste-as-reuse is the single worst drift pattern found in + the 2026-07 audit (2–6 diverging copies of the same machinery per hotspot file). +- No speculative generality: no parameters "for forward-compatibility", no reserved + branches that `pass`, no dead flags. Build it when it is needed. +- No defensive bloat: do not guard states that cannot occur (e.g. `try/except` around + importing a hard dependency). Guards imply the state is reachable; false guards lie. +- Before writing a helper, look for the existing one: `uw.function`, `uw.maths`, and + `src/underworld3/utilities/` are the discovery paths. Prefer a battle-tested + package over a hand-rolled implementation. + +## 6. API Conventions + +These settle the June-2026 drift (see `docs/reviews/2026-07/API-CONSISTENCY-REVIEW.md`). + +| Topic | Rule | +|---|---| +| BC argument order | `add__bc(value, boundary, ...)` — value first, matching the original trio, the published examples/benchmarks, and previous major versions (maintainer decision 2026-07-04). The newer boundary-first methods (`add_nitsche_bc`, `add_rotated_freeslip_bc`, `add_constraint_bc`) migrate to it; their current signatures are shimmed. | +| BC value parameter | ONE name across all BC methods ⟨pending maintainer confirmation: `conds` (implied by the original-order decision) vs `value`⟩. Until decided, do not introduce a third spelling. | +| Direction selection | Component masking via `None`/`sympy.oo` entries in the value vector; `direction=` only for a scalar constraint along a vector (defaults to outward normal); `normal=` strictly overrides the geometric surface-normal source. `components=` is deprecated — never in new code. | +| Solver capabilities | Anything that configures or reads one solver is a METHOD on that solver, lazily importing its `utilities/*` implementation (the `boundary_flux` pattern) — never a free function as the documented entry point. | +| Namespaces | Every user-facing module is exported from its subpackage `__init__`/`__all__` in the PR that creates it. No deep-import-only features. | +| Docstrings | NumPy/Sphinx style with RST `:math:`. This SUPERSEDES the Markdown-for-pdoc prescription still printed in `UW3_Style_and_Patterns_Guide.md` — that section is wrong; do not follow it. | + +## 7. Data Access + +- Variable data in new code uses the `array` property with the three-index shapes: + scalars `(N, 1, 1)`, vectors `(N, 1, dim)`, tensors `(N, dim, dim)`. +- Mesh coordinates are `mesh.X.coords`. +- NEVER in new code: `with mesh.access(...)` / `with swarm.access(...)`, `mesh.data`, + or `mesh.points`. They exist only so old code keeps running. +- The flat `.data` property carries ONE sanctioned exception: it bypasses all units + evaluation and re-packing, so it may be used for a raw variable-to-variable copy + inside the non-dimensionalisation boundary (where values are already consistent). + Any other use in new code goes through `.array`. + +```python +temperature.array[:, 0, 0] = values # GOOD — scalar, three indices +temperature.data[:, 0] = values # BAD — compatibility layer in new code +``` + +## 8. Tests + +- Every bug fix ships the regression test that would have caught the bug — written + first, shown to fail, then fixed. +- Test files follow `tests/test_NNNN_description.py` numbering and carry both markers: + a level (`level_1`/`level_2`/`level_3`) and a tier (`tier_a`/`tier_b`/`tier_c`). +- Validate a new test's own correctness before changing library code to satisfy it. +- NOTE: test tiers A,B,C ... A are the hardened tests that have been explicitly reviewed. You can build code around tier A tests, but tier C are tests that are not mature enough to drive coding. + +## 9. Scope Discipline for AI Sessions + +Fix what you were asked to fix. No drive-by refactors, renames, or "while I was here" +cleanups — they turn a reviewable diff into an unreviewable one and have been the +main vector of drift. When you find a real problem outside your task, mark the exact +location with the project's `# TODO(BUG): description` format (see CLAUDE.md) and +mention it in your report; do not fix it silently. + +## 10. Authority + +This Charter wins on any conflict. One governing document per topic: + +| Topic | Governing document | +|---|---| +| Data access | `docs/developer/subsystems/data-access.md` | +| Units | `docs/developer/design/UNITS_SIMPLIFIED_DESIGN_2025-11.md` | +| Testing tiers | `docs/developer/TESTING-RELIABILITY-SYSTEM.md` | +| Branching & releases | `docs/developer/guides/branching-strategy.md` | + +`UW3_Style_and_Patterns_Guide.md` remains as the detailed reference, but where it and +this Charter disagree (notably docstring format and coordinate access), the Charter wins. + +## 11. Things to Remember + +Underworld is a parallel code — no feature is complete if it only works in serial. + +Underworld functions are a better choice than PETSc, PETSc is a better choice than MPI, +and serial libraries like scipy are usually a poor choice, generally only for +prototypes. Do not leave them in code without checking. + +Examples: + +- `uw.print` or `uw.timing` are better than PETSc because they wrap the best of the + PETSc tools in a uw-styled interface. Avoid `.npz` or `.csv` logging when we have + good parallel output that is properly flushed. +- The user should never see parallel / mpi calls. `uw.mpi.rank` or `mpi.rank` is a + mistake — there should be a wrapper already and it will work better. Guide the users! +- Scripts and drivers written during development sessions use uw's own machinery, not + hand-rolled equivalents. Option parsing is the canonical example: use `uw.Params` / + `Param` (notebook-editable defaults, units-aware, `-uw_name value` CLI overrides via + PETSc options) — not `argparse`, not ad-hoc `sys.argv` handling, not a config dict at + the top of the file. A session script is a model a reader should be able to run and + read; the founding rule applies to it in full. diff --git a/docs/reviews/2026-07/API-CONSISTENCY-REVIEW.md b/docs/reviews/2026-07/API-CONSISTENCY-REVIEW.md new file mode 100644 index 000000000..2726e22b3 --- /dev/null +++ b/docs/reviews/2026-07/API-CONSISTENCY-REVIEW.md @@ -0,0 +1,334 @@ +# API Consistency Review — July 2026 Quality Campaign (Dimension 3) + +**Status**: audit complete; findings adversarially verified 2026-07-03 +**Base**: `development` @ `1d003481` (audit worktree, campaign index at `e848d131`) +**Scope**: public API surface — the boundary-condition family, solver +constructors, solver-scoped capabilities (rotated BC, boundary flux, custom +multigrid, update callbacks), mode attributes, quantity factories, meshing +constructors, and namespace exposure (`uw.*`, subpackage `__init__`/`__all__`). + +Abbreviation used throughout: `pyx` = `src/underworld3/cython/petsc_generic_snes_solvers.pyx`. + +## Overview + +June 2026 added new boundary-condition entry points, three solver-scoped +capability families, and two new utility modules across many non-overlapping +AI-assisted sessions. Each addition is individually reasonable; collectively +they have split the public API into two argument-order camps, two +value-parameter names, two custom-multigrid entry points with different names +and different correctness envelopes, and a set of user-facing modules that +resolve only via deep import. + +This review (a) proposes THE single convention set for the codebase and +(b) lists every verified deviation with a deprecation-shim design. Per the +maintainer's constraints, **every fix harmonizes via a shim** — old signatures +keep working for at least one release cycle with a `DeprecationWarning`; +nothing here is a hard break, and nothing touches solver numerics. All shims +are zero-cost when unused (one inline `isinstance`/`type` check or a +`None`-default keyword alias; no wrapper layers). + +Every location cited below was read directly in this worktree; line numbers +are exact at `1d003481`. Nine findings are adversarially verified (API-01 … +API-09); three lower-severity observations were evidence-checked by the author +but did not go through the adversarial pass (API-10 … API-12) and are tabled +separately. Details refuted during verification are recorded in the appendix +so the same false leads are not re-found. + +**Scale note (API-01)**: the migration surface for the BC argument-order fix +is far larger than early estimates. At this ref the legacy order appears at +roughly **920 call sites under `docs/`** (~40 in `.md`, ~695 in `.py` +examples, ~185 in `.ipynb`) **plus ~450 in `tests/`** — a ~1,370-site wave. +The shim makes this safe, but the deprecation-window length and the eventual +removal of the old order need explicit maintainer sign-off: this is the +most-used user-facing API in the codebase. + +## Changes Made + +None — audit only. Proposed changes are listed as findings and are scheduled +for Wave C (API harmonization) and Wave E (docs alignment). + +## System Architecture + +### The API landscape as found (all read at `1d003481`) + +**Boundary-condition family** (methods on solver classes): + +| Method | Signature (as found) | Location | +|---|---|---| +| `add_essential_bc` | `(conds, boundary, components=None)` | `pyx:1530` | +| `add_natural_bc` | `(conds, boundary, components=None)` | `pyx:1573` | +| `add_dirichlet_bc` | `(conds, boundary, components=None)` | `pyx:1631` | +| `add_nitsche_bc` (SNES_Vector) | `(boundary, g=None, direction=None, gamma=10.0, theta=1, local_h=True)` | `pyx:3317` | +| `add_rotated_freeslip_bc` (Stokes) | `(boundary, normal=None)` | `pyx:5170` | +| `add_nitsche_bc` (SNES_Stokes_SaddlePt) | `(boundary, g=None, direction=None, normal=None, gamma=10.0, theta=1, mask=None, local_h=True)` | `pyx:5293` | +| `add_constraint_bc` (Stokes_Constrained) | `(boundary, g=0.0, normal=None, screening=None, augmentation=None, augmentation_base=1.0e4, degree=None)` | `systems/solvers.py:2335` | + +The legacy trio is value-first; every method added since is boundary-first. +These seven are the only BC-adder definitions in `src/` (an +`add_essential_p_bc` exists only as a commented-out stub near `pyx:5149`). +The prescribed datum is spelled `conds` in the trio and `g` in exactly three +newer methods (the two Nitsche overloads and `add_constraint_bc`); +`add_rotated_freeslip_bc` takes no value at all (free-slip `g=0` implicit) and +participates in the drift only through `normal=`. The `components=` selector +already carries a live `DeprecationWarning` (`pyx:1449-1455`) in favour of +`None`/`sympy.oo` entries in the value vector (`pyx:1393-1398, 1458-1464`); +`direction=` (defaulting to the outward surface normal, `pyx:3369-3376`) and +`normal=` (three per-method meanings: rotation source at `pyx:5170`, Nitsche +consistency normal at `pyx:5293`, `Gamma_P1` default at +`systems/solvers.py:2396`) overlap with it. + +**Solver-scoped capabilities.** `add_update_callback` (`pyx:301`), +`boundary_flux` / `boundary_flux_field` (`pyx:2165`/`pyx:2183`, lazily +importing `utilities/boundary_flux.py` at `pyx:2180, 2190`), and the rotated +free-slip family (`pyx:5170/5253/5272`, delegating to +`utilities/rotated_bc.py`) all follow the good pattern: **a method on the +solver that lazily imports its implementation module**. Custom multigrid is +the outlier: the solver *method* `set_custom_mg(coarse_meshes, kind=...)` +(`pyx:259`) configures the **legacy serial-only, finest-only-reduction, +single-field-only** path (`_require_serial` called at +`utilities/custom_mg.py:640`; `NotImplementedError` for non-empty PC prefixes, +i.e. Stokes blocks, at `custom_mg.py:646-647`), while the correct +parallel-capable BC-per-level path is the free function +`set_custom_fmg(solver, coarse_meshes, *, builder=..., field_id=...)` +(`custom_mg.py:605`) reachable only by deep import. The method's own docstring +(`pyx:280-291`) advertises Stokes velocity-block support that only the +deep-import path delivers. + +**Mode attributes.** `consistent_jacobian` is set as a bare attribute +(`pyx:92`) documented only by a code comment block (`pyx:71-91`). It is +dispatched by exact match in two places: `_jacobian_source` (`pyx:160-199`; +`if not mode` → Picard, `mode == "continuation"` → blended, **any other truthy +value falls through to full Newton**) and the solve path +(`pyx:1096`, `== "continuation"` selects the two-phase continuation solve). +`utilities/rotated_bc.py:396` branches on the same unvalidated tri-state. + +**Namespace exposure.** `utilities/__init__.py` (86 lines) imports none of +`rotated_bc`, `boundary_flux`, `custom_mg` — all in-package uses are +method-local lazy imports, and docstring cross-references such as +`:mod:`underworld3.utilities.custom_mg`` (`pyx:290-291`) resolve only after a +deep import. `meshing/__init__.py` (`__all__` at line 68) neither imports nor +lists `bounding_surface.py`'s `BoundingSurface` (`:41`) and +`register_radial/plane/box_face_surfaces` (`:213/:224/:233`), so +`uw.meshing.BoundingSurface` raises `AttributeError` even though +`Mesh.register_tangent_slip_provider` requires a `BoundingSurface` instance +(`discretisation/discretisation_mesh.py:2719-2730`). + +**Quantity factories.** `uw.quantity` (`__init__.py:223` → +`function/quantities.py:883`) returns a `UWQuantity`; `uw.create_quantity` +(`__init__.py:253` → `units.py:1115`, implemented as `value * ureg(units)` at +`units.py:60-64`) returns a **raw Pint Quantity**. Same conceptual purpose, +silently different objects downstream; `UWQuantity` itself is not exposed at +top level, so `isinstance` checks require a deep import. +`tests/test_0640_api_consistency_regression.py:140,254,274` currently pins +`create_quantity` as public. + +**Constructors.** Base `SNES_Scalar` (`pyx:2265`) and every sibling — +`SNES_Vector` (`pyx:3200`), `SNES_Darcy`, `SNES_Stokes`, all projections — +order `(mesh, u_Field, degree, verbose, ...)`. `SNES_Poisson` +(`systems/solvers.py:190-198`) is uniquely inverted: +`(mesh, u_Field, verbose=False, degree=2, ...)`. Meshing constructors are +internally consistent in their grandfathered camelCase (`minCoords`, +`cellSize`); the one substantive drift is that the `units=` kwarg exists only +on the two Cartesian constructors (`meshing/cartesian.py:43, 970`). +`uw.function.evaluate` / `global_evaluate` were checked and found mutually +consistent — no finding. + +### THE convention set (proposed) + +> **Maintainer decision 2026-07-04 — supersedes C1 and amends C2 below.** This review +> proposed boundary-first BC ordering; L. Moresi ruled the other way: the ORIGINAL +> value-first order `add__bc(value, boundary, ...)` is canonical — it is the +> most widely used in the published examples and benchmarks and matches previous +> major versions. Wave C therefore migrates the NEWER boundary-first methods +> (`add_nitsche_bc`, `add_rotated_freeslip_bc`, `add_constraint_bc`) to value-first, +> shimming their current signatures; the legacy trio is already conforming. The +> canonical datum name is pending confirmation (`conds`, implied by the original-order +> decision, vs the `value` proposed here). The normative statement lives in +> `docs/developer/UW3_STYLE_CHARTER.md` §6, which wins over this review on conflict. +> The analysis below is preserved as written for the audit record. + +| # | Convention | Rule | +|---|---|---| +| C1 | **BC argument order** | `add__bc(boundary, value, ...)` — boundary label (str) first, prescribed datum second. Matches every method added since the trio; the trio is shimmed. | +| C2 | **Value-parameter name** | ONE name: **`value`** — the prescribed boundary datum in every `add_*_bc`. Self-documenting for non-FEM readers (founding readability rule). `conds=` and `g=` accepted as `None`-default deprecated keyword aliases for one cycle. | +| C3 | **Direction selection** | ONE mechanism: component masking via `None`/`sympy.oo` entries in `value` (as today); a scalar constraint along a vector uses `direction=` (defaults to outward normal). `normal=` is reserved strictly for overriding the *geometric surface-normal source* and is never a component mask. `components=` completes its already-warned deprecation. | +| C4 | **Method-on-solver rule** | Any capability that configures or reads one specific solver is a method on that solver, delegating via lazy import to a `utilities/*` implementation module (the `boundary_flux` pattern, `pyx:2180`). Free functions remain the implementation layer, not the documented entry point. | +| C5 | **Namespace exposure** | Every user-facing module is added to its subpackage `__init__`/`__all__` in the PR that creates it. Top-level `uw.*` keeps one canonical factory per concept. | +| C6 | **Constructor parameters** | `(mesh, , , , DuDt, DFDt)` — discretisation before behaviour flags, everywhere. New parameters are `snake_case`; existing camelCase is grandfathered but never extended. | +| C7 | **Mode/state attributes** | Multi-state switches (e.g. `consistent_jacobian`) are validated properties with NumPy docstrings; unrecognized values raise `ValueError` at assignment, never silently select a behaviour. Falsy values normalize to the canonical `False`. | +| C8 | **Docstrings** | NumPy/Sphinx style with RST `:math:` (settled maintainer decision). Google-style `Args:` blocks are nonconforming. | +| C9 | **Deprecation shims** | Old signature detected inline (one type check on a positional, or a `None`-default keyword alias); `warnings.warn(..., DeprecationWarning, stacklevel=2)`; forward to the new path. No wrappers, no decorators — zero cost on the new signature. Removal only after an explicitly signed-off window (see API-01). | + +### Shim designs (per deviation class) + +**Arg-order shim (API-01)** — unambiguous because a boundary label is always a +`str` and a BC value never is (`add_condition` at `pyx:1359` raises +`ValueError` for string conds; all repo callers pass string boundaries, enum +users all use `.name`; no caller uses `conds=`/`boundary=` keywords): + +```python +def add_essential_bc(self, boundary, value=None, components=None, **kw): + if not isinstance(boundary, str): # old order: (conds, boundary) + boundary, value = value, boundary # value slot holds the label + warnings.warn("add_essential_bc(conds, boundary) is deprecated; " + "use add_essential_bc(boundary, value)", + DeprecationWarning, stacklevel=2) + ... +``` + +One `isinstance` on the hot path; new-signature calls pay nothing else. + +**Keyword-alias shim (API-02)** — `conds=None` / `g=None` kept in the +signature one cycle; if supplied and `value` is not, forward with a warning. +`None` is not a meaningful datum for these methods, so the sentinel is safe. + +**Constructor-order shim (API-08)** — the swapped pair is `(verbose, degree)` +vs `(degree, verbose)`; the legacy detection must be `type(third) is bool`, +**not** `isinstance`, because `isinstance(True, int)` is `True`. + +**Attribute-validation shim (API-04)** — `consistent_jacobian` becomes a +property; the setter accepts `{False, True, "continuation"}`, normalizes other +falsy values (`None`, `0`) to `False` (today `if not mode` treats them as +Picard), and raises `ValueError` for anything else. Invalid values currently +mis-select full Newton silently, so raising is a bug-fix, not a break. + +## Findings — verified + +All nine adversarially verified against this worktree; ranked +most-severe-first. + +| ID | Location | Severity | Effort | Finding | Proposed fix | +|---|---|---|---|---|---| +| API-01 | `pyx:1530,1573,1631` vs `pyx:3317,5170,5293`, `systems/solvers.py:2335` | high | M | BC family split into two argument-order camps: the legacy trio takes `(conds, boundary)`, every later method takes `(boundary, ...)`. The migration surface is ~920 call sites in `docs/` plus ~450 in `tests/` (~1,370 total — an early "~101 docs sites" estimate was a ~13× undercount). This is the most-used user-facing API in the codebase. | Adopt boundary-first (C1) on the trio with the `isinstance(boundary, str)` arg-order shim; update docs/examples in the same wave (Wave C + Wave E). **Deprecation-window length and eventual removal of the old order require explicit maintainer sign-off** — do not hard-remove after one cycle by default. | +| API-02 | `pyx:1530/1573/1631` (`conds`) vs `pyx:3317,5293`, `solvers.py:2335` (`g`); `pyx:1449-1455` (`components`), `pyx:3369-3376` (`direction`), `pyx:5170/5293`, `solvers.py:2396` (`normal`) | medium | S | Value-parameter and direction vocabulary drift: the datum is `conds` in the trio and `g` in exactly three newer methods (`add_rotated_freeslip_bc` takes no value; there is no fifth `g` method). Direction/masking is spread across four overlapping mechanisms: `components=` (deprecated), `None`/`oo` masking, `direction=`, and `normal=` with three per-method meanings. | ONE datum name `value` (C2) with `conds=`/`g=` as `None`-default deprecated aliases; ONE direction convention (C3): masking in `value`, `direction=` for scalar-along-vector, `normal=` strictly for geometric-normal override; finish the `components=` deprecation. Lands in the same edit as API-01. | +| API-03 | `pyx:3317` vs `pyx:5293` | medium | S | Two same-named `add_nitsche_bc` methods diverge: the `SNES_Vector` version lacks `normal=` and `mask=`, so code written against the Stokes variant raises `TypeError` when moved to a vector solver. The vector docstring (`pyx:3336-3349`) does not itself document those parameters, but cross-references `SNES_Stokes_SaddlePt.add_nitsche_bc` twice ("See ... for details") with no note that the vector variant lacks them. | Align the `SNES_Vector` signature: accept `normal=` with the same geometric-normal-override semantics (Stokes `pyx:5391-5395`); accept `mask=` and raise a clear `NotImplementedError` naming the limitation if genuinely unsupported. Signature/docs only — no numerics. | +| API-04 | `pyx:92` (set), `pyx:71-91` (comment), `pyx:160-199` (`_jacobian_source`), `pyx:1096` (solve dispatch) | medium | S | `consistent_jacobian` is a bare, undocumented tri-state `False\|True\|"continuation"` with **no validation anywhere**. `_jacobian_source` falls through to the full-Newton tangent for any unrecognized truthy value — `"picard"`, `"Continuation"`, `1` all silently select Newton; the exact-string match at `pyx:1096` likewise silently skips the continuation solve on a typo. | Validated property (C7): setter accepts `{False, True, "continuation"}`, normalizes falsy to `False`, raises `ValueError` otherwise; NumPy docstring lifted from the `pyx:71-91` comment block. Validation covers both dispatch sites. Behaviour bit-identical for the three valid values (within the pyx no-numerics ground rule). | +| API-05 | `pyx:259` (`set_custom_mg`) vs `utilities/custom_mg.py:605` (`set_custom_fmg`), `:640` (`_require_serial` call), `:646-647` (single-field gate) | medium | S | Custom multigrid has two entry points with different names, parameter vocabularies (`kind=` vs `builder=`/`field_id=`), and correctness envelopes: the discoverable solver *method* is the legacy serial-only, finest-only-reduction, single-field-only path — and its docstring (`pyx:280-291`) promises Stokes velocity-block support that only the deep-import `set_custom_fmg` path delivers. | Add `SolverBaseClass.set_custom_fmg(coarse_meshes, *, builder=..., field_id=None, verbose=False)` delegating lazily to `utilities.custom_mg` (C4, `boundary_flux` pattern at `pyx:2180`); export `set_custom_fmg` from `utilities/__init__`; `set_custom_mg` gains a `DeprecationWarning` (behaviour preserved one cycle); unify on `builder=` with `kind=` as deprecated alias. | +| API-06 | `utilities/__init__.py` (86 lines); `meshing/__init__.py:68` (`__all__`); `meshing/bounding_surface.py:41,213,224,233` | medium | S | Exposure gaps: `utilities/__init__` imports none of `rotated_bc`/`boundary_flux`/`custom_mg` (docstring cross-refs like `pyx:290-291, 497` resolve only via deep import); `meshing/__init__` neither imports nor lists `BoundingSurface` and the three `register_*_surfaces` helpers, so `uw.meshing.BoundingSurface` raises `AttributeError` despite being required by `Mesh.register_tangent_slip_provider` (`discretisation_mesh.py:2719-2730`). | Add `from . import rotated_bc, boundary_flux, custom_mg` (+ `set_custom_fmg` re-export) to `utilities/__init__`; add the four `bounding_surface` names to `meshing/__init__` imports and `__all__` (C5). Pure additions, no shim; adopt C5 as the go-forward convention. | +| API-07 | `__init__.py:223,253`; `function/quantities.py:883`; `units.py:60-64,1115` | medium | S | Duplicate top-level quantity factories with different return types: `uw.quantity` → `UWQuantity`; `uw.create_quantity` → raw Pint `Quantity` (`value * ureg(units)`). Same purpose, silently different objects downstream; `UWQuantity` not exposed at top level (deep import needed for `isinstance`); `test_0640_api_consistency_regression.py:140,254,274` pins `create_quantity` as public. | `uw.quantity` is THE factory (matches CLAUDE.md units principles). `create_quantity` keeps exact behaviour/return type one cycle but emits `DeprecationWarning` naming `uw.quantity`; expose `uw.UWQuantity`; update test_0640 in the same PR. | +| API-08 | `systems/solvers.py:190-198` vs `pyx:2265` and siblings (`solvers.py:396,1114,2618,2933,3165,3316`) | medium | S | `SNES_Poisson.__init__` is `(mesh, u_Field, verbose=False, degree=2, ...)` while its base and every sibling put degree before verbose — SNES_Poisson is uniquely inverted. A positional `SNES_Poisson(mesh, None, 3)` intended as `degree=3` silently sets `verbose=3` and auto-creates a degree-2 field; with an existing field `T`, `SNES_Poisson(mesh, T, 3)` silently enables verbose (degree is unused then — see refuted-claims appendix). No in-repo caller is currently bitten (all keyword calls). | Reorder to `(mesh, u_Field, degree, verbose, ...)` (C6) with the constructor-order shim: `type(third) is bool` → legacy `verbose` + `DeprecationWarning`. Add a positional-call regression test. | +| API-09 | `pyx:2183` (`boundary_flux_field(..., scale=)`), `pyx:5272` (`dynamic_topography(..., buoyancy_scale=)`), `systems/solvers.py:2461` (`topography(..., buoyancy_scale=, reference=)`), `utilities/boundary_flux.py:256` (`boundary_flux_to_field`) | medium | S | Vocabulary drift across the three dynamic-topography/flux-recovery paths: field-write vs expression-return, `scale=` vs `buoyancy_scale=`, and a free function (`boundary_flux_to_field`) spelled differently from the method it backs (`boundary_flux_field`). **Do not alias `scale=` to `buoyancy_scale`**: they are not the same factor — `buoyancy_scale` is Δρg used as a divisor with the sign internal (`rotated_bc.py:775`, `-s/buoyancy_scale`), while `scale` is a generic multiplier whose topography value is `-1/(Δρg)` (negated reciprocal; primary tested use is heat flux/Nusselt, `test_1019:38`, `parallel/test_1065:44`). | Safe scope: (1) rename the free function `boundary_flux_to_field` → `boundary_flux_field` to match the method, old name aliased one cycle; (2) document the expression-return (`topography`) vs field-write (`dynamic_topography`/`boundary_flux_field`) distinction; (3) document in `boundary_flux_field`'s docstring that `scale = -1/buoyancy_scale` for topography — do not rename the parameter. | + +## Findings — unverified + +Evidence-checked by the author at the cited lines but **not** adversarially +verified; treat as candidates pending a verify pass before entering the +remediation worklist. + +| ID | Location | Severity | Effort | Finding | Proposed fix | +|---|---|---|---|---|---| +| API-10 | `pyx:3200-3212` vs `pyx:2277-2281` (SNES_Scalar auto-create), `pyx:735-737` (`Unknowns.u` setter ignores `None`) | low | S | `SNES_Vector.__init__` advertises `u_Field=None` like `SNES_Scalar` but never auto-creates the variable (`self.Unknowns.u = u_Field` directly at `pyx:3212`, no `if u_Field is None` branch); the setter silently ignores `None`, so construction appears to succeed and the solver fails later with an obscure `AttributeError`. Same-named parameter, different contract between sibling bases. | Either auto-create the vector variable as `SNES_Scalar` does, or raise `ValueError("u_Field is required")` at construction. No shim needed — current behaviour is already a crash, just delayed and cryptic. | +| API-11 | `meshing/cartesian.py:43,378,970` (accept `units=`, thread it to `Mesh` at `:352,909,1367`) vs `meshing/annulus.py`/`spherical.py` (no `units=` anywhere; `Mesh` itself accepts it, `discretisation_mesh.py:274`) | low | M | The `units=` kwarg exists on **three** Cartesian constructors only (an earlier "two" was a miscount — `BoxInternalBoundary` at `:378` also has it), so the units system's mesh entry point is geometry-dependent: annulus/spherical auto-non-dimensionalise UWQuantity geometry args (e.g. `annulus.py:116-121`) but never pass coordinate units to `Mesh`. Worse, the three Cartesian docstrings disagree with themselves: two label `units=` "**Deprecated**" (`:83-85`, `:1005-1006`) while one documents it as live ("Coordinate units for unit-aware arrays", `:421-422`) and `StructuredQuadBox` actively auto-detects units from UWQuantity inputs (`:1106-1117`). | First decide the parameter's status (live vs deprecated) — the docstrings must stop contradicting the code; if live, thread `units=` through the remaining constructors (pure addition, default `None` preserves behaviour). Can be scheduled with a units wave rather than Wave C. | +| API-12 | `units.py:100,349,420,517,571,1119` (representative) | low | S | Top-level-exported units API uses Google-style `Args:` docstrings — including `uw.create_quantity` — against the settled NumPy/RST standard (C8); newer solver/BC docstrings already conform, making the exported units module the visible outlier. | Mechanically convert `units.py` public-function docstrings to NumPy style during Wave E (`scripts/docstring_sweep.py` exists). Full docstring census belongs to Dimension 6. | + +## Testing Instructions + +Validation plan for the eventual Wave C/E fixes (each PR cites its finding ID): + +- **Baseline**: `./uw build` in the wave worktree, then + `pytest -m "level_1 and tier_a"` green before and after; `tier_a or tier_b` + before merge (campaign ground rule). +- **Every shim** lands with two tests: (1) the OLD signature produces + identical results and emits exactly one `DeprecationWarning` + (`pytest.warns(DeprecationWarning)`); (2) the NEW signature emits none + (`warnings.simplefilter("error")` inside the test). +- **API-01/02**: run the arg-order shim tests on all three trio methods, + including a mixed case (`add_dirichlet_bc("Top", 0.0)` new vs + `add_dirichlet_bc(0.0, "Top")` old). Grep docs/tests after the Wave E sweep: + `grep -rE "add_(essential|natural|dirichlet)_bc\(" docs tests` call sites + must all be new-order. +- **API-03**: `vector_solver.add_nitsche_bc(bdy, normal=n)` must not raise + `TypeError`; `mask=` on the vector solver raises `NotImplementedError` with + a message naming the limitation (if that branch is taken). Existing Nitsche + tier_a tests bit-identical. +- **API-04**: `solver.consistent_jacobian = "picard"` (and `"Continuation"`) + raises `ValueError`; `False`/`True`/`"continuation"` round-trip; `None`/`0` + normalize to `False`; existing users + `tests/test_1018_rotated_freeslip.py:237` and + `tests/parallel/test_1064_rotated_freeslip_parallel.py:191` (bare-attribute + assignment of valid values) stay green; tier_a solver results bit-identical. +- **API-05/06**: + `python -c "import underworld3 as uw; uw.utilities.custom_mg.set_custom_fmg; uw.meshing.BoundingSurface"` + succeeds without deep imports; `set_custom_mg` warns but reproduces its + current results; custom-MG tests (test_1015-1017 range) and rotated + free-slip tests (`test_1018`, `parallel/test_1064`) green; **np2/np4 + parallel runs required** for anything touching `custom_mg`. +- **API-07**: update `tests/test_0640_api_consistency_regression.py` in the + same PR that adds the `create_quantity` warning (it calls it bare at + `:254`); assert `uw.UWQuantity` importable and + `isinstance(uw.quantity(1, "m"), uw.UWQuantity)`. +- **API-08**: positional regression test — `SNES_Poisson(mesh, None, 3)` after + the fix warns nothing and creates a degree-3 unknown; + `SNES_Poisson(mesh, None, True)` (legacy verbose) warns and preserves + degree=2. +- **API-09**: pure rename/docs — heat-flux tests + `tests/test_1019_boundary_flux.py` and + `tests/parallel/test_1065_boundary_flux_parallel.py` green; old + free-function name importable with warning for one cycle. + +## Known Limitations + +- Line numbers are exact at `development@1d003481` and will drift as waves + land; finding IDs, not line numbers, are the stable reference. +- This review covers the *public* API surface; internal call sites on + deprecated data-access patterns (~41 sites, Wave B) are Dimension 1/4 + territory. After Wave B, internal code must not exercise the shims added + here (shim-warning tests enforce this incidentally). +- The `value` canonical name (C2) is a proposal; the zero-churn alternative + (standardize on `g`) flips only the alias direction — shim mechanics + identical. Maintainer's call. +- API-01's eventual removal timeline is explicitly **not** decided here: with + ~1,370 old-order call sites in docs and tests, "one release cycle then + remove" would hard-break the most-used API; the shim can be kept + indefinitely at negligible cost. +- `uw.function.evaluate`/`global_evaluate` and the meshing constructor family + (beyond API-11 and grandfathered camelCase) were checked and found + internally consistent — no findings; recorded so coverage is explicit. +- All `pyx` proposals are confined to naming/docs/validation/shims; no + numerics change is proposed anywhere in this review (campaign ground rule). + +## Appendix: refuted or corrected claims + +No whole finding was refuted, but these sub-claims were corrected during +adversarial verification — recorded so they are not re-found: + +1. **"~101 documentation call sites teach the old BC order"** — WRONG (~13× + undercount). True surface at this ref: ~920 in `docs/` + ~450 in `tests/` + (≈1,370 total; re-counted by this author: + `grep -rEoh "add_(essential|natural|dirichlet)_bc\(" docs | wc -l` → 925 + across all file types, 920 restricted to `.md`/`.py`/`.ipynb`; same over + `tests` → 449. No `docs/_build` exists at this ref to inflate the count). +2. **"The datum is spelled `g` in all five newer BC methods"** — WRONG. + Exactly **three** methods take `g` (`pyx:3317`, `pyx:5293`, + `solvers.py:2335`); `add_rotated_freeslip_bc` (`pyx:5170`) takes no value + parameter, and there is no fifth method (`add_essential_p_bc` is + commented out near `pyx:5149`). +3. **"The vector `add_nitsche_bc` docstring documents `normal`/`mask` + parameters it does not accept"** — WRONG as stated. It never names them + (`pyx:3318-3350` documents only its six actual parameters); the problem is + the unqualified cross-references to the Stokes docstring (`pyx:3340,3344`) + where they ARE documented. +4. **"`scale=` on `boundary_flux_field` is the same physical factor as + `buoyancy_scale` and should be renamed/aliased"** — WRONG and dangerous: + `buoyancy_scale` is Δρg used as a divisor with the minus sign internal + (`rotated_bc.py:775`), `scale` a generic multiplier whose topography value + is `-1/(Δρg)` — treating them as one factor invites sign/reciprocal errors. + Document the relationship instead (see API-09). +5. **"`SNES_Poisson(mesh, T, 3)` silently changes the discretisation"** — + WRONG for an existing field `T`: `degree` is only used when `u_Field is + None` (auto-create at `pyx:2277-2281`; the `Unknowns.u` setter ignores a + `None` reassignment at `pyx:735`), so `T`'s own degree governs; the call + only silently sets `verbose=3`. The genuine wrong-discretisation case is + `SNES_Poisson(mesh, None, 3)` (see API-08). +6. **"The `consistent_jacobian` setter should accept exactly + `{False, True, "continuation"}`"** — needs one refinement: other falsy + values (`None`, `0`) currently behave as Picard via `if not mode` + (`pyx:179`), so the setter must normalize falsy to `False` rather than + reject it (see API-04 fix). + +## Sign-Off + +| Role | Name | Date | Status | +|------|------|------|--------| +| Maintainer | Louis Moresi | 2026-07-05 | Pending review | +| Author | Claude (audit session, Dimension 3 — api) | 2026-07-03 | Complete | \ No newline at end of file diff --git a/docs/reviews/2026-07/BRANCH-TRIAGE-LEDGER.md b/docs/reviews/2026-07/BRANCH-TRIAGE-LEDGER.md new file mode 100644 index 000000000..417d88b51 --- /dev/null +++ b/docs/reviews/2026-07/BRANCH-TRIAGE-LEDGER.md @@ -0,0 +1,829 @@ +# Branch Triage Ledger — Quality Audit 2026-07 + +## Overview + +This ledger triages every local branch and worktree of the underworld3 repository as part of +the July 2026 quality audit. The June 2026 development surge left ~40 investigated branches +plus ~40 merged-and-clean worktrees behind. Each investigated branch was checked against +`development` (audit baseline `1d003481`; development tip at verification time `9bd6c8ee`) +for: unmerged commits (`git merge-base --is-ancestor`, `git cherry`, content diffs against +squash merges), unpushed commits (local vs `origin`), and uncommitted work in the attached +worktree (`git status --porcelain`, cross-branch grep for the dirty content). + +Verdict taxonomy: + +- **LAND** — complete, validated, unmerged work that should be pushed and PR'd to development. +- **EXTRACT** — the branch as a whole is dead, but specific commits, uncommitted files, or + documents exist nowhere else and must be rescued before deletion. +- **KEEP_ACTIVE** — in-flight work; do not touch. +- **ARCHIVE_DELETE** — fully superseded by merged PRs; tag `archive/` then delete + branch (and worktree, if any). +- **REMOVE_WORKTREE_ONLY** — branch fully merged and worktree clean; remove the worktree and + delete the branch after the archive tag. + +Verdict counts (investigated branches): **LAND 4, EXTRACT 11, KEEP_ACTIVE 2, +ARCHIVE_DELETE 15, REMOVE_WORKTREE_ONLY 8**, plus a bulk list of 40 merged+clean +REMOVE_WORKTREE_ONLY candidates. **14 rows carry risk_of_loss = HIGH** — every one of them +holds work that exists in exactly one place (unpushed commits or uncommitted worktree files). + +## Changes Made + +None — audit only; proposed changes are listed as findings (verdicts and the execution +protocol below). No branch, worktree, or file outside `docs/reviews/2026-07/` was modified. + +## System Architecture + +The repository uses a hub of worktrees under `.claude/worktrees/`, one per work stream, each +with its own pixi environment. `development` is the integration trunk; `main` is the release +branch. Session isolation means abandoned worktrees accumulate: some are pure leftovers of +squash-merged PRs, some hide the only copy of uncommitted work. Three worktrees are +**misnamed relative to the branch they hold** (a hazard for any cleanup script that assumes +worktree name == branch name): + +| Worktree directory | Actual branch checked out | +|---|---| +| `.claude/worktrees/custom-mg-prolongation` | `feature/rotated-freeslip-bc` | +| `.claude/worktrees/vep-loading-unloading` | `feature/exp-integrator-investigation` | +| `.claude/worktrees/integrate-surface-submesh` | `feature/fs-surface-smoother-driver` | +| `.claude/worktrees/in-memory-checkpoint` | `docs/snapshot-toolkit-changelog` | +| `.claude/worktrees/winslow-mesh-smoother` | `feature/anisotropic-metric-mover` | +| `.claude/worktrees/snesfas-spike` | `docs/snesfas-investigation` | + +All cleanup actions must therefore key on the **branch**, verified with `git worktree list`, +never on the directory name. + +--- + +## HIGH-RISK ROWS — read these first + +These 14 rows hold work that exists in **exactly one place**. Deleting the branch or worktree +without the listed rescue step loses it permanently. + +| # | Branch | Verdict | What is at risk | +|---|--------|---------|-----------------| +| 1 | `feature/adapt-on-top` | LAND | Entire Layer-2 NVB/SBR adapt-on-top engine — local-only, **no origin remote** | +| 2 | `bugfix/custom-mg-parallel` | KEEP_ACTIVE | Continuation of the above + today's np>1 cross-partition transfer, **unpushed**, dirty design docs | +| 3 | `bugfix/yield-homotopy` | LAND | The planned PR-2 after #258 (yield homotopy + FMG monitor-leak fix) — **local-only, unpushed** | +| 4 | `worktree-product-system` | LAND | Canonical workflows package + only copy of its tests/examples/guides; uncommitted doc polish | +| 5 | `feature/adaptive-convection` | KEEP_ACTIVE | 13 commits ahead of origin (kinematic fault, budget metric) — **not pushed** | +| 6 | `docs/blog-posts` (worktree) | EXTRACT | Uncommitted 2026-06-04 revision of finding-particles post + figure set in no branch | +| 7 | `feature/cetz-figures` (worktree) | EXTRACT | Uncommitted element-location blog figure set + two doc/figure polish edits | +| 8 | `feature/elliptic-ma` (worktree) | EXTRACT | Uncommitted `mover=` selector for `Mesh.OT_adapt`, parallel RBF-cloud allgather fix, MA study scripts | +| 9 | `feature/exp-integrator-freesurface` | EXTRACT | Unmerged free-surface paper draft + integrator-zoo supplementary (only copy) | +| 10 | `feature/fault-convection` (worktree) | EXTRACT | Uncommitted `adaptive-fault-convection-reimplementation.md` spec — exists nowhere else | +| 11 | `feature/gradient-plasticity` (worktree) | EXTRACT | Entirely-uncommitted `add_smoothing_field()` gradient-plasticity spike (solver pyx + solvers.py) | +| 12 | `feature/petsc-cell-hint` | EXTRACT | Unmerged tip (cell-plane projection), uncommitted `build-petsc.sh` macOS SDK fix, FE-vs-RBF demo | +| 13 | `feature/snes-update-callbacks` | EXTRACT | Unmerged tip commit `b82acea7` (all-solver final-iterate dispatch, post-#250 review fix) | +| 14 | `feature/vep-two-stokes` | EXTRACT | Unmerged, validated `ViscoPlasticExplicitElastic` constitutive class + Phase G post-mortem | + +--- + +## Triage Ledger — Investigated Branches + +### LAND — push and PR to development + +| Branch | Worktree | Risk of loss | One-liner | +|--------|----------|--------------|-----------| +| `feature/adapt-on-top` | `adapt-on-top` | **HIGH** | Layer-2 NVB/SBR adapt-on-top engine (`mesh.adapt` child meshes, native parallel newest-vertex bisection, `Surface.remap_to`, post-#290 custom_mg fixes) — complete and validated, local-only with no remote backup, and required by the already-shipped adapt-on-top-faults skill; push and PR to development. | +| `bugfix/yield-homotopy` | `yield-homotopy` | **HIGH** | The planned follow-up PR to #258: unified delta-soft-min/power-mean yield law + residual-paced yield homotopy + FMG monitor-leak fix, with tests and design docs — complete, local-only, unpushed. | +| `worktree-product-system` | `product-system` | **HIGH** | The canonical workflows/product-system package (api 0.2) with its only copy of the tests, convection+H2Ex examples, guides and scaffold command — unmerged, plus uncommitted doc/config polish that exists nowhere else; finish (commit dirty polish, drop .baks, fold in adaptive-convection's mesh_updates tweak, re-verify ddt fix against development) and PR to development so feature/adaptive-convection can rebase onto it. | +| `feature/parallel-point-eval` | `parallel-point-eval` | low | Manifold-PDE (dim != cdim) campaign, 6/7 commits already in development; only the tip — cell-plane projection enabling SLCN advection on SphericalManifold — remains to PR, and the petsc-cell-hint follow-up branch builds directly on it. | + +**Evidence (LAND):** + +- **`feature/adapt-on-top`** — 49 commits ahead of development, not merged (merge-base + check), local-only branch with NO origin remote and no PR; worktree clean. Holds the + Layer-2 adapt-on-top system: `mesh.adapt(engine="nvb"/"sbr")` returning a refined child + (`discretisation_mesh.py` +506, engine param verified at branch lines 6015/6122), NVB + graded-bisection engine incl. native uwnvb DMPlexTransform (`utilities/nvb.py` 341 lines, + `nvb_transform.c` 1207 lines, `_nvb_transform.pyx`), parallel Route B with SF cross-rank + closure, `Surface.remap_to`/director (`surfaces.py` +266), custom_mg fixes on top of merged + Layer-1 (scalar-solver auto-inject guard `a4add4b7`; maybe_->auto_inject rename `1e1fabdd`), + tests test_0835–0839 (~1150 lines), design docs LAYER2_SBR_ADAPT_ON_TOP.md + + NVB_GRADED_ADAPT.md. NOT superseded: PR #290 merged only Layer-1 custom-P FMG; `git grep` + shows zero NVB code in development. Dependency inversion: the adapt-on-top-faults skill + already shipped to development (`.claude/skills/adapt-on-top-faults` via #299) documents + `mesh.adapt(engine="nvb")`, which exists only here. Design doc marks Stages 2a/2b/2c DONE + (2026-07-01) with parallel acceptance tests; tip commit dated 2026-07-03 is naming polish. + Only recorded open item: marker-replay checkpoint. +- **`bugfix/yield-homotopy`** — 32 commits ahead of development (merge-base `25a4388e`, + #262); worktree clean (`git status --porcelain` empty). Holds the "PR-2 after #258" VEP + convergence work: unified delta-soft-min yield law + power-mean smoother + (`constitutive_models.py` +599; power-mean at branch line 977, absent from development), + residual-paced `enable_yield_homotopy`/`yield_smoother` API (absent from development src), + FMG monitor-leak fix (`snes.monitorCancel` at `petsc_generic_snes_solvers.pyx` + :7461/:7490/:7501/:7512, absent from development), flux_jacobian surrogate-tangent hook, + `tests/test_1053_yield_homotopy.py` (+189 lines, new), 3 design docs + (VISCOPLASTIC_YIELD_HOMOTOPY.md 386L, yield-homotopy-convergence-study.md 233L, + jacobian-consistent-tangent.md 99L) and Spiegelman benchmark upgrades. NOT superseded: + PR #258 (`c63cd707`) merged only the consistent-Jacobian half (carried here via merged + sub-branch bugfix/jacobian-unwrap-to-constants); no other merged PR touches + homotopy/power-mean/monitorCancel. Branch exists ONLY locally: no + origin/bugfix/yield-homotopy, `git branch -r --contains` tip is empty. Last commit + 2026-06-30 wraps up "THE RECIPE" — study looks complete and PR-ready modulo a development + catch-up merge (base predates #265/#290/#293/#294/#298/#304, so expect conflicts in the + solver pyx and constitutive_models.py). +- **`worktree-product-system`** — Branch holds the ORIGIN of the `underworld3.workflows` + product system: ~27 real commits (rest of the 47 are development merges + + feature/fault-system-workflow ancestry) adding `src/underworld3/workflows/` (10 modules + ~3.0k lines: _base/_products/_run/_runner/_cache/_cli/_diagram/scaffold, api 0.2 with + cache_key freshness), `./uw workflow scaffold`, `tests/test_0810_workflow_runner.py` + (248 lines), full convection-sweep + H2Ex example workflows (~4k lines under + `docs/examples/workflows/`) and user+developer guides (tip `bb054b77`, 2026-05-07); plus + cleanup commits (`model.py` −130 demo ThermalConvectionConfig classes; + constitutive_models_new.py removal — the latter independently done on development, + `995ec767`). NOT superseded: `git log development` shows no workflow-package merge and + `src/underworld3/workflows/` is absent from development. feature/adaptive-convection + carries a committed near-identical copy of the src package (deliberate local port per + session records, unmerged, plus one improvement: `_run.py` mesh_updates kwarg) but NOT the + tests, examples, guides, or scaffold — those exist only here. Worktree is DIRTY (14 files, + post-tip polish): NEW docs found nowhere else (`docs/developer/guides/workflow-concepts.md`, + `docs/api/workflows.md`, `docs/examples/workflows/index.md`), real config improvements + (`convection_config.py` qdegree model_validator; `simulate.py` hidden-field CLI), Sphinx + wiring (`docs/conf.py`, `api/index.md`), 2 junk .bak files, and a `ddt.py` SemiLagrangian + `_clamp_to_domain` fix whose core intent (clamp SL back-trace to domain) development + already has at `ddt.py:2627-2628/2689-2692` via return_coords_to_bounds — only its kd-tree + fallback for meshes lacking that hook is unique. +- **`feature/parallel-point-eval`** — 7 commits ahead of development; 6 are patch-equivalent + to commits already in development (`2d421388`, `6f4d64b0`, `fbf3df3e`, `2f892ca3`, + `fd6070d3`, `6423c73f`, landed ~2026-05-21 — `git cherry` flags `9f7a1fa0` but its patch + differs from merged `fd6070d3` only in hunk offsets). Only tip `7232bedb` (2026-05-24) is + unmerged: `Mesh._project_to_nearest_cell_plane` (+139 lines `discretisation_mesh.py`) + + `SphericalManifold.return_coords_to_bounds` cell-plane composition (+26 lines + `spherical.py`) — both absent from development per `git grep` — plus 86-line + INVESTIGATION.md diagnosis of the PETSc DMInterpolation 2-manifold cell-misroute, 12 probe + scripts, and an advection animation. This commit is what makes SLCN advection run on the + sphere (validated, tier-A unchanged per commit message). Worktree is clean. Tip is pushed + to origin at the same SHA and is also the base of feature/petsc-cell-hint (dirty sibling + worktree carrying the planned DMInterpolation cell-hint follow-up), so landing must + coordinate with that successor: PR the single tip commit (rebase onto current development, + ~5 weeks behind), then rebase petsc-cell-hint onto development and this branch/worktree can + be removed. + +### EXTRACT — rescue named content, then delete + +| Branch | Worktree | Risk of loss | One-liner | +|--------|----------|--------------|-----------| +| `docs/blog-posts` | `blog-posts` | **HIGH** | Merged blog-posts branch whose worktree hides an unsaved 4-June revision of the finding-particles post plus its new element-location figure set — commit/PR the dirty files, then delete branch and worktree. | +| `feature/cetz-figures` | `cetz-figures` | **HIGH** | Superseded skill-shipping commit (merged as #299), but the dirty worktree uniquely holds a finished element-location blog figure set plus two small doc/figure polish edits worth rescuing before deleting. | +| `feature/elliptic-ma` | `elliptic-ma` | **HIGH** | The MMPDE/Monge-Ampere mover branch — fully merged into development, but its worktree holds uncommitted, unique follow-on work: the `mover=` selector for `Mesh.OT_adapt` (dropped from dev in a merge), a parallel RBF-cloud allgather fix for `_winslow_mmpde`, and the MA+arc-length study driver/renderer scripts. | +| `feature/exp-integrator-freesurface` | `exp-integrator-freesurface` | **HIGH** | Free-surface integrator-zoo campaign (May 2026): unique unmerged paper draft + benchmark supplementary + 1.5k-line investigation record, docs-only; dirty src prototypes already upstreamed to development, throwaway scripts should not land as-is. | +| `feature/fault-convection` | `fault-convection` | **HIGH** | Merged/never-diverged branch whose only value is one uncommitted design doc (the fault+adaptive-convection re-implementation spec, found nowhere else) plus 5 named reference scripts — commit the doc (and optionally those scripts) to development, then remove worktree and delete branch. | +| `feature/gradient-plasticity` | `gradient-plasticity` | **HIGH** | Gradient-plasticity research spike living entirely as uncommitted changes: a monolithically-coupled implicit-gradient (screened-Poisson) smoothing field for Stokes_Constrained, found nowhere else — commit it to preserve before removing the worktree; the accompanying Jacobian-bug handoff doc is superseded by PR #258. | +| `feature/petsc-cell-hint` | `petsc-cell-hint` | **HIGH** | Manifold/cdim stack mostly landed in development; extract the unmerged tip (cell-plane projection into return_coords_to_bounds), the uncommitted build-petsc.sh SDK-fallback fix, and the FE-vs-RBF demo, then archive — the dirty petsc_tools.c bypass prototype is superseded by feature/dminterp-bypass-element-check. | +| `feature/snes-update-callbacks` | `snes-update-callbacks` | **HIGH** | SNES per-iteration update callbacks — merged as #250 except the tip review-address commit (all-solver final-iterate dispatch + hook rename + test), a small genuine improvement to cherry-pick before archiving. | +| `feature/vep-two-stokes` | `vep-two-stokes` | **HIGH** | VEP integrator campaign mostly merged via PR #161; uniquely holds the unmerged, validated ViscoPlasticExplicitElastic constitutive class (operator-split yield-on-total, 7x SNES speed-up) plus the Phase G post-mortem design doc — extract those, discard the rest. | +| `bugfix/deform-cache-invalidation` | `deform-cache-invalidation` | low | Cache-invalidation fix for direct `_deform_mesh` calls; two of three commits already landed on development, and the remaining 31-line `_mesh_version` bump addresses a still-live stale-kdtree gap in `mesh.deform()` but must be reworked around the snapshot version-gate (e.g. a separate geometry-version counter) — extract that finding as an issue/small PR, then archive-tag and delete branch + remove the clean worktree. | +| `feature/fault-system-workflow` | `fault-system-workflow` | low | Origin of the workflows/product-system idea plus the H2Ex geographic-fault pipeline examples; package superseded by worktree-product-system and src fixes by PR #131, but the W1–W5/Step5–6 example suite (~3.5k lines) exists only here — cherry-pick it onto product-system, then archive. | + +**Evidence (EXTRACT):** + +- **`docs/blog-posts`** — Branch tip `29a2029d` is fully merged into development + (`merge-base --is-ancestor` confirms; `development..docs/blog-posts` is empty, 0 ahead) and + development has not touched `publications/blog-posts/` since, so the BRANCH itself is dead + weight. But the WORKTREE holds a coherent uncommitted editing session dated 2026-06-04: a + 145-line prose revision of `publications/blog-posts/finding-particles.md` (KDTree + control-point cell location + `dm.migrate()` negotiation write-up), 33 lines in + particles-as-symbols.md, a README tweak, and 6 untracked figure assets + (element-location-demo.png/.typ, 3 data JSONs, generate-element-location-data.py) that + appear in NO branch (`git log --all` on those paths is empty). The revised text embeds the + new figure, so markdown + figures belong together. Extraction = commit the 9 dirty paths to + a fresh docs branch and PR to development; then the worktree can be removed and the merged + branch deleted. +- **`feature/cetz-figures`** — 1 commit ahead (`a8f87f87`: track .claude/skills + + cetz-figures skill) — byte-identical to merged PR #299 (`aa3e383e` on development, + 2026-07-03), so the branch itself is fully superseded. But the worktree is dirty with 18 + files, and 3 items exist NOWHERE else: (a) a complete new element-location blog figure set + (typ + rendered png + generator .py + 3 data JSONs, dated June 3-4) for + `publications/blog-posts/finding-particles.md`, which the merged blog post does not yet + reference; (b) an unmerged edit to `docs/advanced/curved-boundary-conditions.md` swapping + the ASCII-art normals sketch for the already-committed + `figures/curved-bc/facet-vs-true-normals.png`; (c) cosmetic polish to + `docs/advanced/figures/cuboid-3d/cuboid-faces.typ` (+regenerated png). The other 12 dirty + files are byte-identical duplicates of content already on development (blog figures / + cuboid-3d generators). +- **`feature/elliptic-ma`** — Branch tip `4dccafb7` is 0 ahead and a verified ancestor of + development (`git merge-base --is-ancestor` passes; listed in `--merged`). Its work — + anisotropic MMPDE mover, elliptic Monge-Ampere mover, slip surfaces, RBF metric eval — is + live in development (`smoothing.py:2992` `_winslow_mmpde`, `:3617` "ma", `:3642` "mmpde"; + `_ot_adapt.py:250` says "grafted from feature/elliptic-ma"). Note: the branch's committed + `mover=` kwarg on `_ot_adapt_step` (`c977e485`) was silently dropped from current + development by merge `b7da8f29`, so dev's OT_adapt hardcodes the "ot" mover. The worktree + is DIRTY with 5 files, none junk: (1) pyx +8 `_pre_solve_hook` — redundant, duplicated on + feature/gmg-geometric-interp and superseded by custom_mg/set_custom_fmg (#290); + (2) `discretisation_mesh.py` +2 exposing `mover=` through `Mesh.OT_adapt` — exists in no + current branch; (3) `smoothing.py` +12 parallel allgather of the RBF reference cloud in + `_winslow_mmpde` (fixes rank-local KDTree → non-SPD metric at partition boundaries) — no + equivalent anywhere; (4) `scripts/stagnant_lid_adapt_loop.py` +94 study flags + (--adapt-method ot-reset-ma, --metric-choice arc-length, --dt-basis mean) plus an OT-reset + resume-reference correctness fix; (5) untracked `scripts/render_ma_arclen_frames.py` + (167 lines, P3-faithful MA/arc-length animation renderer). Items 2–5 found nowhere else by + cross-branch grep. +- **`feature/exp-integrator-freesurface`** — 6 commits ahead of development, +12,399 lines, + ALL docs/publications, zero src changes. Holds: publication-track paper draft + `publications/free-surface-paper/draft.md` (494 lines: semi-Lagrangian kinematic update + + amplitude-invariant relaxation CFL for free-surface viscous flow) and + integrator_zoo_supplementary.md (819 lines, FE/RK2/RK4/AB2/BDF2-SL + Cathles benchmark); + investigation record `docs/developer/design/EXPONENTIAL_FREE_SURFACE.md` (1496 lines) + + 2 handoff docs + deformable_surface_metronome_design_note.md (Model.advance timekeeper + architecture sketch); plus ~20 throwaway `_phase_i_*`/`_plot_*` scripts (~9k lines) of + exactly the docs/design contamination class the audit flags. NOT superseded: + `git log --all -- publications/free-surface-paper/` shows the draft exists only on this + branch; no merged PR contains any of it. Dirty worktree (25 files): the 3 modified src + files (`discretisation_mesh.py` +25, `ddt.py` +93, `solvers.py` +3) are prototypes of fixes + that ARE now on development — SL monotone limiter + theta landed via `d09f1a3b`/`75bf61af` + (`ddt.py:1466-1507` on development), `_deform_mesh` cache invalidation via `93d2501f` + (`discretisation_mesh.py:3213-3225`) — so the src diffs are safely discardable. The 22 + untracked files are post-tip debugging/viz scripts plus + free-surface-convection-session-2026-05-13.md (session note, exists nowhere else; its + conclusions were upstreamed as the above commits). Extract = land the paper draft, + supplementary, EXPONENTIAL_FREE_SURFACE.md and metronome note (relocating throwaway scripts + per audit policy), optionally rescue the session note, then archive-tag and delete; the + free-surface effort has since pivoted to the 3-number held-lid integrator, so the branch + itself is not in-flight. +- **`feature/fault-convection`** — Branch tip `25a4388e` equals the merge-base with + development (zero ahead commits; `git log development..feature/fault-convection` is empty) + — the branch holds no unique commits and the real fixes it explored landed via PRs + #259/#264/#266 already on development. The worktree is dirty with 58 untracked files: 57 + exploratory diagnostic scripts (`scripts/fault_*.py` etc.) whose quantitative results are + explicitly declared suspect/superseded, plus ONE valuable file — + `docs/developer/design/adaptive-fault-convection-reimplementation.md` (73-line spec dated + 2026-06-22: validated mmpde recipe, anisotropic fault-metric formula + M = rho·I + (Rf²−1)·exp(−(d/w)²)·n nᵀ, gmsh refine_lines + carrier approach, diagnostics + inventory, 4-step landing plan). Verified this doc exists nowhere else: no git history on + any branch, absent from the main checkout and ~/+Simulations; the adapt-on-top-faults and + adaptive-meshing skills cover different/partial recipes. The doc cites 5 of the scratch + scripts by name as its reference material (fault_convection_adapt_loop.py, + mmpde_metric_proof.py, adapt_vs_uniform_compare.py, bc_leak_check.py, + multi_history_plot.py). The previously recorded uncommitted smoothing.py dup is no longer + present (no modified tracked files). +- **`feature/gradient-plasticity`** — Branch has 0 unique commits (tip `7696d89f` = + development ancestor, PR #240; trivially merged). ALL value is uncommitted in the worktree: + (1) `solvers.py` +51 — new `SNES_Stokes_Constrained.add_smoothing_field()`: + screened-Poisson implicit-gradient smoothing field coupled monolithically into the saddle + solve (gradient-plasticity regularization); (2) `petsc_generic_snes_solvers.pyx` +105/−9 — + its assembly (Helmholtz F0/F1, G0/G3, su/us cross-coupling Jacobian blocks, fieldsplit + grouping of all multipliers). `git grep` across every branch head: + `add_smoothing_field`/`_is_smoothing` exist NOWHERE else. (3) Untracked + `docs/developer/design/jacobian-unwrap-constants-bug.md` (190 lines, "fix not yet started") + IS superseded — the fix landed as PR #258 (`c63cd707`) and development has + jacobian-consistent-tangent.md — but the smoothing-field spike is a distinct capability NOT + covered by #258. Spike has no tests and changes solver-pyx numerics, so + commit-to-preserve (branch or archive tag), don't PR as-is. +- **`feature/petsc-cell-hint`** — Holds the cdim/manifold stack (SphericalManifold, + embedded-coord plumbing, parallel point-eval on 2-manifolds). 6 of 7 ahead commits already + landed in development via cherry-picks (`git cherry`: + `3a09fa02`/`107de9b6`/`fff550eb`/`166cf5af`/`09138cc0` patch-equivalent; `9f7a1fa0` + identical to dev `fd6070d3` modulo hunk offsets). UNMERGED: tip `7232bedb` — + `Mesh._project_to_nearest_cell_plane` (139 lines) + + `SphericalManifold.return_coords_to_bounds` cell-plane composition (26 lines) + + INVESTIGATION.md DMInterpolation-misroute diagnosis + 12 probe scripts; absent from + development and from successor branch feature/dminterp-bypass-element-check (which has the + real bypass fix, `17a5a8d3`+`7168a0ae`, off an older base without the manifold files). + Dirty worktree: petsc_tools.c/_function.pyx = earlier prototype of the bypass, superseded + by dminterp-bypass branch commits; `build-petsc.sh` 18-line macOS SDK xcrun-fallback fix + exists NOWHERE else (`git log --all -S` empty); untracked compare_fe_rbf_lonlat.py + FE-vs-RBF validation demo + rendered results exist nowhere else. +- **`feature/snes-update-callbacks`** — 4 commits ahead of development (+589/−56 across + `petsc_generic_snes_solvers.pyx`, `systems/solvers.py`, test_1016, + `docs/advanced/solver-iteration-callbacks.md`). First three commits were squash-merged as + PR #250 (development `54b815c3`, 2026-06-18 20:26). Tip commit `b82acea7` (20:54, 28 min + AFTER the merge) is UNMERGED and exists only on this branch (local + origin): it + centralises the final-iterate callback dispatch in `_snes_solve_with_retries` so non-Stokes + solvers get it too (verified: development pyx:8070 still Stokes-only, pyx:325 still + `_maybe_install_snes_update`), removes the spurious `_needs_function_rewire` on callback + registration, renames `_maybe_install_snes_update` to `_attach_snes_update_hook` (the + no-maybe_-prefix rule), and adds test_final_iterate_dispatch_on_scalar_solver. Worktree is + clean (`git status --porcelain` empty). +- **`feature/vep-two-stokes`** — 91 ahead of development, but `git cherry` shows 73/91 + commits patch-equivalent — landed via PR #161 (feature/exp-integrator-investigation, merged + 2026-05-05): Phase A–F ETD work, integrator='bdf'/'etd' API, DDt.set_initial_history, + test_1052_* regression tests (verified in development tree). The 17 unique commits + (Phase G, 2026-04-29..05-01) are NOT superseded: two-Stokes operator split rejected with + post-mortem, then the working v5b result — class ViscoPlasticExplicitElastic at + `constitutive_models.py:2008` (~260 net src lines, bdf_blend/etd_blend damping, 4-sig-fig + baseline match, ~7x fewer SNES iters on first-order integrators), a 316-line completed + VEP_TWO_STOKES_OPERATOR_SPLIT.md (development only has the plan stub), a VE convergence + study, plus ~15 throwaway `_phase_g_*` scripts/traces/pngs in docs/developer/design/. + `git grep` confirms ViscoPlasticExplicitElastic absent from development; June's VEP work + (PR #258 consistent Jacobian + yield-homotopy) is a different route, not a replacement. + Worktree clean (`status --porcelain` empty), branch pushed to origin. Extract = cherry-pick + the class + design-doc post-mortem + convergence study onto a fresh branch (skip the + `_phase_g_` contamination), then archive-tag and delete. +- **`bugfix/deform-cache-invalidation`** — 3 substantive commits ahead. Two (lambdify + `_expr_hash` fix closing #194 + cache-contract test rewrite) already on development as + cherry-picks `ca5c3efb`/`16103071` — net unique diff is one 31-line hunk (`c641dfe8`): bump + `_mesh_version` and clear `_restore_kdt`/`_restore_coords_id` in `_deform_mesh`. PR #191 + was CLOSED 2026-07-02 as superseded, but the closure claim ("deform() bumps the version, + lines 863-865") is wrong — verified those lines are the mesh.X.coords callback, while + public `deform()` (`discretisation_mesh.py:3092`) calls `_deform_mesh` directly + (`:3123-3127`) which bumps only `_topology_version` (`:3224`). So `_mesh_version`-keyed + caches (`_get_domain_kdtree` `:5571-5582`, `_owned_cells_mask_cache` `:4794`, variable + `_get_kdtree` used by rbf_interpolate, `discretisation_mesh_variables.py:998-1006`) stay + frozen after `mesh.deform()` on development@1d003481 — the gap looks still live. The fix + cannot cherry-pick as-is: it collides with the snapshot `_mesh_version` gate + (`:4191-4219`; PR #191 CI failed with SnapshotInvalidatedError), and the `_restore_kdt` + half targets attributes appearing nowhere in src/. Worktree clean; all commits pushed to + origin and preserved in closed PR #191. +- **`feature/fault-system-workflow`** — 31 ahead of development, last commit 2026-04-22, + worktree clean, tip `c567b311` == origin/feature/fault-system-workflow (fully pushed). + Holds three things: (1) the original underworld3.workflows "make-like product system" + package — SUPERSEDED: worktree-product-system branched from these very commits + (`af3778ef`/`f9970c1e` are ancestors, verified with `merge-base --is-ancestor`) and + extended it to api 0.2 (+1895 lines _cache/_cli/_diagram/_run/_runner/scaffold, last commit + 2026-05-07); (2) src fixes (ellipsoid quantities, checkpoint round-trip, Darcy projection + sign, commit `5668edb4`) — SUPERSEDED by merged PR #131 (`3829b271`/`62bebb3f` on + development; `_checkpoint_ellipsoid_pending` present in dev coordinates.py); (3) UNIQUE + unmerged work found nowhere else except its own origin remote: + `docs/advanced/h2ex-workflow.md` + ~3.5k lines of H2Ex geographic fault pipeline examples + in `docs/examples/WIP/Geographical/` (W1–W5b, W5-Interactive trame visualiser, + Step5-RunPipeline, Step6-Visualise), Louis-authored, absent from both development and + worktree-product-system (`git cat-file` checked). Those scripts import + underworld3.workflows (WorkflowProducts), so they must be carried onto product-system (or + land with it), then tag archive/feature/fault-system-workflow and delete. + +### KEEP_ACTIVE — in-flight, do not touch + +| Branch | Worktree | Risk of loss | One-liner | +|--------|----------|--------------|-----------| +| `bugfix/custom-mg-parallel` | `custom-mg-parallel` | **HIGH** | In-flight continuation of the custom-MG program past merged PR #290: the whole Layer-2 NVB/adapt-on-top engine (native uwnvb transform, mesh.adapt child, 5 test files) plus today's non-nested np>1 cross-partition transfer — none of it merged or pushed, with uncommitted design-doc updates for today's work; keep and finish (should be committed/pushed and PR'd soon). | +| `feature/adaptive-convection` | `adaptive-convection` | **HIGH** | Active adaptive-convection-with-faults study: workflows-package port (+adaptive-mesh checkpoint fix), Annulus refine_lines, and the kinematic-fault example — in-flight, 13 commits not yet pushed to origin. | + +Per the execution protocol, `feature/numpy2-support` (main-checkout work stream) and the +quality-audit branch/worktree (`feature/quality-audit-2026-07`) are also KEEP_ACTIVE. + +**Evidence (KEEP_ACTIVE):** + +- **`bugfix/custom-mg-parallel`** — 52 commits ahead of development (merge-base `417d18b5`, + 2026-07-02; tip `7311fe6d` dated 2026-07-03), +5274/−115 lines across 25 files. Despite the + branch name, only a small fraction is "custom-mg parallel bugfix": the branch has absorbed + feature/adapt-on-top (merge `29890cd0`) and carries the ENTIRE Layer-2 program that is NOT + in development: (1) NVB graded adapt engine — new `src/underworld3/utilities/nvb.py` + (341 lines), `_nvb_transform.pyx` + `nvb_transform.c` (native uwnvb DMPlexTransform, 1249 + lines), mesh.adapt SBR/NVB adapt-on-top child in `discretisation_mesh.py` (+506), + `Surface.remap_to` + director in `meshing/surfaces.py` (+266); (2) five new test files + test_0835–0839 (~1144 lines) + test_1017 parallel additions; (3) two brand-new commits + (`5ac985ef` operator-faithful finest reduced map on adapt children; `7311fe6d` + cross-partition transfer for non-nested coarse tails at np>1) extending `custom_mg.py` + (+262 over development's copy). Supersession check: PR #290 (`f6a75ef0`) merged only + Layer-1 custom-P FMG; PR #297 merged the FMG lockout — nothing in development contains + nvb.py, the NVB tests, adapt-on-top, or the cross-partition transfer (`git ls-tree + development` confirms nvb.py absent; grep of development log finds no NVB/adapt-on-top + merges). No remote copy exists (origin has only feature/custom-mg-prolongation); the tip + commits live only on this branch and local feature/adapt-on-top. Worktree dirty: 2 files, + both docs/developer/design/ notes (GENERALIZED_FMG_HIERARCHY_AND_ADAPT.md +24/−8, + LAYER2_SBR_ADAPT_ON_TOP.md 1-line rename fix) — real, uncommitted documentation of the + step-4 work (non-nested np>1 DONE) that exists nowhere else. The NVB adapt engine is also + load-bearing for the active adapt-on-top-faults workflow/skill and the annulus fault + studies. +- **`feature/adaptive-convection`** — 17 commits ahead of development, +6,224 lines, worktree + clean. Holds: (1) local port of the underworld3.workflows package from + worktree-product-system plus a unique 9-line mesh_updates fix in `workflows/_run.py` + (per-step mesh geometry checkpointing for adaptive meshes) — diffed the two branches, only + `_run.py` differs, so product-system does NOT have this fix; (2) `meshing/annulus.py` + refine_lines gmsh line-refinement (+56 lines, absent from development); + (3) `docs/examples/workflows/adaptive_convection/` study (~3,900 lines: fault_config.py, + kinematic fault/ridge advection, budget multi-monitor metric, diagnostics/render scripts). + NOT superseded by any merged PR: development has no workflows package, no refine_lines, no + adaptive_convection example (verified via `git ls-tree`/grep and `log --grep`). CAUTION: + local tip `22deae75` is 13 commits ahead of origin/feature/adaptive-convection (remote + stuck at `4686b86e`) — the kinematic-fault and budget-metric commits exist only in the + local repo and should be pushed. Project records mark this as in-flight (Task 2 = fault + friction is next); workflows duplication with worktree-product-system is deliberate, + coordinate before PR. + +### ARCHIVE_DELETE — superseded; tag `archive/` then delete + +| Branch | Worktree | Risk of loss | One-liner | +|--------|----------|--------------|-----------| +| `feature/boundary-flux` | `boundary-flux` | none | CBF boundary_flux primitive (heat flux/Nusselt/traction) — merged verbatim as PR #294 and since improved on development; branch and clean worktree are safe to archive-tag and delete. | +| `feature/rotated-freeslip-bc` | `custom-mg-prolongation` (misnamed) | none | Source branch of merged PR #293 (rotated strong free-slip + sigma_nn dynamic topography); tip byte-identical to the squash, worktree clean — archive-tag and delete both branch and the misnamed custom-mg-prolongation worktree. | +| `feature/dminterp-bypass-element-check` | `dminterp-bypass-element-check` | low | The DMLocatePoints-bypass prototype that development already absorbed (`c52201b8` cites it by name); only residue is a cosmetic one-line sentinel-cast guard in petsc_tools.c that is a no-op on real platforms. | +| `feature/fs-stress-equilibrium` | `fs-stress-equilibrium` | none | Docs-only snapshot of the ETD stress-equilibrium free-surface investigation (design notes + phase-I experiment scripts), identically contained in and superseded by feature/fs-surface-smoother-driver. | +| `feature/fs-surface-smoother-driver` | `integrate-surface-submesh` (misnamed) | low | Mid-June free-surface etd_topo exploration record (12.6k lines of design docs/scripts, method since superseded); every src/test change already merged via PRs #237/#238/#246/#249/#251 — archive-tag (after committing 3 dirty exploration files) and delete. | +| `feature/projection-smoothing-length` | `projection-smoothing-length` | none | v1 of the projection smoothing_length API — fully superseded by the v2 rewrite merged as PR #234; clean worktree, nothing unique left. | +| `feature/snes-constant-nullspace` | `snes-constant-nullspace` | none | Original standalone constant-nullspace-for-singular-Poisson branch; its feature, guard, and test all landed on development (PR #236 + dedup `5bb72c4c`) in stronger form — safe to tag archive/ and delete with its worktree. | +| `feature/exp-integrator-investigation` | `vep-loading-unloading` (misnamed) | low | Merged (PR #161) exponential-integrator branch whose single unmerged commit is a superseded Phase I kinematic-ETD free-surface investigation doc + handoff — the handoff was fulfilled by the merged FREESLIP_DYNAMIC_TOPOGRAPHY_FREESURFACE.md scheme; tag archive/ and delete. | +| `api/surface-quantity-support` | — | none | Feb-2026 unit-aware Surface API branch (quantity args for refinement_metric/influence_function + dimensionalised pv_mesh) whose both commits were cherry-picked verbatim into development in March — fully superseded, safe to tag archive/ and delete. | +| `feature/custom-mg-prolongation` | — | none | The generalized custom-P FMG hierarchy work (Phase 1 complete), squash-merged to development as PR #290 — every commit's content verified present in development, safe to tag archive/ and delete. | +| `feature/snesfas-spike` | — | none | Raw commit trail of the SNESFAS/Vanka multilevel nonlinear-Stokes feasibility spike — its final docs (plus curated prototypes) already merged via PR #245. | +| `feature/vep-loading-unloading` | — | low | April-2026 VEP loading/unloading investigation whose real fixes (divergence_retries, mixin recursion) already re-landed on development; leftover is an A/B evidence trail (test-only plastic-basis option, benchmark scripts, figures) preserved by the archive tag. | +| `fix/darcy-sign-convention` | — | none | The rejected PR #243 flipped-flux Darcy sign fix — same bug fixed correctly (with f!=0 tests) by merged #255; approach explicitly superseded, nothing to salvage. | +| `ss2098/fix-thermal-convection-units-tutorial` | — | none | External contributor's units-tutorial fix, fully squash-merged (and slightly improved) as PR #263 — the branch is now redundant. | +| `test-darcy-rebase` | — | none | Stale local rebase-test of the Darcy/Richards feature branch, fully merged to development in April (`4895c433`); safe to tag archive/test-darcy-rebase and delete. | + +**Evidence (ARCHIVE_DELETE):** + +- **`feature/boundary-flux`** — Holds the Consistent Boundary Flux (CBF) primitive: 3 commits + ahead (`4dbd30bd` feature, `40e86996` np4 partition-cut reaction fix, `787c0bde` Copilot + review fixes) adding `utilities/boundary_flux.py` (+255), solver pyx methods (+108), tests + test_1019 + parallel test_1065 (+144). Fully superseded by merged PR #294 (squash + `ae647025` on development): `git diff feature/boundary-flux ae647025` shows ZERO + differences in any boundary-flux file (only pre-existing #290 custom_mg files differ). The + prior session record's "OPEN np4 boundary-cut issue" was fixed in `40e86996` and IS in the + merge. Development has since evolved the module further (partial_reaction mode + + write_boundary_scalar_field factored out by #293). Worktree `.claude/worktrees/boundary-flux` + is clean (`git status --porcelain` empty). +- **`feature/rotated-freeslip-bc`** — Branch holds 11 commits (`41c64819`..`f4d217f9`): + rotated strong free-slip BC with sigma_nn reaction, geometric-FMG solve path, + parallel-safety fixes, CBF sigma_nn topography, dynamic_topography hand-off, plus tests + test_1018/test_1064. Fully superseded by merged PR #293 (squash `51b19182` "Rotated strong + free-slip BC + sigma_nn dynamic-topography (parallel), free-surface hand-off") — verified: + `git diff 51b19182 f4d217f9` on all 5 touched files (rotated_bc.py, boundary_flux.py, + petsc_generic_snes_solvers.pyx, both test files) is EMPTY, and merge-base `ae647025` is the + commit directly before the squash. Development has since moved further on these files + (#298 rotated-SNES). Worktree + `/Users/lmoresi/+Underworld/underworld3-pixi/.claude/worktrees/custom-mg-prolongation` is + CLEAN (empty `status --porcelain`). Worktree name is a leftover from the #290 custom-MG + work; the directory was reused for this branch. +- **`feature/dminterp-bypass-element-check`** — 2 commits ahead of merge-base `44c2f472`: + `17a5a8d3` (DMLocatePoints bypass on simplex/manifold meshes given a kdtree cell hint) and + `7168a0ae` ((size_t)-1 sentinel guard before PetscInt cast). Commit 1 is EXPLICITLY + superseded: development's `petsc_tools.c:73-74` says "ported from + feature/dminterp-bypass-element-check, 17a5a8d", brought in by `c52201b8` (2026-05-28, + "remesh field-transfer redesign + parallel locator hardening"), and development has since + evolved past the branch (PR #203 serial bypass, hint policy centralized in + `mesh._eval_use_robust_location`, out-of-domain best-claim block in `_function.pyx`). + Commit 2's one-line guard is the only unported residual, and it is practically inert: + `(PetscInt)(size_t)-1 == -1` on all supported compilers and development's recovery path + (`petsc_tools.c:142`) already rejects negatives via `recovery_cells[k] >= 0` — a + portability nicety, not a live bug. Worktree is clean (`git status --porcelain` empty; root + vep_*.png are tracked at merge-base, not stray work). +- **`feature/fs-stress-equilibrium`** — 1 commit ahead of development (`9a5b5037`, + 2026-06-14): docs-only, +12,406 lines in docs/developer/design/ — 5 design/session notes + (EXPONENTIAL_FREE_SURFACE.md, EXPONENTIAL_FREE_SURFACE_HANDOFF.md, + STRESS_EQUILIBRIUM_FREESURFACE.md, deformable_surface_metronome_design_note.md, + free-surface-convection-session-2026-05-13.md) plus ~20 throwaway + `_phase_i_fs_*`/`_plot_*`/`_probe_*` experiment scripts. No src/ changes. Not merged to + development, but SUPERSEDED by branch feature/fs-surface-smoother-driver: `git cherry` + marks `9a5b5037` as already-applied there, and `git range-diff` confirms it equals + `a0ef4df8` byte-for-byte; that branch adds 4 further commits (surface-smoother driver + wiring, deform identity-gate fix, gmsh spacedim-leak fix, zoo port with tests). No merged + PR contains it; #293's free-surface hand-off on development is different content. Worktree + `.claude/worktrees/fs-stress-equilibrium` is clean (`git status --porcelain` empty). + Caveat: "no loss" depends on feature/fs-surface-smoother-driver being kept or archived — it + is the only other holder of this patch. +- **`feature/fs-surface-smoother-driver`** — 5 commits ahead of development (tip `b72c6521`, + 2026-06-15). The two src fixes (`c383aa9d` gmsh dm_plex_gmsh_spacedim leak; `ff2bddab` + coord identity-gate) merged to development via PRs #238 (`84294f2c`) and #237 (`0ed93123`) + — `_clear_gmsh_import_options` and the gate are present in dev's discretisation_mesh.py. + The remaining 3 commits (etd_topo stress-equilibrium free-surface integrator + zoo driver) + are ~12.6k lines entirely in docs/developer/design/ (EXPONENTIAL_FREE_SURFACE.md 1496 + lines, HANDOFF, STRESS_EQUILIBRIUM_FREESURFACE.md, ~19 `_phase_i_`/`_plot_`/`_probe_` + exploration scripts) — exploration whose method was superseded by the free-slip + + three-number topography integrator line (PR #293 hand-off, Crameri-benchmarked write-up). + Worktree dirty (13 files): ALL src/test modifications verified present in development + (capability gate + ephemeral_coords via #246/#249; old_frame_traceback via + #251/`a708e8c4`; topography(reference="mean") + parallel-safe Stokes_Constrained; SL-BDF2 + docstrings); untracked test_0826/test_0855/SL docs byte-identical to dev. Unique + uncommitted leftovers are exploration-only: +232-line zoo script extension, + `_phase_i_fs_deform_demo.py`, and a ~45-line prototype-provenance section in + lagged-clone-sl-history.md (mechanism already merged+documented in dev ddt.py). Recommend + committing the 3 dirty exploration files onto the branch before tagging + archive/feature/fs-surface-smoother-driver so the tag captures everything, then delete + branch + worktree. +- **`feature/projection-smoothing-length`** — Holds 2 commits (`16cb46ef`, `1c5ef1c7`): + unit-aware smoothing_length property on the four SNES Projection classes plus a Pint + round-trip fix, with `tests/test_0505_projection_smoothing_length.py`; touches only + systems/solvers.py and that test. Superseded by merged PR #234 (development merge + `da4573a0`, from feature/projection-smoothing-length-v2, squash `31e7fd2f`) — the + CI-passing rewrite of this v1 branch (its own PR #200 had 5 CI failures). Test file is + byte-identical to what #234 merged; current development has smoothing_length live (27 refs + in solvers.py). Worktree is clean (`git status --porcelain` empty). Sibling worktree + projection-smoothing-length-v2 sits at the merged commit (bulk list). +- **`feature/snes-constant-nullspace`** — 2 commits ahead (`d99e8723` adds + SNES_Scalar.petsc_use_constant_nullspace; `f5b53ad4` warns on nullspace+Dirichlet, adds + test_1055, 129 lines); worktree clean (`status --porcelain` empty). Fully superseded by PR + #236 (merge `5c790acd` of feature/manifest-constant-nullspace, commit `dc6dacd0`) plus + dedup commit `5bb72c4c` on development: development's petsc_generic_snes_solvers.pyx + carries the canonical constant_nullspace with a STRONGER guard (raises ValueError on + Dirichlet BCs vs this branch's warning), cached nullspace object, near-nullspace for GAMG, + and keeps petsc_use_constant_nullspace as a documented back-compat alias (pyx ~2429-2442). + The branch's test survives as `tests/test_1056_constant_nullspace.py` (same file ported to + canonical flag name, plus an extra setter test). `d99e8723`'s own message anticipated this: + "once that merges to development the duplicate will be resolved on rebase." +- **`feature/exp-integrator-investigation`** — Main work (Phase A–I VE/VEP exponential + integrator, ETD-1/ETD-2 API, EXPONENTIAL_VE_INTEGRATOR.md) merged to development via PR + #161 (`03473c62`). Sole ahead-commit `2d7205bb` (2026-05-06) adds only + docs/developer/design/ material: EXPONENTIAL_FREE_SURFACE.md (899 lines, self-marked + "investigation in progress"), a 284-line session HANDOFF doc with open problems (large-dt + drift, gamma estimation), and 5 throwaway `_phase_i_*`/`_plot_*` scripts (2,619 lines + total, no src/ changes). That handoff was consumed: its open questions are resolved by the + merged, Crameri-benchmarked three-number h-infinity scheme in + docs/developer/design/FREESLIP_DYNAMIC_TOPOGRAPHY_FREESURFACE.md on development (landed + with PR #293/#264 work), which rejects the kinematic-ETD approach the Phase I doc explores. + Ahead-commit exists on no other branch; worktree vep-loading-unloading is completely clean + (empty `git status --porcelain`). Archive tag preserves the investigation narrative for the + record. +- **`api/surface-quantity-support`** — Holds 2 commits touching only + `src/underworld3/meshing/surfaces.py` (+86/−17): `9ffff618` + "Surface.refinement_metric()/influence_function() accept uw.quantity values" (adds + `_to_nd_length` helper) and `38fb30b9` "Surface.pv_mesh returns dimensionalised coordinates + for visualisation". Both are EXACT patch-id matches to commits already in development — + `9ffff618` ≡ `4c16f5a5` and `38fb30b9` ≡ `d37a76b0` (both cherry-picked to development + 2026-03-20; verified with `git patch-id`, identical hashes). Current development + surfaces.py confirmed to contain `_to_nd_length` (line 58, now used at ~15 call sites) and + the dimensionalised pv_mesh property (lines 921-935). No worktree exists for this branch; + nothing dirty. Zero unique content remains. +- **`feature/custom-mg-prolongation`** — 16 commits ahead of merge-base `a7f0b11f`: custom-P + geometric-MG prolongation (`utilities/custom_mg.py` 653 lines, set_custom_mg hook in + petsc_generic_snes_solvers.pyx, tests 1015/1016/1017 + parallel, design doc). Fully + superseded by merged PR #290 (squash `f6a75ef0`, 2026-07-02 12:00, includes branch tip + `b7d1c4fd` from 11:28): all 6 non-pyx files byte-identical to development, pyx hooks + confirmed present in development (lines 156-158/259/3046). Local tip == origin tip; no + worktree on this branch (the `.claude/worktrees/custom-mg-prolongation` directory is + checked out on feature/rotated-freeslip-bc, a different branch). +- **`feature/snesfas-spike`** — 16 docs-only commits ahead of development, netting to 3 + design docs (+847 lines): multilevel-nonlinear-stokes-strategy.md, snesfas-feasibility.md, + snesfas-vanka-feasibility-study.md (SNESFAS/Vanka nonlinear-multigrid feasibility spike, no + src/ changes). Fully superseded by merged PR #245 (`34a9dd46`, "docs(solvers): preserve + SNESFAS / Vanka / grid-sequencing investigation") via sibling branch + docs/snesfas-investigation, which contains these 16 commits in its history: two of the + three docs are byte-identical on development, the third is a strict superset there (adds + pointer to in-repo prototypes docs/examples/snesfas_investigation/ that the branch lacks). + No worktree on this branch; nothing uncommitted. +- **`feature/vep-loading-unloading`** — 14 commits ahead of development (last 2026-04-24), no + worktree attached (the vep-loading-unloading worktree dir is now on + feature/exp-integrator-investigation). All functional src changes are already on + development: divergence_retries re-landed as `ee667376` (present at + `petsc_generic_snes_solvers.pyx:1068` `_snes_solve_with_retries`), MathematicalMixin + recursion fix and timestep restore are patch-equivalent per `git cherry` (dev + `c09f6281`/`20c2cbec`). Remaining unique content is exploratory evidence only: a test-only + `plastic_strain_rate_basis="effective"|"instantaneous"` option in constitutive_models.py + (notes file states "Default is unchanged — read-only evidence, not a proposal"), 3 A/B + benchmark scripts + notes, and 6 figure-only commits (retake/reactive-dt strategy PNGs; the + retake code itself was never committed). Development later replaced this investigation with + its own vardt benchmark suite (docs/advanced/benchmarks/bench_ve_square_vardt*.py) and + solved variable-dt VEP drift via snapshot projection (`5b3548c8`/`beb4f2e3`), superseding + the branch's line of inquiry. +- **`fix/darcy-sign-convention`** — 2 commits ahead of development (`a807b552`, `c50aaac4`; + last 2026-06-16, author J.C. Graciosa), no worktree, clean. They flip DarcyFlowModel.flux + to −q and re-plumb SNES_Darcy/TransientDarcy velocity projection and DFDt around the + flipped sign; the branch's own F1 docstring concedes the f≠0 source term is left wrong. + Explicitly superseded by merged PR #255 (development commit `46b8f16c`, 2026-06-19, + "Closes #214. Supersedes #243 (which flipped the flux instead, introducing an untested + f != 0 source-sign regression)") which fixes the same transient-velocity sign bug the + opposite way (assembly flux unchanged, velocity = −darcy_flux) with regression tests + test_1004b including f≠0. Current development `solvers.py:804` confirms the branch's + approach was rejected, not merely unlanded. +- **`ss2098/fix-thermal-convection-units-tutorial`** — One commit ahead (`374d1e68` "Fix + thermal convection units tutorial", sshukla2@alaska.edu, 2026-06-19) touching only + `docs/examples/Tutorial_Thermal_Convection_Units.py` (+238/−232). Superseded by merged PR + #263 (development commit `4185a768`, same author, merged 2026-07-01): the merged file + version equals the branch version plus 18 extra lines (kelvin_to_celsius helper, unit-aware + dt cap) — `git diff` branch→`4185a768` on the tutorial file shows only additions on the + merged side, so nothing unique remains on the branch. No worktree, no dirty files. +- **`test-darcy-rebase`** — Local rebase-test of origin/feature/darcy-richards-solvers: 7 + commits (2026-02/03) adding TransientDarcy + Richards solvers (solvers.py +465), + `utilities/retention_curves.py` (Haverkamp/van Genuchten retention curves), tutorials + 16/17, Tracy (2006) benchmark, porous-flow.md docs, tests 1005/1006. Fully superseded by + merge `4895c433` "Merge TransientDarcy and Richards solvers" (development, 2026-04-08): + `git cherry` marks 6/7 commits patch-equivalent; the 7th (`aa45e0dc`) differs from merged + `1dfa914f` by only 2 lines of conflict resolution; every branch-created file + (retention_curves.py, both notebooks, both tests, porous-flow.md, Tracy benchmark) is + byte-identical in current development. Remaining solvers.py delta is development having + advanced past the branch (June docstring/descriptor-pattern refactors, Darcy sign fix #255) + — branch holds nothing development lacks. No worktree, no dirty files. + +### REMOVE_WORKTREE_ONLY — investigated: merged, worktree clean (or holding only junk) + +| Branch | Worktree | Risk of loss | One-liner | +|--------|----------|--------------|-----------| +| `bugfix/fault-influence-edge` | `fault-influence-edge` | none | Finite-edge fix for Surface.influence_function, fully merged to development as PR #241 — safe to remove the worktree and delete the branch. | +| `bugfix/gamma-p1-deformed-normal` | `gamma-p1-deformed-normal` | none | Source branch of merged PR #264 (deformed-mesh normals/membership fix); 100% contained in development, worktree clean — remove worktree, delete branch. | +| `docs/snapshot-toolkit-changelog` | `in-memory-checkpoint` (misnamed) | none | Merged docs branch (PR #199) for the snapshot toolkit changelog/API docs; worktree contains only stale demo PNGs — remove worktree, branch safe to delete. | +| `feature/rotated-snes` | `rotated-snes` | none | Rotated free-slip SNES-integration branch, fully merged as PR #298 — remove clean worktree and delete branch; nothing unmerged or uncommitted. | +| `docs/ship-claude-skills` | `ship-claude-skills` | none | Ships the UW3 Claude skills in-repo — already landed verbatim via squash-merge PR #299; branch and clean worktree are pure leftovers. | +| `worktree-sl-field-carry-on-deform` | `sl-field-carry-on-deform` | none | Spent staging area for the merged coord-mutation gate + mesh.deform() work (PR #246): every staged byte already in development, untracked fault scripts superseded by newer copies in the fault-convection worktree — remove worktree and delete the merged branch. | +| `docs/snesfas-investigation` | `snesfas-spike` (misnamed) | none | Docs-only preservation of the SNESFAS/Vanka/grid-sequencing feasibility spike, already merged verbatim to development as PR #245 — safe to remove worktree and delete the branch. | +| `feature/anisotropic-metric-mover` | `winslow-mesh-smoother` (misnamed) | low | The merged (PR #209) Winslow/anisotropic mesh-smoother + OT_adapt branch; worktree dirt is May-2026 exploratory scripts plus two tracked edits already superseded in development — remove worktree, branch safe to delete. | + +**Evidence (REMOVE_WORKTREE_ONLY, investigated):** + +- **`bugfix/fault-influence-edge`** — One ahead commit `6291cab6` (Surface.influence_function + finite-edge fix: +75 lines `src/underworld3/meshing/surfaces.py`, +103-line + `tests/test_0851_surface_influence_edge.py`). Superseded by merged PR #241 (squash + `44aff945` on development, identical title); `git cherry` marks the commit + patch-equivalent ("−") and both files are byte-identical between branch tip and current + development. Worktree is clean (`status --porcelain` empty). +- **`bugfix/gamma-p1-deformed-normal`** — Holds the deformed-mesh boundary-normals / + domain-membership fix: 2 ahead commits (`8a9d2ff2` core fix, `e086e72a` Copilot-review + follow-up) touching `discretisation_mesh.py` (+177), `petsc_generic_snes_solvers.pyx` + (26 lines), and new tests test_0056_projected_normals_deform.py / + test_0057_deformed_domain_membership.py. Fully superseded by merged PR #264 (development + squash commit `51ecd292`, same title): the merge-base..tip diff md5 is byte-identical to + the squash commit's diff, and branch-tip file trees match development's merged state + exactly. Worktree is clean (`git status --porcelain` empty). +- **`docs/snapshot-toolkit-changelog`** — Branch holds the snapshot-toolkit documentation + commit (CHANGES entry, current API names, toctree wiring) on top of the snapshot-disk + feature (PR #198). It is 0 commits ahead of development and was merged via PR #199 (merge + commit `44c2f472` on development). Worktree + `/Users/lmoresi/+Underworld/underworld3-pixi/.claude/worktrees/in-memory-checkpoint` is + dirty only with 2 untracked demo render PNGs (snapshot_backstepping_demo.png, + snapshot_backstepping_spatial.png, dated 2026-05-12/13) plus a few gitignored vep_*.png + renders — no code or doc changes. Nothing exists only here except throwaway figures. +- **`feature/rotated-snes`** — Holds the rotated strong free-slip SNES integration: 9 commits + ahead of development (merge-base `af537d56`), 5 files (+605/−54): utilities/rotated_bc.py, + cython/petsc_generic_snes_solvers.pyx, systems/solvers.py, + tests/test_1018_rotated_freeslip.py, tests/parallel/test_1064_rotated_freeslip_parallel.py. + Fully superseded by merged PR #298 (squash commit `1d003481` on development, 2026-07-03, + "Rotated strong free-slip inside the nonlinear Stokes solve"), whose message enumerates the + branch's commit subjects. Content containment verified directly (`git cherry` '+' is a + squash-merge artifact): rotated_bc.py, solvers.py, and both test files are identical + between branch tip and development; the only .pyx difference is development-side additions + from the later PR #297 FMG guard. Worktree `.claude/worktrees/rotated-snes` is clean + (`git status --porcelain` empty). +- **`docs/ship-claude-skills`** — Single ahead commit `2349ed12` adds `.claude/skills/` + (6 skills, 24 files, 9,766 lines incl. cetz-figures examples/PNGs) plus a .gitignore + allow-list. Squash-merged to development as `aa3e383e` (#299, 2026-07-03, + ancestor-verified); `git diff 2349ed12..development` over .claude/skills is empty (only an + unrelated later Expt_Grains/ gitignore line differs). Worktree at + `.claude/worktrees/ship-claude-skills` is clean (`status --porcelain` empty). Pending + follow-up outside this repo: delete duplicate ~/.claude/skills copies now that #299 merged. +- **`worktree-sl-field-carry-on-deform`** — Branch is 0 ahead and a verified ancestor of + development (`git merge-base --is-ancestor` passes); its intended work merged as PR #246 + (merge `865e7027`, commit `f99c8aa2` "Foolproof mesh-coordinate mutation: capability gate + + public deform() + SL-field CARRY transfer", from bugfix/sl-field-carry-on-deform), and the + code is live in development (`discretisation_mesh.py:3022-3092`, `solvers.py:3711-3712`). + Worktree dirty state: 5 STAGED files whose blobs hash byte-identical to `f99c8aa2`'s + versions (nothing unmerged), plus 2 UNTRACKED scripts (fault_convection_adapt_loop.py, + fault_render.py) that are older June-16 snapshots of the larger June-21 copies living in + the fault-convection worktree (newer copies add + --fault-passive/--old-frame/--fault-base-smin/--sim-dir; sl-field copies hold nothing + unique). +- **`docs/snesfas-investigation`** — Branch holds exactly 1 commit (`fcb9692b`, 2026-06-16): + 9 new docs-only files, +1451 lines — 3 design notes + (docs/developer/design/snesfas-feasibility.md, snesfas-vanka-feasibility-study.md, + multilevel-nonlinear-stokes-strategy.md) plus docs/examples/snesfas_investigation/ + (README + 5 prototype scripts). No src/ changes. Fully superseded by merged PR #245 + (development commit `34a9dd46`, 2026-06-18, identical title): `git cherry` reports the + commit patch-equivalent ("− fcb9692b"), and `git diff development docs/snesfas-investigation` + over all 9 files is empty — every file is byte-identical on development. Worktree + `.claude/worktrees/snesfas-spike` is clean (`git status --porcelain` empty). The spike's + run scripts live outside the repo (~/+Simulations/snesfas_spike/) and are unaffected. +- **`feature/anisotropic-metric-mover`** — Branch holds the anisotropic/Winslow MMPDE + mesh-smoother rewrite (meshing/smoothing.py) plus mesh.OT_adapt() (commit `17b98a5d`) and + the constant-nullspace guard (tip `d4e1cbf6`). Fully merged: 0 commits ahead of + development; PR #209 merge commit `989d69ae` is in development, and the work was further + evolved by successor branch feature/anisotropic-mover-adapt-transfer (PR #228, merge + `59775f97`). Worktree is dirty with 112 files: ~108 are untracked throwaway + OT-investigation scripts (scripts/_ot_*, _sl_*, plot_*, launch/watch shells). The two + tracked modifications are superseded: (1) `discretisation_mesh.py` +28 adds + get_mean_radius() — development already has the refined version at + `discretisation_mesh.py:5600-5661`; (2) stagnant_lid_adapt_loop.py +270/−100 is a CLI reorg + whose substance (mesh.OT_adapt call, mover dispatch) exists in development's evolved + harness (script line 388, PR #228 commits `b6cbe0f1`/`40be67b7`); only cosmetic argparse + grouping is unique, sitting on a stale pre-#228 base (blob `238dda87` never committed + anywhere). + +--- + +## Bulk REMOVE_WORKTREE_ONLY — merged branches with clean worktrees + +The following 40 branches were pre-screened as fully merged into development with clean +worktrees (where a worktree exists). Spot verification performed 2026-07-03 on three +randomly chosen entries before writing this ledger: + +- `git merge-base --is-ancestor bugfix/swarm-empty-partition-read development` → **passes** (merged) +- `git merge-base --is-ancestor feature/advdiff-theta-exposure development` → **passes** (merged) +- `git merge-base --is-ancestor bugfix/jit-c-cache development` → **passes** (merged) +- Worktrees `advdiff-estimate-dt-percentile`, `advdiff-theta-exposure`, `jit-c-cache`: + `git status --porcelain` empty (0 dirty files) in each. + +Before executing the batch, the same two checks (`merge-base --is-ancestor` + clean +`status --porcelain`) MUST be run on every entry — do not trust this list blind. + +| Worktree (`.claude/worktrees/…`) | Branch | +|---|---| +| `advdiff-estimate-dt-percentile` | `bugfix/swarm-empty-partition-read` | +| `advdiff-monotone-mode-kwarg` | `feature/advdiff-monotone-mode-kwarg` | +| `advdiff-theta-exposure` | `feature/advdiff-theta-exposure` | +| `analytic-solcx-stress` | `bugfix/analytic-solcx-stress` | +| `boundary-slip-surfaces` | `feature/boundary-slip-surfaces` | +| `constant-nullspace-test` | `feature/manifest-constant-nullspace` | +| `crameri-benchmark` | `feature/crameri-benchmark` | +| `freesurface-material-advection` | `bugfix/freesurface-material-advection` | +| `global-evaluate-parallel-extrapolation` | `bugfix/global-evaluate-parallel-extrapolation` | +| `gmg-geometric-interp` | `feature/anisotropic-mover-adapt-transfer` | +| `in-cell-test-loose-semantics` | `bugfix/in-cell-test-loose-semantics` | +| `integrals-revert` | `bugfix/integrals-revert` | +| `jit-c-cache` | `bugfix/jit-c-cache` | +| `jit-constant-recompilation` | `worktree-jit-constant-recompilation` | +| `lambdify-caching-cachehit` | `bugfix/lambdify-caching-cachehit` | +| `memprobe-diagnostics` | `bugfix/memprobe-diagnostics` | +| `mesh-deform-cache-invalidation` | `feature/mesh-deform-cache-invalidation` | +| `multi-component-projection` | `feature/multi-component-projection` | +| `ns-ddt-projection-source` | `bugfix/ns-ddt-projection-source` | +| `parallel-singular-corruption` | `bugfix/parallel-singular-corruption` | +| `project-work-var-vtype` | `bugfix/project-work-var-vtype` | +| `projection-smoothing-length-v2` | `feature/projection-smoothing-length-v2` | +| `refined-submesh-pair` | `feature/refined-submesh-pair` | +| `region-ds-cell-labels` | `bugfix/region-ds-cell-labels-quarantine` | +| `sl-traceback-monotone-limiter` | `feature/sl-traceback-monotone-limiter` | +| `surface-submesh` | `feature/surface-submesh` | +| `swarm-advection-migrate` | `bugfix/swarm-advection-migrate` | +| `swarm-routed-point-eval` | `feature/swarm-mesh-adapt-transfer` | +| `ve-stokes-hang-130` | `bugfix/ve-stokes-hang-130` | +| `worktree-policy` | `docs/worktree-policy` | +| — (no worktree) | `archive/docs-legacy-quarto` | +| — (no worktree) | `bugfix/dminterp-vector-multifield` | +| — (no worktree) | `bugfix/vep-investigation-fixes` | +| — (no worktree) | `docs/maturity-gated-release` | +| — (no worktree) | `feature/constant-nullspace-test` | +| — (no worktree) | `feature/gmg-geometric-interp` | +| — (no worktree) | `feature/integrate-surface-submesh` | +| — (no worktree) | `investigate/issue-151` | +| — (no worktree) | `test-multicomp` | + +--- + +## Execution Protocol + +1. **Archive tag before any delete.** For every branch slated for deletion (ARCHIVE_DELETE, + REMOVE_WORKTREE_ONLY, EXTRACT-after-rescue, and every bulk entry), first: + `git tag archive/ ` — tags are cheap and make every deletion + reversible. Push tags (`git push origin --tags`) so the archive survives local cleanup. +2. **Re-verify at execution time.** For each branch in a batch: confirm + `git merge-base --is-ancestor development` (or, for squash merges, re-run the + content-diff check recorded in the evidence above) and confirm the attached worktree is + clean (`git -C status --porcelain` empty). Any mismatch → pull the branch out + of the batch and escalate. +3. **Maintainer signs off deletions per batch.** Louis Moresi approves each deletion batch + before it runs. Suggested batching: (a) bulk REMOVE_WORKTREE_ONLY list; (b) investigated + REMOVE_WORKTREE_ONLY; (c) ARCHIVE_DELETE with risk "none"; (d) ARCHIVE_DELETE with risk + "low"; (e) EXTRACT branches only AFTER their rescue commits/PRs have landed and been + verified on development. +4. **EXTRACT rescues come first.** No EXTRACT-verdict branch or worktree may be touched until + the specific files/commits named in its evidence are committed somewhere durable (a fresh + branch pushed to origin, or a merged PR). High-risk EXTRACT rows hold uncommitted work — + removing the worktree destroys it unrecoverably. +5. **LAND branches are not cleanup targets.** They go through the normal PR flow + (push to origin first for the local-only ones: feature/adapt-on-top, + bugfix/yield-homotopy) and their worktrees are removed only after merge. +6. **KEEP_ACTIVE — do not touch:** `feature/numpy2-support` (the main-checkout work stream; + per campaign instructions), the quality-audit branch/worktree + (`feature/quality-audit-2026-07` at `.claude/worktrees/quality-audit-2026-07`), + `bugfix/custom-mg-parallel`, and `feature/adaptive-convection`. Both of the latter two + should be pushed to origin promptly as a safety measure (they contain unpushed, + locally-unique commits) — pushing is not "touching". +7. **Worktree removal command:** use `./uw worktree remove ` only where the branch + should also die; where the branch must survive (LAND before merge), use + `git worktree remove ` alone. Never key on worktree directory names — six worktrees + are misnamed relative to their branch (see System Architecture). + +## Testing Instructions + +This is an audit document; there is no code to test. To re-verify any row: + +```bash +cd /Users/lmoresi/+Underworld/underworld3-pixi/.claude/worktrees/quality-audit-2026-07 + +# merged-by-ancestry? +git merge-base --is-ancestor development && echo merged + +# merged-by-squash? (content check against the squash commit named in the evidence) +git diff -- + +# unique commits +git log --oneline development.. + +# unpushed? +git log --oneline origin/.. # (or: git branch -r --contains ) + +# worktree dirty? +git -C /Users/lmoresi/+Underworld/underworld3-pixi/.claude/worktrees/ status --porcelain +``` + +The bulk-list spot checks recorded above were run on 2026-07-03 against development tip +`9bd6c8ee` from the audit worktree. + +## Known Limitations + +- **Not every branch was triaged.** Two items known from prior records were NOT investigated + in this pass and are absent from the ledger: `bugfix/rotated-freeslip-schur-pc` (worktree + `.claude/worktrees/rotated-freeslip-schur-pc`, clean at verification time, tip `b8e4a053`) + and `origin/bugfix/stokes-constrained-parallel` (remote-only; prior records flag an OPEN + release-blocking item — 0.4% 3D velocity discrepancy). Both need a follow-up triage before + any repo-wide cleanup claims completeness. Other remote-only branches + (origin/feature/stokes-constrained, origin/bugfix/numpy2-followup-cross-and-artifact, …) + were out of scope. +- **State drift.** Verdicts were formed against development@`1d003481`..`9bd6c8ee` on + 2026-07-03. Any merge, push, or worktree edit after that date can invalidate a row — hence + the mandatory re-verification step in the execution protocol. Notably, at final + verification the main checkout was on `development` (clean) and no local + `feature/numpy2-support` branch existed (it exists on origin); the KEEP_ACTIVE instruction + for numpy2-support is carried from the campaign brief regardless. +- **Bulk list verified by sampling.** Only 3 of 40 bulk entries were individually re-verified + (all passed); the remaining 37 rely on the pre-screening and MUST be re-checked at + execution time (protocol step 2). +- **Evidence line numbers** refer to the branch/commit named in each evidence block, not + necessarily to current development; development moves daily. +- Dirty-worktree inventories describe file-level content, not semantic completeness — an + EXTRACT rescue should be reviewed by someone who knows the sub-project before landing. + +## Sign-Off + +| Role | Name | Date | Status | +|------|------|------|--------| +| Maintainer | Louis Moresi | 2026-07-05 | Pending review | +| Author | Claude (audit session, Dimension 2 — branch triage) | 2026-07-03 | Complete | \ No newline at end of file diff --git a/docs/reviews/2026-07/DOCS-STANDARDS-COHERENCE.md b/docs/reviews/2026-07/DOCS-STANDARDS-COHERENCE.md new file mode 100644 index 000000000..57eda7837 --- /dev/null +++ b/docs/reviews/2026-07/DOCS-STANDARDS-COHERENCE.md @@ -0,0 +1,467 @@ +# Docs & Standards Coherence Review — July 2026 Quality Campaign (Dimension 6) + +**Status**: audit complete; findings adversarially verified 2026-07-03 +**Base**: `development` @ `1d003481` (audit worktree, campaign index at `e848d131`) +**Scope**: coherence of the documentation system itself — which documents claim +authority, whether they agree with the code and with each other, the state of +the docstring backlog and its triage queue, the changelog as the quarterly +record, and hygiene of the design-doc directory. + +Abbreviation used throughout: `pyx` = `src/underworld3/cython/petsc_generic_snes_solvers.pyx`. + +## Overview + +The June 2026 development burst (~95 first-parent commits May–June, +12.6k +lines to `src/`) moved the *code* forward much faster than the documents that +govern it. The result is not missing documentation so much as **contradictory +authority**: the document CLAUDE.md names as the "Authoritative Reference" for +data access (`UW3_Style_and_Patterns_Guide.md`) teaches a pattern the code +deprecates at runtime, prescribes a docstring format (markdown-for-pdoc) that +the settled NumPy/Sphinx standard replaced, and mandates a file format (Quarto +`.qmd`) of which zero instances exist in the repository. Meanwhile the +machinery meant to drive documentation remediation is itself stale: the +docstring review queue predates every June feature, and the changelog's +"Q2 (April – June)" section ends in April. + +Seven findings were adversarially verified (DOC-01 … DOC-07); one +lower-severity finding was evidence-checked by the author — every cited line +was read directly in this worktree — but did not go through the adversarial +pass (DOC-08). All fixes are docs-only (plus two docstring edits); none +touches solver numerics or public API, consistent with the campaign ground +rules. Details refuted or corrected during verification are recorded in the +appendix so the same leads are not re-chased. + +### Verified findings (adversarially checked) + +| ID | Sev | Effort | File | Finding | +|----|-----|--------|------|---------| +| DOC-01 | high | M | `docs/developer/UW3_Style_and_Patterns_Guide.md` | The CLAUDE.md-declared authoritative style guide is stale on four normative topics: docstring format, doc file format, data-access examples, and Quarto front matter | +| DOC-02 | high | S | `docs/docstrings/review_queue.md` | Docstring triage queue is six months stale — flags now-complete items as missing and omits the entire June 2026 public API | +| DOC-03 | medium | M | `docs/developer/CHANGELOG.md` | Quarterly changelog (CIG/stakeholder record) has no entry after April 2026 despite ~95 May–June first-parent commits and a dozen headline PRs | +| DOC-04 | medium | S | `CLAUDE.md:324` | Authority pointer for data access names the stale guide; four overlapping documents cover one topic with no governing-doc map | +| DOC-05 | medium | M | `src/underworld3/swarm.py:4559` | The genuinely-undocumented public API (`uw.function.evaluate` params, `Swarm.advection`, the checkpoint trio) is not what the stale queue would have a docstring wave work on | +| DOC-06 | medium | S | `docs/developer/design/` | 30 tracked experiment artifacts (24 `_*.py`, 6 `_*.trace.txt`) plus 5 result PNGs sit beside ~36 genuine design docs | +| DOC-07 | medium | M | `docs/developer/design/jacobian-consistent-tangent.md` | Merged/parked/superseded design docs lack the status headers the directory's own convention establishes | + +### Author-verified findings (not adversarially checked) + +| ID | Sev | Effort | File | Finding | +|----|-----|--------|------|---------| +| DOC-08 | low | S | `docs/advanced/mesh-adaptation.md:247` | One deprecated `mesh.data` survives in user-facing docs, and the `mesh.adapt()` docstring's own Examples use `with mesh.access(...)` + `mesh.data` | + +## Changes Made + +None — audit only. Proposed fixes are recorded per finding below and are +scheduled for **Wave E (docs alignment)**, with DOC-05 feeding the docstring +wave and DOC-06 a candidate for **Wave A (deletions)**. + +## System Architecture + +What this dimension's audit established about how the documentation system is +put together, for the maintainer: + +**Three layers claim normativity, and they disagree.** (1) `CLAUDE.md` is the +operational contract for AI-assisted sessions — its "Documentation Requests" +section correctly mandates MyST `.md`/Sphinx, and its Data Access quick table +correctly marks `mesh.data` and the access-context deprecated. (2) The +document CLAUDE.md itself points to as "Authoritative Reference" +(`UW3_Style_and_Patterns_Guide.md`, also routed to contributors via +`docs/developer/guides/contributing.md:57`) contradicts layer 1 on docstring +format, file format, and coordinate access. (3) The code is the ground truth: +`mesh.data` emits a `DeprecationWarning` directing to `mesh.X.coords` +(`discretisation_mesh.py:3563-3578`), and the solver docstrings that exist +follow NumPy/Sphinx style (e.g. `solve` at `pyx:2923`, `SNES_Scalar` at +`pyx:2197`). A reader who follows the declared chain of authority ends at the +one document that is wrong. Where a topic has a single dated design doc with a +supersession banner — units, via `UNITS_SIMPLIFIED_DESIGN_2025-11.md:3` ("This +document supersedes all previous units planning documents") — coherence is +fine; the failures cluster exactly where multiple undated documents share a +topic (data access has four: the Style Guide, `subsystems/data-access.md` +["Current Implementation (2025+)"], `UW3_Developers_NDArrays.md`, and +`design/ARCHITECTURE_ANALYSIS.md`). + +**The remediation machinery is itself documentation, and it rotted the same +way.** The docstring backlog is managed through a generated queue +(`docs/docstrings/review_queue.md`, from `scripts/docstring_sweep.py`) last +committed 2026-01-13 (`cdf5bb21`); the quarterly changelog is hand-maintained +and last touched 2026-05-04 (content ending April). Both are "pull" documents +with no per-release regeneration step, so a six-month development burst +invalidated them silently. The design directory has the opposite dynamic: +things get *added* (experiment scripts, solver traces, result PNGs beside the +write-up that cites them) and never re-filed once the investigation closes. +Some closed docs do carry status markers (`multilevel-nonlinear-stokes-strategy.md` +frontmatter `status: PARKED (2026-06-16)`; `EXPONENTIAL_VE_INTEGRATOR.md` has +a `**Status**` line) — roughly 20 of the 36 design docs — so the convention +exists and merely needs completing, not inventing. + +**Consequence for the campaign**: Wave E should fix the *authority map* first +(DOC-04), then the documents it points at (DOC-01), then regenerate the queue +(DOC-02) before any docstring-wave slot is allocated (DOC-05) — otherwise the +wave will be triaged from January data and re-document what June already +documented. + +## Findings in detail + +### DOC-01 — Style Guide: the declared authority is stale on four normative topics + +**Severity: high · Effort: M · Category: docs-standards** +**File**: `docs/developer/UW3_Style_and_Patterns_Guide.md` + +The guide carries Quarto YAML front matter (L1–21: `format: html/pdf`, +`code-fold`, `theme: cosmo`) and is named "Authoritative Reference" +(`CLAUDE.md:324`) and the contributor style reference +(`docs/developer/guides/contributing.md:57`). Four of its normative sections +contradict the current standards: + +1. **Docstring format** — "## Markdown Docstrings for pdoc/pdoc3" (L113, with + a full worked markdown-headers example, and summary item 4 at L459) + contradicts the settled NumPy/Sphinx RST standard: the conversion plan + exists at `docs/plans/docstring-conversion-plan.md`, and shipped docstrings + follow it (e.g. `solve` at `pyx:2923` — Parameters/Returns/Examples/Notes; + `SNES_Scalar` at `pyx:2197`). +2. **Doc file format** — "**Format**: Quarto markdown (`.qmd`)" (L422–423) and + the migration-table row "Plain markdown → Quarto markdown" (L470) + contradict CLAUDE.md's Documentation Requests section (MyST `.md` for + Sphinx). Zero `.qmd` files exist in the repository. +3. **Data access** — `mesh.data[0] = new_position` is presented under + "# Preferred: Direct array access" (L205) and `mesh.data += displacement` + appears in the delay-callback example (L270), while the code deprecates + `mesh.data` at runtime (`discretisation_mesh.py:3563` property; warning at + 3576–3578, "use mesh.X.coords instead"). The "Coordinate System + Transformations" migration block (L217–221) directs readers to *private* + attributes `swarm._particle_coordinates` (`swarm.py:3039`) and + `mesh._deform_mesh` (`discretisation_mesh.py:3148`) as the "NEW" pattern. +4. **Front matter** — the Quarto YAML header itself (L1–21) is dead weight in + a Sphinx/MyST build. + +**Scope guard**: `swarm.data` (L206, L271) is **not** stale — it remains the +current documented pattern for swarm variables per CLAUDE.md's module-boundary +table and must be kept. Likewise the `with mesh.access(var)` example at L251 +is correctly filed under a deprecated-legacy heading — it is documentation +*of* the deprecation, not an instance of staleness. + +**Proposed fix**: rewrite the four sections — (a) replace the pdoc section +with the NumPy/Sphinx RST standard (one worked example with `:math:` and +Parameters/Returns, referencing `docs/plans/docstring-conversion-plan.md`); +(b) replace the `.qmd` file-convention section and migration-table row with +MyST `.md`/Sphinx guidance matching CLAUDE.md; (c) fix the coordinate examples +to `mesh.X.coords` and delete the private-attribute migration advice; (d) +replace the Quarto front matter with a plain MyST title. Keep as-is (verified +still consistent with current code): Naming Conventions (~L55–60), +NDArray_With_Callback patterns, array-vs-data shapes (matches +`subsystems/data-access.md`), MPI patterns, callback patterns, testing +patterns, and test-file naming (L425–429). + +### DOC-02 — Docstring review queue is six months stale in both directions + +**Severity: high · Effort: S · Category: docstrings** +**File**: `docs/docstrings/review_queue.md` (last commit `cdf5bb21`, 2026-01-13) + +The queue that would drive the docstring-backlog wave misrepresents the +codebase both ways: + +- **False negatives (already fixed)**: the queue flags `❌ solve [method] + L1264` and `❌ SNES_Scalar [class] L807` as needing overview/parameters; + both now carry full NumPy docstrings (`pyx:2923` and `pyx:2197` + respectively — read directly). +- **Missing the newest API entirely**: `grep -cE "add_nitsche_bc| + add_rotated_freeslip_bc|boundary_flux|set_custom_fmg|consistent_jacobian"` + over the queue returns **0**, while all five exist in current source + (`pyx:3317`, `pyx:5170`, `pyx:2165`, `utilities/custom_mg.py:605`, + `pyx:92/2165ff`). + +A wave triaged from this file would spend slots re-fixing fixed items and skip +the least-documented, newest API. + +**Proposed fix**: rerun `scripts/docstring_sweep.py` (exists, verified) +against `development@1d003481` to regenerate `review_queue.md` (and +`inventory.json`) **before** allocating any docstring-wave slots; add the +sweep to the release checklist so the queue cannot go stale unnoticed again. + +### DOC-03 — Changelog stops in April; ~95 May–June commits unrecorded + +**Severity: medium · Effort: M · Category: changelog** +**File**: `docs/developer/CHANGELOG.md:7` + +The changelog declares itself the source for quarterly CIG/stakeholder +reporting (line 3). Its "2026 Q2 (April – June)" section contains exactly +three entries, all April (DDt.set_initial_history, Multi-Component Projection, +PETSc Jacobian layout fix); the file's last commit is `aed517f6` (committer +date 2026-05-04). Measured in this worktree: **95** first-parent commits with +commit dates 2026-05-01 … 2026-06-30 on `development@1d003481`, and zero of +the headline PRs appear anywhere in the file (grep for #216, #250, #251, +#258, #259, #264, #265, #266, #275, #276, #290, #293, #294, #297, #298 → 0 +hits). + +One scoping nuance (verified against `git log --first-parent`): several +headline features merged just *after* Q2 closed — #298 on 2026-07-03; #297, +#216, #258, #293, #294, #290 all on 2026-07-02 — while #275, #265 (06-24), +#266/#264 (06-22), #259 (06-20), #251 (06-19), #250 (06-18) are genuinely +June. The backfill should therefore read **"May – early July"**, either as an +extended Q2 section or a Q2 section plus the first Q3 entries. + +**Proposed fix**: backfill at the changelog's existing conceptual granularity +(roughly 10–14 grouped entries; the PR descriptions already contain the +prose). Add a release-checklist item in +`docs/developer/guides/release-process.md` requiring a changelog sweep of +`git log --first-parent` since the last entry. + +### DOC-04 — Authority map: repoint CLAUDE.md and adopt one-governing-doc-per-topic + +**Severity: medium · Effort: S · Category: docs-authority** +**File**: `CLAUDE.md:324` + +`CLAUDE.md:324` reads `**Authoritative Reference**: +\`docs/developer/UW3_Style_and_Patterns_Guide.md\`` under "## Data Access +Patterns" — yet CLAUDE.md's own quick table eight lines below correctly marks +`mesh.data` deprecated, and the guide it crowns teaches `mesh.data` as +"Preferred" (DOC-01). Four documents cover data access: +the Style Guide, `subsystems/data-access.md` (heading "## Current +Implementation (2025+)", accurate on the `array`-property / +NDArray_With_Callback model), `UW3_Developers_NDArrays.md` (internals), and +`design/ARCHITECTURE_ANALYSIS.md` (2025 analysis of the same machinery). By +contrast, units has a clean single authority +(`design/UNITS_SIMPLIFIED_DESIGN_2025-11.md:3`, explicit supersession banner) +— that is the pattern to replicate. + +**Proposed fix**: adopt a one-governing-doc-per-topic map and repoint +`CLAUDE.md:324` accordingly — data access → `subsystems/data-access.md` +(governing), `UW3_Developers_NDArrays.md` demoted to "internals reference", +`ARCHITECTURE_ANALYSIS.md` given a "historical analysis (2025), see +data-access.md" banner; docstring style → the corrected Style Guide section +(with `docs/plans/docstring-conversion-plan.md` as implementation plan); doc +file format → CLAUDE.md Documentation Requests (Style Guide `.qmd` section +superseded); units → `UNITS_SIMPLIFIED_DESIGN_2025-11.md` (add subordination +notes to `ai-notes/units-system-guide.md` and +`ai-notes/MESHVARIABLE_UNITS_GUIDE.md`); branching → +`guides/branching-strategy.md` (no conflict found); testing tiers → +`TESTING-RELIABILITY-SYSTEM.md` with `ai-notes/TEST-CLASSIFICATION-2025-11-15.md` +marked as a dated snapshot. Record the table in `docs/developer/index.md` as +the master authority index. + +### DOC-05 — The real docstring gaps are not where the stale queue points + +**Severity: medium · Effort: M · Category: docstrings** +**File**: `src/underworld3/swarm.py:4559` (and companions below) + +Verified undocumented public API, in priority order for the wave: + +1. **`uw.function.evaluate`** (`function/functions_unit_system.py:789`, + confirmed as the `uw.function` export) — 14 parameters; the docstring + documents only `monotone` and defers the other ~13 to the *private* + `_evaluate_impl`. Inline the full parameter documentation (or hoist + `_evaluate_impl`'s docs into the public wrapper). +2. **`Swarm.advection`** (`swarm.py:4559`) and **`NodalPointSwarm.advection`** + (`swarm.py:4961`) — no docstrings. NumPy docstrings should cover `V_fn`, + `delta_t` units handling (`delta_t` is non-dimensionalised internally, + `swarm.py:4573`), `order`, `corrector`, and `step_limit` — noting the + defaults differ (`Swarm.advection` `step_limit=False` at 4567; + `NodalPointSwarm.advection` `step_limit=True` at 4969). +3. **The checkpoint trio** — `Swarm.read_timestep` (`swarm.py:3879`, + signature `base_filename/swarm_id`), `SwarmVariable.read_timestep` + (`swarm.py:1971`, signature `data_filename/swarmID/data_name`), and + `SwarmVariable.write_proxy` (`swarm.py:1960`) — all undocstringed; the two + `read_timestep` signatures genuinely disagree and each constructs + undocumented filename conventions that only the source reveals. + +**Do not** spend slots on items the January queue flags that are already +complete: `solve`, `add_essential/natural/dirichlet_bc`, `add_nitsche_bc`, +`add_rotated_freeslip_bc`, `estimate_dt`, `boundary_flux` (all in `pyx`) and +`set_custom_fmg` (`utilities/custom_mg.py:605`) all have docstrings. + +### DOC-06 — 35 experiment artifacts tracked inside the design-doc directory + +**Severity: medium · Effort: S · Category: docs-hygiene** +**File**: `docs/developer/design/` (e.g. `_exp_integrator_phase_a.py`) + +Counted directly in the worktree: **24** underscore-prefixed `.py` experiment +scripts (8 matching `_exp_integrator_*` plus `_exp_jury_rig_minimal.py` and +15 others incl. `_repro_dminterp_multifield_bug.py`), **6** `_*.trace.txt` raw +solver logs, and **5** `exp_integrator_*.png` result images — 35 tracked +artifacts beside the ~36 genuine design `.md` docs. +`EXPONENTIAL_VE_INTEGRATOR.md` cites the scripts and PNGs at ~10 points +(verified at lines 50, 89, 218, 279, 375 and further), so they are provenance +and cannot be blindly deleted; grep across `src/`, `tests/`, and all docs +finds **no other consumer** of any artifact, and nothing at all references the +6 trace files. + +**Proposed fix**: `git mv` the 24 scripts and 5 PNGs to +`docs/developer/design/experiments/exp-integrator/` (one batch; +`_exp_jury_rig_minimal.py` moves with them since `EXPONENTIAL_VE_INTEGRATOR.md` +cites it) and update the ~10 relative references; delete the 6 `.trace.txt` +logs outright (reproducible from the scripts, preserved in git history); +relocate `_repro_dminterp_multifield_bug.py` next to the test suite or attach +it to the corresponding issue — it is a bug reproduction, not a design +artifact. Candidate for Wave A. + +### DOC-07 — Design docs missing status headers the directory convention expects + +**Severity: medium · Effort: M · Category: docs-hygiene** +**File**: `docs/developer/design/jacobian-consistent-tangent.md:1` + +The directory holds 36 `.md` docs; roughly 20 carry a status-style marker (the +convention is real: `multilevel-nonlinear-stokes-strategy.md` has frontmatter +`status: PARKED (2026-06-16)`; `EXPONENTIAL_VE_INTEGRATOR.md` a `**Status**` +line). Verified missing and misleading without one: + +- `jacobian-consistent-tangent.md` — no status header, never mentions that it + merged as PR #258 (`c63cd707`, opt-in `solver.consistent_jacobian`). +- `snesfas-feasibility.md` and `snesfas-vanka-feasibility-study.md` — open + with GO verdicts, preserved via PR #245, but contain zero reference to + PR #290, whose custom-prolongation FMG is the shipped geometric-MG path. +- `MATHEMATICAL_MIXIN_DESIGN.md:4` — "**Status**: Design Phase" (2025-10-26) + although `utilities/mathematical_mixin.py` ships and CLAUDE.md cites the doc + as its internals reference. +- No doc-level status: `COORDINATE_MIGRATION_GUIDE.md`, + `ARCHITECTURE_ANALYSIS.md`, `fmg-checkpoint-hierarchy`, + `fault-refinement-simplification`, `mesh-adaptation-formulation`, + `petsc-dmplex-checkpoint-reload-plan`, `ND_UNITS_BOUNDARY_CONTRACT`. + +**Proposed fix**: add a one-to-three-line `**Status:**` header per doc +following the existing convention — e.g. jacobian doc → "Implemented — PR #258 +(opt-in `solver.consistent_jacobian`, default off)"; snesfas docs → +"Investigation record (preserved via PR #245); production geometric-MG path is +custom prolongation, PR #290"; MATHEMATICAL_MIXIN → "Implemented +(`utilities/mathematical_mixin.py`)"; COORDINATE_MIGRATION_GUIDE and +ARCHITECTURE_ANALYSIS → historical/2025 snapshots. Requires a short +verification pass per doc against git history before stamping. + +### DOC-08 — Last deprecated coordinate patterns in user-facing docs (author-verified) + +**Severity: low · Effort: S · Category: docs-standards** +**Files**: `docs/advanced/mesh-adaptation.md:247`, +`src/underworld3/discretisation/discretisation_mesh.py:5940-5942` + +Read directly in this worktree: the mesh-adaptation guide's complete worked +example prints `mesh.data.shape[0]` (line 247) — `mesh.data` raises a +`DeprecationWarning`; current API is `mesh.X.coords`. Worse, the +`mesh.adapt()` docstring's own Examples section (`discretisation_mesh.py`, +`with mesh.access(metric):` at 5940 and `fault.distance_from(mesh.data)` at +5942) demonstrates both deprecated patterns — the API teaching what its own +deprecation warnings forbid. Beginner tutorials spot-checked clean: no +`mesh.access`/`swarm.access` in any of the 18 notebooks (see appendix). + +**Proposed fix**: replace `mesh.data.shape[0]` with `mesh.X.coords.shape[0]` +in `mesh-adaptation.md:247`; rewrite the adapt-docstring example to direct +`metric.data[:, 0] = ...` plus `mesh.X.coords` per the current pattern table. +(The docstring edit is a comment-only change to `discretisation_mesh.py`, not +`pyx`; no numerics.) + +## Testing Instructions + +Wave E fixes are docs and docstrings only; validation is mechanical: + +1. **Docs build** — `pixi run docs-build` must complete without new warnings + after every batch (Style Guide rewrite, banner additions, artifact moves). + Broken relative links from the DOC-06 `git mv` batch will surface here; + additionally `grep -rn "_exp_integrator\|_exp_jury_rig\|.trace.txt" + docs/` must show only paths under `design/experiments/exp-integrator/`. +2. **Pattern regression** — run `/check-patterns` (deprecated-pattern scanner) + over `docs/` after DOC-01/DOC-08 land: zero hits for `mesh.data` as a + coordinate accessor and for `with mesh.access(` outside explicitly-labelled + legacy/deprecation sections. Confirm no over-correction: `swarm.data` usages + must remain. +3. **Docstring queue regeneration (DOC-02)** — run + `scripts/docstring_sweep.py`; assert the regenerated queue (a) no longer + flags `solve`/`SNES_Scalar` in `pyx`, and (b) contains entries for + `Swarm.advection`, `read_timestep`, `write_proxy`, and + `uw.function.evaluate`. That double-check validates DOC-05's target list. +4. **Docstring wave (DOC-05, DOC-08)** — docstrings render via the docs build; + spot-check `help(uw.function.evaluate)` and `help(uw.swarm.Swarm.advection)` + in `pixi run -e default python`. Run `pytest -m "level_1 and tier_a"` + before/after as the campaign gate (changes are comment-only; this guards + against accidental code edits, especially for the `discretisation_mesh.py` + and any `pyx` docstring touches, which require a rebuild via `./uw build`). +5. **Changelog (DOC-03)** — review-only; verify each backfilled entry cites a + real merged PR (`git log --first-parent --oneline` since `aed517f6`). +6. **Authority map (DOC-04)** — after repointing, grep for the old claim: + `grep -n "Authoritative Reference" CLAUDE.md` should point at + `subsystems/data-access.md`; confirm `docs/developer/index.md` carries the + authority table and builds into the toctree. + +## Known Limitations + +- **Line numbers are exact at `1d003481`/`e848d131` only** and will drift as + waves land; findings identify sections by heading/content as well for that + reason. +- **DOC-08 was not adversarially verified** (author-read only). Its evidence + is small and was re-read directly for this document, but it kept its + original tier per campaign rules. +- **Coverage**: this audit checked the documents that claim authority, the + remediation machinery, the design directory, and spot-checked the 18 + beginner tutorial notebooks for deprecated patterns. It did **not** + line-audit `docs/advanced/` beyond mesh-adaptation.md, the ~695 example + `.py` files under `docs/`, or notebook prose accuracy — the API-consistency + review (dimension 3) covers the BC-argument-order sweep of those files. +- **DOC-07 requires per-doc git-history verification before stamping** status + headers; the suggested wordings above are grounded in the PRs cited but each + stamp should be confirmed at fix time. +- **Commit-count convention**: the "95 commits" figure counts first-parent + commits by commit date 2026-05-01…2026-06-30 at `1d003481` (measured both + via explicit `--since/--until` datetime bounds and via `%cs` date strings — + both give 95). A bare `--until=2026-07-01` gives 93 (boundary/timezone + artifact); including early-July merges gives ~105. Use 95. + +## Sign-Off +| Role | Name | Date | Status | +|------|------|------|--------| +| Maintainer | Louis Moresi | 2026-07-05 | Pending review | +| Author | Claude (audit session) | 2026-07-03 | Complete | + + + +- **Auditor**: Claude (docs dimension subagent), 2026-07-03. All `file:line` + evidence above read directly in the audit worktree at `e848d131` (content + identical to `development@1d003481` for every cited file). +- **Adversarial verification**: DOC-01 … DOC-07 passed an independent + adversarial pass; corrections from that pass are incorporated and logged in + the appendix. DOC-08 author-verified only. +- **Maintainer sign-off (L. Moresi)**: ☐ pending — required before Wave E + scheduling; DOC-06 deletions additionally require the campaign's + per-batch deletion sign-off (Wave A rules). + +--- + +## Appendix — Refuted and corrected claims (do not re-find) + +No finding was refuted outright, but the following details from earlier drafts +were corrected during verification, and several suspects were checked clean. +Recording them here so later waves don't rediscover the same false leads: + +1. **"Only 54 commits May–June"** — wrong; measured 95 first-parent commits + (commit dates 2026-05-01…2026-06-30) at `1d003481`. The changelog gap is + *larger* than first claimed (DOC-03). +2. **"Headline June features"** — several actually merged 2026-07-02/03 + (#298, #297, #216, #258, #290, plus unlisted #293, #294); genuinely-June + are #275, #265, #266, #264, #259, #251, #250. Backfill scope is + "May – early July", not "May–June" (DOC-03). +3. **"23 of 38 design docs have status markers"** — not reproducible; the + directory holds 36 `.md` docs, ~20 with a status-style marker. The + convention-completion argument stands; the precise count does not (DOC-07). +4. **"9 `_exp_integrator_*.py` files"** — actually 8, plus + `_exp_jury_rig_minimal.py` which is cited from `EXPONENTIAL_VE_INTEGRATOR.md` + and must move with the batch. Totals (24 scripts / 6 traces / 5 PNGs / 30 + underscore-tracked artifacts) are exact (DOC-06). +5. **Style Guide line drift** — the pdoc heading is at L113 (not 111); + `mesh.data += displacement` at L270 (L271 is `swarm.data`); the `mesh.data` + deprecation property is at `discretisation_mesh.py:3563` with the + `warnings.warn` call at 3576–3578; `solve` is defined at `pyx:2923` + (DOC-01, DOC-02). +6. **Checked clean — not staleness**: the Style Guide's `with mesh.access` + example (L251) is correctly filed under a deprecated-legacy heading; + `swarm.data` (L206/L271) is the *current* swarm pattern and must not be + "fixed" (DOC-01). +7. **Checked clean — beginner tutorials**: no `mesh.access`/`swarm.access` in + any of the 18 notebooks; `mesh.points` grep hits there are PyVista + `pvmesh.points`, not UW3; `uw.non_dimensionalise` in tutorials 12/15 is the + current public `units.py` function, not the deprecated `model.py` methods + (DOC-08 scope check). +8. **Checked clean — already documented**: `solve`, the BC family + (`add_essential/natural/dirichlet_bc`, `add_nitsche_bc`, + `add_rotated_freeslip_bc`), `estimate_dt`, `boundary_flux` (all `pyx`) and + `set_custom_fmg` (`utilities/custom_mg.py:605`) have docstrings — despite + what the January queue says (DOC-02, DOC-05). +9. **`evaluate` parameter count** — 14 parameters total, so the public wrapper + defers ~13 (not ~14) to `_evaluate_impl` (DOC-05). + +*Underworld development team with AI support from Claude Code.* \ No newline at end of file diff --git a/docs/reviews/2026-07/LOOSE-ENDS-AUDIT.md b/docs/reviews/2026-07/LOOSE-ENDS-AUDIT.md new file mode 100644 index 000000000..20e63d92c --- /dev/null +++ b/docs/reviews/2026-07/LOOSE-ENDS-AUDIT.md @@ -0,0 +1,434 @@ +# Loose-Ends Audit — July 2026 Quality Campaign (Dimension 1) + +**Status**: audit complete; findings adversarially verified 2026-07-03 (where marked) +**Base**: `development` @ `1d003481` (audit worktree, campaign index at `e848d131`) +**Scope**: TODO/TODO(BUG) markers, stubs, skipped/xfailed tests, disabled logic +(`if 0`, commented-out code) across `src/` and `tests/`. + +Abbreviation used throughout: `pyx` = `src/underworld3/cython/petsc_generic_snes_solvers.pyx`. + +## Overview + +June 2026's ~132 commits left a large but **highly bimodal** population of loose +ends. At one extreme, several `TODO(BUG)` comments are among the best engineering +records in the codebase (the partition-seam natural-BC note at `pyx:2545` documents +three tested-and-rejected workarounds and three fix options). At the other, a +one-line dead ternary (`comp = 0 if ... == 1 else 0`, `_function.pyx:845`) silently +returns wrong derivative values for every multi-component variable on the rbf +evaluation path — a real correctness bug hiding behind a `TODO` that reads like a +note-to-self. + +The skipped-test population divides cleanly into three families: + +1. **Skips masking real, reproducible bugs** — the skip is honest, the bug is live + (LE-04, LE-06, LE-07, LE-08). Two were reproduced live in this worktree's own + built environment during verification. +2. **Aspirational placeholders** — ~30+ skips across the units test suite + (`test_0620/0630/0730/0800/0803/0804/0811`) test interfaces that were *proposed* + and never implemented, or implemented with different semantics. They provide + false coverage: several now pass vacuously rather than skip (LE-10, LE-11). +3. **Legitimately-conditional markers** — correctly-annotated xfails guarding + tracked open bugs (LE-26) or acknowledged fragile tests (LE-21). These should + stay. + +Every `file:line` cited below was read directly in this worktree; line numbers are +exact at `1d003481`. Twelve findings (LE-01 … LE-12) are adversarially verified; +fourteen (LE-13 … LE-26) had their evidence lines personally read by the author but +did not go through the full adversarial pass. Two claims were **refuted** during +verification and are recorded in the appendix so the same false leads are not +re-found — one of them (the `ptest_0762` skip) turned out to be *stale* rather than +masking anything: the bug it cites was fixed in April 2026 (issue #151, commit +`453e5063`). + +## Changes Made + +None — audit only. Proposed changes are classified per finding and are scheduled +for Wave A (deletions & dead code), the bug-fix track (real bugs found behind +TODO/skips), and the dimension-5 swarm review (API-inconsistency evidence). + +## System Architecture + +What this dimension's survey revealed about the affected subsystems, for the +maintainer: + +**Function evaluation (`src/underworld3/function/`)** has two derivative-evaluation +paths: an accurate L2-projection path (`_function.pyx:973`) and a fast Clement/rbf +path (`_clement_to_work_variable`, reached via `rbf=True, force_l2=False`, +`_function.pyx:959-969`). The rbf path computes per-component Clement gradients +correctly and stores them keyed by `(source_var, component)` (lines 832-841), and +each derivative expression carries its correct data-column index +(`diffcls.component = data_loc`, line 845 in the registration block at 843-847) — +but the retrieval loop discards it (LE-01). The evaluation gateway for units +(`functions_unit_system.py`) accepts quantity *values* but not quantity-valued +*coordinates* in list form (LE-08); a whole family of unit-aware-coordinate +features (`coord_units`, `UnitAwareArray` returns) exists only as test-suite +placeholders (LE-11). + +**Diagnostics (`src/underworld3/cython/petsc_maths.pyx`)** — `CellWiseIntegral` +still carries the PR #172 clone-DM pattern that was reverted out of `Integral` +(`88807c26`): a cloned DM with a single fresh P1 field is fed a global vector +packed for the *multi-field* mesh layout, over-counting by ~2× (LE-02; reproduced +live: `CellWiseIntegral(mesh, fn=1.0)` on the unit square sums to 2.0). It is +exported to users at `src/underworld3/maths/__init__.py:56`. + +**Swarm subsystem (`src/underworld3/swarm.py`, `swarms/pic_swarm.py`)** is the +densest cluster of loose ends and feeds directly into the dimension-5 review. +Proxy staleness is handled *lazily by design* — the #216 fix (commit `af537d56`) +deliberately kept invalidation lazy to avoid an O(100 MiB) eager-refresh memory +regression (comment at `swarm.py:2699-2707`), which leaves one documented hole: +solvers that read the proxy's DM directly, without touching `.sym`, consume stale +data (LE-03). The array-view reduction interface diverges from MeshVariable's +contract (scalar vs per-component-tuple returns, LE-07). Four identical +`if sync: pass` placeholders make a threaded-through `sync=` parameter a silent +no-op (LE-14). + +**Solvers (`pyx`)** — the loose ends here are mostly *records*, not bugs: the +`pyx:2545` block is a high-value account of a genuine parallel assembly defect +(≈0.027% natural-BC load under-count at partition seams) and its dead-end +workarounds (LE-05). The remainder is pre-refactor commented-out code (old setup +flow, superseded `clearDS`/`createDS` block whose 3-line rationale — "destroys +field registrations" — is worth keeping, old f0/F1 constructions, debug prints; +LE-13, LE-15). Per the campaign ground rule, only comment-level changes are +proposed for this file. + +**Units test suite** — the largest false-coverage surface. The proposed +`mesh.units` interface was implemented with *different* semantics (model-owned +coordinate units; a conflicting `units=` parameter warns and is ignored, +`discretisation_mesh.py:274, 282-301`), so `test_0620`'s try/except-skip tests +now partly pass vacuously and partly skip — both outcomes test nothing (LE-10). +Sixteen skip markers across seven files guard the never-implemented +`coord_units`/quantity-coordinate features (LE-11); `UnitAwareExpression` tests +outlived the architecture that removed the class (LE-19). Separately, the units +system has one real regression-coverage casualty: all three mesh-variable-ordering +("Batman") regression tests die in `Projection.solve()` because +`UnitAwareDerivativeMatrix` defines no `__mul__`/`__neg__` and sympy's +`DomainMatrix.scalarmul` bypasses its `_sympy_()` protocol — so a DM-state +corruption guard has zero active coverage (LE-06). + +**Time-derivative machinery (`systems/ddt.py`)** exposes a documented constructor +parameter (`preserve_moments`) whose entire implementation is dead behind +`if 0 and ...` (LE-09) — a user setting it gets a silent no-op. + +**Compat shims** — `discretisation/persistence.py` is a pure re-export shim with +three "TODO: Implement" stubs for features that live nowhere else; its only +importer is `__init__.py:209` (LE-12). + +## Findings + +### Verified findings (adversarially checked) + +Ranked most-severe-first. All dispositions respect the campaign ground rules (no +solver numerics, no hard API breaks). + +| ID | Sev | Effort | Location | Category | Summary | +|----|-----|--------|----------|----------|---------| +| LE-01 | high | S | `src/underworld3/function/_function.pyx:845` | todo-real-bug | Dead ternary always selects component 0: rbf-path derivative evaluation of multi-component variables returns the component-0 gradient for every component | +| LE-02 | high | M | `src/underworld3/cython/petsc_maths.pyx:303` | todo-real-bug | `CellWiseIntegral` clone-DM/section layout mismatch over-counts ~2× (reproduced: fn=1.0 on unit square → 2.0); user-facing export | +| LE-03 | high | M | `src/underworld3/swarm.py:1075` | todo-real-bug | Stale proxy DM after swarm write: lazy refresh fires only on `.sym` access; solvers reading the proxy DM directly get stale data (issue #215 Bug 3, deferred by #216) | +| LE-04 | high | L | `tests/parallel/test_1017_custom_mg_parallel_mpi.py:129` | skipped-test-masking-bug | Skip masks issue #291: Stokes_Constrained segfaults at np>1 in the interior-multiplier section reduction, independently of custom-P | +| LE-05 | medium | L | `src/underworld3/cython/petsc_generic_snes_solvers.pyx:2545` | todo-real-bug | Partition-seam natural-BC under-count (~0.027% load, ~0.1% velocity in augmented Stokes_Constrained); TODO(BUG) block is a high-value engineering record | +| LE-06 | medium | M | `tests/test_0813_mesh_variable_ordering_regression.py:32` | skipped-test-masking-bug | Three skips kill ALL "Batman" DM-corruption regression coverage; root cause is `UnitAwareDerivativeMatrix * NegativeOne` TypeError in the Projection residual template (reproduced live) | +| LE-07 | medium | M | `tests/test_0850_comprehensive_reduction_operations.py:32` | skipped-test-masking-bug | Swarm array-view reductions return scalars where MeshVariable returns per-component tuples (reproduced live); 2 skips mask it, 1 skip is stale, 1 is blocked only by the xfailed missing global `std()` | +| LE-08 | medium | M | `tests/test_0812_poisson_with_units.py:25` | skipped-test-masking-bug | Three skips mask a real gap: `evaluate()` at quantity-valued `[(x, y)]` coordinates raises TypeError in `non_dimensionalise` (reproduced live); substantive Poisson-with-units coverage lost | +| LE-09 | medium | S | `src/underworld3/systems/ddt.py:2750` | disabled-logic | Documented `preserve_moments` parameter is a silent no-op — entire implementation dead behind `if 0 and ...` (lines 2750, 2760) plus "TODO: DELETE" access-pattern remnants | +| LE-10 | medium | M | `tests/test_0620_mesh_units_interface.py:64` | skipped-test-obsolete | 11 try/except-skip tests of a PROPOSED mesh-units interface; actual semantics differ (model-owned units) so several now pass vacuously — false coverage either way. `test_0630` (4 decorator skips) is a pure demonstration of the superseded proposal | +| LE-11 | medium | M | `tests/test_0804_backward_compatibility_units.py:17` | skipped-test-obsolete | 16 skip markers across 7 files are placeholders for the never-implemented `coord_units`/`UnitAwareArray`-return/quantity-coordinate feature family | +| LE-12 | medium | S | `src/underworld3/discretisation/persistence.py:51` | stub-verdict | Pure backward-compat re-export shim + three "TODO: Implement" stubs; only importer is `__init__.py:209`; module name promises functionality that does not exist | + +#### LE-01 — rbf-path multi-component derivatives are wrong (`_function.pyx:845`) + +Evidence (read at `1d003481`): + +```python +comp = 0 if source_var.num_components == 1 else 0 # TODO: handle multi-component +grad = gradient_at_nodes[(source_var, comp)] # shape (n_nodes, dim) +``` + +Both ternary branches are `0`. Per-component Clement gradients are correctly +computed and stored keyed `(source_var, c)` (lines 832-841), and each +`UnderworldAppliedFunctionDeriv` carries the correct data-column index +(`diffcls.component = data_loc`, registration block 843-847). Only retrieval is +broken: `v[1].diff(x)` with `rbf=True` returns the component-0 gradient. The L2 +path (`:973`) is unaffected. + +**Fix**: in the loop at 848-854, key by each expression's own component — +`grad = gradient_at_nodes[(source_var, deriv_expr.component)]` — and delete the +ternary. Add a regression test evaluating `v[1].diff(x)` with `rbf=True` on a +vector MeshVariable. rbf-path only; no solver numerics. + +#### LE-02 — CellWiseIntegral ~2× over-count (`petsc_maths.pyx:303`) + +The TODO(BUG) at 303-309 is accurate: lines 310-319 clone `mesh.dm`, attach a +fresh single-P1-field DS, and hand `DMPlexComputeCellwiseIntegralFEM` a global +vector packed for `mesh.dm`'s multi-field layout (`:298`). Reproduced live in the +worktree env: `CellWiseIntegral(mesh, fn=1.0)` on the unit square sums to 2.0 +(control `Integral` = 1.0). The over-count reproduces even with a constant +integrand, so the corruption is the DM/section layout mismatch generally, not +only wrong-offset DOF reads. History: `57ec0176` (PR #172) introduced the pattern, +`88807c26` reverted it from `Integral`, `d65fe9a2` added the TODO + the two xfails +at `tests/test_0501_integrals.py:182,195`. + +**Fix**: rewrite `evaluate()` to integrate against `mesh.dm` + `mesh.dm.getDS()` +(the current `Integral` pattern), then remove the two xfails. Caveat: this +reinherits the issue-#171 linear-time-growth behaviour `Integral` has — acceptable +for correctness parity, note it in the fix. + +#### LE-03 — stale swarm proxy on DM-level access (`swarm.py:1075`) + +`_update()` (1060-1073) only marks `_proxy_stale=True`; `_update_proxy_if_stale()` +is called in production only from `.sym`/`.sym_1d` (1627, 1643; same pattern at +2203). `Mesh.update_lvec()` (`discretisation_mesh.py:2924-2960`) reads `var.vec` +for every mesh variable including swarm proxies, with no staleness check, and no +solver-entry refresh exists. The reproducer tests work around it manually +(`tests/test_0112_swarm_add_particles.py:110, 821`). Issue #215 is **closed** +(by PR #216) with Bug 3 explicitly deferred — cite it as "closed; Bug 3 deferred", +not as an open issue. + +**Fix constraint**: hooking the refresh into every DM-level access is exactly the +eager pattern #216 rejected (O(100 MiB) regression — `swarm.py:2699-2707`, +`tests/test_0006_memory_leak.py`). The maintainer-consistent variant is a single +eager refresh at solve entry (iterate swarm-proxied aux variables before +assembly). Verify against #215's reproducer and the memory-leak test. + +#### LE-04 — skip masks release-relevant #291 segfault (`test_1017...mpi.py:129`) + +The skip (129-135) is legitimate and self-un-blocking ("auto-enables once #291 is +fixed"), but the underlying defect is user-facing breakage of a shipped-on- +`development` solver: Stokes_Constrained segfaults at np=2 with plain GAMG +(canonical `test_1062_constrained_solcx` also segfaults), isolated to +`_constrain_interior_multipliers_in_section` (`pyx:5014` default-on flag, `:6925` +definition, `:7345` call site). Issue #291 is open. + +**Disposition**: keep the skip and wording; keep #291 on the campaign's +release-blocking list; on fix, remove this skip and the sibling serial-only +restriction. + +#### LE-05 — partition-seam natural-BC record (`pyx:2545-2583`) + +Verbatim-verified TODO(BUG) block above the natural-BC registration loop (`:2584`): +interior facets with one local support cell at partition seams lose non-owned +closure DOFs at global assembly (~0.027% load under-count → ~0.1% velocity in +augmented Stokes_Constrained); documents three rejected cheap workarounds and +three fix options. Introduced by `09e8c734` (PR #265). Cross-ref to +`discretisation_mesh.py:758-761` (non-overlapped assembly DM) is real. + +**Disposition**: keep — high-value engineering record. Track fix option (a) +(partition-independent manual boundary-load assembly) as a campaign follow-on. +Note: the `test_0502_boundary_integrals.py` MPI skips and issue #291 are +*adjacent territory*, not evidence of this bug (see refuted-claims appendix). + +#### LE-06 — Batman regression coverage hole (`test_0813_mesh_variable_ordering_regression.py:32,111,179`) + +All three tier_b tests die at `proj.solve()` (the Poisson solve succeeds) with +`TypeError: unsupported operand type(s) for *: UnitAwareDerivativeMatrix and +NegativeOne` — reproduced live by bypassing the skips. Fires in the Projection +residual template `(self.u.sym - self.uw_function) * self.uw_weighting_function` +(`systems/solvers.py:2648`); `UnitAwareDerivativeMatrix` +(`utilities/mathematical_mixin.py:795-1016`) defines no `__mul__`/`__neg__` and +sympy's `DomainMatrix` element-wise path bypasses `_sympy_()`. This is the only +test file covering the "batman" DM-corruption regression — zero active coverage. + +**Fix (two parts)**: (1) implement/route `UnitAwareDerivativeMatrix.__mul__`/ +`__neg__` (units subsystem) — the repair point is the DomainMatrix +scalar-multiplication path; (2) until then, rewrite the three tests *without +units* so the ordering/no-Batman coverage is restored immediately (the units +dependency — `units="metre"` at test setup — is incidental to what they guard). + +#### LE-07 — swarm reduction-interface skips (`test_0850_comprehensive_reduction_operations.py`) + +Measured live: `SimpleSwarmArrayView.max/min/mean/sum/std` (`swarm.py:665-706`) +return scalars for vector variables; `SimpleMeshArrayView` +(`discretisation_mesh_variables.py:2287-2360`) returns per-component tuples. Per- +skip triage: **line 32** and **line 72** mask this real bug (line 72 additionally +needs `vtype=uw.VarType.TENSOR` — requiring explicit vtype for ambiguous component +counts is *shared, deliberate* design, not a SwarmVariable bug: MeshVariable +raises the identical ValueError, `swarm.py:197-200`); **line 335** is stale — its +body passes today as written; **line 268** is blocked only by the legitimately- +xfailed missing global `MeshVariable.std()` (xfails at 163, 294, 313 are correct +unimplemented-feature markers). The file header (13-19) already declares the +implementation wrong. + +**Fix**: align the SwarmVariable array-view reductions to the MeshVariable +per-component-tuple contract, then unskip 32/72; unskip 335 as-is. Note this is a +return-type behaviour change for callers relying on scalar returns from +multi-component swarm reductions. Feed into the dimension-5 swarm review. + +#### LE-08 — quantity-coordinate evaluate gap (`test_0812_poisson_with_units.py:25,106,183`) + +Reproduced live: Poisson with quantity BCs converges, then +`uw.function.evaluate(T.sym, [(x_qty, y_qty)])` raises +`TypeError: Cannot non-dimensionalise object of type ` for both +`pint.Quantity` and `UWQuantity`. Mechanism: `functions_unit_system.py:306-309` +passes the raw list to `uw.non_dimensionalise()`, whose protocol chain +(`units.py:643-878`) has no list/tuple-of-quantity handling. Adjacent xfails: +`test_0750_unit_aware_interface_contract.py:116,195,249`. + +**Fix**: coerce lists/tuples of UWQuantity in the evaluate() gateway, then unskip; +or declare the format unsupported, document it, and rewrite the tests with +supported coordinate forms so the Poisson-with-units coverage is regained. + +#### LE-09 — `preserve_moments` silent no-op (`ddt.py:2750, 2760`) + +Documented at 1390-1391 ("Use moment-preserving projection (experimental)"), +accepted at 1464, stored at 1480; both implementation blocks are behind +`if 0 and self.preserve_moments and ...` — unreachable by short-circuit. Inner +"TODO: DELETE" commented-out `mesh.access` blocks at 2768-2770 and 2777-2781. +No caller in src/tests/docs ever passes `preserve_moments=True`. + +**Fix (Wave A)**: delete both `if 0` blocks and the remnants (git remembers); +make `preserve_moments=True` raise +`NotImplementedError("preserve_moments is not currently implemented")` rather +than removing the parameter (avoids the API break); update the docstring. + +#### LE-10 / LE-11 — aspirational units-test placeholders + +LE-10: `test_0620` (11 skip sites, e.g. `:64`) tests a "PROPOSED" interface whose +real implementation differs — `Mesh.__init__` accepts `units=` +(`discretisation_mesh.py:274`) but the model owns coordinate units and conflicts +warn-and-ignore (`:282-301`). Because `mesh.units` now exists, several tests pass +vacuously instead of skipping — false coverage either way. `test_0630` carries 4 +unconditional decorator skips ("Demonstrates proposed interface - not yet +implemented"). **Fix**: delete `test_0630`; rewrite `test_0620` to assert the +*implemented* semantics (units accepted, model precedence, warning on conflict). + +LE-11: 16 skip markers, all unimplemented-feature placeholders for the same +family: exact reason "coord_units parameter not implemented" ×7 +(`test_0730:91,117`; `test_0803_simple_workflow_demo:21`; +`test_0803_units_workflow_integration:24,262,328`; `test_0804:17`); sibling +"planned feature" reasons in `test_0800:58,111,162` (UnitAwareArray return; +`points_in_domain`/`get_closest_cells` with Pint coordinates) and +`test_0811:35,59,86,111,134,155` (evaluate with quantity-coordinate lists — +overlaps LE-08). `evaluate()` (`functions_unit_system.py:789-805`) has no +`coord_units` kwarg; no design doc commits to one. **Fix**: one decision for the +family — consolidate into a single clearly-labelled aspirational module if the +feature is on the roadmap, else delete all placeholders and record the proposal +in `docs/developer/design/`. + +#### LE-12 — `persistence.py` verdict: delete + +54-line module: re-export of `EnhancedMeshVariable`/`create_enhanced_mesh_variable` +(lines 40-43) + three "TODO: Implement" stubs (51-53); docstring admits the +2025-01-13 move and the misleading name. Only importer: `__init__.py:209`. +Deletion is behaviour-neutral (`EnhancedMeshVariable` is independently exported +via `discretisation/__init__.py:21` and `__init__.py:605`); the Symbol +Disambiguation note is already in +`docs/developer/design/SYMBOL_DISAMBIGUATION_2025-12.md` (line 365). **Fix**: +delete module + import; update the CLAUDE.md Key Files entry; optionally leave a +one-release warn-on-import shim if external imports are a concern. + +### Unverified findings (evidence lines read by the author; not adversarially checked) + +| ID | Sev | Effort | Location | Category | Summary & disposition | +|----|-----|--------|----------|----------|-----------------------| +| LE-13 | medium | M | `pyx:4026` | dead-code | Stale pre-refactor remnants across the solver file: old setup flow (4026-4038), superseded clearDS/createDS block (4052-4066 — keep its 3-line "destroys field registrations" rationale), old f0/F1 constructions (2684-2688, 3624-3631), debug prints (2746-2747), commented `mesh.access` one-liners (3016, 3057, 4044), dead attribute assignments (3216-3227, 2334, 3280). Wave A: delete (comment-only change per pyx ground rule); tier_a green pre/post since a pyx touch forces a rebuild | +| LE-14 | low | S | `src/underworld3/swarm.py:1235` | todo-stale | Four identical `TODO: Add parallel sync logic here if needed` over `if sync: pass` (1235, 1285, 1342, 1391) — the `sync=` argument is a silent no-op; speculative API. Delete the blocks; drop the parameter or document that DMSwarm getField/restoreField handles sync. Feed to dimension 5 | +| LE-15 | low | S | `pyx:2766` | dead-code | Ten-line commented-out sketch of natural-BC flux Jacobians under the intent note at 2764 ("perhaps a different user-interface altogether is required for flux-like bcs"). Keep the one-line intent comment, delete the code block, record the idea in planning if still wanted | +| LE-16 | low | S | `src/underworld3/cython/petsc_discretisation.pyx:248` | dead-code | Content-free `## Todo !` introducing `petsc_dm_get_periodicity` preserved inside a module-level triple-quoted string (250-285) — can never run, no explanatory note. Wave A: delete; planning-file entry if DM periodicity reading is still wanted | +| LE-17 | low | S | `src/underworld3/swarms/pic_swarm.py:326` | dead-code | Commented-out `setLocalSizes` (with 6-line "existing values are wrong" explanation) + lost-particle debug remnant (~387-390). Delete both; condense to one line: do not pre-set local sizes — `insertPointUsingCellDM` sizes the swarm. Fold into the swarm cleanup wave | +| LE-18 | low | S | `tests/test_quantities_simplified.py:21` | skipped-test-obsolete | Module-level skip: imports `underworld3.function.quantities_simplified`, "which does not exist"; has never tested anything. Delete; first port any uncovered cases (multiplication-order, `.data` non-dimensionalisation) to tests against the real `quantities` module | +| LE-19 | low | S | `tests/test_0754_arithmetic_closure_complete.py:166` | skipped-test-obsolete | Skips citing "UnitAwareExpression class not implemented - feature replaced by simplified units architecture" (0754:166,288; 0756:164) test a class the authoritative redesign removed. Delete; re-express still-relevant closure properties against the simplified API (the neighbouring xfails at 0754:109,146 already do and should stay) | +| LE-20 | low | S | `tests/test_0750_meshing_follow_metric.py:269` | skipped-test-obsolete | Three `strict=False` xfails (269, 324, 344) mark capabilities of the pre-merge elliptic-ma MA mover that the development merge deliberately dropped/replaced. **Corrected during this audit** — the briefing's description (list-of-(metric,weight) composition; pointer to `fault_comb_metric`) does not match the file: actual reasons are (269) a #202 field-transfer skip-threshold nudge, a skip-optimisation only; (324) alignment capture now validated via the OT mover (test_0760); (344) boundary slip now via `smooth_mesh_interior(method='mmpde', slip_surfaces=...)` (test_0855). Disposition unchanged in spirit: delete (269 optionally re-tuned) or keep only with reasons as-is — they already point at the surviving coverage | +| LE-21 | low | M | `tests/test_1100_AdvDiffCartesian.py:98` | skipped-test-broken | xfail on the mesh0 param is a broken test, not a product bug: the file's own header says "not a great test" — step-function IC unrepresentable on the FE mesh, tolerance tuned to the legacy RBF-smoothed boundary path; the corrected in-cell/FE routing (more accurate) exceeds the stale atol. Rework with a representable IC + FE-derived tolerance, then drop the xfail; until then the annotation is correct | +| LE-22 | low | S | `src/underworld3/kdtree.py:8` | todo-unimplemented-feature | TODO(CLEANUP) accurate: pykdtree unused (module re-exports the ckdtree/nanoflann backend) yet remains a dependency whose OpenMP runtime causes fatal double-init crashes on macOS beside PETSc/numpy; tracked in planning (Active, 2026-02-13). Wave A: remove pykdtree from pixi.toml/setup deps, keep this module as the import point, delete the TODO | +| LE-23 | low | S | `src/underworld3/checkpoint/disk_snapshot.py:29` | stub-verdict | Verdict on the "empty stub groups": KEEP — they are the documented phased on-disk format (`write_snapshot_skeleton` at 161 stubs groups with `filled_by` attrs at 192; called by `write_snapshot` at 275/292; exercised by test_0010). Only defect: header still says "Phase 1 (this commit)" (line 29) while phases 2/3a/3b are implemented below (`filled_by` set at 356, 375, 423). Update the docstring only | +| LE-24 | low | S | `src/underworld3/meshing/smoothing.py:2033` | comment-block-verdict | Verdict on the 18 comment blocks ≥10 lines (589-4489, incl. the redistribution-regimes note at 2033 and the envelope-ansatz derivation at 4025): ALL are prose documenting physics/design intent — none is commented-out code. Same for ddt.py's large blocks except the LE-09 `if 0` region. Keep all (the campaign's documented-intent carve-out); optionally promote the two longest derivations to docs/ if dimension 4 flags file length | +| LE-25 | low | S | `src/underworld3/discretisation/discretisation_mesh.py:2584` | todo-legitimate-note | Residual marker triage — legitimate, keep: TODO(parallel) at 2584 and `functions_unit_system.py:721/757`; TODO(follow-up) at 2818 (`_pinned_mask` face-label limitation); "for now" heuristics in `units.py:430/1110/1602`, `swarm.py:1109-1148/2306`; feature/deprecation notes in `model.py:404/701/3788/3910`; rename notes (`discretisation_mesh_variables.py:1133/2809`, `discretisation_mesh.py:4409`); minor notes (`adaptivity.py:862`, `geometry_tools.py:5`, `segmented.py:218`, `__init__.py:656`, `_jitextension.py:924`, `solvers.py:3995`). Cosmetic-only: fix the `TODO: DECPRECATE` typo at `pyx:1450` and normalise marker spelling if a style-charter wave lands | +| LE-26 | low | S | `tests/test_1064_constrained_spherical_shell_response.py:227` | skipped-test-masking-bug | Two strict xfails correctly document that LU sub-solves in the constrained velocity\|[p,h] fieldsplit do not reproduce the validated Nitsche/default velocity response — a tracked open solver bug adjacent to the Stokes_Constrained 3D velocity discrepancy, not a test defect. Keep exactly as-is (`strict=True` surfaces a silent fix as XPASS); cross-reference the reason string so both are triaged together | + +## Testing Instructions + +How to validate the eventual fixes, per the campaign gates (tier_a green pre/post +each wave; `tier_a or tier_b` before merge; np2/np4 for swarm/migration touches). + +**Bug fixes behind verified findings** (each unskip is itself the regression test): + +- **LE-01**: new test — vector MeshVariable with known analytic field; assert + `uw.function.evaluate(v[1].diff(x), coords, rbf=True)` matches the analytic + ∂v₁/∂x and *differs* from ∂v₀/∂x. Must fail before the fix, pass after. Also + confirm the L2 path (`rbf=False`) is byte-identical pre/post. +- **LE-02**: `CellWiseIntegral(mesh, fn=1.0)` on the unit square must sum to 1.0; + remove the two xfails at `tests/test_0501_integrals.py:182,195` and confirm they + pass. Compare against `Integral` for a non-trivial multi-field integrand. +- **LE-03**: run issue #215's reproducer (a Projection consuming a swarm proxy + after a swarm write, *without* touching `.sym`); remove the manual + `_update_proxy_if_stale()` workarounds at + `tests/test_0112_swarm_add_particles.py:110,821` and confirm green. Gate on + `tests/test_0006_memory_leak.py` to prove no eager-refresh memory regression. + Run np2/np4 (swarm code). +- **LE-04/#291**: fix is out of this dimension's scope; on fix, remove the skip at + `test_1017...mpi.py:129` and the serial-only restriction, then + `mpirun -n 2 pytest tests/test_1062_constrained_solcx.py` must pass. +- **LE-06**: after implementing `UnitAwareDerivativeMatrix.__mul__`/`__neg__`, + remove the three skips in `test_0813_mesh_variable_ordering_regression.py` — all + three (including the created-BEFORE-solve control) must pass. Interim: the + units-free rewrites must pass immediately. +- **LE-07**: after aligning swarm reductions to the tuple contract, unskip lines + 32/72 (adding `vtype=uw.VarType.TENSOR` at the line-72 test) and unskip 335 + as-is; the xfails at 163/294/313 stay until global `std()` lands. +- **LE-08**: unskip the three tests in `test_0812_poisson_with_units.py`; also run + the adjacent xfails at `test_0750_unit_aware_interface_contract.py:116,195,249` + looking for XPASS. + +**Wave A deletions** (LE-09, LE-12 … LE-20 deletions, LE-13, LE-22): behaviour- +neutral by construction — full `pytest -m "tier_a"` pre/post must be identical; +`pytest -m "tier_a or tier_b"` before merge. For LE-13 (pyx comments) a rebuild is +forced: `./uw build` then tier_a. For LE-09 add one test asserting +`preserve_moments=True` raises NotImplementedError. For LE-12 confirm +`import underworld3` and `from underworld3.discretisation import MeshVariable` +still work and grep shows no remaining `persistence` importer. For LE-22, a full +test run on macOS after removing pykdtree from the dependency set (the crash it +guards against is load-order dependent). + +**Test-suite hygiene** (LE-10/11/18/19/20): deletions/rewrites — run the affected +files plus `pytest -m "tier_a or tier_b"`; for rewritten `test_0620`, assert the +implemented semantics (units accepted, model precedence, `UserWarning` on +conflict — `discretisation_mesh.py:282-301`). + +**Stale-skip removal from the refuted appendix**: un-skip +`tests/parallel/ptest_0762_read_timestep_swarm_routed.py:75` and run at np≥2; the +#151 fix is already covered by +`tests/parallel/test_0790_swarm_write_timestep_mpi.py`. + +## Known Limitations + +- **LE-13 … LE-26 are not adversarially verified.** Every anchor line was read + directly by the author in this worktree, but the surrounding claims (e.g. "no + other caller", history assertions) were not independently re-derived for these + fourteen. One briefing claim in this group was materially wrong and is corrected + in place (LE-20). +- Line numbers are exact at `development` @ `1d003481` only; any wave that lands + earlier shifts them. +- Live reproductions (LE-02, LE-06, LE-07, LE-08) ran in this worktree's own built + environment (site-packages verified against src); they were not repeated on a + second platform or in parallel. +- Skip/xfail counts for the units placeholder family (LE-11) were grep-verified at + this ref but the family boundary is a judgement call — a few "planned feature" + markers elsewhere may belong to it. +- Overlaps deliberately deferred: swarm API inconsistencies (LE-03, LE-07, LE-14, + LE-17) feed dimension 5; file-length/readability observations (LE-24) feed + dimension 4; the BC/API-surface issues are dimension 3's territory. +- GitHub issue states (#151 closed, #215 closed, #291 open) were checked 2026-07-03 + and may move. + +## Refuted claims (do not re-find) + +| Claimed finding | Why it is wrong | +|-----------------|-----------------| +| `tests/parallel/ptest_0762_read_timestep_swarm_routed.py:75` — "skip masks an untracked parallel `Swarm.write_timestep` hang" | The hang is GitHub issue #151, **closed 2026-04-29**, fixed by commit `453e5063` (an ancestor of `1d003481`): both `Swarm.save` (`swarm.py:3768-3804`) and `SwarmVariable.save` (`swarm.py:1861-1892`) carry `BUGFIX(#151)` allgather/global-shape/per-rank-slab fixes for exactly the hypothesised collective-create root cause. Dedicated np≥2 regression coverage exists (`tests/parallel/test_0790_swarm_write_timestep_mpi.py`). The only real defect is that the **skip itself is stale** (authored on a branch parallel to the fix, never re-validated) — remove it and re-run at np≥2; do NOT open a new issue. | +| `tests/test_0502_boundary_integrals.py:141` — "six MPI skips are the same partition-seam assembly family as `pyx:2545`; add an np=2 characterisation test of the known wrong value" | Wrong diagnosis. The underlying bug is an **UnboundLocalError in the `BoxInternalBoundary` constructor**: `boundaries`/`boundary_normals` are bound only inside the `if uw.mpi.rank == 0:` gmsh block (`meshing/cartesian.py:536`, bindings ~547-548 / ~644-648) while `Mesh(...)` at ~896-901 runs on all ranks — rank>0 raises before any mesh exists, exactly as the test file's own comment (115-117) says. The `pyx:2545` bug concerns natural-BC *residual* scatter and its comment notes the pure integral is machine-precision identical in parallel. The skips also do not remove all parallel internal-boundary coverage (Annulus/spherical internal tests at 240-293, 325 carry no MPI skip; test_0502 is not in the parallel CI suite — `scripts/test_levels.sh:179,190`, `scripts/test.sh:99`). A "known wrong value" characterisation test is impossible: at np=2 the constructor raises. The real fix is a one-liner mesh-construction bug (broadcast/bind the Enum on all ranks), unrelated to `pyx:2545`. | + +## Sign-Off +| Role | Name | Date | Status | +|------|------|------|--------| +| Maintainer | Louis Moresi | 2026-07-05 | Pending review | +| Author | Claude (audit session) | 2026-07-03 | Complete | + + + +- **Author**: loose-ends audit (dimension 1), 2026-07-03, on the audit worktree + at `development` @ `1d003481`. +- **Verification**: LE-01 … LE-12 adversarially verified (independent re-read + + live reproduction where noted); LE-13 … LE-26 author-read anchors only; two + claims refuted and recorded above. +- **Maintainer sign-off**: pending (L. Moresi) — required before Wave A deletions + execute and before LE-07's return-type behaviour change is scheduled. \ No newline at end of file diff --git a/docs/reviews/2026-07/READABILITY-REVIEW.md b/docs/reviews/2026-07/READABILITY-REVIEW.md new file mode 100644 index 000000000..dfd5f19c3 --- /dev/null +++ b/docs/reviews/2026-07/READABILITY-REVIEW.md @@ -0,0 +1,471 @@ +# Readability Review — July 2026 Quality Campaign (Dimension 4) + +**Base**: `development` @ `1d003481` (audit worktree `.claude/worktrees/quality-audit-2026-07`) +**Date**: 2026-07-03 +**Scope**: readability of the June-2026 change hotspots — `meshing/smoothing.py`, +`cython/petsc_generic_snes_solvers.pyx`, `discretisation/discretisation_mesh.py`, +`systems/ddt.py`, `systems/solvers.py`, `utilities/rotated_bc.py`, +`utilities/custom_mg.py`, `discretisation/remesh.py`. +**Standard applied**: the founding rule — *anyone should be able to read the code and +understand it* — assessed for a working geodynamicist reading each file cold. + +--- + +## Overview + +June 2026 added ~132 commits / +12.6k lines to `src/`, largely AI-assisted across many +non-overlapping sessions. This audit read the eight hotspot files and produced +**118 findings**: 14 adversarially verified (every `file:line` re-read in the worktree, +several with corrections recorded below) and 104 unverified observations queued for +verification before remediation. No refuted findings survived to the tables; nine +refuted *sub-claims* are recorded in the appendix so they are not re-found. + +The consistent picture: **micro-readability is good to excellent, macro-structure is +poor**. Individual June additions carry intent-stating comments a geodynamicist can +follow (Nitsche local-h, FMG sidecar splice, rotated-BC lifecycle notes, DDt class +docstrings, remesh REMAP/CARRY/REINIT narrative). But each session added its feature by +copy-paste rather than extraction, so the same machinery now exists in 2–6 diverging +copies; dead code and stale docstrings from superseded designs were left in place; and +two files (`smoothing.py` at 4,518 lines, `petsc_generic_snes_solvers.pyx` at 8,150 +lines) have grown past the point where any reader can hold them. + +Three findings are severe enough to flag individually: + +1. **READ-01** — `_winslow_mmpde`'s 3D branch references an undefined function + (`_signed_volumes`, `smoothing.py:3124`); the docstring's "Dimension-general + (d = 2, 3)" claim is false and any 3D MMPDE call raises `NameError`. +2. **READ-02** — `_winslow_anisotropic` contains a ~100-line inline duplicate + (`smoothing.py:2152–2252`) of its own `_build_M_tensor()` closure (2060–2150), + diverged only by `_`-suffixed variable names. +3. **READ-03** — the four solver classes in `petsc_generic_snes_solvers.pyx` + quadruplicate their setup/BC/solve machinery, and the copies have **silently + diverged in BC-label semantics** (2-vs-2: `str(boundary)` vs `"UW_Boundaries"`), + so any shared-helper extraction must preserve the divergence deliberately. + +--- + +## Changes Made + +None. This is an audit-only deliverable; no source files were modified. This document +is the only artifact. + +--- + +## System Architecture + +What this dimension's reading revealed, per subsystem, for the maintainer. + +### The mesh-mover module (`meshing/smoothing.py`, 4,518 lines) — *not graded; worst structural debt of the set* + +The file is an accumulation of graft events, and its own banners say so +(`# ===== grafted from feature/elliptic-ma =====`, line 2930). It contains **six +movers**, five carrying a `_winslow_` prefix that is wrong for four of them: +`_winslow_spring` (506) is a truss-energy nonlinear-CG spring solver, +`_winslow_elliptic` (1162) is a Benamou–Froese–Oberman Monge–Ampère Picard, +`_winslow_equidistribute` (1453) is an OT-improvement step, `_winslow_mmpde` (2992) is +Huang–Kamenski MMPDE; only `_winslow_anisotropic` (1718) is genuinely Winslow. The +sixth mover (graph-Laplacian Jacobi, `metric=None`) is inline in the dispatch after +line 3657, unprefixed. The public API strings are already honest (`"spring"`, `"ma"`, +`"ot"`, `"anisotropic"`, `"mmpde"`) — the lie is internal-only, which makes the rename +cheap. + +The module docstring (lines 1–59) describes a superseded design: it calls the spring +path Jacobi relaxation, "Status: under development", weak grading 1.03 (lines 33–37) — +directly contradicted by `_winslow_spring`'s own docstring, which explains it replaced +Jacobi sweeps *because* they stalled. The docstring never mentions `mmpde`, yet the +file's own `DeprecationWarning` (3627–3633) tells users to "Prefer 'mmpde' for +production adaptive meshing". Note the API **default** is still `method="spring"` +(2623, 3525) and in-tree callers use `"ma"`/`"ot"` — mmpde is *recommended*, not +default. + +Duplication is the dominant failure mode: ~130 lines of verbatim ≥8-line blocks +(~260 counting both copies) plus substantial near-duplication — the signed-area +backtrack exists three times (only one copy got the sliver-floor improvement), the +per-vertex step cap twice, the `_build_M_tensor` body twice (READ-02), the +displacement reweighting twice, mean-edge-length twice, and ~19 inline +`from mpi4py import MPI as _MPI` imports guard 43 allreduce sites. Eight module-level +caches coexist. The long-term fix is the package split (READ-05), but note that +private names (`_edge_pairs`, `_tri_cells`, `_pinned_mask`, …) are imported by +production code in `meshing/surfaces.py`, `meshing/_ot_adapt.py`, +`systems/solvers.py`, `discretisation/discretisation_mesh.py` and by tests +0750/0760/0762/0763/0855 — a split must re-export those, not just the public API. + +One finding here crosses from readability into defect: the 3D MMPDE branch is +un-executable (READ-01). + +### The solver core (`cython/petsc_generic_snes_solvers.pyx`, 8,150 lines) — *not graded; under the no-numerics constraint* + +Four classes (`SNES_Scalar`, `SNES_Vector`, `SNES_MultiComponent`, +`SNES_Stokes_SaddlePt`) each carry a private copy of: the xxhash +coordinate-change preamble, the `PetscDSAddBoundary_UW` BC-registration loops, the +`_setup_solver` copyFields/copyDS/createClosureIndex/SNES tail, and the solve() +epilogue (whose identical "sync `_gvec` / invalidate cached views" stanza proves bug +fixes are already being applied four times). Critically, the copies are **not** +verbatim: essential BCs register under `str(boundary)` in Scalar (2648) and SaddlePt +(6895) but under `"UW_Boundaries"` in Vector (3570, comment `# was: str(boundary)`) and +MultiComponent (4402), and SaddlePt's versions interleave multiplier-constraint and +fieldsplit logic. Any helper extraction (READ-03) must parameterize this 2-vs-2 +divergence or it will silently change BC behaviour — hence the campaign rule that this +file gets naming/docs/dead-code changes only, with structural extraction in a +separately benchmarked wave. + +Beyond duplication, the file reads as a fossil record: `from xmlrpc.client import +Boolean` is line 1; four error paths `raise("string")` (a `TypeError` at runtime, not +the intended message — 1383/1386/1466/2022); two `if True: # c_label and label_val +!= -1:` guards wrap entire BC-wiring blocks (3884, 7140); 40-line commented-out +rebuild paths and unresolved design musings ("LM: this is probably not something we +need…") interrupt the narrative; and the hedging name `_maybe_install_snes_update` +(325) violates the maintainer's twice-stated naming rule. + +### The mesh god-module (`discretisation/discretisation_mesh.py`, 6,616 lines) — **grade C+** + +Individual June methods (boundary_normal, cell_size, boundary_slip, the FMG sidecar +splice) are excellently narrated, but `__init__` alone spans ~830 lines mixing eight +separable concerns, and the load-bearing coordinate-update callback exists in **three +silently diverged copies** (834 in `__init__` with the identity gate and teardown +guard; 2027 without the identity gate; 6086 in `adapt()` without either). Four dead +methods, a dead-and-broken `meshVariable_mask_from_label` (bare `MeshVariable` +NameError), `if False:` chains, and a duplicated boundary-label extraction where one +copy uses the exact pattern the other copy's comment warns is a hard-abort hazard. + +### Time-derivative machinery (`systems/ddt.py`, 3,676 lines) — **grade C+** + +Class-level docstrings are the best in the audit (BDF/AM pairing, SLCN vs SL-BDF2 +with citations), but `SemiLagrangian.update_pre_solve` is a 515-line method +interleaving six concerns, the UnitAwareArray→ND unwrap dance is copy-pasted six times +with accidental branch-order differences, ~250 lines of per-class boilerplate are +quintuplicated across the five DDt flavors, `preserve_moments` is documented but dead +(`if 0 and …`), and two `_object_viewer`s reference attributes that don't exist +(AttributeError on view). + +### Solver front-end (`systems/solvers.py`, 5,067 lines) — **grade B** + +Excellent physics narrative throughout; graded down for triplicated +`smoothing_length` properties, duplicated 55-line `delta_t` setters opening with bare +`except: pass`, four copies of the diffusivity-max block in `estimate_dt`, +`_apply_unit_aware_scaling` swallowing all exceptions around two redundant branches, +another banned `_maybe_` name (2264), and +`SNES_Vector_Projection.projection_problem_description` double-counting smoothing and +penalty terms its own F1 Template already contains (2985–2991) — a reader cannot tell +what weak form is assembled. + +### Rotated free-slip (`utilities/rotated_bc.py`, 795 lines) — **grade B** + +Genuinely well-narrated (rotate→constrain→solve→rotate-back→gauge→σ_nn is followable), +marked down for compression style: the module docstring still calls itself a +"Development version … productizes the validated prototypes" (it *is* the module), +`_zero_rows_local` exists precisely to encapsulate the np>1 overflow subtlety yet its +body is hand-inlined three more times, and pervasive semicolon-chained +create/use/destroy lines obscure the PETSc object lifecycle the file's own comments +call the trickiest part. + +### Custom multigrid (`utilities/custom_mg.py`, 653 lines) — **grade A-** + +Exemplary narrative: design rationale in the module docstring, the BC-per-level +invariant explicitly marked load-bearing, every PETSc quirk explained. Remaining debt +is small: mode-dependent `maps` element types read via magic index `lay[3]` (557–559), +duplicated DM-prep and zero-column guards, semicolon packing, terse parallel-path +parameter names. + +### Remesh transfer (`discretisation/remesh.py`, 494 lines) — **grade B** + +REMAP/CARRY/REINIT intent and the sagitta/parallel-leak mechanism are clearly stated; +loses marks for two dead functions (`_remap_one_var`, `_new_coord_cache`), the +`REMESH_MONOTONE` env knob buried inside a per-variable loop inside a try block, and +a thrice-repeated guarded write-back `except Exception: pass` whose sanctioned failure +mode is stated only once. + +### Cross-cutting patterns (for the Style Charter follow-on) + +- **Copy-paste is the reuse mechanism**: every multi-session file shows 2–6 copies of + shared machinery, several already diverged. Wave D should prioritize extraction + where divergence is accidental and *document* it where deliberate. +- **Silent `except Exception: pass`** appears in every file, almost never stating the + sanctioned failure mode. The remesh/model-registration blocks that *do* document + their swallow are the template to mandate. +- **Stale self-description**: docstrings and comments describing deleted designs + (smoothing module docstring, rotated_bc "prototype" framing, ddt theta comment, + drifted line-number references). Mechanical CI gates can't catch these; review can. +- **Hedging names** (`_maybe_*`) re-appeared twice in June despite the standing rule — + a lint-able pattern. + +--- + +## Findings + +Ranked most-severe-first. **Wave** column maps to the campaign's remediation waves +(A = deletion/dead code, D = readability rewrite, D-doc = comment/docstring-only). +`S/M/L` = effort. + +### Verified findings (adversarially verified; every line re-read in the audit worktree) + +| ID | Sev | Eff | Wave | Location | Finding | Fix | +|----|-----|-----|------|----------|---------|-----| +| READ-01 | High | S | A | `meshing/smoothing.py:3124` | `_winslow_mmpde` docstring claims "Dimension-general (d = 2, 3)" but the cdim==3 branch assigns `signed_vol = _signed_volumes`, which is **defined nowhere** in the package (src-wide grep) — any 3D MMPDE call raises `NameError`; the 3D branch is dead code contradicting the docstring. | Implement `_signed_volumes(coords, tets)` (tet analogue of `_signed_areas`; `_tet_cells` at 2931 exists to feed it) **or** raise `NotImplementedError` for cdim==3 and make the docstring 2D-only; add a level_1 test for the chosen behaviour. | +| READ-02 | High | M | D | `meshing/smoothing.py:2152–2252` | `_winslow_anisotropic` inlines a ~100-line density/eigen-clamped tensor-assembly block that is line-for-line identical (modulo `_`-suffixed names) to the density/assembly portion of its own `_build_M_tensor()` closure at 2080–2150; the closure additionally refreshes `Dcoords`/`gproj` (2067–2078), so calling it once pre-loop repeats a deterministic re-solve and makes the pre-loop `gvec/gn/gmax` at 1996–2019 redundant (`old0`, `h0` must stay). | Delete the inline block; call `_build_M_tensor()` once before the outer loop (it already reads/writes the same state via `nonlocal`); remove the now-redundant pre-loop `gvec/gn/gmax` computation. | +| READ-03 | High | L | D (benchmarked wave) | `cython/petsc_generic_snes_solvers.pyx:2473ff` | Four-way duplication across `SNES_Scalar`/`Vector`/`MultiComponent`/`Stokes_SaddlePt`: xxhash coord preamble (2479, 3456, 4299, 6697), BC-registration loops (2584, 3507, 4346, 6785), `_setup_solver` tail (2892, 3943, 4721, 7326), solve epilogue (3054, 4088, 4791, 8072) — the identical "sync `_gvec`/invalidate cached views" stanza in all four epilogues proves fixes are applied ×4. **Caution**: copies are NOT verbatim — essential-BC labels diverge 2-vs-2 (`str(boundary)` at 2648/6895 vs `"UW_Boundaries"` at 3570/4402), and SaddlePt interleaves multiplier/fieldsplit logic. | Extract base-class helpers (`_coords_unchanged_since_setup`, `_register_essential_bcs`/`_register_natural_bcs`, `_finalise_dm_and_snes`, `_copy_solution_to_fields`) that **parameterize or deliberately preserve** the label divergence and give SaddlePt hook points. No-behavior-change, benchmarked wave only. | +| READ-04 | Med | L | D | `meshing/smoothing.py:794` (whole file) | 4,518-line graft-accumulation module: 6 movers, MA solver wiring, Hessian recovery, metric builders, adapters, 8 module-level caches, explicit `grafted from feature/elliptic-ma` banner (2930). | Split into a `meshing/smoothing/` package (`graph.py`, `spring.py`, `monge_ampere.py`, `mmpde.py`, `metrics.py`, `api.py`). **Must also re-export the private names** imported cross-module and by tests (`_edge_pairs`, `_tri_cells`/`_tet_cells`, `_pinned_mask`, `_auto_pinned_labels`, `_owned_vertex_mask`, `_signed_areas`, …) and keep the smoothing↔`_ot_adapt` mutual imports lazy. | +| READ-05 | Med | M | D | `meshing/smoothing.py:2403` | The coherent global signed-area backtrack exists ×3 (1412–1437, 1665–1691, 2403–2444); only the anisotropic copy got the sliver-floor improvement (`a_min_floor = 0.01·median`, 2422/2435) — the other two remain sliver-blind (`a1min > 0.0`). Per-vertex step cap duplicated at 1403–1410 / 1656–1663. **Do not** fold in mmpde's structurally different cap (3424–3430) or the spring/mmpde line searches. | Extract `_backtracked_move(..., area_floor=0.0)` and `_cap_step_to_edge_fraction(...)`; `area_floor=0.0` default keeps the two older movers bit-identical. | +| READ-06 | Med | M | D | `meshing/smoothing.py:506` | Misleading `_winslow_` prefix on 4 of 5 prefixed movers: spring equilibrium (506), Monge–Ampère (1162), OT step (1453), MMPDE (2992); only `_winslow_anisotropic` (1718) is Winslow. Public method strings are already honest, so the lie is internal — but the blast radius includes docstring references in `remesh.py:229`, `_ot_adapt.py:8`, `smoothing.py:805`, 37 files under `scripts/`, and design docs; `_WINSLOW_CACHE` shares the prefix. | Rename to `_spring_equilibrium_mover`, `_monge_ampere_mover`, `_ot_improvement_step`, `_mmpde_mover` (keep `_winslow_anisotropic`); sweep scripts/docs or keep aliases for one cycle. | +| READ-07 | Med | M | D | `meshing/smoothing.py:1362` | ~19 inline `from mpi4py import MPI as _MPI` imports (e.g. 1361–1364, 2075–2078) guarding 43 allreduce sites and 29 `if uw.mpi.size > 1:` blocks; no module-top import despite mpi4py being a hard dependency. | Import `_MPI` at module top; add `_global_min/_global_max/_global_sum` (and `_global_mean` for the `allreduce/size` sites at 1612/2014/3238) that no-op in serial; each mover shrinks ~30 lines. | +| READ-08 | Med | S | D-doc | `meshing/smoothing.py:1–59` | Module docstring describes the deleted design: spring = Jacobi relaxation "Status: under development", weak grading 1.03, "can stall against the tangle guard" (33–37) — contradicted by `_winslow_spring`'s own docstring (nonlinear CG adopted *because* Jacobi stalled). `mmpde` never mentioned though the file's own DeprecationWarning (3627–3633) recommends it for production. Corrections recorded: 5 (not 6) movers carry the prefix; mmpde is *recommended*, not the default (API default is `spring`, 2623/3525); duplication is ~130–260 lines, not ~300. | Rewrite: one paragraph per mover (what it IS, status, when to use); delete stale spring status; state mmpde is recommended-for-production; move the 2026-05-16 MA note to the section banner at 794 where it is already duplicated. | +| READ-09 | Med | S | D-doc | `meshing/smoothing.py:3931–3932` | Dead amp-inversion: `amp = refinement**(cdim/power) − 1` is never used — every `refinement is not None` path enters the envelope branch (4063) which exhaustively returns/raises before `amp`'s only consumer (4155). The `refinement` param docstring (3827–3840) and comment (3925–3930) still document the superseded override semantics; actual envelope behaviour lives only in an inline comment (4025–4062). `'arc-length'` is accepted (4124–4133) but missing from the option lists (4045–4053, 4294). | Delete 3925–3932; rewrite the `refinement` doc to describe the envelope branch and which params it ignores; add `'arc-length'` to both option lists. | +| READ-10 | Med | S | D-doc | `meshing/smoothing.py:2693, 3646–3654` | `smooth_mesh_interior` docstring says `method : {"spring", "ma"}`; the ValueError (3646–3654) lists spring/ma/ot/anisotropic but **not** `mmpde` — the mover the same function's DeprecationWarning recommends. | Docstring → `{"spring", "ma", "anisotropic", "mmpde"}` with mmpde described as recommended production mover and `'ot'` noted deprecated-only; add mmpde to the ValueError text. | +| READ-11 | Med | S | D | `meshing/smoothing.py:2999` | `_winslow_mmpde(..., **_ignored)` silently swallows typo'd `method_kwargs` (no other mover has a catch-all). **But** it is not a pure hedge: the strategy path legitimately forwards `resolution_ratio` (injected 3565–3566, consumed pre-dispatch 3581–3583, forwarded via `**mk` 3643–3644) — bare removal breaks `strategy=` + mmpde with TypeError. | Accept `resolution_ratio=None` explicitly (or pop it in dispatch), then remove `**_ignored` and warn on any remaining unknown kwargs. | +| READ-12 | Med | S | D-doc | `meshing/smoothing.py:408` | `mesh_metric_mismatch` docstring (380, 408–411) documents a 3-key return dict; the actual return (476–480) has 5 keys — and the undocumented `misalignment` key is exactly what the `skip_threshold` machinery consumes (3595). Precision: misalignment = `sqrt(1 − max(0, r)²)` (negative correlation clamps to 0 → misalignment 1.0). | Document the full 5-key dict, one line each for `alignment` (Pearson r, globally reduced) and `misalignment` (the skip criterion, with the clamp). | +| READ-13 | Med | S | D | `cython/petsc_generic_snes_solvers.pyx:325` | Hedging name `_maybe_install_snes_update` (introduced 54b815c3, 2026-06-18) violates the twice-enforced no-`maybe_`/`try_` naming rule; docstring already states the condition ("iff callbacks are registered"). One call site (1095), no external references. | Rename to `_attach_snes_update_dispatcher` (or similar does-what-it-says name); update the call site. | +| READ-14 | Med | S | A | `cython/petsc_generic_snes_solvers.pyx:3884, 7140` | `if True: # c_label and label_val != -1:` wraps the entire natural-BC DS-wiring block in SNES_Vector and SNES_Stokes_SaddlePt — a constant-literal guard adding a spurious indent level and implying a condition that no longer exists (no else/elif; Cython constant-folds it). | Delete both lines and dedent — byte-identical semantics. | + +### Unverified findings (anchors sampled, not adversarially verified — verify before remediation) + +I personally re-read the anchor lines for a sample of these in the worktree +(READ-15, READ-24, READ-27/28/33/35/37/39/41/43/47/49/50/52/54/56/58/60/62/64/66/89/97, +and the import blocks) and all sampled anchors match; the full adversarial pass +(callers, reachability, already-fixed-at-HEAD checks) has **not** been run on this +table. + +#### High severity + +| ID | Sev | Eff | Wave | Location | Finding | Fix | +|----|-----|-----|------|----------|---------|-----| +| READ-15 | High | L | D | `discretisation/discretisation_mesh.py:255` | FILE GRADE C+. `Mesh.__init__` (255–1084, ~830 lines) mixes unit-scale derivation, file-format dispatch, boundary-enum patching, FMG sidecar splice, refinement/coarsening, coordinate-callback wiring, and sympy setup in one flat block. | Extract 8 named private methods called in sequence (`_derive_length_scale_from_model`, `_load_dm_from_file`, `_patch_boundary_enum`, `_splice_hierarchy_from_sidecar`, `_build_refined_hierarchy`, `_build_coarsened_hierarchy`, `_install_coordinate_array`, `_setup_symbolic_coordinates`); pure code motion. | +| READ-16 | High | M | D | `discretisation/discretisation_mesh.py:834` | The `mesh_update_callback` closure exists ×3 (834, 2027, 6086) and has silently diverged: only the `__init__` copy has both the teardown guard and the load-bearing `array is not mesh._coords` identity gate; `_re_extract_from_parent` drops the gate; `adapt` drops both. | One module-level `_mesh_coords_update_callback` (the `__init__` version, comments included) + a `Mesh._install_coords_array` helper; call from all three sites. | +| READ-17 | High | L | D | `systems/ddt.py:2286` | FILE GRADE C+. `SemiLagrangian.update_pre_solve` (2286–2801, 515 lines) interleaves ≥6 concerns; the RK2 midpoint characteristic trace is buried under unit bookkeeping. | Extract intention-named helpers (`_shift_history_with_blend`, `_record_current_field_into_history`, `_trace_departure_points`, `_sample_history_at_departure`, …) so the method reads shift → record → trace → sample. | +| READ-18 | High | M | D | `systems/ddt.py:2593` | The UnitAwareArray/`.magnitude`/non-dimensionalise unwrap block is copy-pasted ≥6× (2069, 2400, 2593, 2657, 2904, 2916) with slightly different branch ordering that is not intentional. | One `_to_nondim_ndarray(value, ...)` helper next to `_as_float`, docstring stating the ND-space invariant (issue #267); replace all six sites. | +| READ-19 | High | M | D | `systems/ddt.py:2630` | Node-velocity (2560–2613) and midpoint-velocity (2630–2675) blocks in the trace-back loop are near-identical (evaluate vs global_evaluate + coord array) — DRY violation on the hottest path; unit-handling fixes must be applied twice. | Extract `_velocity_nd_at(coords, use_global, subtract_v_mesh)`; combines with READ-18. | +| READ-20 | High | L | D | `systems/ddt.py:3453` | ~250 lines of verbatim per-class boilerplate quintuplicated across the five DDt flavors: model-registration try/except (incl. the identical 7-line "#195" comment ×5), coefficient init, `effective_order`, `bdf_coefficients`, `bdf()`, `adams_moulton_flux()`, state-setter validation. | Shared `_DDtBase` (or module helpers as step one); each flavor keeps only its storage. | + +#### Medium severity + +| ID | Sev | Eff | Wave | Location | Finding | Fix | +|----|-----|-----|------|----------|---------|-----| +| READ-21 | Med | S | A | `…snes_solvers.pyx:1383` | Four error paths `raise("message")` (1383, 1386, 1466, 2022) → runtime `TypeError: exceptions must derive from BaseException` instead of the diagnostic. | `raise ValueError(...)`/`TypeError(...)` keeping the message text (error-path only). | +| READ-22 | Med | S | D | `…snes_solvers.pyx:2278` | `SNES_Scalar.__init__` creates a default var then unconditionally `self.Unknowns.u = u_Field` (possibly None) — works only because the setter silently ignores None; SNES_Vector uses truthiness `if not u_Field:`; the default scalar var uses `num_components=mesh.dim`. | Explicit if/else in both constructors; `num_components=1` (or comment if load-bearing). | +| READ-23 | Med | M | A | `…snes_solvers.pyx:5620` | Stokes `strategy` setter: 'robust'/'fast' branches are `pass` "(Reserved)"; remainder re-duplicates the `__init__` option bundle with one silent divergence (`pc_mg_type` kaskade vs additive); `is_setup = False` commented out so post-setup calls silently no-op. | Delete empty branches (ValueError on unknown); extract shared option bundle; comment on the kaskade/additive difference — do not change values without benchmarking. | +| READ-24 | Med | S | A | `…snes_solvers.pyx:4026` | Fossilised commented-out blocks: 40-line dead rebuild path (4026–4066), whole `add_essential_p_bc` (5148–5168), dead flux-BC block (2766–2778), stale alternates (2684, 3624, 5119, 6355, 6470). | Delete (git keeps them); keep 2–3-line prose notes for documented pitfalls (e.g. the clearDS/createDS note). | +| READ-25 | Med | M | D | `…snes_solvers.pyx:4500` | Explicit-index Jacobian construction (G0–G3 flat-layout loops) duplicated between SNES_Vector (3684–3714, BC 3746–3790) and SNES_MultiComponent (4500–4531, BC 4556–4596); the valuable layout commentary exists only in the MultiComponent copy. | Extract `_petsc_pointwise_jacobians` / `_petsc_bd_jacobians` module helpers carrying the comment once. | +| READ-26 | Med | M | D | `…snes_solvers.pyx:3888` | Bd-residual/jacobian registration written as three parallel `_has_f1` if/else pairs, duplicated Vector (3888–3931) vs MultiComponent (4664–4710), ~120 lines varying only in pointer-vs-NULL slots. | Compute optional pointers once (small cdef helper) or extract one shared `_wire_bd_terms`. | +| READ-27 | Med | M | D | `…snes_solvers.pyx:7659` | `compute_volume_residual_fields` / `compute_boundary_residual_fields` duplicate ~70 lines each (preamble 7469–7498 vs 7597–7636; per-field IS copy-out 7534–7569 vs 7659–7694); `_assemble_volume_reaction` (2086–2163) is a third overlapping variant. | Extract `_gather_state_for_residual` and `_split_local_residual_by_field`; the public methods become ~20-line wrappers. | +| READ-28 | Med | S | D-doc | `…snes_solvers.pyx:4617` | `SNES_MultiComponent._setup_pointwise_functions` omits `self._current_jit_cache_key = …` (its three siblings set it at 2815/3826/6682), so `_build`'s constants-only fast path (1175) can never fire for MultiComponent; nothing says whether that is intentional. | Document-first: comment stating the fast path is disabled here and why; align with siblings in the benchmarked wave. | +| READ-29 | Med | S | D | `…snes_solvers.pyx:8025` | Stokes solve: picard and else branches end with a byte-identical 12-line tail (8026–8039 vs 8043–8056); the `# Now go back to the original plan` comment at 8025 sits at the wrong indent, misleading about control flow. | Dedent the common tail to run once after the optional Picard warmup (verbatim code motion). | +| READ-30 | Med | S | D-doc | `…snes_solvers.pyx:7973` | Magic `snes_max_it = 50` silently pushed to petsc_options each solve (8029/8046), clobbering user settings; neighbouring options are read from current state, this one hardcoded, uncommented. | Comment the intent now; reading the current option is a behavior change → queue for benchmarked wave. | +| READ-31 | Med | S | A | `…discretisation_mesh.py:4492` | Four dead methods, zero callers: `_build_kd_tree_index_DS` (4492), `_build_kd_tree_index_PIC` (4606), `get_min_radius_old` (5584), `_get_mesh_centroids` (5544, self-labelled deprecated). | Delete; note in historical-notes.md if archaeologically valuable. | +| READ-32 | Med | S | A | `…discretisation_mesh.py:5699` | `meshVariable_mask_from_label` is dead AND broken: bare `MeshVariable` (5703) never imported → NameError on any call; no callers repo-wide. | Delete (or fix + test if the capability is wanted). | +| READ-33 | Med | S | A | `…discretisation_mesh.py:4548` | ~30-line commented-out face-normal block (4548–4578) duplicating `_mark_faces_inside_and_out` logic, plus commented `build_index()` scratch at 4510/4592/4645/4652 — the dead block is longer than the live loop. | Delete; one line pointing at git history if needed. | +| READ-34 | Med | S | A | `…discretisation_mesh.py:1063` | Dead `if False:`/`elif False:` chain (1063–1078) for removed native coordinate-system calculus; commented scratch at 527–535, 695, 711. | Replace with the single live line + one note; delete scratch. | +| READ-35 | Med | S | D | `…discretisation_mesh.py:316` | Length-scale derivation duplicates an identical 12-line try/except for `domain_depth` and `length` (only the key differs), both with bare `except:` silently swallowing `to_base_units()` failures. | Collapse to a loop over the two keys; narrow the except; comment why fallback-not-raise. | +| READ-36 | Med | M | D | `…discretisation_mesh.py:1668` | Surviving-boundary-label enum construction duplicated between `extract_region` (1668–1689) and `extract_surface` (1836–1866) with **different safety idioms** — extract_region uses the direct `getStratumIS` pattern the newer copy's comment warns can hard-abort on submesh DMs. | Extract `_surviving_labels(...)` implementing the safe getValueIS-first idiom; use in both. | +| READ-37 | Med | S | D | `…discretisation_mesh.py:1888` | `extract_surface` inlines a KDTree vertex map (1893–1901) with a stale comment claiming `_build_vertex_map` is broken (issue #197) — the method directly below (1908–1938) was already fixed (071c5636) with the same code. | Delete the inline block + stale comment; call `_build_vertex_map()` as extract_region does (verify tuple ordering). | +| READ-38 | Med | M | D | `…discretisation_mesh.py:2052` | `_re_extract_from_parent` duplicates `adapt()`'s variable teardown/reinit/transfer machinery (2052–2107 vs 6107–6174) with small differences (IDW vs `uw.function.evaluate`) a reader cannot tell are intentional. | Extract shared teardown/reinit/invalidate helpers; keep the two transfer strategies as small named functions so the difference is visible. | +| READ-39 | Med | M | D | `…discretisation_mesh.py:4734` | Four-way facet outward-normal dispatch duplicated with near-identical code+comments in `_mark_faces_inside_and_out` (4734–4766) and `_mark_local_boundary_faces_inside_and_out` (5015–5047). | Extract `_facet_outward_unit_normal(...)` carrying the dimension-case comments once. | +| READ-40 | Med | M | D | `…discretisation_mesh.py:1445` | `view()` level 0 computes `gather_data` three times and never uses the result (collective calls, dead results; 1449 gathers a loop-leftover `i`); variable/boundary tables triplicated across view level 0/level 1/view_parallel. | Delete dead gathers; extract `_print_variable_table`/`_print_boundary_table`; `uw.pprint` at 1469. | +| READ-41 | Med | S | D | `…discretisation_mesh.py:537` | `all_edges_IS_dm` assigned only inside `if all_edges_label_dm:` yet referenced after (NameError if no 'depth' label); name says "edges" but `getStratumIS(0)` fetches depth-0 **vertices**. | Single guarded block; rename to `vertex_stratum_is`; one intent comment (Null_Boundary = every vertex, value 666). | +| READ-42 | Med | S | D-doc | `…discretisation_mesh.py:3247` | `_legacy_access` docstring example is UW2 copy-paste that cannot run (`FeMesh_Cartesian`, `with someMesh._deform_mesh():` as a context manager — which the file spends 100 lines saying you must not do). | Replace with a real UW3 snippet or delete the example. | +| READ-43 | Med | S | D-doc | `…discretisation_mesh.py:3689` | Deprecated `points` setter ends with `self._coords = model_coords` — rebinding to a plain ndarray, silently discarding the NDArray_With_Callback wrapper: no deform callback, PETSc coords never updated; contradicts its docstring. | `# TODO(BUG)` per project convention (working path is `self._coords[...] = …`); follow-up fixes or removes the setter. | +| READ-44 | Med | S | A | `systems/ddt.py:2750` | Dead moment-preservation: two `if 0 and self.preserve_moments …` blocks (2750–2785), yet the parameter is accepted, stored (1480), documented "experimental" (1390), and `self.I` (1774) exists solely to serve the dead blocks. | Delete the blocks + `self.I`; remove the param or make it raise NotImplementedError; fix docstring. | +| READ-45 | Med | S | D-doc | `systems/ddt.py:1933` | Comment contradicts code: state setter says SemiLagrangian "doesn't take a theta argument" and hardcodes `_update_am_values(..., 0.5)` — but `__init__` takes theta (1467, PR #187) and update_pre_solve uses `self.theta` (2351); restore on a theta=1.0 instance silently re-derives CN coefficients. | Delete stale comment; pass `self.theta` (flag the one-line value change to the maintainer — next update_pre_solve overwrites anyway). | +| READ-46 | Med | S | D | `systems/ddt.py:3167` | Broken display code ×2: `Lagrangian._object_viewer` (3167–3174) and `Lagrangian_Swarm._object_viewer` (3519–3526) reference `self.psi`/`self.dt_physical` which are never set → AttributeError on view; Eulerian carries the same lines commented out (1023–1030). | `self.psi_fn`, drop/guard the dt_physical line, delete Eulerian's dead copy (or reduce all three to the working line). | +| READ-47 | Med | S | D | `systems/ddt.py:1116` | Nested bare `except:` clauses as dispatch (copy → evaluate → projection) with no types or comment; a genuine evaluate() bug silently becomes an expensive projection solve. | Explicit `if self._psi_meshVar is not None:` first; narrow the remaining except; one-line intent comment. | +| READ-48 | Med | S | D | `systems/solvers.py:2264` | FILE GRADE B. `_maybe_install_auto_gauge` (2026-06-24) uses the banned `maybe_` prefix (rule enforced twice before). | Rename `_install_auto_gauge_if_eligible`; update call site (2230) and cross-references (2132, 2385). | +| READ-49 | Med | M | D | `systems/solvers.py:2760` | `smoothing_length` getter/setter + `smoothing` property triplicated verbatim across the three projection classes (2760–2853, 3018–3071, 3376–3423) incl. `_smoothing_is_dimensional` bookkeeping. | `_SmoothingLengthMixin` or module helpers; keep per-class docstrings as thin wrappers. | +| READ-50 | Med | M | D | `systems/solvers.py:4338` | 55-line `delta_t` setter duplicated verbatim in SNES_AdvectionDiffusion (3832–3884) and SNES_Diffusion (4338–4390); both open with bare `except: pass` (swallows KeyboardInterrupt). | Extract `_nondimensionalise_timestep(value)`; narrow the except to `(TypeError, ValueError)`. | +| READ-51 | Med | M | D | `systems/solvers.py:4414` | Diffusivity-max block in `estimate_dt` duplicated ×4 (TransientDarcy 736, AdvDiff 3926, Diffusion 4414, NavierStokes 4992); centroid-velocity block ×3 (Stokes 1885, AdvDiff 3963, NS 5029). | Extract `_global_max_diffusivity` and `_centroid_velocities_nd`; the `.magnitude` rationale lives once. | +| READ-52 | Med | M | D | `systems/solvers.py:100` | `_apply_unit_aware_scaling`: one try/except silently discards every exception (147–149, unused `e`); its two branches are redundant — both multiply by `fundamental_scales['time']`; the field-units inspection changes nothing. | Collapse to a single path; delete or justify the dead branch; specific exceptions; drop redundant `import sympy` (103). | +| READ-53 | Med | S | D | `systems/solvers.py:2985` | `SNES_Vector_Projection.projection_problem_description` adds smoothing+penalty to `self._f1` that the F1 Template (2963–2970) already contains — doubling the terms; unmarked deprecated while its siblings carry deprecation notes. | Delete if unused (grep first); else mark deprecated and assign `self._f1 = self.F1.sym` only; at minimum `# TODO(BUG)`. | +| READ-54 | Med | M | D | `systems/solvers.py:4119` | Transient solve() choreography repeated near-identically ×3 (TransientDarcy 799–839, AdvDiff 4101–4138, Diffusion 4490–4526); the 3-line cache-invalidation idiom verbatim ×3. | Extract `_invalidate_solution_cache`; consider a `_transient_solve_template` with hooks. | +| READ-55 | Med | S | D-doc | `utilities/rotated_bc.py:1` | FILE GRADE B. Module docstring opens "Development version of underworld3.utilities.rotated_bc — … Productizes the validated prototypes" — but this file IS that module on the integration branch; stale provenance framing misleads about status. | Rewrite the first two lines to describe what the module IS. | +| READ-56 | Med | S | D | `utilities/rotated_bc.py:317` | `_zero_rows_local` exists to encapsulate the ownership-relative zeroing (np>1 overflow subtlety) yet its exact body is hand-inlined ×3 more (244–246, 610–612, 652–655). | Call the helper at all three sites; move its definition above first textual use. | +| READ-57 | Med | S | D | `utilities/rotated_bc.py:736` | Boundary-spec normalization written two different ways: nested-comprehension dict one-liner (736–737) vs build_rotation's loop (148). | Extract `_boundary_spec(spec) -> (name, normal)`; use in both. | +| READ-58 | Med | M | D | `utilities/rotated_bc.py:479` | `solve_rotated_freeslip_nonlinear` is 177 lines (338–514); the backtracking line search + interleaved manual `destroy()` bookkeeping forces the reader to track 6 live Vecs/Mats across three exit paths. | Extract the line search into a helper owning its temporaries' destroys; group per-iteration destroys at one point. | +| READ-59 | Med | M | D | `utilities/custom_mg.py:557` | `maps` element type is mode-dependent (parallel: 4-tuples read via magic `lay[3]` at 559; serial: bare index arrays) — reader must trace two return shapes. | `LevelLayout` NamedTuple from `_level_dof_layout`/`_coarse_dof_layout`; `lay.n_full`; optionally split serial/parallel loop bodies into named helpers. | +| READ-60 | Med | S | A | `discretisation/remesh.py:190` | FILE GRADE B. `_remap_one_var` is dead: body is `raise NotImplementedError`, docstring admits "not currently used", no callers. | Delete; its pointer text already lives in the module docstring. | +| READ-61 | Med | S | A | `discretisation/remesh.py:200` | `_new_coord_cache` has zero callers and duplicates the new-DOF-coordinate capture inline in `_remap_var_set` (411–416) — dead code doubling as a which-copy-is-live trap. | Delete; if the inline block wants a name, extract it there as the single implementation. | +| READ-62 | Med | S | D | `discretisation/remesh.py:447` | `REMESH_MONOTONE` env knob read via `import os as _os` **inside the per-variable loop inside a try block** (447–450) — a module behaviour switch invisible from the module top, re-parsed per variable. | Hoist to a documented module-level constant with the falsy-string normalisation done once. | +| READ-63 | Med | M | D | `discretisation/remesh.py:421` | Guarded write-back `try: …[...] = X / except Exception: pass` ×3 (349–353, 422–426, 467–471) + the twin in `_snapshot_remap_data` (177–187); each silently drops a variable's transfer with no stated failure mode. | One `_write_var_data(var, values)` helper with the legitimate failure mode named once (unallocated/size-0 storage); preserve the swallow (behaviour unchanged). | +| READ-64 | Med | S | D-doc | `discretisation/remesh.py:272` | Re-entrancy comment says it "surfaces the outer scratch dict" but the code does nothing of the sort (the outer wrapper set `_remesh_pending_scratch` at 280); the nested branch returns True unconditionally, contradicting the docstring contract. | Rewrite the comment to state what is true; note the return value is meaningless for nested calls. Comment-only. | + +#### Low severity + +| ID | Sev | Eff | Wave | Location | Finding | Fix | +|----|-----|-----|------|----------|---------|-----| +| READ-65 | Low | S | D | `meshing/smoothing.py:507` | Dead params `relax=None, step_frac=None` ("kept only for signature stability"); `n_sweeps` misnames the nonlinear-CG iteration cap (741). | Drop the dead params; rename to `max_cg_iters` with one deprecation cycle for `n_sweeps`. | +| READ-66 | Low | S | D | `meshing/smoothing.py:1342` | `_zig` computed identically ×3 (1342, 1584, 1968) each with a 5-line re-explanation; `_wire` closure pair duplicated ×3 (1228, 1545, 1888). | `_warm_start_krylov(...)` + `_solver_wiring(...)` helpers; rename `_zig` → `zero_init_guess`. | +| READ-67 | Low | S | D | `meshing/smoothing.py:2305` | `move_anisotropy` radial/tangential reweighting duplicated verbatim (1387–1400 vs 2305–2318). | Extract `_reweight_displacement_radial_tangential(...)`. | +| READ-68 | Low | S | A | `meshing/smoothing.py:1277` | Dead locals: `_cdim` ×3 (1277, 1593, 2271); `Lbar`/`L0`/`L0_mean` triple-name one constant in `_winslow_spring` (606–608). | Delete `_cdim`s; use `Lbar` directly in the verbose diagnostic. | +| READ-69 | Low | S | D | `meshing/smoothing.py:4405` | Global mean-edge-length h0 implemented twice (2004–2014, 4405–4416), both with matching compounding-refinement commentary. | Extract `_mean_edge_length(dm, coords)`; keep the warning once at the cache declaration (73–83). | +| READ-70 | Low | S | A | `…snes_solvers.pyx:1` | Line 1 of the flagship solver file is `from xmlrpc.client import Boolean` (unused); `sympify`, `TypeAlias`, `class_or_instance_method` also unused. | Delete the four imports. | +| READ-71 | Low | S | D | `…snes_solvers.pyx:1691` | F1 guard property's RuntimeError says "F0 is being used" (copy-paste of the F0 property at 1687) — points a developer at the wrong term. | Fix the message to F1. | +| READ-72 | Low | S | D | `…snes_solvers.pyx:1022` | Two divergent SNES convergence-reason tables in one class (866–888: 11 codes; 1022–1035: 8 codes; −9/−10/−11 already missing from the compact map). | One class-level table `code -> (NAME, explanation)`; both consumers format from it. | +| READ-73 | Low | S | D | `…snes_solvers.pyx:2115` | Time non-dimensionalisation snippet copy-pasted ×6 (2115, 3005, 7471, 7611, 7929, 7962). | One `_nondimensional_time(time)` base-class helper. | +| READ-74 | Low | S | D-doc | `…snes_solvers.pyx:3130` | Unresolved editorial musings as comments (3130 "LM: probably not something we need", 5508 "Why is this here??", 6220 "uf0, uF1 are redundant", 5029). | Delete or convert to `# TODO(DESIGN):` per CLAUDE.md policy. | +| READ-75 | Low | M | D | `…snes_solvers.pyx:2317` | Verbose-monitor option toggle copy-pasted ×4 (2317, 3250, 4254, 5035); GAMG default bundle ×3 (2304, 3241, 4245; incl. a literally repeated line at 2315). | Extract `_set_monitor_options(verbose)` / `_set_gamg_defaults()`. | +| READ-76 | Low | S | D | `…snes_solvers.pyx:3614` | Local named `dim` assigned `mesh.cdim` (3614–3615, 4439) — embedding dim, not topological; plus a stale "~line 173" comment reference. | Rename to `gdim`/`cdim`; replace the line reference with the method name. | +| READ-77 | Low | S | A | `…discretisation_mesh.py:8` | Dead imports: `mpi4py.MPI.Info` (8), the baffling `blockmatrix.bc_dist` (11), unused `gather_data` (18); CoordSys3D imported at 28 and re-imported at 878. | Delete all four. | +| READ-78 | Low | S | D-doc | `…discretisation_mesh.py:5236` | Self-contradictory stale docstring: two consecutive sentences state opposite semantics for `tol > 0` (first should describe `on_boundary=True`). | Fix the first sentence. | +| READ-79 | Low | S | D | `…discretisation_mesh.py:4894` | `_test_if_points_in_cells_internal` repeats the control-point loop ×3 with only the final comparison differing. | Compute once; select the threshold; one comment on strict/non-strict. | +| READ-80 | Low | M | D | `…discretisation_mesh.py:1274` | `quality()` is single-letter soup (`a, b, cl_, A, q, et, jr, rel`) under a superb docstring. | Rename locals to what they are; one comment on the edge-to-tris build; formulas unchanged. | +| READ-81 | Low | S | D-doc | `…discretisation_mesh.py:2649` | Blanket `except Exception: pass` in the refresh paths (2649, 3133–3145, 6156) — a failed boundary-normal refresh silently leaves a Nitsche BC on stale geometry, the exact bug class these features prevent. | Comment the sanctioned reason at each swallow or route through debug-level pprint; narrow types in a follow-up. | +| READ-82 | Low | S | D | `…discretisation_mesh.py:4170` | `Dict` used in an annotation but never imported (latent-NameError smell; harmless at runtime). | `dict[str, …]` builtin generic or add to the typing import. | +| READ-83 | Low | S | D | `…discretisation_mesh.py:3855` | Inverted no-op guards: `if os.path.exists(...): pass else: raise` ×2. | `if not …: raise`, hoist `abs_dir` once. | +| READ-84 | Low | S | D | `…discretisation_mesh.py:193` | `_hierarchy_sidecar_name` keeps a `level` arg "for forward-compatibility"; every call site passes 0 and the design persists only the coarsest level. | Drop the param (or delete the forward-compat sentence and state "always 0 today"). | +| READ-85 | Low | S | A | `systems/ddt.py:2508` | Dead diagnostic: `coords_template`/`has_units` (2508–2509, "# DIAGNOSTIC") computed, never read — leftover from the #267 rework. | Delete 2507–2509. | +| READ-86 | Low | S | A | `systems/ddt.py:1690` | `self._nswarm_psi = None` kept alive with an eight-line apology that itself says nothing reads it. | Delete; one-line note or rely on git history. | +| READ-87 | Low | S | D | `systems/ddt.py:1660` | Work variable named `f"W_{instance}_{i}"` where `i` is a **leaked loop index** from a loop ending at 1614 — the suffix looks meaningful but is accidental; `W` says nothing. | Rename e.g. `psi_work_sl_{instance}`; note why it exists (different degree/continuity than psi_star). | +| READ-88 | Low | S | A | `systems/ddt.py:3022` | Dead class attr `instances = 0` ("to create unique … ids") in Lagrangian (3022) and Lagrangian_Swarm (3404) — never incremented or read; ids come from `uw_object.instance_number`; the comment misleads. | Delete both. | +| READ-89 | Low | S | D-doc | `systems/ddt.py:2488` | Comments cite drifted hard-coded line numbers ("~line 1540" → actually ~2092; "lines 703-709" → ~2419–2424). | Method-name references instead of line numbers. | +| READ-90 | Low | S | D | `systems/ddt.py:2173` | Redundant inline imports of module-scope names: sympy ×4, numpy, RemeshPolicy ×4, UnitAwareArray ×5 — falsely suggesting circularity. | Hoist to module level (both are leaf modules); comment any import that genuinely must stay local. | +| READ-91 | Low | S | D-doc | `systems/ddt.py:534` | `Symbolic` accepts/stores/documents `bcs`, `smoothing` (and threads ignored `evalf`/`verbose`) that only make sense for projection-backed flavors. | Remove, or state "accepted for interface parity; unused here" in one comment. | +| READ-92 | Low | S | D | `systems/ddt.py:1801` | `try: register_remesh_hook … except Exception: pass` ×2 with no stated tolerated failure — reads as fear, contrasting with the well-documented guard just above. | Narrow to `except AttributeError` + one-line comment ("older Mesh without the hook registry"). | +| READ-93 | Low | S | A | `systems/solvers.py:4273` | Five stale commented-out blocks (4273–4289 "??? unable to solve after n timesteps", 556–561, 1849–1856, 3200–3203, 549–550). | Delete; replace 4273-block with a one-line known-limitation note if worth keeping. | +| READ-94 | Low | S | D-doc | `systems/solvers.py:3887` | `estimate_dt` takes `percentile: float = 0.0` but the docstring documents only `direction_aware`; the semantics live in an inline comment invisible to help()/Sphinx. | Lift the inline explanation into the Parameters section. | +| READ-95 | Low | S | A | `systems/solvers.py:4043` | `np.maximum(h_per_element, 0.0)` is a no-op (h = max−min ≥ 0 by construction) and its comment describes the *following* `np.where`. | Delete the line; move/reword the comment onto the np.where. | +| READ-96 | Low | S | D | `systems/solvers.py:76` | Module-level `expression = lambda …` shadows a common name, hides why `_unique_name_generation=True` is needed, and gives useless tracebacks. | `def expression(...)` with a one-line docstring citing SYMBOL_DISAMBIGUATION_2025-12.md. | +| READ-97 | Low | S | D | `systems/solvers.py:1313` | `_prev_effective_order` lazily created via hasattr-guard inside solve() instead of `__init__` — solver state invisible from `__init__`. | Initialise to None in `__init__` with a one-line comment; drop the guard. | +| READ-98 | Low | S | D | `systems/solvers.py:284` | Six shadow imports of module-scope names (`uw` ×2, `sympy` ×3, `np` ×1) inside methods. | Delete (disappears with READ-49 for three of them). | +| READ-99 | Low | S | D | `systems/solvers.py:335` | `CM_is_setup` abbreviated, defined twice identically (335, 1485), and raises AttributeError (rather than False) before a constitutive model is assigned. | Define once on the base as `constitutive_model_is_setup` (+ alias); document/handle the not-yet-assigned case. | +| READ-100 | Low | S | D | `utilities/rotated_bc.py:33` | `_velocity_field_id(solver)` ignores its argument and returns 0, while the pressure id is a bare inline `PRE = 1` (230) — inconsistent ceremony for two constants. | Module constants `_VELOCITY_FIELD = 0`, `_PRESSURE_FIELD = 1` with one comment; use at all call sites. | +| READ-101 | Low | S | D | `utilities/rotated_bc.py:229` | Pressure-datum search runs unconditionally but `pin` is consumed only in the opt-in LU branch (252–258); its parallel-unsafety is noted in prose, not `TODO(BUG)`. | Move the search inside the LU branch (pure motion); convert the note to `# TODO(BUG):`. | +| READ-102 | Low | S | D | `utilities/rotated_bc.py:89` | `try: import sympy … except Exception: sym_fn = None` guards an impossible state (sympy is a hard dep) and would silently misinterpret a lambdify error as a constant-array normal. | Import sympy plainly; drop the try/except. | +| READ-103 | Low | S | D-doc | `utilities/rotated_bc.py:692` | `info` parameter is the untyped result dict whose keys differ between linear and nonlinear paths; the contract is learnable only by diffing two return statements (272–274 vs 510–514). | Rename to `solve_result`; add Returns key listings to both solve functions' docstrings. | +| READ-104 | Low | M | D | `utilities/rotated_bc.py:436` | Pervasive semicolon-chained lines compress create/use/destroy of PETSc objects (215–220, 230, 259–261, 386, 422, 433, 436, 450, 461–462, 473, 486–488, 582), obscuring the lifecycle the file's own comments call trickiest. | Mechanically split to one statement per line; keep lifecycle comments attached. | +| READ-105 | Low | S | D | `utilities/rotated_bc.py:444` | `from underworld3 import mpi` repeated inline ×4 (444, 457, 501, 614) just for `mpi.pprint` — reader cannot tell if load-bearing. | Import once at module top (uw.mpi is a leaf) or one commented inline import per function. | +| READ-106 | Low | S | D | `utilities/custom_mg.py:268` | FILE GRADE A-. 4-line DM-prep block (clone/copyFields/copyDS/createDS) duplicated between `_coarse_reduced_map` (268–271) and `_coarse_dof_layout` (318–321). | Extract `_clone_dm_with_solver_discretisation(...)`; move the copyDS 'trick' explanation onto it. | +| READ-107 | Low | S | D | `utilities/custom_mg.py:586` | Zero-column singular-coarse-operator guard exists twice (inline scipy 586–591; `_assert_no_zero_columns_parallel` 362–375); the physics rationale documented only on one. | Extract `_assert_no_zero_columns_serial` next to the parallel one; share the wording. | +| READ-108 | Low | S | D | `utilities/custom_mg.py:228` | Bare `except Exception: return dm` around `getNumFields()` with no stated raising state — silently falling back to the whole DM would mask a multi-field mistake. | Narrow to `PETSc.Error` + one-line comment, or drop if it cannot fail on a built DM. | +| READ-109 | Low | S | D | `utilities/custom_mg.py:325` | `_build_parallel_transfer` uses `cc, fc, lay_c, lay_f` while the serial `_reduced_transfer` (275) spells out full names — reader re-derives the convention. | Rename to match the serial vocabulary (call site at 578 updates in the same commit). | +| READ-110 | Low | S | D-doc | `utilities/custom_mg.py:87` | `r = np.where(r == 0.0, 1e-30, r)` — uncommented magic clamp; the intent (r²log r → 0; clamp only keeps log finite) unstated. | One-line comment. | +| READ-111 | Low | S | D | `utilities/custom_mg.py:66` | Semicolon-packed multi-statement lines (66, 73, 196, 366, 569). | Split; pure formatting. | +| READ-112 | Low | S | D-doc | `utilities/custom_mg.py:639` | Legacy finest-only path in `inject_custom_mg` is live (set_custom_mg, test_1015) but duplicates degree/continuity extraction from `build` and carries no lifespan marker. | `# TODO(deprecate): remove with SolverBaseClass.set_custom_mg …` + state the one case where legacy differs. | +| READ-113 | Low | S | D | `utilities/custom_mg.py:489` | `sub` holds an options-prefix string but collides with `sub` = sub-DM elsewhere in the file. | Rename `vel_prefix`; move inside the `if verbose:` block. | +| READ-114 | Low | S | D | `discretisation/remesh.py:173` | `_snapshot_remap_data` is also called on the managed-vars bucket (304) — the name lies about half its uses. | Rename `_snapshot_var_data`; two call sites. | +| READ-115 | Low | S | A | `discretisation/remesh.py:168` | Unreachable `else: buckets["remap"].append(var)` (policy already normalised; all members matched) and an impossible `if var is None:` guard; the deliberate ValueError→REMAP fail-safe (159–161) doesn't say so. | Delete both; half-line comment pointing at the fail-safe rationale in the RemeshPolicy docstring. | +| READ-116 | Low | S | D | `discretisation/remesh.py:474` | `remap_var_set = _remap_var_set` alias 90 lines after the private definition; grep for the public name lands on an assignment (external caller: ddt.py:1840). | Rename the function itself to `remap_var_set`; delete the alias. | +| READ-117 | Low | S | D | `discretisation/remesh.py:380` | Redundant `if live else []` tail on a comprehension; `RemeshContext.scratch` docstring says "keys are by convention" without listing the exactly-two conventions. | Drop the tail; enumerate `ale_opt_out` and `v_mesh` with producer/consumer. | +| READ-118 | Low | S | D-doc | `discretisation/remesh.py:329` | Operator `on_remesh` hook exceptions swallowed, reported only under `verbose` — a failing ALE history update vanishes silently, with no stated rationale (unlike the file's other guards). | Comment the intended contract (best-effort hooks; REMAP already secured the fields) — or Wave-D decision to warn unconditionally. | + +--- + +## Testing Instructions + +How to validate the eventual fixes, per the campaign ground rules +(`tier_a` green pre/post each wave; `tier_a or tier_b` before merge; np2/np4 for +anything touching partition-sensitive code). + +1. **Baseline before any wave**: from the remediation worktree, `./uw build` then + `pytest -m "level_1 and tier_a"`; capture the pass list as the reference. +2. **Pure code-motion / dedent / rename claims** (READ-02, -05, -13, -14, -16, -29, + and most Wave-D items): assert no behavioural diff mechanically — + - For `.py` files: `python -m py_compile` + run the owning test files + (smoothing/mover: `test_0750, test_0760, test_0762, test_0763, test_0855`). + - For the `.pyx`: rebuild with `rm -rf build/` first (stale `build/src/*.c` is a + known trap — confirm the change landed via `strings <.so> | grep `), + then run `test_1010, test_1015, test_1016, test_1018, test_1053, test_1064`. +3. **READ-01 (`_signed_volumes`)**: whichever option is chosen, add a level_1 test + that calls `smooth_mesh_interior(method="mmpde")` on a small 3D mesh and asserts + either successful movement or a clean `NotImplementedError` — today it must + raise `NameError`, which the new test should first reproduce (test-before-fix per + project policy). +4. **READ-02**: after deleting the inline block, run the anisotropic mover on a + deterministic metric with `metric_refresh_per_iter` both True and False and + diff final coordinates against pre-change output (expect bit-identical for the + refresh path; identical-to-tolerance for the once-before-loop path since the + closure re-runs a deterministic `gproj.solve()`). +5. **READ-03 and all `.pyx` structural extractions**: benchmarked wave only. Beyond + tests, verify BC-label behaviour explicitly: solve one Scalar/Poisson and one + Vector problem with a Dirichlet BC pre- and post-change and compare solutions + bit-for-bit; run np2 and np4. The 2-vs-2 label divergence must survive + extraction unchanged unless a separate decision unifies it. +6. **Mesh-mover findings (READ-04..-12, -65..-69)**: run at np1/np2/np4 — many + duplicated blocks are MPI-reduction code where a wrong extraction deadlocks + (rank-local early return before a collective). Any hang = failure. +7. **Package split (READ-04)**: after the move, verify every cross-module private + import still resolves: `python -c "from underworld3.meshing.smoothing import + _edge_pairs, _tri_cells, _tet_cells, _pinned_mask, _auto_pinned_labels, + _owned_vertex_mask, _signed_areas"` and run the five mover test files plus + `surfaces`/`_ot_adapt` consumers. +8. **Doc-only fixes** (D-doc rows): `pixi run docs-build` clean; no test impact + expected, but run the owning test file once as a smoke check. +9. **Unverified findings**: each must pass the same adversarial verification as the + verified table (re-read lines at the remediation base, confirm callers/ + reachability, confirm not already fixed) **before** its fix is applied. + +--- + +## Known Limitations + +- **104 of 118 findings are unverified.** Their anchors were sampled (all sampled + anchors matched at `1d003481`) but the full adversarial pass — caller analysis, + reachability, fixed-at-HEAD checks — was performed only for READ-01..READ-14. + Treat the unverified tables as a triage queue, not a worklist. +- **Line numbers are pinned to `development@1d003481`.** Any commit to these files + invalidates them; remediation PRs should re-verify against their own base. +- **READ-01 is a functional defect, not just readability** (3D MMPDE cannot run). + It is reported here because the audit found it via a docstring contradiction, but + it may deserve routing to the bugs track / external planning file. +- **READ-45 and READ-53 include one-line behavioural corrections** (restore-time + theta; double-counted smoothing/penalty terms). Both are flagged for explicit + maintainer sign-off rather than being folded silently into readability waves. +- `petsc_generic_snes_solvers.pyx` findings respect the campaign constraint + (naming/docs/dead-code only); the structural extractions (READ-03, -25, -26, -27, + -75) are queued for the separately benchmarked wave and are **not** Wave-D-safe. +- Two hotspot files (`smoothing.py`, the `.pyx`) were not assigned letter grades by + the grading pass; their state is characterized narratively instead. +- The dimension briefing document was not available at its expected path; campaign + context was taken from `docs/reviews/2026-07/README.md` (ground rules, waves, + constraints) and the existing Dimension-3 review's section conventions. + +--- + +## Appendix — Refuted claims (do not re-find) + +No whole findings were refuted, but nine sub-claims were refuted or corrected during +adversarial verification. Recorded so later audits do not resurrect them: + +1. **"mmpde is the production default"** — refuted. The API default is + `method="spring"` (`smoothing.py:2623, 3525`); in-tree callers use `"ma"`/`"ot"`. + mmpde is *recommended* by the file's own DeprecationWarning, not the default. +2. **"Six movers carry the `_winslow_` prefix"** — refuted; five do. The sixth + (graph-Laplacian Jacobi) is inline in the dispatch and unprefixed. +3. **"~300 lines of literal duplicates in smoothing.py"** — overstated; ~130 lines + of verbatim ≥8-line blocks (~260 counting both copies) plus near-duplication. +4. **"The `_build_M_tensor` closure exactly matches the inline block"** — refuted; + the closure is a superset (additionally refreshes `Dcoords`/`gproj`, 2067–2078). + The line-for-line match is the density/assembly portion only. +5. **"Renaming the `_winslow_*` movers touches only the dispatch plus scripts"** — + refuted; docstrings in `remesh.py`/`_ot_adapt.py`/`smoothing.py:805`, 37 script + files, and design docs also reference the names. +6. **"`**_ignored` in `_winslow_mmpde` is a pure hedge; removing it is safe"** — + refuted; the strategy path legitimately forwards `resolution_ratio` + (3565–3566 → 3643–3644). Bare removal breaks `strategy=` + mmpde. +7. **"The four solver classes' helpers are verbatim-shared modulo the class-name + string"** — refuted; essential-BC label semantics diverge 2-vs-2 + (`str(boundary)` vs `"UW_Boundaries"`), and SaddlePt interleaves + multiplier/fieldsplit logic. Extraction must preserve this deliberately. +8. **"The four-times-applied fix stanza in the .pyx came from PR #216 + (af537d56)"** — refuted attribution; that commit touched `swarm.py` / + `_function.pyx`. The evidence is the identical stanza itself, whatever PR + introduced it. +9. **"Re-exporting only the public names suffices for the smoothing package + split"** — refuted; private helpers are imported by production modules and by + tests 0750/0760/0762/0763/0855 and must keep resolving under + `underworld3.meshing.smoothing.`. + +--- + +## Sign-Off +| Role | Name | Date | Status | +|------|------|------|--------| +| Maintainer | Louis Moresi | 2026-07-05 | Pending review | +| Author | Claude (audit session) | 2026-07-03 | Complete | + + + +- Audit dimension: **4 — Readability of change hotspots** +- Base audited: `development` @ `1d003481` +- Findings: **118** total — 14 verified (3 high / 11 medium), 104 unverified + (6 high / 44 medium / 54 low); 0 refuted findings, 9 refuted sub-claims recorded. +- All verified-table `file:line` anchors personally re-read in the audit worktree + on 2026-07-03; unverified-table anchors sampled. +- Prepared for the July 2026 quality campaign, Phase 1. + +*Underworld development team with AI support from [Claude Code](https://claude.com/claude-code)* \ No newline at end of file diff --git a/docs/reviews/2026-07/README.md b/docs/reviews/2026-07/README.md new file mode 100644 index 000000000..afc51c2d9 --- /dev/null +++ b/docs/reviews/2026-07/README.md @@ -0,0 +1,47 @@ +# July 2026 — Post-AI-Development Quality Campaign + +**Status**: Phase 1 (audit) complete 2026-07-03 — remediation worklist ready for maintainer review +**Base**: `development` @ `1d003481` +**Campaign plan**: approved 2026-07-03 (L. Moresi) + +## Why + +June 2026 brought ~132 commits and +12.6k lines to `src/`, largely AI-assisted across +many non-overlapping sessions. This campaign audits the result against the project's +founding quality rule — *anyone should be able to read the code and understand it* — +and remediates in reviewed, tested waves. + +## Audit dimensions + +| # | Dimension | Review document | Status | +|---|-----------|-----------------|--------| +| 1 | Loose ends (TODOs, stubs, skipped tests, disabled logic) | `LOOSE-ENDS-AUDIT.md` | complete | +| 2 | Branch & worktree triage ledger | `BRANCH-TRIAGE-LEDGER.md` | complete | +| 3 | API consistency & convention proposal | `API-CONSISTENCY-REVIEW.md` | complete | +| 4 | Readability of change hotspots | `READABILITY-REVIEW.md` | complete | +| 5 | Swarm/particle subsystem architecture | `SWARM-SUBSYSTEM-REVIEW.md` | complete | +| 6 | Docs & standards coherence | `DOCS-STANDARDS-COHERENCE.md` | complete | + +Cross-dimension synthesis and the ranked remediation worklist: `REMEDIATION-WORKLIST.md`. + +## Remediation waves (post-audit, each its own PR) + +- **Wave A** — deletions & dead code +- **Wave B** — internal migration off deprecated access patterns (~41 sites) +- **Wave C** — API harmonization (deprecation shims, no hard break) +- **Wave D** — readability rewrites of hotspot files +- **Wave E** — docs alignment +- **Branch-triage execution** (parallel track; deletions signed off per batch) + +Follow-ons: swarm modernization refactor (design doc from dimension 5); +guardrails — UW3 Style Charter + mechanical CI gates. + +## Ground rules + +- Every finding carries `file:line` evidence and is adversarially verified before + entering the worklist. +- Every remediation PR cites the finding(s) it resolves. +- `petsc_generic_snes_solvers.pyx`: naming/docs/dead-code changes only — no numerics + without separate benchmarking. +- Gate per wave: `tier_a` tests green pre/post; `tier_a or tier_b` before merge; + parallel (np2/np4) tests for anything touching swarm/migration code. diff --git a/docs/reviews/2026-07/REMEDIATION-WORKLIST.md b/docs/reviews/2026-07/REMEDIATION-WORKLIST.md new file mode 100644 index 000000000..9fd07661d --- /dev/null +++ b/docs/reviews/2026-07/REMEDIATION-WORKLIST.md @@ -0,0 +1,432 @@ +# Remediation Worklist — July 2026 Quality Campaign (Cross-Dimension Synthesis) + +**Status**: worklist v1, 2026-07-03 — synthesized from all six audit dimensions +**Base**: `development` @ `1d003481` (audit worktree, campaign index at `e848d131`) +**Sources**: `LOOSE-ENDS-AUDIT.md` (LE-01…26), `API-CONSISTENCY-REVIEW.md` (API-01…12), +`READABILITY-REVIEW.md` (READ-01…118), `SWARM-SUBSYSTEM-REVIEW.md` (SWARM-01…24), +`DOCS-STANDARDS-COHERENCE.md` (DOC-01…08), `BRANCH-TRIAGE-LEDGER.md`. + +Abbreviation: `pyx` = `src/underworld3/cython/petsc_generic_snes_solvers.pyx`. + +## Overview + +This is the single ranked worklist for the remediation waves. Every actionable +finding from the six review documents is assigned to exactly one track below; +duplicates across dimensions are merged (the merge is noted on the row). Within +each wave, items are ranked **severity first, then effort** (S before M before L). + +Tracks: + +- **Track 0 — Bug fixes** (added by this synthesis): confirmed correctness defects + the audits found behind TODOs/skips/docstrings. These do not fit the five + cleanup waves and should ship as small individual PRs, generally *before* the + wave that touches the same file. 19 items. +- **Wave A — deletions & dead code**: 27 items. +- **Wave B — internal deprecated-pattern migration**: 6 items (~41 call sites). +- **Wave C — API harmonization with deprecation shims**: 13 items. +- **Wave D — readability rewrites** (incl. the flagged benchmarked `.pyx` sub-wave D2): 94 items. +- **Wave E — docs alignment**: 11 items. +- **Branch-track** (parallel): 6 ordered actions covering ~80 branches/worktrees. +- **Follow-on**: 8 items (swarm modernization refactor, guardrails, deferred solver work). + +Ground rules carried from the campaign: no `pyx` numerics without separate +benchmarking; API changes only via deprecation shims (no hard breaks); `tier_a` +green pre/post each wave, `tier_a or tier_b` before merge, np2/np4 for anything +touching swarm/migration/partition-sensitive code. **Unverified findings +(LE-13…26, API-10…12, READ-15…118, SWARM-15…24, DOC-08) must pass the same +adversarial verification as the verified tables before their fix is applied** — +they are scheduled here but each carries that gate implicitly. + +Eighteen items need the maintainer's judgment before (or as part of) execution; +they are marked `[D#]` on their rows and collected in +**"Decisions needed from Louis"** at the end. + +## Changes Made + +None — planning document only. Each remediation PR will cite the finding ID(s) +it resolves and the track item below. + +## System Architecture + +How the tracks interlock (the cross-wave dependency map): + +1. **Track 0 before Wave B in `swarm.py`/`ddt.py`** — the swarm bug fixes + (BF-02…BF-08) touch the same functions Wave B migrates; fix first, migrate + second, so the migration diff is against correct code. +2. **Wave A before Wave B for the swarm package** — deleting `swarms/pic_swarm.py` + (WA-02) removes 18 of the ~41 deprecated-pattern sites from Wave B's scope. +3. **Wave B before Wave D in the same file** — the `ddt.py` rewrites + (READ-17/18/19/20) and any swarm-file restructuring must wait for the + access-pattern migration (WB-01/WB-02) so each PR stays reviewable. +4. **Wave C before Wave E's call-site sweep** — the BC arg-order shim (WC-01/02) + must land before the ~1,370-site docs/tests sweep (WE-09). +5. **Wave E internal ordering** — authority map (WE-01) → style-guide rewrite + (WE-02) → docstring-queue regeneration (WE-03) → docstring wave (WE-04), + per DOC-04/01/02/05. +6. **Branch-track before file waves where branches collide** — + `bugfix/yield-homotopy` (LAND) conflicts with `pyx` and + `constitutive_models.py`: land it before Wave A/D `pyx` edits (WA-05…09, + D-pyx groups). `bugfix/custom-mg-parallel` (KEEP_ACTIVE) extends + `custom_mg.py` by +262 lines: Wave C's WC-04 and the D-custom_mg group wait + for or coordinate with it. The `feature/elliptic-ma` EXTRACT (smoothing.py + parallel allgather fix) should be rescued before the D-smoothing group edits + the same function. +7. **Follow-on gates** — SWARM-12's lazification is *blocked* until BF-08 + (pre-solve proxy refresh) exists; the swarm self-validating-cache refactor + (FO-01) subsumes BF-02's tactical fix but does not replace it. + +## Track 0 — Bug fixes (small individual PRs, most-severe first) + +| # | Finding(s) | Sev | Eff | Location | Action | Deps / notes | +|---|-----------|-----|-----|----------|--------|--------------| +| BF-01 | LE-01 | High | S | `function/_function.pyx:845` | Fix dead ternary so rbf-path derivatives use each expression's own component; regression test `v[1].diff(x)` with `rbf=True` | Standalone | +| BF-02 | SWARM-01, SWARM-02 (+SWARM-17 fold after verify) | High | S | `swarm.py:3451` | Move `_invalidate_canonical_data()` (+ kd-tree drop, proxy-stale mark) ahead of the migrate() no-move early return; add invalidation in `add_particles_with_global_coordinates` and (verify) `populate()` | Before WB-01; np2/np4 gate | +| BF-03 | LE-02 | High | M | `cython/petsc_maths.pyx:303` | Rewrite `CellWiseIntegral.evaluate()` against `mesh.dm`+`getDS()` (the `Integral` pattern); remove xfails `test_0501:182,195`; note the #171 caveat | Standalone | +| BF-04 | SWARM-06 | High | M | `systems/ddt.py:3197,3281,3549,3638` | Rewrite Lagrangian history component writes via `_data_layout` modern interface; add advecting-history tests for both classes | Counts toward Wave B for those lines | +| BF-05 | SWARM-04 | High | M | `swarm.py:480` | Separate "suppress migration" from "suppress PETSc sync" inside `migration_disabled()`; flush pending writes on exit; repair vacuous `ptest_0755:322-329` | np2 gate | +| BF-06 | SWARM-07 | High | M | `swarm.py:1457` | Empty-rank strategy: guard KDTree on 0 particles; starved-rank proxies untouched + warn (not silent zeros) | np4 test, all particles on one rank | +| BF-07 | SWARM-03 | High | M | `swarm.py:3114` | Deferred/context-exit migration for modern coordinate writes (docstring already promises it); never per-write (collective deadlock) | **[D12]**; after BF-02/BF-05 | +| BF-08 | LE-03 = SWARM-05 | High | M | `swarm.py:1075` | Stale-proxy hole (#215 Bug 3): single eager proxy refresh at solve entry; gate on `test_0006_memory_leak` | **[D11]**; unblocks FO-01's SWARM-12 lazification | +| BF-09 | READ-01 | High | S | `meshing/smoothing.py:3124` | 3D MMPDE `NameError`: implement `_signed_volumes` OR `NotImplementedError` + 2D-only docstring; level_1 test either way | **[D8]**; before D-smoothing group | +| BF-10 | LE-06 | Med | M | `tests/test_0813...:32` + `utilities/mathematical_mixin.py:795` | (a) Interim: rewrite the three Batman regression tests units-free (restores DM-corruption coverage now); (b) implement `UnitAwareDerivativeMatrix.__mul__`/`__neg__`, then re-unitize | (a) immediate; (b) units subsystem | +| BF-11 | LE-07 | Med | M | `swarm.py:665-706` | Align SwarmVariable array-view reductions to MeshVariable per-component-tuple contract; unskip `test_0850:32/72/335` | **[D6]** return-type behaviour change | +| BF-12 | LE-08 | Med | M | `function/functions_unit_system.py:306` | Quantity-valued coordinate lists in `evaluate()`: coerce, or document-unsupported and rewrite `test_0812` with supported forms | **[D7]** (units-family decision) | +| BF-13 | LE refuted-appendix #2 | Med | S | `meshing/cartesian.py:536` | `BoxInternalBoundary` rank>0 `UnboundLocalError`: bind/broadcast `boundaries`/`boundary_normals` on all ranks; then revisit the six `test_0502` MPI skips | One-liner + np2 test | +| BF-14 | SWARM-11 (fix part) | Med | S | `swarm.py:4885` | Keyword-explicit `super().__init__` so `verbose` stops landing in `recycle_rate` | Class fate is **[D5]** | +| BF-15 | READ-46 | Med | S | `systems/ddt.py:3167,3519` | Fix `_object_viewer` AttributeError (`self.psi` → `psi_fn`; guard `dt_physical`); delete Eulerian's dead commented copy | Verify first (unverified table) | +| BF-16 | SWARM-16 | Med | M | `swarm.py:4649` | Substep advection: particles crossing partitions evaluated on wrong rank from substep 2 — verify, then migrate (or global-evaluate) between substeps | Verify first; np2 trajectory test | +| BF-17 | SWARM-19 | Med | S | `swarm.py:3810` | `save()` writes different coordinate systems per IO branch — verify, then unify on model-unit coordinates | Verify first; both-branch round-trip test | +| BF-18 | READ-43 | Med | S | `discretisation/discretisation_mesh.py:3689` | Deprecated `points` setter silently discards the NDArray_With_Callback wrapper: fix to in-place write or remove the setter (it is already deprecated) | Verify first; mark `TODO(BUG)` immediately in Wave D if deferred | +| BF-19 | LE-04 | High | L | `tests/parallel/test_1017...:129` | #291 (Stokes_Constrained np>1 segfault): keep skip as-is; keep on the release-blocking list — fix is owned outside the cleanup waves | **[D17]**; on fix, unskip + drop serial-only restriction | + +## Wave A — deletions & dead code (one or few PRs; behaviour-neutral by construction) + +Gate: full `tier_a` identical pre/post; `tier_a or tier_b` before merge; `./uw build` +with `rm -rf build/` for any `pyx` touch. Maintainer signs off deletions per batch. + +| # | Finding(s) | Sev | Eff | Location | Action | Deps / notes | +|---|-----------|-----|-----|----------|--------|--------------| +| WA-01 | LE-12 | Med | S | `discretisation/persistence.py` | Delete module + `__init__.py:209` import; update CLAUDE.md Key Files entry | **[D1]** delete vs warn-on-import shim | +| WA-02 | SWARM-09 (+LE-17 subsumed) | Med | S | `swarms/pic_swarm.py` | Delete the never-installed 1,534-line module + breadcrumbs (`swarm.py:70, 2449-2455`) | **[D4]**; after WA-03 decision; removes 18 Wave-B sites | +| WA-03 | SWARM-08 | Med | M | `swarm.py:3360, 4744` | Recycle/streak feature: excise `recycle_rate>1` machinery + docstring claims (or port working logic → moves to Track 0) | **[D4]**; either way add a guard/test | +| WA-04 | LE-09 = READ-44 | Med | S | `systems/ddt.py:2750-2785` | Delete both `if 0 and preserve_moments` blocks, `self.I`, and access remnants; `preserve_moments=True` raises `NotImplementedError`; test it | | +| WA-05 | READ-21 | Med | S | `pyx:1383,1386,1466,2022` | `raise("string")` → `ValueError`/`TypeError` with the same message (error-path only) | After yield-homotopy lands (pyx conflicts) | +| WA-06 | READ-14 | Med | S | `pyx:3884, 7140` | Delete `if True:` constant guards, dedent — byte-identical | Same pyx PR as WA-05/07 | +| WA-07 | LE-13 + READ-24 + LE-15 | Med | M | `pyx:4026-4066, 5148-5168, 2766-2778, 2684, 3624, 5119, 6355, 6470, 2746, 3216-3227, ...` | Delete fossil commented-out blocks and dead assignments; KEEP the 3-line clearDS/createDS rationale and one-line intent notes | Comment-only per pyx rule; rebuild + tier_a | +| WA-08 | READ-70 | Low | S | `pyx:1` | Delete 4 unused imports (`xmlrpc.client.Boolean` et al.) | Same pyx PR | +| WA-09 | READ-23 (deletion part) | Med | S | `pyx:5620` | Delete empty 'robust'/'fast' `pass` branches (ValueError on unknown); comment the kaskade/additive divergence — **do not change option values** | | +| WA-10 | READ-31 | Med | S | `discretisation_mesh.py:4492,4606,5544,5584` | Delete four dead methods (zero callers) | Verify first | +| WA-11 | READ-32 | Med | S | `discretisation_mesh.py:5699` | Delete dead-AND-broken `meshVariable_mask_from_label` | Verify first | +| WA-12 | READ-33 + READ-34 | Med | S | `discretisation_mesh.py:4548, 1063, 527, 695, 711` | Delete commented face-normal block, `if False:` chains, scratch | Verify first | +| WA-13 | READ-77 | Low | S | `discretisation_mesh.py:8,11,18,878` | Delete dead imports; de-duplicate CoordSys3D import | | +| WA-14 | READ-60 + READ-61 | Med | S | `discretisation/remesh.py:190,200` | Delete dead `_remap_one_var` and `_new_coord_cache` | | +| WA-15 | READ-115 | Low | S | `remesh.py:168` | Delete unreachable else-branch and impossible `if var is None:` guard; half-line fail-safe note | | +| WA-16 | SWARM-21 | Low | S | `swarm.py:4641,4682-4700,1988,1994` | Remove/gate unconditional hot-path prints; delete dead `corrector`/`evalf` signature params (commented implementation) | Param removal via WC-12-style shim if public | +| WA-17 | SWARM-22 = LE-14 | Low | S | `swarm.py:1235,1285,1342,1391,961-967,1136,1499,1184-1200,1242-1263` | Delete no-op sync TODO stubs, `_rbf_reduce_to_meshVar`, legacy/enhanced-array pass-throughs (retire `test_0530` with them), commented blocks | `sync=` kwarg itself deprecates via WC-12 | +| WA-18 | LE-16 | Low | S | `cython/petsc_discretisation.pyx:248-285` | Delete dead triple-quoted `petsc_dm_get_periodicity`; planning-file note if still wanted | | +| WA-19 | READ-68, READ-85, READ-86, READ-88, READ-93, READ-95 | Low | S | smoothing/ddt/solvers dead locals & stale comment blocks | One batch PR of small deletions | Verify each anchor | +| WA-20 | LE-18 | Low | S | `tests/test_quantities_simplified.py` | Delete (imports a nonexistent module); port any uncovered cases to real `quantities` tests first | | +| WA-21 | LE-19 | Low | S | `tests/test_0754:166,288; test_0756:164` | Delete UnitAwareExpression-era skips; keep the neighbouring correct xfails | | +| WA-22 | LE-20 | Low | S | `tests/test_0750_meshing_follow_metric.py:269,324,344` | Delete (or keep with reasons as-is — they already point at surviving coverage) | | +| WA-23 | LE-10 | Med | M | `tests/test_0620, test_0630` | Delete `test_0630`; rewrite `test_0620` to assert the *implemented* mesh-units semantics | **[D7]** | +| WA-24 | LE-11 | Med | M | 16 skip markers across 7 units test files | One decision for the `coord_units`/quantity-coordinate placeholder family: consolidate into one labelled aspirational module, or delete + record the proposal in `docs/developer/design/` | **[D7]**; overlaps BF-12 | +| WA-25 | LE refuted-appendix #1 | Low | S | `tests/parallel/ptest_0762:75` | Remove stale skip (bug #151 fixed 2026-04); re-run at np≥2 | | +| WA-26 | LE-22 | Low | S | `pixi.toml` / setup deps, `kdtree.py:8` | Remove unused `pykdtree` dependency (macOS OpenMP crash hazard); keep module as import point; full macOS test run | **[D13]** | +| WA-27 | DOC-06 | Med | S | `docs/developer/design/` | `git mv` 24 scripts + 5 PNGs to `design/experiments/exp-integrator/`; delete 6 `.trace.txt`; relocate `_repro_dminterp_multifield_bug.py`; update ~10 doc references | **[D16]**; link fixes verified by WE docs build | + +## Wave B — internal migration off deprecated access patterns (~41 sites) + +Gate: np2/np4 parallel tests for every swarm/ddt touch; `tier_a` pre/post. +Do after Track-0 swarm fixes and WA-02 (which deletes pic_swarm's 18 sites). + +| # | Finding(s) | Sev | Eff | Scope | Action | Deps / notes | +|---|-----------|-----|-----|-------|--------|--------------| +| WB-01 | SWARM-13 (part) | Med | M | `swarm.py` — 13 live `with …access(…)` sites (1161, 2381, 3339, 3345, 3376, 3628, 3725, 4719, 4767, 4791, 4949, 4972, 4975) | Migrate to direct `.data` access | After BF-02…BF-08 | +| WB-02 | SWARM-13 (part) | Med | M | `systems/ddt.py` — 7-9 sites | Migrate; BF-04 already converts the Lagrangian blocks | Before READ-17…20 (Wave D same-file) | +| WB-03 | briefing inventory | Med | S | `utilities/adaptivity.py` — 4 sites | Migrate | | +| WB-04 | briefing inventory | Low | S | `pyx` — 7 deprecated-pattern refs | Verify: most are commented one-liners (delete in WA-07); migrate any live ones as comment-safe edits only | pyx rule applies | +| WB-05 | briefing inventory | Med | M | internal `mesh.data` refs (~14) | Migrate to `mesh.X.coords` | | +| WB-06 | SWARM-13 (part) | Low | S | `swarm.py:4317-4456` | Delete `_legacy_access` (zero callers) once WB-01 lands | | + +## Wave C — API harmonization (shims only; zero-cost when unused) + +Every shim lands with the two-test pattern (old signature = identical result + +exactly one DeprecationWarning; new signature = zero warnings). Conventions +C2–C9 from `API-CONSISTENCY-REVIEW.md` are adopted; **C1 is superseded by the +maintainer decision of 2026-07-04** (recorded in `UW3_STYLE_CHARTER.md` §6): the +ORIGINAL value-first order `add__bc(value, boundary, ...)` is canonical, and +the NEWER boundary-first methods migrate to it. + +| # | Finding | Sev | Eff | Action | Deps / notes | +|---|---------|-----|-----|--------|--------------| +| WC-01 | API-01 | High | M | Value-first arg order (per D2, decided 2026-07-04): migrate `add_nitsche_bc`/`add_rotated_freeslip_bc`/`add_constraint_bc` to `(value, boundary, ...)` with shims for their current boundary-first signatures; legacy trio already conforms | **[D2 DECIDED]** shim kept indefinitely pending contrary ruling; WE-09 sweep now targets only the newer methods' call sites | +| WC-02 | API-02 | Med | S | ONE datum name (`value`) with `conds=`/`g=` aliases; ONE direction convention; finish `components=` deprecation | **[D3]** name choice; same edit as WC-01 | +| WC-03 | API-04 | Med | S | `consistent_jacobian` → validated property `{False, True, "continuation"}`, falsy→False, else ValueError; NumPy docstring from the `pyx:71-91` comment | Bug-fix-flavoured (invalid values currently silently select Newton) | +| WC-04 | API-05 | Med | S | `SolverBaseClass.set_custom_fmg(...)` method (lazy-import pattern); deprecate `set_custom_mg`; unify on `builder=` | Coordinate with `bugfix/custom-mg-parallel` (BT); np2/np4 | +| WC-05 | API-06 | Med | S | Export `rotated_bc`/`boundary_flux`/`custom_mg` from `utilities/__init__`; add `BoundingSurface` + `register_*_surfaces` to `meshing/__init__`/`__all__` | Pure additions | +| WC-06 | API-07 | Med | S | `uw.quantity` = THE factory; `create_quantity` warns one cycle; expose `uw.UWQuantity`; update `test_0640` same PR | **[D14]** | +| WC-07 | API-08 | Med | S | `SNES_Poisson.__init__` reordered to `(mesh, u_Field, degree, verbose)` with `type(third) is bool` legacy shim + positional regression test | | +| WC-08 | API-03 | Med | S | Align `SNES_Vector.add_nitsche_bc` signature with the Stokes variant (`normal=`; `mask=` → clear NotImplementedError if unsupported) | Signature/docs only | +| WC-09 | API-09 | Med | S | Rename free function `boundary_flux_to_field` → `boundary_flux_field` (alias one cycle); document `scale = -1/buoyancy_scale` relationship — do NOT alias the parameters | | +| WC-10 | API-10 | Low | S | `SNES_Vector` `u_Field=None` contract: auto-create like `SNES_Scalar` or raise at construction | Verify first; pairs with READ-22 | +| WC-11 | API-11 | Low | M | `units=` mesh-constructor kwarg: settle live-vs-deprecated, fix self-contradictory docstrings, thread through remaining constructors if live | **[D7]**; can ride a units wave | +| WC-12 | SWARM-22 (kwarg part) | Low | S | Deprecate the no-op `sync=` kwarg on the four pack/unpack methods via keyword shim | With WA-17 | +| WC-13 | READ-65 | Low | S | Drop dead `relax`/`step_frac` params; `n_sweeps` → `max_cg_iters` with one deprecation cycle | With D-smoothing group | + +## Wave D — readability rewrites (grouped by file; one PR per group) + +Rule of thumb: pure code motion / rename / dedent, mechanically verified +(bit-identical outputs where the source review specifies). `D-doc` rows are +comment/docstring-only. All unverified rows re-verify at their remediation base. + +### D-smoothing (`meshing/smoothing.py`) — after BF-09; rescue elliptic-ma EXTRACT first + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-01 | READ-02 | High | M | Delete the ~100-line inline duplicate of `_build_M_tensor()`; call the closure once pre-loop | +| D-02 | READ-05 | Med | M | Extract `_backtracked_move(area_floor=0.0)` + `_cap_step_to_edge_fraction` (×3 / ×2 copies; default keeps older movers bit-identical) | +| D-03 | READ-06 | Med | M | Rename the four mis-prefixed `_winslow_*` movers (keep `_winslow_anisotropic`); sweep scripts/docs or alias one cycle | +| D-04 | READ-07 | Med | M | Module-top `_MPI` import + `_global_min/max/sum/mean` helpers (kills ~19 inline imports / 43 allreduce sites) | +| D-05 | READ-11 | Med | S | Accept `resolution_ratio=None` explicitly, then remove `**_ignored` and warn on unknown kwargs | +| D-06 | READ-08 | Med | S | D-doc: rewrite the stale module docstring (one paragraph per mover; mmpde recommended-not-default) | +| D-07 | READ-09 | Med | S | D-doc: delete dead amp-inversion lines; document the envelope branch; add `'arc-length'` to option lists | +| D-08 | READ-10 | Med | S | D-doc: fix `smooth_mesh_interior` method lists to include `mmpde` | +| D-09 | READ-12 | Med | S | D-doc: document the real 5-key `mesh_metric_mismatch` return dict | +| D-10 | READ-66 | Low | S | `_warm_start_krylov` + `_solver_wiring` helpers; rename `_zig` | +| D-11 | READ-67 | Low | S | Extract duplicated radial/tangential displacement reweighting | +| D-12 | READ-69 | Low | S | Extract `_mean_edge_length` (×2) | +| D-13 | READ-04 | Med | L | LAST: split into `meshing/smoothing/` package; MUST re-export the cross-module private names (`_edge_pairs`, `_tri_cells`, `_pinned_mask`, …) | + +### D-pyx-safe (naming/docs/dead-code only, per campaign rule) — after yield-homotopy lands + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-14 | READ-13 | Med | S | Rename `_maybe_install_snes_update` (banned hedging prefix) — but see BT-02: the unmerged `feature/snes-update-callbacks` tip already does this rename; cherry-pick instead of re-doing | +| D-15 | READ-22 | Med | S | Explicit if/else for `u_Field=None` in Scalar/Vector constructors (pairs with WC-10) | +| D-16 | READ-29 | Med | S | Dedent the byte-identical 12-line Picard/else solve tail to run once; fix the misleading comment | +| D-17 | READ-71 | Low | S | Fix F1-guard error message (says F0) | +| D-18 | READ-72 | Low | S | One class-level SNES convergence-reason table; both consumers format from it | +| D-19 | READ-73 | Low | S | One `_nondimensional_time` helper (×6 copies) | +| D-20 | READ-76 | Low | S | Rename `dim`→`cdim` locals; fix stale line-number comment | +| D-21 | READ-74 | Low | S | D-doc: delete or `TODO(DESIGN):`-ify unresolved editorial musings | +| D-22 | READ-30 (comment part) | Med | S | D-doc: comment the hardcoded `snes_max_it = 50` clobber now; behaviour change deferred to D2 **[D18]** | + +### D2 — benchmarked `.pyx` structural sub-wave (NOT Wave-D-safe; own benchmark protocol) **[D10]** + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-23 | READ-03 | High | L | Extract four-way-duplicated setup/BC/solve helpers, **parameterizing the 2-vs-2 essential-BC label divergence** (`str(boundary)` vs `"UW_Boundaries"`) unless [D10] unifies it deliberately | +| D-24 | READ-25 | Med | M | Extract explicit-index Jacobian-construction helpers (Vector/MultiComponent) | +| D-25 | READ-26 | Med | M | Extract shared bd-residual/jacobian wiring | +| D-26 | READ-27 | Med | M | Extract `_gather_state_for_residual` / `_split_local_residual_by_field` | +| D-27 | READ-28 | Med | S | Align MultiComponent's missing `_current_jit_cache_key` with siblings (document first) | +| D-28 | READ-75 | Low | M | Extract monitor-toggle / GAMG-defaults option bundles (×4 / ×3) | + +### D-mesh (`discretisation/discretisation_mesh.py`) + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-29 | READ-15 | High | L | Split the ~830-line `Mesh.__init__` into 8 named private methods (pure code motion) | +| D-30 | READ-16 | High | M | One module-level coords-update callback (the `__init__` version) used at all 3 diverged sites — coordinate with the `bugfix/deform-cache-invalidation` EXTRACT (`_mesh_version` gap, same territory) | +| D-31 | READ-36 | Med | M | Extract `_surviving_labels` with the safe getValueIS-first idiom (one copy uses the hard-abort-prone pattern) | +| D-32 | READ-37 | Med | S | Delete the stale inline KDTree vertex map; call the already-fixed `_build_vertex_map()` | +| D-33 | READ-38 | Med | M | Extract shared teardown/reinit/invalidate helpers for `_re_extract_from_parent` vs `adapt()` | +| D-34 | READ-39 | Med | M | Extract `_facet_outward_unit_normal` (×2 four-way dispatch) | +| D-35 | READ-40 | Med | M | `view()`: delete dead gathers; extract variable/boundary table printers | +| D-36 | READ-35 | Med | S | Collapse the duplicated length-scale try/except; narrow the bare excepts | +| D-37 | READ-41 | Med | S | Fix `all_edges_IS_dm` NameError-risk + misnaming | +| D-38 | READ-80 | Low | M | Rename the single-letter locals in `quality()` | +| D-39 | READ-79 | Low | S | De-triplicate the `_test_if_points_in_cells_internal` loop | +| D-40 | READ-82 | Low | S | Fix un-imported `Dict` annotation | +| D-41 | READ-83 | Low | S | Fix inverted no-op guards | +| D-42 | READ-84 | Low | S | Drop the forward-compat `level` param (or state "always 0 today") | +| D-43 | READ-42 | Med | S | D-doc: replace the UW2 `_legacy_access` docstring example | +| D-44 | READ-78 | Low | S | D-doc: fix the self-contradictory `tol > 0` docstring | +| D-45 | READ-81 | Low | S | D-doc: state the sanctioned reason at each `except Exception: pass` in the refresh paths | + +### D-ddt (`systems/ddt.py`) — after WB-02 + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-46 | READ-17 | High | L | Decompose the 515-line `update_pre_solve` into intention-named helpers | +| D-47 | READ-18 | High | M | One `_to_nondim_ndarray` helper for the ×6 unwrap dance | +| D-48 | READ-19 | High | M | Extract `_velocity_nd_at` for the near-identical node/midpoint velocity blocks | +| D-49 | READ-20 | High | L | Shared `_DDtBase`/module helpers for the ~250-line quintuplicated boilerplate | +| D-50 | READ-45 | Med | S | D-doc + one-line change: delete stale theta comment; pass `self.theta` on restore **[D9]** | +| D-51 | READ-47 | Med | S | Replace nested bare-except dispatch with explicit branch + narrowed except | +| D-52 | READ-87 | Low | S | Rename the leaked-loop-index work-variable name | +| D-53 | READ-90 | Low | S | Hoist redundant inline imports | +| D-54 | READ-92 | Low | S | Narrow the `register_remesh_hook` excepts + one-line comments | +| D-55 | READ-89 | Low | S | D-doc: replace drifted line-number comments with method names | +| D-56 | READ-91 | Low | S | D-doc: document (or remove) `Symbolic`'s interface-parity-only params | + +### D-solvers (`systems/solvers.py`) + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-57 | READ-48 | Med | S | Rename `_maybe_install_auto_gauge` (banned prefix) | +| D-58 | READ-49 | Med | M | `_SmoothingLengthMixin` for the triplicated properties | +| D-59 | READ-50 | Med | M | Extract `_nondimensionalise_timestep`; narrow the bare `except: pass` | +| D-60 | READ-51 | Med | M | Extract `_global_max_diffusivity` / `_centroid_velocities_nd` (×4 / ×3) | +| D-61 | READ-52 | Med | M | Collapse `_apply_unit_aware_scaling` to a single path; specific exceptions | +| D-62 | READ-53 | Med | S | `projection_problem_description` double-counts smoothing/penalty: grep-first delete, else deprecate + fix assignment **[D9]** | +| D-63 | READ-54 | Med | M | Extract `_invalidate_solution_cache`; consider a transient-solve template | +| D-64 | READ-96 | Low | S | `expression` lambda → def with docstring | +| D-65 | READ-97 | Low | S | Initialise `_prev_effective_order` in `__init__` | +| D-66 | READ-98 | Low | S | Delete shadow imports | +| D-67 | READ-99 | Low | S | Define `CM_is_setup` once on the base; handle the pre-assignment case | +| D-68 | READ-94 | Low | S | D-doc: document `percentile` in `estimate_dt` | + +### D-rotated_bc (`utilities/rotated_bc.py`) + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-69 | READ-56 | Med | S | Call `_zero_rows_local` at its three hand-inlined sites | +| D-70 | READ-57 | Med | S | Extract `_boundary_spec` normalizer | +| D-71 | READ-58 | Med | M | Extract the line search from the 177-line nonlinear solve; group destroys | +| D-72 | READ-55 | Med | S | D-doc: rewrite the stale "Development version / productizes prototypes" opening | +| D-73 | READ-100 | Low | S | Module constants for field ids | +| D-74 | READ-101 | Low | S | Move pressure-datum search into the LU branch; `TODO(BUG)` the parallel-unsafety | +| D-75 | READ-102 | Low | S | Plain sympy import (drop impossible-state try/except) | +| D-76 | READ-104 | Low | M | Split semicolon-chained PETSc lifecycle lines | +| D-77 | READ-105 | Low | S | One module-top `mpi` import | +| D-78 | READ-103 | Low | S | D-doc: rename `info`→`solve_result`; document return keys | + +### D-custom_mg (`utilities/custom_mg.py`) — wait for `bugfix/custom-mg-parallel` landing + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-79 | READ-59 | Med | M | `LevelLayout` NamedTuple replacing magic `lay[3]` indices | +| D-80 | READ-106 | Low | S | Extract `_clone_dm_with_solver_discretisation` | +| D-81 | READ-107 | Low | S | Extract the serial zero-column guard beside the parallel one | +| D-82 | READ-108 | Low | S | Narrow the bare except around `getNumFields()` | +| D-83 | READ-109 | Low | S | Rename terse parallel-path parameters to match serial vocabulary | +| D-84 | READ-111 | Low | S | Split semicolon-packed lines | +| D-85 | READ-113 | Low | S | Rename `sub` prefix-string collision | +| D-86 | READ-110 | Low | S | D-doc: comment the RBF r=0 clamp | +| D-87 | READ-112 | Low | S | D-doc: lifespan marker on the legacy finest-only path | + +### D-remesh (`discretisation/remesh.py`) + +| # | Finding | Sev | Eff | Action | +|---|---------|-----|-----|--------| +| D-88 | READ-62 | Med | S | Hoist `REMESH_MONOTONE` env knob to a documented module constant | +| D-89 | READ-63 | Med | M | One `_write_var_data` helper naming the sanctioned swallow (×4 copies) | +| D-90 | READ-114 | Low | S | Rename `_snapshot_remap_data` → `_snapshot_var_data` | +| D-91 | READ-116 | Low | S | Rename `_remap_var_set` to its public alias; delete the alias line | +| D-92 | READ-117 | Low | S | Drop redundant comprehension tail; enumerate the two scratch-key conventions | +| D-93 | READ-64 | Med | S | D-doc: fix the wrong re-entrancy comment | +| D-94 | READ-118 | Low | S | D-doc: state the best-effort `on_remesh` hook contract (or decide to warn unconditionally) | + +## Wave E — docs alignment (ordering matters: WE-01 → WE-02 → WE-03 → WE-04) + +| # | Finding | Sev | Eff | Action | Deps | +|---|---------|-----|-----|--------|------| +| WE-01 | DOC-04 | Med | S | Adopt the one-governing-doc-per-topic authority map; repoint `CLAUDE.md:324`; record the table in `docs/developer/index.md` | First | +| WE-02 | DOC-01 | High | M | Rewrite the Style Guide's four stale normative sections (NumPy/RST docstrings, MyST `.md`, `mesh.X.coords`, drop Quarto front matter); also fix the non-running `swarm.data += …` "Preferred" example (SWARM-13 evidence) | After WE-01 | +| WE-03 | DOC-02 | High | S | Regenerate `review_queue.md` via `scripts/docstring_sweep.py`; add to release checklist | Before WE-04 | +| WE-04 | DOC-05 | Med | M | Docstring wave on the *verified* gaps: `uw.function.evaluate` params, `Swarm.advection` ×2, the checkpoint trio | After WE-03 | +| WE-05 | DOC-03 | Med | M | Backfill the changelog "May – early July" (~10-14 grouped entries); add the release-checklist sweep | | +| WE-06 | DOC-07 | Med | M | Status headers on the ~16 unmarked design docs (per-doc git verification before stamping) | | +| WE-07 | DOC-08 | Low | S | Fix `mesh-adaptation.md:247` + the `mesh.adapt()` docstring Examples (deprecated patterns) | | +| WE-08 | API-12 | Low | S | Convert `units.py` public docstrings Google→NumPy (`docstring_sweep.py` assists) | | +| WE-09 | API-01/02 sweep | Med | M | Update call sites of the NEWER methods (nitsche/rotated/constraint) to value-first order; the ~1,370 legacy-order sites already conform (per D2 decision) | After WC-01/02 **[D2 DECIDED]** | +| WE-10 | LE-23 | Low | S | Update the `disk_snapshot.py` "Phase 1 (this commit)" header (phases 2/3 shipped) | | +| WE-11 | SWARM-24 | Med | S | Interim: banner on the misleading 26-line `swarm-system.md` stub pointing at the swarm review; real doc is FO-01's deliverable | | + +## Branch-track (parallel; per the ledger's execution protocol) + +| # | Action | Scope | Deps / notes | +|---|--------|-------|--------------| +| BT-01 | **Safety pushes first** (pushing is not "touching") | Unpushed locally-unique commits: `bugfix/custom-mg-parallel`, `feature/adaptive-convection` (13 ahead of origin), `feature/adapt-on-top` (no remote), `bugfix/yield-homotopy` (local-only) | Immediate; zero risk | +| BT-02 | **EXTRACT rescues** (11 branches) — commit/PR the named unique content before anything is deleted | blog-posts revision+figures; cetz-figures figure set; elliptic-ma `mover=`/allgather fix/scripts; exp-integrator-freesurface paper draft; fault-convection reimplementation spec; gradient-plasticity spike (commit-to-preserve, do NOT PR); petsc-cell-hint tip + build-petsc.sh fix; snes-update-callbacks tip `b82acea7` (cherry-pick — also resolves READ-13/D-14); vep-two-stokes `ViscoPlasticExplicitElastic` + post-mortem; deform-cache-invalidation `_mesh_version` gap (rework as issue/small PR — feeds D-30); fault-system-workflow H2Ex examples (onto product-system) | **[D15]**; blocks deletion batches | +| BT-03 | **LAND PRs** (4) | `feature/adapt-on-top`; `bugfix/yield-homotopy` (expect pyx/constitutive conflicts — land before Wave A/D pyx edits); `worktree-product-system` (commit dirty polish, fold adaptive-convection's `_run.py` fix, coordinate workflows-package ownership); `feature/parallel-point-eval` tip | **[D15]** landing order/ownership | +| BT-04 | **Deletion batches** with archive tags (`git tag archive/` + push tags), re-verifying merge-ancestry + clean worktree per entry | (a) bulk 40 REMOVE_WORKTREE_ONLY; (b) investigated 8 REMOVE_WORKTREE_ONLY; (c) ARCHIVE_DELETE risk-none (15); (d) ARCHIVE_DELETE risk-low; (e) EXTRACT branches after BT-02 verified on development | **[D15]** per-batch sign-off; never key on worktree dir names (6 misnamed) | +| BT-05 | **Follow-up triage** of the two un-triaged items | `bugfix/rotated-freeslip-schur-pc` (local, clean) and `origin/bugfix/stokes-constrained-parallel` (holds the OPEN release-blocking Item2, 0.4% 3D velocity) | **[D17]** | +| BT-06 | **KEEP_ACTIVE guard list** — exclude from all cleanup | `feature/numpy2-support`, `feature/quality-audit-2026-07`, `bugfix/custom-mg-parallel`, `feature/adaptive-convection` | Standing | + +## Follow-on (post-wave workstreams) + +| # | Item | Scope | +|---|------|-------| +| FO-01 | **Swarm modernization design doc + refactor** (this campaign's dimension-5 deliverable) | Self-validating canonical cache (SWARM-10, generation-stamp discipline), migration trigger matrix (SWARM-03/18), shared array-view refactor for BOTH swarm and mesh variables (SWARM-14, incl. unifying the three `_data_layout` copies and deleting dead `_array_cache`), rank-local RBF seam behaviour (SWARM-15), `_get_map` stale-cache trap (SWARM-23), KDTree copy-in-`__cinit__` question (SWARM-20), checkpoint/restore fidelity audit; replaces the SWARM-24 stub with a real subsystem doc | +| FO-02 | **SWARM-12 safe parts** (can ship early) | `sym_1d` staleness check + `np.add.at` vectorization (bit-identical guard); lazification of `IndexSwarmVariable._update()` stays BLOCKED until BF-08 lands | +| FO-03 | **Guardrails: UW3 Style Charter + mechanical CI gates** | no-`maybe_`/hedging-name lint; deprecated-pattern scanner over src+docs in CI; shim-warning tests as the Wave-B/C regression net; `docstring_sweep` + changelog sweep on the release checklist; black/format check promotion; commented-out-code and bare-`except: pass` review checklist items (READ cross-cutting patterns) | +| FO-04 | **Partition-seam natural-BC fix, option (a)** (LE-05) | Partition-independent manual boundary-load assembly; the `pyx:2545` TODO(BUG) record stays until then | +| FO-05 | **Issue #291 fix** (BF-19) | Stokes_Constrained np>1 interior-multiplier section reduction; then unskip `test_1017` + serial-only restrictions | +| FO-06 | **Stokes_Constrained Item2** (0.4% 3D velocity, gauge-independent) | Release-blocking; lives on `origin/bugfix/stokes-constrained-parallel` (BT-05) | +| FO-07 | **Verification pass service** | Adversarial verification of every unverified finding (LE-13…26, API-10…12, READ-15…118, SWARM-15…24) at each wave's remediation base, before its fix is applied | +| FO-08 | **Units-family feature work** (if [D7] decides "implement") | `coord_units` / quantity-coordinate evaluate / UnitAwareArray returns, replacing the placeholder tests deleted in WA-23/24 | + +## Decisions needed from Louis + +1. **persistence.py fate (LE-12 / WA-01)** — delete `discretisation/persistence.py` outright, or leave a one-release warn-on-import shim? (Either way the CLAUDE.md Key Files entry is updated.) +2. **BC argument-order migration (API-01 / WC-01, WE-09)** — **DECIDED 2026-07-04**: the ORIGINAL value-first order `(value, boundary)` is canonical (most used in examples/benchmarks; matches previous major versions). The newer boundary-first methods migrate to it with shims. Remaining sub-decision: confirm the canonical datum name (`conds` implied vs `value`) — see #3. +3. **Canonical BC value-parameter name (API-02 / WC-02)** — `value` (proposed, self-documenting) vs `g` (zero-churn alternative); shim mechanics are identical either way. +4. **Recycle/streak swarms (SWARM-08/09 / WA-02, WA-03)** — port the working recycle logic from the dead `swarms/pic_swarm.py` into `Swarm`, or excise `recycle_rate > 1` and its docstring claims; `pic_swarm.py` (1,534 never-installed lines) is deleted either way. +5. **NodalPointSwarm fate (SWARM-11 / BF-14)** — keep with a smoke test, or deprecate; it has zero remaining instantiation sites and a positional-argument bug. +6. **Swarm reduction return types (LE-07 / BF-11)** — approve the behaviour change aligning SwarmVariable array-view reductions to MeshVariable's per-component-tuple contract (callers relying on scalar returns from multi-component reductions will see tuples). +7. **Units-family scope decision (LE-08, LE-10, LE-11, API-11 / BF-12, WA-23, WA-24, WC-11)** — one decision for the family: implement quantity-valued-coordinate `evaluate()` and the `coord_units` feature set, or declare them unsupported, delete the ~16 placeholder skips, delete `test_0630`, rewrite `test_0620` to the implemented mesh-units semantics, and settle whether the Cartesian-only `units=` constructor kwarg is live or deprecated (its docstrings currently contradict each other). +8. **3D MMPDE (READ-01 / BF-09)** — implement `_signed_volumes` (restoring the docstring's d=3 claim) or raise `NotImplementedError` and make the docstring 2D-only. +9. **Two one-line behavioural corrections found by the readability audit (READ-45, READ-53 / D-50, D-62)** — (a) pass `self.theta` when restoring SemiLagrangian state instead of the hard-coded 0.5; (b) stop double-counting the smoothing/penalty terms in `SNES_Vector_Projection.projection_problem_description` (delete if unused). Both are flagged rather than folded silently into Wave D. +10. **Benchmarked `.pyx` structural sub-wave (READ-03/25/26/27/28/75, READ-30 behaviour part / D2)** — approve running it at all, and rule on the 2-vs-2 essential-BC label divergence (`str(boundary)` in Scalar/SaddlePt vs `"UW_Boundaries"` in Vector/MultiComponent): preserve it deliberately, or unify it — unification is a solver-behaviour change needing its own validation. +11. **Stale-proxy fix shape (LE-03 = SWARM-05 / BF-08, issue #215 Bug 3)** — approve a single eager proxy refresh at solve entry (the #216-consistent design), gated on the memory-leak test, rather than per-access refresh. +12. **Automatic migration on modern coordinate writes (SWARM-03 / BF-07)** — approve deferred/context-exit migration semantics (the class docstring already promises automatic migration; per-write migration is collective-deadlock-prone and is ruled out). +13. **pykdtree dependency removal (LE-22 / WA-26)** — packaging/environment change: remove the unused dependency whose OpenMP runtime can crash macOS beside PETSc. +14. **`uw.create_quantity` deprecation (API-07 / WC-06)** — approve `uw.quantity` as THE factory, `create_quantity` warned for one cycle, and `UWQuantity` exposed at top level (`test_0640` currently pins `create_quantity` as public). +15. **Branch-track sign-offs (BT-02/03/04)** — per the ledger's execution protocol: (a) approve the rescue plan for the 14 HIGH-risk rows (work existing in exactly one place); (b) set the landing order and ownership for `feature/adapt-on-top`, `bugfix/yield-homotopy`, and the workflows package (`worktree-product-system` vs `feature/adaptive-convection` carry deliberate duplicates — who lands it); (c) sign off each deletion batch (bulk 40, investigated 8, ARCHIVE_DELETE none/low, EXTRACT-after-rescue). +16. **Design-directory cleanup deletions (DOC-06 / WA-27)** — approve deleting the 6 `.trace.txt` solver logs outright and moving the 29 experiment scripts/PNGs to `design/experiments/exp-integrator/`. +17. **Release-blocker confirmation (LE-04/#291, Stokes_Constrained Item2 / BF-19, BT-05, FO-05/06)** — confirm both stay on the release-blocking list and assign owners; neither is fixable inside the cleanup waves. +18. **`snes_max_it = 50` clobber (READ-30 / D-22)** — the solve path silently overwrites user `petsc_options` each solve: approve changing it to respect user settings (behaviour change, D2 benchmarked sub-wave) or keep the behaviour and only document it. + +## Testing Instructions + +Per-wave gates (campaign ground rules) plus the finding-specific validation +recorded in each source review's own Testing Instructions section: + +- **Every wave**: `./uw build` in the wave worktree; `pytest -m "level_1 and tier_a"` + green pre/post; `pytest -m "tier_a or tier_b"` before merge. +- **Track 0 / anything touching swarm, migration, or partition-sensitive code**: + `mpirun -np 2` and `-np 4` parallel suites; each bug fix lands with the named + regression test from its source review (the unskip IS the regression test where + a skip masked the bug). +- **Wave A**: behaviour-neutral by construction — full tier_a identical pre/post; + `pyx` touches force `rm -rf build/` + rebuild (stale `build/src/*.c` trap); + confirm via `strings <.so> | grep ` where applicable. +- **Wave B**: `/check-patterns` over `src/` shows the migrated sites gone; + shim-warning tests (Wave C) later enforce that internal code no longer + exercises deprecated paths. +- **Wave C**: two-test shim pattern per item (old signature identical + exactly one + DeprecationWarning; new signature zero warnings under `simplefilter("error")`). +- **Wave D**: pure-motion claims verified mechanically (bit-identical outputs per + READABILITY-REVIEW Testing Instructions §2–8); D2 additionally requires the + benchmark protocol and bit-for-bit BC-behaviour comparison at np1/np2/np4. +- **Wave E**: `pixi run docs-build` clean; `/check-patterns` over `docs/`; + regenerated docstring queue sanity checks (DOC-02/05 cross-validation). +- **Branch-track**: re-verify `merge-base --is-ancestor` (or squash content-diff) + and clean `status --porcelain` per entry at execution time; archive tag before + every delete; push tags. + +## Known Limitations + +- Line numbers throughout are pinned to `development@1d003481`; every wave + re-verifies its anchors at its own base. Finding IDs are the stable reference. +- The majority of Wave D rows (READ-15…118) and several rows in other waves are + **unverified findings** — scheduled here as a triage queue, each gated on the + FO-07 verification pass before its fix is applied. +- Item counts double-assign nothing: where two dimensions found the same defect + the row lists all IDs and lives in one track only. +- The Branch-track verdicts drift with repository state; the ledger's execution + protocol (re-verification per batch) is mandatory, not advisory. +- Wave sizing is intentionally uneven: Wave D is ~94 rows but mostly S-effort + grouped by file into ~9 PRs; Wave B is 6 rows but its np-parallel gates make + it the riskiest cleanup wave. + +## Sign-Off + +| Role | Name | Date | Status | +|------|------|------|--------| +| Maintainer | Louis Moresi | 2026-07-05 | Pending review — 18 decision items above require explicit sign-off | +| Author | Claude (audit synthesis session) | 2026-07-03 | Complete | + +*Underworld development team with AI support from Claude Code.* \ No newline at end of file diff --git a/docs/reviews/2026-07/SWARM-SUBSYSTEM-REVIEW.md b/docs/reviews/2026-07/SWARM-SUBSYSTEM-REVIEW.md new file mode 100644 index 000000000..96485762f --- /dev/null +++ b/docs/reviews/2026-07/SWARM-SUBSYSTEM-REVIEW.md @@ -0,0 +1,517 @@ +# Swarm Subsystem Review — July 2026 Quality Campaign (Dimension 5) + +**Status**: audit complete; findings adversarially verified 2026-07-03 (where marked) +**Base**: `development` @ `1d003481` (audit worktree, campaign index at `e848d131`) +**Scope**: `src/underworld3/swarm.py` (4,996 lines), `src/underworld3/swarms/pic_swarm.py` +(1,534 lines, uninstalled), the swarm-facing parts of `src/underworld3/systems/ddt.py`, +`src/underworld3/ckdtree.pyx`, and `docs/developer/subsystems/swarm-system.md`. + +Every `file:line` cited below was read directly in this worktree; line numbers are +exact at `1d003481`. Findings SWARM-01 … SWARM-14 passed the full adversarial +verification pass (including two independent runtime reproductions in this +worktree's built environment); SWARM-15 … SWARM-24 had their evidence lines +personally read but did not go through the adversarial pass. Sub-claims refuted +during verification are recorded in the appendix so the same false leads are not +re-found. + +## Overview + +The swarm subsystem is the single densest cluster of correctness risk found by +this campaign, and the risk has one common shape: **caches and proxies that are +invalidated by call-site discipline rather than by self-validation**, in a module +where the discipline has already been broken at least four separate times. + +The subsystem's data pipeline (PETSc DMSwarm field → cached canonical NumPy copy +→ per-access array view → RBF-projected proxy MeshVariable → solver JIT) has +*five* hand-maintained consistency contracts: pack-to-PETSc on write, cache +invalidation on layout change, kd-tree invalidation on coordinate change, proxy +staleness on data change, and migration on coordinate change. Each contract has +at least one confirmed hole at this baseline: + +- `Swarm.migrate()` skips **all** invalidation when no particle changes rank — + the *common* case in serial and for small in-domain moves — leaving stale + canonical arrays, a stale kd-tree over a mutated buffer, and fresh-looking + proxies (SWARM-01/02; runtime-reproduced: after adding 2 particles, + `dm.getLocalSize()` = 398 but `var.data.shape[0]` = 396). +- Writing particle coordinates through the modern API (`swarm.coords` setter or + `_particle_coordinates.data`) never migrates at all, despite the class + docstring promising automatic migration (SWARM-03). +- Inside `migration_disabled()` — the context the docstrings *recommend* for + batch writes — variable writes are silently discarded before ever reaching + PETSc (SWARM-04), and the parallel test that covers this pattern + (`ptest_0755`) passes vacuously. +- Solvers that consume a proxy without touching `.sym` read stale proxy data — + the known issue #215, still open, `TODO(BUG)` at `swarm.py:1075` (SWARM-05). + +Beyond the cache/staleness cluster: user-configured `Lagrangian`/`Lagrangian_Swarm` +history terms crash on first use (runtime-reproduced AttributeError, SWARM-06); +empty MPI ranks get silent zeros or a hard crash from RBF proxy updates +(SWARM-07); and the advertised streak-swarm feature (`recycle_rate > 1`) is dead +at two independent `NameError`s (SWARM-08), with the only working copy of that +logic stranded in a 1,534-line module that is never installed (SWARM-09). + +The #216 fixes (commit `af537d56`) were correct but treated symptoms: they added +invalidation calls at three call sites while the underlying single-cache-no- +validation design remained. The highest-leverage remediation is architectural — +make the canonical cache self-validating the way mesh variables already are +(SWARM-10) — with the individual holes patched as fast, testable interim fixes. + +## Changes Made + +None — audit only. Proposed fixes are classified per finding below and feed the +campaign remediation waves: the SWARM-01/02/04 invalidation holes and SWARM-06 +crash belong to the bug-fix track; SWARM-08/09/11 dead code to Wave A; the 13 +internal `access()` sites (SWARM-13) to Wave B; SWARM-10/14 to the swarm +modernization design doc that is this dimension's follow-on deliverable. + +## Findings — verified (adversarial pass complete) + +| ID | Sev | Effort | Location | Finding | +|----|-----|--------|----------|---------| +| SWARM-01 | High | S | `swarm.py:3451` | `migrate()` early-returns when no particles are globally unclaimed *without* invalidating caches; `add_particles_with_global_coordinates` relies on migrate for invalidation → stale wrong-sized `.data` in serial (runtime-confirmed) | +| SWARM-02 | High | S | `swarm.py:3451` | Same early return leaves the cached kd-tree built over a mutated coordinate buffer → internally inconsistent NN queries feeding `rbf_interpolate` and proxy updates | +| SWARM-03 | High | M | `swarm.py:3114` | Coordinate writes via `swarm.coords` setter / `_particle_coordinates.data` never trigger migration, contradicting the class docstring → silent wrong-rank particles in parallel | +| SWARM-04 | High | M | `swarm.py:480` | Variable/coordinate writes inside `migration_disabled()` are silently discarded (callbacks early-return, nothing re-packs, invalidation destroys the only copy); `ptest_0755` asserts vacuously | +| SWARM-05 | High | M | `swarm.py:1075` | Stale-proxy hole (issue #215): `_update()` only sets `_proxy_stale`; refresh fires only via `.sym` — solvers reading the proxy DM directly consume stale data | +| SWARM-06 | High | M | `systems/ddt.py:3197` | `Lagrangian`/`Lagrangian_Swarm` history crashes with AttributeError on first update (runtime-confirmed): `psi_star_0[i, j].data` — `__getitem__` returns a sympy function with no `.data` | +| SWARM-07 | High | M | `swarm.py:1457` | Empty/starved ranks: `rbf_interpolate` silently writes zeros into proxies at ≤1 local particles; unguarded KDTree construction on 0 particles raises IndexError (empirically confirmed) → MPI abort/hang | +| SWARM-08 | Med | M | `swarm.py:3360` | Streak/recycle swarm (`recycle_rate > 1`, docstring-advertised) is dead: `populate()` NameError on `all_local_cells` (runtime-confirmed); `advection()` NameError on `cellid` (`getField` commented out at 4733, use at 4744). Zero tests | +| SWARM-09 | Med | S | `swarms/pic_swarm.py:22` | Entire 1,534-line module is dead: no `__init__.py`, never installed, broken relative import, sole importer commented out (`swarm.py:2450`); already diverging from `Swarm` (step_limit default, missing #216 fixes) | +| SWARM-10 | Med | M | `swarm.py:1546` | Architecture: `SwarmVariable.data` trusts the cached `_canonical_data` with no size/generation revalidation — the root cause behind #216 and SWARM-01; mesh variables already self-validate (`discretisation_mesh_variables.py:2721-2727`) | +| SWARM-11 | Med | S | `swarm.py:4885` | `NodalPointSwarm.__init__` passes `verbose` positionally into the `recycle_rate` slot (`Swarm.__init__` signature at 2523); class also has zero remaining instantiation sites (~130 lines dead public surface) | +| SWARM-12 | Med | M | `swarm.py:2176` | `IndexSwarmVariable` breaks the lazy-`_update()` contract (eager re-projection, special-cased in `_invalidate_canonical_data` at 2699-2707 to avoid an O(100 MiB) leak); `sym_1d` (2209-2226) returns `_MaskArray` with **no** staleness check. Eagerness is load-bearing — see Known Limitations | +| SWARM-13 | Med | L | `swarm.py:4458` | Deprecated-pattern debt: 13 live internal `with …access(…)` sites in `swarm.py` + 7 in `ddt.py`; `_legacy_access` (4317-4456) has zero callers; the style guide's "Preferred" swarm example doesn't run (getter-only property → AttributeError) | +| SWARM-14 | Med | L | `swarm.py:1592` | `.array` creates a fresh closure-defined view class + instance per access (~415 lines at 529-944; tensor reads do a full PETSc unpack copy each time); `_data_layout` exists in three near-copies (`swarm.py:983`, `swarm.py:4510` — "cut'n'pasted from the MeshVariable class" — `discretisation_mesh_variables.py:1654`). Mesh side shares the same design (see appendix, R-2) | + +### SWARM-01 — `migrate()` no-move early return skips cache invalidation + +`Swarm.migrate()` returns at `swarm.py:3451-3452` (`if global_unclaimed_points +== 0: return`) with the only `_invalidate_canonical_data()` call on the +fall-through path at 3525. `add_particles_with_global_coordinates` (def 3644) +calls `addNPoints` at 3706 — changing the DM local size — and its only +invalidation route is `if migrate: self.migrate(...)` at 3729-3730; with +`migrate=False` nothing ever invalidates. In serial (or whenever all points are +already claimed) the early return fires, so every SwarmVariable's cached `.data` +keeps the old particle count. Runtime-reproduced on this worktree's build: +`dm.getLocalSize()` = 398 vs `var.data.shape[0]` = 396 after adding 2 in-domain +points. New particles are invisible to reads; writes pack wrong-sized arrays. +The sibling `add_particles_with_coordinates` invalidates explicitly at 3635 +(part of #216), confirming the asymmetry — the same bug class as the three #216 +fixes, one path missed. + +**Fix (S)**: move `_invalidate_canonical_data()` ahead of the early return; add +an explicit invalidation in `add_particles_with_global_coordinates` after +`addNPoints` (mirroring 3635). Serial regression test. Subsumed longer-term by +SWARM-10. + +### SWARM-02 — same early return leaves a poisoned kd-tree + +Advection writes `_particle_coordinates.data` in place (4674/4705); the +coordinate variable is a plain SwarmVariable (2588) whose canonical-data +callback (472-501) only syncs to PETSc — nothing drops `swarm._kdtree` (cleared +only at 2710, inside `_invalidate_canonical_data`). `KDTree` (`ckdtree.pyx:103-104`) +stores a **no-copy memoryview** and builds its nanoflann index once, so after an +in-place coordinate mutation followed by a no-move `migrate()` (serial, or any +parallel step where no particle changes rank — the common case), the cached tree's +topology reflects old positions while its stored points show new ones: queries are +internally inconsistent, not merely "frozen". `_get_kdtree` (2712-2723) returns it +unconditionally; `rbf_interpolate` consumes it at 1476. Silently wrong NN +interpolation into every proxy. + +**Fix (S)**: same invalidation move as SWARM-01 (at minimum `self._kdtree = None` ++ proxy-stale marking before the early return). Regression test: build tree, +advect in-domain, assert `rbf_interpolate` reflects new positions. + +### SWARM-03 — modern coordinate writes never migrate + +The `Swarm` docstring (2510-2513) promises automatic migration when particles +move. Reality: `swarm.coords` setter (3114) routes to +`canonical_data_callback` (472-505) → `pack_raw_data_to_petsc` (1306-1349), +which packs the field and updates proxies but never migrates (its `sync=True` +branch is literally `pass # TODO`). Only the *deprecated* `points` property +migrates (2948-2951); the `access()` shim (4458+) drops the migrate-on-exit that +`_legacy_access` performed (4427-4430); internal advection compensates with an +explicit `self.migrate()` at 4806. In parallel, user code that moves particles +through the supported API leaves them on the wrong rank — silent corruption. +Nuance: `migration_control()` (deferred mode) and `dont_clip_to_mesh()` do +migrate on exit (3275-3280 / 3209), but plain direct writes do not. + +**Fix (M)**: post-write migrate hook on the coordinate variable — but +**deferred/context-exit form**, not per-write: `migrate()` is collective, so a +hook firing from a write callback risks MPI deadlock when ranks write unevenly. +Extend `migration_control` semantics into the coords setter and the `access()` +shim. Add an np2 test moving particles across the partition via +`._particle_coordinates.data`. + +### SWARM-04 — writes inside `migration_disabled()` silently discarded + +Both sync callbacks early-return while `_migration_disabled` is set +(`swarm.py:429-430`, `480-481`); every user write path funnels through them +(`.data` → canonical NDArray_With_Callback 1546-1550; `.array` setter → 1607; +`points` in-place → 2946). Nothing re-packs afterwards. `migrate()` reads +coordinates from the PETSc field (3428) — i.e. the *stale* ones — and its +trailing `_invalidate_canonical_data()` (3525) nulls the canonical copy, +destroying the only record of the writes. The docstrings (~3220-3253) explicitly +recommend writing inside the context; `tests/parallel/test_0755_swarm_global_stats.py:322-329` +does exactly that and asserts vacuously (its perturbation never reaches the +DMSwarm). Note: outright data destruction requires migrate to proceed past the +SWARM-01 early return; with stale in-domain coords the canonical copy survives +but PETSc silently never sees the writes — wrong either way. + +**Fix (M)**: separate "suppress migration" from "suppress PETSc sync" — pack +while the flag is set, gate only the `migrate()` call; or accumulate a dirty +flag and flush pending canonical arrays in `_MigrationControlContext.__exit__` +before migrating. Fix `ptest_0755` to actually assert the values survive. + +### SWARM-05 — stale proxy consumed by solvers (issue #215) + +`_update()` (1060-1073) only sets `_proxy_stale`; `_update_proxy_if_stale()` is +invoked solely from `.sym`/`.sym_1d` (1627/1643; IndexSwarmVariable inline at +2203-2205) — a repo-wide grep finds no other callers. The solve path has no +enforcement: `mesh.update_lvec()` (`discretisation_mesh.py:2924-2960`) pulls the +proxy's vec directly into assembly; `_jitextension.py` has zero swarm/proxy +handling; `Projection` captures `uw_function` as a sympy Matrix at construction +(`solvers.py:2636/2667`) so `solve()` never re-touches `.sym`. Write +`material.data` → solve → stale proxy data. `write_proxy()` (1960-1966) has the +same hole. The unresolved `TODO(BUG)` at 1075-1080 documents it. + +**Fix (M)**: enforce freshness at consumption — pre-solve/JIT walk for +swarm-proxy UnderworldFunctions calling `_update_proxy_if_stale()`, and/or have +the proxy MeshVariable's `.data`/vec access consult the owner's `_proxy_stale`. +Any hook must respect the `_updating_proxy` reentrancy guard +(`_rbf_to_meshVar` writes `meshVar.data[...]`, 1132). Regression test: write +`material.data`, solve a Projection without touching `.sym`, assert freshness. + +### SWARM-06 — Lagrangian ddt history terms crash on first update + +`ddt.py:3197, 3281, 3549, 3638-3639` all do `psi_star_0[i, j].data[:] = ...` +where `psi_star_0` is a SwarmVariable. `MathematicalMixin.__getitem__` +(`mathematical_mixin.py:74-109`) returns `sym[index]` — an +UnderworldAppliedFunction (`_function.pyx:51`) with no `.data`. Runtime-confirmed: +`Lagrangian.initialise_history()` raises AttributeError at exactly `ddt.py:3197`. +The default `DuDt` is SemiLagrangian, so only user-configured +`Lagrangian`/`Lagrangian_Swarm` histories hit it — but those are advertised +(`systems/solver_template.py:92-93`, `solvers.py:196-197`). Coverage gap: the +only test (`test_0007`) round-trips `.state` without ever advecting. The blocks +also use deprecated `swarm.access()` / `swarm.data` internally. + +**Fix (M)**: rewrite the component update via the modern interface +(`psi_star_0.data[:, psi_star_0._data_layout(i, j)]`, coordinates from +`swarm._particle_coordinates.data`); add a minimal advecting-history test for +both classes. + +### SWARM-07 — empty ranks: silent zeros or hard crash + +`rbf_interpolate` (`swarm.py:1457-1458`) returns `np.zeros(...)` whenever the +rank holds ≤1 particles; `_rbf_to_meshVar` (1130-1132) writes that straight into +the proxy — silent wrong nodal values on starved ranks, no warning. Two verbatim +TODOs ("some form of global fall-back…", 1110 and 1148) confirm the known gap. +Separately, `IndexSwarmVariable._update_proxy_variables` reaches unguarded +KDTree construction (2374 via `_get_kdtree`; direct at 2409), and +`KDTree(np.zeros((0,2)))` raises `IndexError` (empirically confirmed; +boundscheck is on — a clean crash, not memory corruption, see appendix R-1) — +one rank raising during a collective proxy update means MPI abort/hang. + +**Fix (M)**: explicit empty-rank strategy — guard tree construction on +`local_size == 0`; for starved ranks either leave the proxy untouched + warn or +implement the documented global fallback. At minimum, warn loudly instead of +writing zeros. + +### SWARM-08 / SWARM-09 — dead recycle feature and its stranded working copy + +`populate()`'s `recycle_rate > 1` branch (starts 3338) uses `all_local_cells` +(3360) — the name occurs exactly once in the file, at its use site → NameError +for any recycle swarm (runtime-confirmed). `advection()`'s cycle block indexes +`cellid[swarm_size::]` (4744) while the only binding is commented out (4733). +The feature is docstring-advertised (2483, streak example 2502-2503), +propagated by `adaptivity.py:710`, and has zero tests. The only *working* copy +of the recycle logic lives in `swarms/pic_swarm.py` — which is never installed +(no `__init__.py`; `find_packages()` excludes it; confirmed absent from +site-packages), has a broken import (`from .swarm` at line 22 — no such sibling), +and is already divergent (step_limit default True at 1086 vs False at +`swarm.py:4567`; #216 invalidation fixes absent; stray `print("Peace")` at 1169). + +**Fix (M/S)**: decide once — either port the working recycle implementation into +`Swarm` (using `mesh.get_closest_local_cells()` instead of `DMSwarm_cellid`, +which doesn't exist on a BASIC swarm) and add a `level_1` streak test, or excise +`recycle_rate` machinery and the docstring claim. Either way delete +`swarms/pic_swarm.py` (git history preserves it) and the breadcrumbs at +`swarm.py:70, 2449-2455`. Do not leave it advertised and crashing. + +### SWARM-10 — the architectural root: no self-validating cache + +`SwarmVariable.data` (1546-1550) returns the cached `_canonical_data` on a bare +existence check. The mesh side already solved this: `_BaseMeshVariable.data` +self-validates against `id(self._lvec)` (`discretisation_mesh_variables.py:2721-2727`, +documented in `docs/developer/subsystems/data-access.md:131-150`). The swarm +equivalent — compare cached row count against `dm.getLocalSize()` plus a +`_population_generation` stamp (the counter exists at 2555 and is already bumped +at 3382, 3416, 3639, 3710, 4273, 4731; its own comment invites this use) — would +have prevented #216 and SWARM-01 by construction. Honest caveat: it cannot make +`_invalidate_canonical_data()` purely optional — same-local-size migrations swap +equal counts between ranks with values changed, and bare `dm.migrate` sites +(e.g. `_route_by_nearest_centroid`, 2754-2756) don't bump the generation — so +the refactor reduces the discipline to "bump the generation at every +layout-mutating site" rather than eliminating it. Still the single +highest-leverage swarm-modernization refactor. + +### SWARM-11 — `NodalPointSwarm` positional-argument bug, dead class + +`swarm.py:4885`: `super().__init__(mesh, verbose, clip_to_mesh=False)` against +`Swarm.__init__(self, mesh, recycle_rate=0, verbose=False, clip_to_mesh=True)` +(2523) — `verbose` lands in `recycle_rate`: silently discarded, and a truthy +value enables (broken, per SWARM-08) recycling. Zero instantiation sites remain +(SemiLagrangian dropped its nodal-swarm cache, `ddt.py:1683`); the constructor +also uses field arrays after `restoreField`/around `migrate` (~4946-4952). +**Fix (S)**: keyword-explicit super call now; decide keep-with-smoke-test vs +deprecate in the modernization doc. + +### SWARM-12 — IndexSwarmVariable contract inconsistency + +Base `_update()` is lazy; `IndexSwarmVariable._update()` (2176-2182) eagerly +runs a pure-Python O(N × nnn) re-projection (2348-2446), forcing the +special-case in `_invalidate_canonical_data` (2699-2707, documenting the +O(100 MiB) leak it avoids). `sym_1d` (2209-2226) returns `_MaskArray` with no +staleness check despite being documented as an alias for `sym` (which checks, +2203-2206). **Safe fixes only** (see Known Limitations): route `sym_1d` through +the staleness check; vectorize the per-particle loop (`np.add.at`). Do **not** +lazify `_update()` until a pre-solve refresh hook (SWARM-05's fix) exists — +the eager calls after migration (2951, 3280, 4443; `adaptivity.py:818`) are the +only mechanism refreshing material masks in a time loop. + +### SWARM-13 / SWARM-14 — modernization debt (Wave B / design doc) + +Thirteen live internal `with …access(…)` sites in `swarm.py` (1161, 2381 — +which stacks mesh and swarm access — 3339, 3345, 3376, 3628, 3725, 4719, 4767, +4791, 4949, 4972, 4975) plus seven in `ddt.py`; `_legacy_access` (4317-4456) has +zero callers. The style guide's "Preferred" swarm example +(`UW3_Style_and_Patterns_Guide.md:206`) is `swarm.data += displacement` — which +raises AttributeError (`swarm.data` is getter-only; only `points` has a setter, +3022-3024): the documented replacement pattern cannot run at all. On `.array`: +both SwarmVariable (1592, view classes 529-944) and MeshVariable +(`discretisation_mesh_variables.py:2012`) rebuild closure-defined view classes +per access — a *shared-view* refactor workstream, not swarm-parity; tensor swarm +reads additionally pay a full PETSc unpack copy per access (1280-1295). Unify +the three `_data_layout` near-copies; remove the dead `_array_cache` +(`discretisation_mesh_variables.py:446, 1888` — assigned None, never read). + +## Findings — unverified (evidence read, no adversarial pass) + +| ID | Sev | Effort | Location | Finding | +|----|-----|--------|----------|---------| +| SWARM-15 | Med | L | `swarm.py:1476` | Proxy RBF interpolation is strictly rank-local (`rbf_interpolator_local`; no ghost-particle exchange / DMSwarmCollect anywhere) → proxy nodes near partition seams interpolate from one-sided neighbourhoods; results deterministic but np-count dependent | +| SWARM-16 | Med | M | `swarm.py:4649` | `advection` with `substeps > 1`: launch-point velocity is evaluated **locally** each substep but no migration happens between substeps (only at 4806) → particles that crossed a partition are evaluated on the wrong rank from substep 2 on; only the midpoint uses `global_evaluate` (4666) | +| SWARM-17 | Med | S | `swarm.py:3384` | `populate()` adds particles via `addNPoints` + direct field writes but never calls `_invalidate_canonical_data()` before returning — same stale-cache class as #216/SWARM-01 | +| SWARM-18 | Med | S | `swarm.py:3277` | `migration_control(disable=False).__exit__` silently skips the promised deferred migration if `local_size` changed inside the context; `add_particles_with_coordinates` (3632) does a raw `dm.migrate` that bypasses `_migration_disabled` entirely while `Swarm.migrate` honours it (3410) | +| SWARM-19 | Med | S | `swarm.py:3810` | `save()` writes different coordinate systems per IO path: parallel-HDF5 branch saves `_particle_coordinates.data` (model units, 3779) but the sequential fallback saves deprecated `self.points` (physically scaled, plus DeprecationWarning and possible migration side-effect) → checkpoints differ by the length scale depending on h5py MPI support | +| SWARM-20 | Med | S | `ckdtree.pyx:103` | `KDTree` stores a no-copy memoryview and indexes it once; any later in-place mutation of the source array leaves index and data inconsistent — every cached-tree consumer depends on perfect invalidation discipline (which SWARM-01/02 show is not upheld). Copying in `__cinit__` makes trees immutable-by-construction | +| SWARM-21 | Low | S | `swarm.py:4641` | Unconditional hot-path prints: `"Advection (2nd): …"` every substep from every rank (flush=True, not gated on `self.verbose`; contrast 4582); three more per substep on the order-1 path (4682-4700); `read_timestep` prints too (1988, 1994). Also dead signature params: `corrector`'s implementation is commented out (4598-4631) and `evalf` is referenced only inside that block | +| SWARM-22 | Low | S | `swarm.py:1235` | Dead API surface: the `sync=True` kwarg on all four pack/unpack methods is a no-op with four identical TODO stubs (1235, 1285, 1342, 1391) — DMSwarm fields are rank-local, nothing to sync; `use_legacy_array`/`use_enhanced_array` pass-through stubs (961-967, pinned only by test_0530); `_rbf_reduce_to_meshVar` (1136) zero callers; `old_data` "TESTING" leftover (1499); large commented-out blocks (1184-1200, 1242-1263) | +| SWARM-23 | Low | S | `swarm.py:4539` | `_get_map`/`_nnmapdict` cache keys nearest-particle indices only by coordinate hash and is never cleared by `migrate()`/`_invalidate_canonical_data()` (only by the dead legacy access manager, 4434) — a stale-cache trap if `_rbf_reduce_to_meshVar` is ever revived. `migrate()`'s `remove_sent_points` parameter is accepted but ignored (hardcoded True at 3477) | +| SWARM-24 | Low | M | `docs/developer/subsystems/swarm-system.md:7` | The subsystem's only architecture doc is a 26-line stub claiming "Well-Documented Subsystem … Priority Low … Integration with NDArray system complete" and citing a nonexistent `swarm/` module of "4,484 total lines" (reality: 4,996 + 1,534 dead) — it actively misdirects reviewers away from the subsystem this audit found most in need of attention | + +Two unverified submissions duplicated verified findings and are subsumed: +the recycle-NameError claim (into SWARM-08, adding the `adaptivity.py:710` +propagation detail) and the dead-`pic_swarm.py` claim (into SWARM-09, adding +the missing-#216-fixes and `print("Peace")` details). Two low-severity +duplicate submissions (debug prints; sync-TODO no-ops) are merged into +SWARM-21/22. + +## System Architecture + +What this dimension's survey established about the swarm subsystem, for the +maintainer: + +**The data pipeline.** A particle field lives in the PETSc DMSwarm. User access +goes through a single cached *canonical* NumPy copy per variable +(`NDArray_With_Callback`, created lazily at `swarm.py:439-505`): reads return +the cache; writes fire callbacks that pack back to PETSc +(`pack_raw_data_to_petsc`, 1306-1349) and mark the proxy stale. `.array` wraps +the same cache in a units/shape view built fresh on every access. Variables +with `proxy_degree > 0` own a proxy MeshVariable refreshed by rank-local RBF +interpolation over a cached kd-tree; solvers consume the proxy's DM directly. + +**The consistency model is discipline, not validation.** Unlike mesh variables +— whose `.data` self-validates against the live PETSc vec identity — the swarm +canonical cache, the kd-tree, and the proxy staleness flag are all invalidated +only where someone remembered to call `_invalidate_canonical_data()` (its own +docstring, 2683-2691, says bare `dm.migrate` callers must do so manually). The +verified findings are all instances of forgotten or unreachable invalidation: +the `migrate()` no-move early return (SWARM-01/02), `populate()` (SWARM-17), +and the `migration_disabled` write-discard (SWARM-04). A +`_population_generation` counter exists and is bumped at most mutation sites +but is consumed by nothing — the self-validating design (SWARM-10) is half +built. + +**Migration is opt-in in practice.** Despite the docstring's promise of +automatic migration, the trigger matrix at this baseline is: deprecated +`points` writes → migrate; `advection()` → explicit internal migrate; +`migration_control()` / `dont_clip_to_mesh()` → migrate on exit (the former +only if local size is unchanged); modern `coords` / `_particle_coordinates.data` +writes and the `access()` shim → **never**. The one route that migrates +automatically is the one being deprecated. + +**Proxies are lazy on the read side only.** Data writes reliably mark proxies +stale, but refresh happens only via `.sym` — nothing on the solve path enforces +it (issue #215). `IndexSwarmVariable` bypasses laziness entirely with an eager, +pure-Python re-projection that is simultaneously a performance problem, a +special case in the invalidation path, and — because nothing else refreshes +material masks after migration — load-bearing. + +**Parallel edges are unhandled.** RBF proxy updates are strictly rank-local +(np-dependent seams, SWARM-15), empty ranks produce zeros or a crash +(SWARM-07), and multi-substep advection evaluates launch velocities locally +without inter-substep migration (SWARM-16). + +**Dead mass distorts the picture.** A never-installed 1,534-line near-duplicate +of `Swarm` (`swarms/pic_swarm.py`) holds the only working copy of the +advertised-but-broken recycle feature; `NodalPointSwarm` (~130 lines) has no +callers and a positional-argument bug; `_legacy_access`, `_rbf_reduce_to_meshVar`, +no-op `sync` kwargs, and dead signature parameters (`corrector`, `evalf`) pad +the API surface. The subsystem doc claims all of this is in good shape. + +## Testing Instructions + +Per the campaign ground rules, anything touching swarm/migration code gates on +`tier_a` green pre/post, `tier_a or tier_b` before merge, and np2/np4 parallel +tests. Run from inside the fix worktree after `./uw build`: + +```bash +pytest -m "level_1 and tier_a" tests/ # quick gate +pytest -m "tier_a or tier_b" tests/ # pre-merge gate +mpirun -np 2 pytest tests/parallel/ # swarm/migration changes +mpirun -np 4 pytest tests/parallel/ +``` + +Finding-specific validation (each fix PR should add the named regression test): + +- **SWARM-01/02/17 (stale cache/kdtree)** — serial test: create swarm + + variable, touch `var.data` (populate the cache), call + `add_particles_with_global_coordinates` (both `migrate=True` and `False`) / + `populate()`, assert `var.data.shape[0] == swarm.dm.getLocalSize()`; and: + build the kd-tree, advect particles in-domain, assert `rbf_interpolate` + reflects the new positions. +- **SWARM-03 (migration on write)** — np2 test: move particles across the + partition via `swarm._particle_coordinates.data`, then assert re-ownership + (each rank's particles inside its local domain; global count preserved). +- **SWARM-04 (`migration_disabled` writes)** — np2 test: write `var.data` + inside `migration_control()`/`migration_disabled()`, exit, migrate, assert + values survive. Repair `ptest_0755:322-329` so its perturbation demonstrably + reaches the DMSwarm (currently vacuous — its passing must not be taken as + coverage). +- **SWARM-05 (stale proxy)** — write `material.data`, run a `Projection` solve + without touching `.sym`, assert the solve consumed fresh values. +- **SWARM-06 (Lagrangian ddt)** — minimal advecting-history test for + `Lagrangian` and `Lagrangian_Swarm` (currently `test_0007` sets + `_history_initialised` manually and never advects — do not rely on it). +- **SWARM-07 (empty ranks)** — np4 test with all particles seeded in one + rank's subdomain; assert no zeros written into proxies on starved ranks and + no crash from `IndexSwarmVariable._update_proxy_variables`. +- **SWARM-08 (recycle)** — if repaired: `level_1` smoke test + `Swarm(recycle_rate=2)` → populate → advect. If excised: assert constructor + raises/warns on `recycle_rate > 1`. +- **SWARM-16 (substep advection)** — np2 trajectory test with + `step_limit=True` forcing substeps > 1 across a partition boundary, compared + against the np1 trajectory. +- **SWARM-19 (save)** — with coordinate scaling active, write via both IO + branches (`force_sequential=True`/`False`) and assert byte-identical particle + coordinates on read-back. + +Behavioral guards for fixes that must NOT change results: SWARM-12's +vectorization should be verified bit-identical against the Python loop on a +random material distribution; SWARM-21's print removal and SWARM-22's dead-API +deletions are behavior-neutral by construction (verify test_0530 is retired +together with the stubs it pins). + +## Known Limitations + +- **Line numbers are exact at `1d003481` only.** The campaign index commit + (`e848d131`) does not touch `src/`; `swarm.py` is byte-identical between the + two. +- **SWARM-15 … SWARM-24 are not adversarially verified.** Their evidence lines + were personally read, but no independent re-derivation or runtime + reproduction was performed; treat severity/mechanism as provisional until the + fix PR reproduces each. +- **SWARM-12's obvious fix is out of scope.** Lazifying + `IndexSwarmVariable._update()` would silently freeze material masks across + advection (the eager calls at `swarm.py:2951, 3280, 4443` and + `adaptivity.py:818` are the only refresh mechanism on the solve path today) — + a solver-numerics change. It becomes safe only *after* SWARM-05's pre-solve + staleness hook lands. Until then, only the `sym_1d` staleness check and the + loop vectorization should ship, and the `_invalidate_canonical_data` + special-case (2699-2707) must be kept. +- **SWARM-10 reduces but does not eliminate invalidation discipline.** + Same-local-size migrations defeat a size check, and bare `dm.migrate` sites + do not bump `_population_generation`; the generation bump must be pushed into + every layout-mutating site (or all migrate wrappers) as part of the refactor. +- **Fix-shape constraint for SWARM-03**: `migrate()` is collective. Any + automatic-migration fix must be deferred (context-exit / explicit flush), not + a per-write callback, or it will deadlock when ranks write coordinates + unevenly. +- **Severity inflation risk in serial-only workflows**: several High findings + (SWARM-03/04/07) manifest only under MPI; serial users see SWARM-01/02/06 + first. +- This review did not audit swarm **checkpoint/restore round-trip fidelity** + beyond SWARM-19, nor the `uw.Model` serialization of swarms; both belong to + the follow-on design doc. + +## Appendix — refuted claims (do not re-find) + +No whole finding was refuted, but four sub-claims failed adversarial +verification and are recorded so they are not resubmitted: + +- **R-1: "`ckdtree.pyx:104` performs an out-of-bounds memory read on an empty + points array."** Refuted: `setup.py` sets only `language_level`, so Cython's + default `boundscheck=True` applies; `&points[0][0]` on a `(0, dim)` buffer + raises a clean `IndexError: Out of bounds on buffer access (axis 0)` + (empirically confirmed). Consequence is a hard crash/MPI hang (SWARM-07), not + memory corruption or UB. +- **R-2: "SwarmVariable `.array` lags behind a cached single-view MeshVariable + architecture."** Refuted: `MeshVariable.array` + (`discretisation_mesh_variables.py:2012`) has the identical + fresh-closure-class-per-access design, and its `_array_cache` attribute + (446, 1888) is dead — assigned `None`, never read. The correct framing is a + *shared* view refactor on both sides (SWARM-14), not swarm-parity. +- **R-3: "IndexSwarmVariable's eager `_update()` is a mere contract violation; + make it lazy like the base."** Refuted as a safe fix: `createMask()` builds + solver expressions from `self._MaskArray` directly (2277-2278) without the + `.sym` refresh, and no solve-time code checks `_proxy_stale` — the eagerness + is currently the only thing keeping material masks fresh in a time loop (see + Known Limitations). +- **R-4: "A size+generation self-check makes `_invalidate_canonical_data()` a + fast-path optimization only."** Overstated: same-local-size migrations change + values without changing `dm.getLocalSize()`, and bare `dm.migrate` sites + (e.g. `_route_by_nearest_centroid`, 2754-2756) do not bump the generation + counter. The refactor (SWARM-10) narrows the discipline; it does not remove + it. +- Also corrected en route: the style guide's "Preferred" swarm example is worse + than "deprecated" — `swarm.data` is getter-only, so `swarm.data += + displacement` raises `AttributeError` and cannot run at all (folded into + SWARM-13). + +## Sign-Off +| Role | Name | Date | Status | +|------|------|------|--------| +| Maintainer | Louis Moresi | 2026-07-05 | Pending review | +| Author | Claude (audit session) | 2026-07-03 | Complete | + + + +- **Audit dimension**: 5 — Swarm/particle subsystem architecture +- **Reviewer**: Claude (Fable 5), AI-assisted audit under the July 2026 quality + campaign; adversarial verification pass completed 2026-07-03 for SWARM-01 … + SWARM-14 (including runtime reproductions of SWARM-01 and SWARM-06 in this + worktree's built environment). +- **Evidence standard**: every cited `file:line` read directly in the audit + worktree at `development@1d003481`; no line numbers trusted from prior notes + without re-reading. +- **Constraint compliance**: no proposed fix touches + `petsc_generic_snes_solvers.pyx` numerics; all fixes are Python-level cache/ + staleness/dead-code changes or additive tests; API-affecting items (SWARM-08 + excise option, SWARM-22 `sync` kwarg removal) are routed through Wave C + deprecation shims. +- **Follow-on deliverable**: swarm modernization design doc (self-validating + cache SWARM-10, migration trigger matrix SWARM-03/18, shared array-view + refactor SWARM-14, real subsystem doc replacing the SWARM-24 stub). + +*Underworld development team with AI support from Claude Code.* \ No newline at end of file