Add executor-backed and MPI batch evaluators

[hmgaudecker]

Stacked on #684 (the batch_evaluator kwarg). Review/merge #684 first; this PR's diff is against that branch.

What

Adds two batch evaluators so a single-driver optimizer can fan its batched criterion
evaluations across a concurrent.futures.Executor:

• executor_batch_evaluator(executor) — a generic factory returning a
  BatchEvaluator-protocol callable backed by any executor (ProcessPoolExecutor,
  ThreadPoolExecutor, Dask, mpi4py.futures.MPIPoolExecutor, …). Order-preserving via
  executor.map; reuses the existing unpack/catch machinery so error_handling
  ("raise" reraises, "continue" → traceback string) matches the other evaluators.
• mpi_batch_evaluator — a named peer to joblib/pathos/threading, registered
  so batch_evaluator="mpi" works (added to BatchEvaluatorLiteral and
  process_batch_evaluator). Lazily imports mpi4py (clear error → optional
  optimagic[mpi] extra), configures cloudpickle as the MPI serializer, and caches a
  module-level MPIPoolExecutor (one pool per process; a per-batch pool would be
  catastrophic).

Plus an optimagic[mpi] optional extra and a how-to (how_to_distributed_optimization)
that explains the single-driver model and the python -m mpi4py.futures launch
precondition.

Why

The batched evaluations an algorithm like tranquilo requests were locked to in-process
backends. On a cluster you want them spread across nodes. The right MPI pattern is a
single optimizer on the driver rank with the worker ranks parked by the
mpi4py.futures launcher — not running the optimizer on every rank (that diverges
under floating-point nondeterminism). An executor's single-submitter model makes that
the only expressible pattern, so the trap is structurally avoided.

Pickling (the crux)

stdlib ProcessPoolExecutor serializes tasks with plain pickle regardless of start
method, so closures/locally-defined criteria (optimagic's partial_func_of_params
output) would be rejected under spawn/forkserver and under MPI. A module-level
_CloudpickleTask wrapper carries a cloudpickle payload that plain pickle transports
intact — giving joblib/loky-level parity for any executor. cloudpickle is already a hard
dependency, so this adds nothing new.

Tests

37 pass. Notably a closure evaluated through ProcessPoolExecutor(mp_context=spawn)
(the case that fails without the wrapper), the ThreadPool path, raise/continue parity,
process_batch_evaluator("mpi") resolution, and the no-mpi4py clear-error path. All
local/deterministic — no cluster needed.

Points for reviewer attention

1. executor_batch_evaluator has no string alias (unlike "mpi"), so it's only
   reachable via from optimagic.batch_evaluators import executor_batch_evaluator.
   Should it be re-exported at the top level (optimagic.executor_batch_evaluator) for
   discoverability? Left as-is for now.
2. Cached MPI executor lifecycle: mpi_batch_evaluator caches one module-level
   MPIPoolExecutor, never explicitly shut down (dies with the process). No API to reset
   between independent optimizations in one process — fine for the
   one-optimization-per-process HPC model, but worth a conscious call.
3. "No workers" detection uses executor.num_workers == 0. Best-effort: launched
   without -m mpi4py.futures, mpi4py may fall back to dynamic MPI_Comm_spawn rather
   than reporting 0. The how-to and the error message both state the launcher precondition
   explicitly. Could use a maintainer's eye with cluster access — I couldn't run real MPI.

Notes

• CHANGES.md references {gh}NNN`` — I'll replace with this PR's number once assigned.
• pixi.lock intentionally not regenerated: the mpi extra isn't pulled into any
  pixi environment (so pixi install --frozen passes against the current lock), and a
  pixi lock here wanted an unrelated repo-wide v6→v7 format upgrade. Left for the
  maintainer.

🤖 Generated with Claude Code