Skip to content

Add user-suppliable batch_evaluator argument to minimize/maximize#684

Open
hmgaudecker wants to merge 2 commits into
mainfrom
feat/user-batch-evaluator
Open

Add user-suppliable batch_evaluator argument to minimize/maximize#684
hmgaudecker wants to merge 2 commits into
mainfrom
feat/user-batch-evaluator

Conversation

@hmgaudecker

Copy link
Copy Markdown
Member

What

Threads a batch_evaluator keyword from the public minimize/maximize API through
create_optimization_problem and the OptimizationProblem container to
InternalOptimizationProblem, replacing the hardcoded process_batch_evaluator("joblib")
in _optimize. Users can now pass either a built-in name ("joblib", "pathos",
"threading") or a callable conforming to the BatchEvaluator protocol.

Also:

  • Removes the dead batch_evaluator Literal field from the Tranquilo / TranquiloLS
    dataclasses. It was never read — the evaluator lives on the optimization problem, not
    the algorithm — so setting it silently no-op'd.
  • Exports BatchEvaluator from the top-level namespace so users can type their custom
    evaluators against the protocol.

Why

The batched criterion evaluations a trust-region optimizer like tranquilo requests were
locked to the in-process joblib evaluator. Some workloads need a different execution
backend — e.g. a distributed, MPI-backed pool that fans each batch of candidate points
out across nodes/GPUs for an expensive simulation-based criterion. There was already a
process_batch_evaluator resolver and a BatchEvaluator protocol; this just lets the
caller reach them.

This is also the foundation for a follow-up: a first-class executor-backed
(concurrent.futures / mpi4py.futures) distributed evaluator, where minimize itself
parks non-root ranks in a worker pool so the optimizer runs on a single driver. That
addresses a real footgun — under MPI, srun starts N copies of the program, so a naive
"run minimize on every rank with a collective evaluator" design ends up with N
optimizers kept in lockstep, which diverges under floating-point nondeterminism. This PR
is the minimal, backward-compatible plumbing that makes the safe design expressible.

Tests

Adds test_minimize_uses_custom_batch_evaluator,
test_maximize_uses_custom_batch_evaluator (a spy callable is the one optimagic actually
invokes), and test_minimize_rejects_unknown_batch_evaluator (clear error on an unknown
name).

Backward compatibility

Fully backward compatible — batch_evaluator defaults to "joblib", the prior behavior.


Happy to add a CHANGES.md entry referencing this PR number under the appropriate
next-version heading (0.5.4 vs 0.6.0 — your call).

🤖 Generated with Claude Code

Threads a `batch_evaluator` keyword (a BatchEvaluatorLiteral string or a callable
conforming to the BatchEvaluator protocol) from the public minimize/maximize API
through create_optimization_problem and the OptimizationProblem container to
InternalOptimizationProblem, replacing the hardcoded joblib evaluator in _optimize.
This lets users plug in a custom evaluator (e.g. an MPI-backed pool) for the batched
criterion evaluations that trust-region optimizers like tranquilo request.

Removes the dead `batch_evaluator` Literal field from the Tranquilo and TranquiloLS
dataclasses: it was never read (the evaluator lives on the problem, not the
algorithm), so it silently no-op'd. Exports BatchEvaluator so users can type custom
evaluators against the protocol.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/optimagic/__init__.py 93.75% <100.00%> (+0.20%) ⬆️
...imagic/optimization/create_optimization_problem.py 88.28% <100.00%> (+0.09%) ⬆️
src/optimagic/optimization/optimize.py 92.10% <100.00%> (ø)
src/optimagic/optimizers/tranquilo.py 99.13% <ø> (-0.02%) ⬇️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant