Cache residuals with stateless evaluators

[dossett]

I used CODEX to analyze this problem and create this PR. I've reviewed the code and tests and stand by them. This summary is written completely by a human (me) other than very light copy editing by an LLM.

This PR does two important things. First, it makes the _ResidualEvaluation object stateless to allow for more aggressive reuse. Second, it caches the residual calculated for a given partition spec and partition values. Currently the residual is calculated on a per file basis, even though the residual isn't a function of the file. Collectively, these changes make residual evaluation safer and faster.

_ResidualEvaluation was mutable because its self.struct member was mutated when evaluating a residual. This change moves the mutability into the associated visitor object, which is created on a per file basis and then discarded. Making _ResidualEvaluation stateless therefore makes it safe to reuse concurrently.

The caching is relatively straight forward. A 128 starting size was somewhat arbitrary, but the benefits of caching are significant so I would not want to start too small.

CODEX-generated summary follows:

---

Residual planning currently creates a mutable evaluator per data file because evaluating one partition changes evaluator state. That avoids unsafe sharing, but it repeatedly prepares equivalent evaluation state and recalculates identical residuals for files whose relevant partition values are the same.

This change makes residual evaluators safe to share by keeping partition-specific state in a short-lived visitor. Scan planning then reuses one prepared evaluator per partition spec and maintains a bounded residual cache keyed by the spec and only the partition fields that can affect the filter. This reduces repeated planning work without allowing high-cardinality, unrelated partition fields to grow or defeat the cache.

Commit structure

1. bca46fc3 — Make residual evaluation stateless: moves partition-local state into _ResidualEvaluationVisitor, prepares the partition schema once, and adds state-isolation and concurrency tests.
2. 5676096d — Cache residuals during scan planning: reuses one evaluator per spec and adds a bounded 128-entry cache keyed by relevant partition values, with planner tests covering spec isolation, irrelevant fields, multiple transforms, and eviction.
3. e9fc9107 — Benchmark residual planning: adds a benchmark using a realistic 15-leaf predicate and both repeated and unique relevant partition values.

Performance

Mean of three runs planning 2,000 files with a 15-leaf predicate:

Relevant partition values	main	This PR	Improvement
7 repeated values	1.117 s	0.004 s	~99.6%
2,000 unique values	1.170 s	0.456 s	~61.0%

The unique-values case intentionally forces cache misses and exercises evaluator reuse independently of cache hits.

Testing

• pytest tests/expressions tests/table -q — 896 passed
• pytest tests/benchmark/test_residual_evaluator_benchmark.py -q -s -m benchmark — 2 passed
• UV_NO_CONFIG=1 .venv/bin/prek run -a — all hooks passed

[rambleraptor]

I think this one looks good, but I've got a big question around public renaming.

In general, we should try and find a way to not have to monkey patch these tests.

[rambleraptor]

We're making this class private. That may be a breaking change. I'll let a maintainer decide if we can do this.

[Fokko]

Thanks for flagging. I don't think the rename is worth breaking possible imports

[Fokko]

I see that the new class has a very similar signature. I'm fine with that

[dossett]

@rambleraptor I reworked the PR to avoid the scope change of ResidualVisitor thank you for calling that out. I also added a property for the cache size and correspondingly simplified the tests.

[Fokko]

Re-creating this evaluator seems expensive on each call 🤔

[dossett]

It is recreated in the case of cache misses, but the visitor is fairly lightweight. There's still a net win because the heaviest work (building the partition StructType) is now done once per partition spec in the now immutable ResidualEvaluator. That's a net savings even when there are no repeated values and no cache hits, per the performance measurement code I ran (that's in the third commit).

[Copilot]

Pull request overview

This PR improves scan planning by making residual evaluation safe to reuse (stateless across calls) and by caching residuals per partition spec and relevant partition values, reducing repeated residual computation across many data files.

Changes:

• Refactors residual evaluation so per-call partition state lives in a short-lived visitor, enabling safe reuse of prepared evaluators.
• Adds a bounded LRU residual cache in ManifestGroupPlanner.plan_files, keyed by spec ID and only the partition fields relevant to the scan filter.
• Adds planner tests for caching behavior, concurrency/state isolation tests for residual evaluation, and a benchmark for residual planning.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
pyiceberg/expressions/visitors.py	Refactors residual evaluation to isolate per-partition state and enable evaluator reuse.
pyiceberg/table/__init__.py	Reuses residual evaluators per spec and adds a bounded residual cache configurable via table options.
tests/expressions/test_residual_evaluator.py	Adds tests for evaluator immutability, concurrency safety, and partition-schema reuse.
tests/table/test_residual_evaluator_planning.py	Adds tests validating residual cache behavior and cache keying rules in planning.
tests/benchmark/test_residual_evaluator_benchmark.py	Adds benchmark coverage for repeated vs unique relevant partition values.

Suppressed comments (1)

pyiceberg/expressions/visitors.py:2020

• ResidualEvaluator no longer exposes the eval(partition_data) method that it previously inherited from ResidualVisitor (before this refactor). That is a potentially breaking public API change for any downstream code that called ResidualEvaluator.eval(...).

To preserve backwards compatibility, consider keeping eval as an alias that delegates to residual_for.

    def __init__(self, schema: Schema, expr: BooleanExpression):

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.