Skip to content

fix: stop benching indexers for offers we never landed - #684

Merged
MoonBoi9001 merged 13 commits into
mainfrom
mb9/stop-benching-indexers-for-our-own-submission-failures
Jul 30, 2026
Merged

fix: stop benching indexers for offers we never landed#684
MoonBoi9001 merged 13 commits into
mainfrom
mb9/stop-benching-indexers-for-our-own-submission-failures

Conversation

@MoonBoi9001

@MoonBoi9001 MoonBoi9001 commented Jul 30, 2026

Copy link
Copy Markdown
Member

This PR stops dipper excluding an indexer from a deployment for 30 days when the reason the agreement expired was dipper's own failure to get the offer on chain.

An indexing agreement expires when no on-chain acceptance arrives inside its window, and that has 2 very different causes: dipper never landed the offer, so the indexer had nothing to accept, or dipper did land it and the indexer let the window close. The exclusion query treated both as a refusal. Expired rows carry no rejection reason, so they missed every tuned bucket, including the 5 minute one that exists for dipper-side faults, and fell into the 30 day catch-all meant for explicit rejections.

The row already tells the 2 apart: offer_tx_hash is written only after a submission mines, so its absence means no offer ever reached the chain. Those expiries now get the same short window as dipper's other faults.

Observed on testnet on 2026-07-29: an indexer accepted a proposal off-chain, the RPC provider dipper submits through answered HTTP 500, and the indexer was excluded from that deployment until late August.

These tests need a containerised Postgres I could not start locally, so CI runs them first.

An agreement that expires without an offer reaching the chain was treated like a refusal and
excluded that indexer from the deployment for 30 days, though it had no offer to accept and had
already agreed. Give that case the short window our own faults get, keyed on the missing hash.
An indexing agreement that expires without its offer reaching the chain is dipper's own fault,
so it gets a short exclusion window rather than the 30 day default. Restrict that to expiries
with no rejection reason, so a reason that later appears on one keeps the window it earns.
The tests around a dipper-caused agreement expiry only checked that the indexer stops counting
as declined once the 5 minute window has passed, which an implementation that dropped the row
from the query altogether would also satisfy. Check the indexer still counts inside the window.
The test for an agreement that expired after its offer reached the chain only checked that some
indexer somewhere came back as declined, so a result naming the wrong deployment or the wrong
indexer would have passed. Name both, matching the neighbouring tests.
The test that stands in for an agreement whose offer reached the chain wrote a single byte into
a column that holds a 32 byte transaction hash in production. Nothing reads the column back yet,
so widen it now rather than leave a trap for whoever starts reading it.
Every other test covering how long an indexer stays excluded is named for the window it checks,
which is what makes the set readable as a grid of reason against window. The 2 newest ones were
named for their conclusion instead, so they sat outside that grid.
How long an indexer stays excluded from a deployment used to be decided purely by the reason it
gave for declining, and the description still said so. An agreement that expired without its
offer reaching the chain now gets a window too, from the absence of that transaction.
The description of how dipper decides how long to keep an indexer out of a deployment said the
window comes purely from the reason the indexer gave for declining. An agreement that expired
without its offer reaching the chain gets one too, from the absence of that transaction.
This setting decides how long an indexer stays out of a deployment after declining, and its
description claimed every expired agreement falls under it. Only expiries whose offer reached
the chain do; the rest are dipper's own failure to submit and clear in minutes.
Dipper used to ask a subgraph whether an on-chain offer had already been recorded before sending
another, and that check went away when every agreement became contract funded. The description
of this step and one of its log lines still described it, which flatters what the code does.
An agreement that expired with no record of its offer reaching the chain is treated as dipper's
own failure, on short exclusion. That record can also be missing when the offer did land, so say
so where the decision is made, along with the fact that this query is what makes it matter.
One test asserts an indexer who let an agreement expire stays out of a deployment for 30 days,
but the agreement it uses has no record of its offer reaching the chain, which now reads as
dipper's own failure and a wait of minutes. Record the offer so the test covers what it claims.
@MoonBoi9001
MoonBoi9001 marked this pull request as ready for review July 30, 2026 14:20
@MoonBoi9001
MoonBoi9001 merged commit 26272f5 into main Jul 30, 2026
11 checks passed
@MoonBoi9001
MoonBoi9001 deleted the mb9/stop-benching-indexers-for-our-own-submission-failures branch July 30, 2026 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant