fix: make a fatal ASM worker exit shut the runner down

[prajwolrg]

Description

A fatal ASM sync failure used to leave the runner alive but permanently stuck. The worker reported the failure as Ok(Response::ShouldExit), which the service framework treats as a clean exit, and strata-tasks only raises a TaskError for an Err or a panic. So nothing signalled the task manager, the process stayed up, the block watcher kept submitting blocks nobody would process, the RPC kept serving stale state, and no supervisor had a reason to restart the binary.

Returning the error instead makes the fatal path actually fatal. The runner exits non-zero and gets restarted.

Fixes SECFIND-339 and SECFIND-341.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature/Enhancement (non-breaking change which adds functionality or enhances an existing one)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactor
• New or updated tests
• Dependency update
• Security fix

Notes to Reviewers

The fix is three lines in crates/worker/src/service.rs. Everything else in this PR is documentation.

Fixed at the worker, not the watcher. SECFIND-339 proposes propagating the error out of the block watcher instead. Both routes end at a TaskError, but the watcher only learns the worker is dead on its next submission, which is the next ZMQ block. That is potentially ten minutes of a node that is up, answers RPC, and is silently not advancing. Fixing it at the worker makes shutdown synchronous with the failure, and keeps the "my exit is terminal" contract with the component that knows it rather than re-implementing it in every caller.

Nothing else needs to change to cooperate with it. An earlier version of this PR also made the watcher fail its own task on a submission error. That turned out to be redundant: once the worker returns Err, the shutdown signal reaches the watcher through the guard it already selects on, so it stops on its own after at most one extra log line. Worse, the watcher change created a problem of its own. Shutdown reaches the worker before the watcher, and the worker drops the completion for an input it has already dequeued, so an ordinary Ctrl-C during a submission comes back as WorkerExited and would have failed the watcher's critical task on every clean stop. Fixing that needed a second commit to un-break what the first one broke. Both are dropped.

The two findings are related but the second one is a docs bug. SECFIND-341 is that subscriptions never close, so followers park forever. They only park forever because the process outlives the worker, which is SECFIND-339. Closing the sender registry does not fix the availability problem on its own: it turns "Moho parks" into "Moho's stream ends and its task exits cleanly", and a cleanly exiting critical task still does not trigger shutdown. That is a quieter zombie, not a restart.

What is left once SECFIND-339 is fixed is that Subscription::recv documented a contract the type does not honour. It claimed to return None when the worker shuts down. It does not, because the worker handle holds a clone of the registry for the whole process, so the senders outlive the emitting worker. No consumer is affected today, since both are service workers whose message loop is raced against the shutdown guard, but the docs invited a bare recv loop that would park forever. The second commit corrects the contract rather than changing the behaviour. Closing the registry explicitly would be the behavioural fix, but with SECFIND-339 fixed there is nothing left for it to prevent, so there is no reason to carry the extra state.

One judgement call worth flagging. WorkerError is not Clone and the caller must keep receiving the typed error through the completion channel, so the error the task exits with is a rendering of it rather than the value itself. The full typed error still reaches the caller, and the worker logs it at the point of failure. Making WorkerError cloneable to avoid this seemed a worse trade than the lost source chain in one log line.

Out of scope, worth a follow-up. The sync worker loop in strata-service only checks its shutdown guard after recv_next() returns, so the ASM worker thread does not wake promptly on a global shutdown. It waits for the next submit_block or the runner's SHUTDOWN_TIMEOUT. The process still exits, just not instantly. That is an upstream strata-common issue.

Checklist

• I have performed a self-review of my code.
• I have commented my code where necessary.
• I have updated the documentation if needed.
• My changes do not introduce new warnings.
• I have added tests that prove my changes are effective or that my feature works.
• New and existing tests pass with my changes.

Related Issues

SECFIND-339, SECFIND-341

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
crates/worker/src/service.rs	99.01% <100.00%> (-0.01%)	⬇️
crates/worker/src/subscription.rs	100.00% <ø> (ø)

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Commit: aef46ad
SP1 Execution Results

program	cycles	gas
asm-stf	136,427,562	134,143,979
moho	5,223,688	5,525,436

[github-actions]

🔒 AI Security Review (claude-opus-5)

✅ No security issues found.

[github-actions]

🔒 AI Security Review (claude-opus-5)

✅ No security issues found.

[github-actions]

🔒 AI Security Review (claude-opus-5)

✅ No security issues found.

[github-actions]

🔒 AI Security Review (claude-opus-5)

✅ No security issues found.

[github-actions]

🔒 AI Security Review (claude-opus-5)

✅ No security issues found.

[github-actions]

🔒 AI Security Review (claude-opus-5)

✅ No security issues found.