fix: --estimate-only on a sweep estimates instead of launching (#524) - #528
Merged
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This was referenced Aug 19, 2026
Contributor
Author
|
#529 is stacked on this branch (fix for #525 — the sweep dropping Worth noting for review: #529 moves the CLI spend controls into |
This was referenced Aug 19, 2026
--estimate-only was checked only inside launchSweepDetached, so a sweep that took the FOREGROUND path launched every row while the user was asking for a preview. That inverts the one command a user runs specifically to avoid spending money. Two ordinary invocations reach the foreground path: --no-detach, which is the documented advice for a heterogeneous sweep because only that path detects an AMI per config (#372), and an explicit --detach with no --max-concurrent, which leaves maxConcurrent at 0 and so fails the `detach && maxConcurrent > 0` dispatch condition. In other words the sweep shape most in need of a cost preview was the one that could not get one. The check now lives once in launchParameterSweep, above the dispatch that chooses between the two paths, so "launches nothing" no longer depends on which path the sweep would have taken. It is deliberately placed before aws.NewClient: EstimateSweepCost reads only the param file, so a cost preview needs no credentials. The budget comparison is factored into reportSweepBudget and shared with the real detached launch so the two cannot drift, and the now-unreachable estimateOnly block inside launchSweepDetached is removed rather than left as dead code that would re-create the per-path shape behind the bug. The regression test asserts on the observable, not the flag: it queries EC2 for every instance in the sweep's region and requires the set to be empty, for all three invocations (default, --no-detach, bare --detach). It does not use `spawn list --state all`, which passes "all" through as a literal instance-state-name filter and therefore returns [] whether or not instances exist (#527) -- using it would have made the test unfailable, which is the defect class the test exists to catch. Fails before, passes after: 2 of the 3 subtests launched 2 instances each pre-fix.
scttfrdmn
force-pushed
the
fix/524-estimate-only-sweep
branch
from
August 19, 2026 06:36
da8e0dd to
4c6e603
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #524.
The bug
--estimate-onlywas checked only insidelaunchSweepDetached, so a sweep thattook the foreground path launched every row while the user was asking for a
preview. That inverts the one command a user runs specifically to avoid spending
money.
Two ordinary invocations reach the foreground path:
--estimate-only--param-file f.yaml(default)--param-file f.yaml --no-detach --ttl 1h--param-file f.yaml --detach(no--max-concurrent)--no-detachis the documented advice for a heterogeneous sweep, because onlythe foreground path detects an AMI per config (#372) — so the sweep shape most
in need of a cost preview was the one that could not get one. The bare
--detachcase is subtler:maxConcurrentonly gets its default inside theauto-enable branch, so an explicit
--detachleaves it at 0 and fails thedetach && maxConcurrent > 0dispatch condition.The fix
--estimate-onlyis now handled once inlaunchParameterSweep, above thedispatch that chooses between the two paths, so "launches nothing" no longer
depends on which path the sweep would have taken. It is deliberately placed
before
aws.NewClient:EstimateSweepCostreads only the param file, so a costpreview needs no credentials.
The budget comparison is factored into
reportSweepBudgetand shared with thereal detached launch so the two cannot drift. The now-unreachable
estimateOnlyblock inside
launchSweepDetachedis removed rather than left as dead code thatwould re-create the per-path shape behind the bug.
The regression test
test/e2e/tier0_sweep_estimate_test.go(Tier 0, Substrate emulator — free, noAWS account). It asserts on the observable, not the flag: it queries EC2 for
every instance in the sweep's region and requires the set to be empty, for all
three invocations above, plus exit 0.
It deliberately does not use
spawn list --state all. That flag passesallthrough as a literalinstance-state-namefilter, which matches nothing,so it returns
[]whether or not instances exist (#527) — using it here wouldhave made the test unfailable, which is the same defect class the test exists to
catch.
Fails before, passes after: 2 of the 3 subtests launched 2 instances each
pre-fix. Post-fix 3/3 green, full unit suite green, full Tier 0 suite green
(158s).
Related
--cost-limitand--ttl, then logsUsing safeguards: ttl=.... Same code path, separate defect; the pre-fix runof this test also reproduces it (instances came up with
ttl:empty despite--ttl 1h).PARAM_*.spawn list --state allreturns[]unconditionally.