Skip to content

fix: --estimate-only on a sweep estimates instead of launching (#524) - #528

Merged
scttfrdmn merged 1 commit into
mainfrom
fix/524-estimate-only-sweep
Aug 19, 2026
Merged

fix: --estimate-only on a sweep estimates instead of launching (#524)#528
scttfrdmn merged 1 commit into
mainfrom
fix/524-estimate-only-sweep

Conversation

@scttfrdmn

Copy link
Copy Markdown
Contributor

Closes #524.

The bug

--estimate-only was checked only inside launchSweepDetached, so a sweep that
took the foreground path launched every row while the user was asking for a
preview. That inverts the one command a user runs specifically to avoid spending
money.

Two ordinary invocations reach the foreground path:

invocation path taken pre-fix --estimate-only
--param-file f.yaml (default) detached honoured
--param-file f.yaml --no-detach --ttl 1h foreground launches every row
--param-file f.yaml --detach (no --max-concurrent) foreground launches every row

--no-detach is the documented advice for a heterogeneous sweep, because only
the foreground path detects an AMI per config (#372) — so the sweep shape most
in need of a cost preview was the one that could not get one. The bare
--detach case is subtler: maxConcurrent only gets its default inside the
auto-enable branch, so an explicit --detach leaves it at 0 and fails the
detach && maxConcurrent > 0 dispatch condition.

The fix

--estimate-only is now handled once in launchParameterSweep, above the
dispatch that chooses between the two paths, so "launches nothing" no longer
depends on which path the sweep would have taken. It is deliberately placed
before aws.NewClient: EstimateSweepCost reads only the param file, so a cost
preview needs no credentials.

The budget comparison is factored into reportSweepBudget and shared with the
real detached launch so the two cannot drift. The now-unreachable estimateOnly
block inside launchSweepDetached is removed rather than left as dead code that
would re-create the per-path shape behind the bug.

The regression test

test/e2e/tier0_sweep_estimate_test.go (Tier 0, Substrate emulator — free, no
AWS account). It asserts on the observable, not the flag: it queries EC2 for
every instance in the sweep's region and requires the set to be empty, for all
three invocations above, plus exit 0.

It deliberately does not use spawn list --state all. That flag passes
all through as a literal instance-state-name filter, which matches nothing,
so it returns [] whether or not instances exist (#527) — using it here would
have made the test unfailable, which is the same defect class the test exists to
catch.

Fails before, passes after: 2 of the 3 subtests launched 2 instances each
pre-fix. Post-fix 3/3 green, full unit suite green, full Tier 0 suite green
(158s).

Related

@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 24 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/launch_sweep.go 0.00% 24 Missing ⚠️

📢 Thoughts on this report? Let us know!

@scttfrdmn

Copy link
Copy Markdown
Contributor Author

#529 is stacked on this branch (fix for #525 — the sweep dropping --ttl/--idle-timeout/--cost-limit). Merge this one first; #529's base will retarget to main automatically.

Worth noting for review: #529 moves the CLI spend controls into paramFormat.Defaults before the --estimate-only early return this PR adds, which also fixes the estimate itself — EstimateSweepCost derives per-row hours from row ttldefaults["ttl"] → a "1h" fallback, so before that injection --ttl 4h was priced at 1h.

--estimate-only was checked only inside launchSweepDetached, so a sweep
that took the FOREGROUND path launched every row while the user was
asking for a preview. That inverts the one command a user runs
specifically to avoid spending money.

Two ordinary invocations reach the foreground path: --no-detach, which
is the documented advice for a heterogeneous sweep because only that
path detects an AMI per config (#372), and an explicit --detach with no
--max-concurrent, which leaves maxConcurrent at 0 and so fails the
`detach && maxConcurrent > 0` dispatch condition. In other words the
sweep shape most in need of a cost preview was the one that could not
get one.

The check now lives once in launchParameterSweep, above the dispatch
that chooses between the two paths, so "launches nothing" no longer
depends on which path the sweep would have taken. It is deliberately
placed before aws.NewClient: EstimateSweepCost reads only the param
file, so a cost preview needs no credentials. The budget comparison is
factored into reportSweepBudget and shared with the real detached
launch so the two cannot drift, and the now-unreachable estimateOnly
block inside launchSweepDetached is removed rather than left as dead
code that would re-create the per-path shape behind the bug.

The regression test asserts on the observable, not the flag: it queries
EC2 for every instance in the sweep's region and requires the set to be
empty, for all three invocations (default, --no-detach, bare --detach).
It does not use `spawn list --state all`, which passes "all" through as
a literal instance-state-name filter and therefore returns [] whether
or not instances exist (#527) -- using it would have made the test
unfailable, which is the defect class the test exists to catch.

Fails before, passes after: 2 of the 3 subtests launched 2 instances
each pre-fix.
@scttfrdmn
scttfrdmn force-pushed the fix/524-estimate-only-sweep branch from da8e0dd to 4c6e603 Compare August 19, 2026 06:36
@scttfrdmn
scttfrdmn merged commit 7e3131b into main Aug 19, 2026
13 checks passed
@scttfrdmn
scttfrdmn deleted the fix/524-estimate-only-sweep branch August 19, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--estimate-only is ignored on the foreground sweep path — --no-detach launches every row instead of estimating

1 participant