Sweep dispatch silently drops --cost-limit and --ttl, then logs "Using safeguards: ttl=..."

[scttfrdmn]

Severity: high — a safeguard is discarded and success-logged in the same breath

On the parameter-sweep path, --cost-limit and --ttl given on the command line are
silently discarded. Each per-row config is built from scratch and only three fields are
copied off the CLI base config, so nothing else survives.

The --no-detach branch then prints:

   Using safeguards: ttl=4h, idle-timeout=

This log line is the most dangerous part of the bug. It affirmatively tells the user a
safeguard is active at the instant that value is being dropped, using the very variable that
is about to go unused. A user reading that line has every reason to believe the sweep is
bounded.

--cost-limit is worse than --ttl, because --ttl has a working alternative
(ttl: under defaults:, a recognised parser key) while --cost-limit has none: there
is no cost_limit: case in the parser, so writing it in the param file produces a
PARAM_cost_limit env var that caps nothing (see #526). The net effect is that a
param-file sweep has no per-instance dollar cap available at all, and --budget is
warning-only, so the only real bound on a sweep is TTL × rate × row count.

Reproduction (spawn v0.100.2, cmd/ at ad8a4e1)

spawn launch sweep-check --param-file two-rows.yaml \
  --no-detach --ttl 4h --cost-limit 5
# stderr says:  Using safeguards: ttl=4h, idle-timeout=
# then, on either launched instance:
aws ec2 describe-tags --filters Name=resource-id,Values=<id> \
  --query 'Tags[?starts_with(Key,`spawn:`)]'
# -> no spawn:cost-limit tag; spawn:ttl reflects the param-file `ttl:`, not the CLI --ttl

A param file with no ttl: under defaults: and only a CLI --ttl 4h is the sharp
case: the CLI check at launch_sweep.go:79 is satisfied, the safeguards line prints, and the
instances get no TTL from that flag.

As with #524, the trace below is source-level rather than a live run — reproducing it costs
money by construction. It is confirmable under the Substrate emulator at no cost.

Evidence

line	what happens
launch_flags.go:223	--ttl binds ttl
launch_flags.go:305	--cost-limit binds costLimit
launch_config.go:292-293	config.CostLimit = costLimit — set on the base config only
launch_single.go:70-77	the sweep path never calls that builder; it hands launchParameterSweep a config with exactly Region and InstanceType
launch_sweep.go:79	--no-detach requires a non-empty CLI ttl or idleTimeout — so the flag is load-bearing for an argument check …
launch_sweep.go:83	… and then Using safeguards: ttl=%s, idle-timeout=%s prints it. This is the misleading affirmation.
sweep.go:74-82	buildLaunchConfigFromParams: "Start with an empty config" — only sweep identity fields are seeded
launch_sweep.go:155-176	the only fields copied off the base config: Region, InstanceType, Name. CostLimit, TTL, and every other launch flag are dropped

Expected behaviour — either of these, not the current third thing

(a) Honour them as per-instance controls. Copy CostLimit/TTL (and any other
per-instance safeguard) from the base config onto each per-row config where the row does not
override it, so --cost-limit 5 means "5 dollars per row". Document which reading applies —
per row or per sweep — because the two differ by a factor of len(params).

(b) Reject them for sweeps. Error out with something actionable:
--cost-limit is not supported with --param-file; set cost_limit: per row (once that key
exists), or simply --cost-limit does not apply to parameter sweeps.

Either is fine. Silent discard with a success log is not. If (b), the safeguards line at
:83 must stop claiming a ttl the instances will not receive — print the value that
actually reaches them, or say nothing.

If param-file fields are the intended interface, then the corollary in #526 is part of
this fix: an unknown key like cost_limit: must error, not become PARAM_cost_limit.
Otherwise the documented workaround for this issue fails as quietly as the bug it replaces.

Regression test to add

given  a 2-row param file with NO ttl: in defaults, launched with --no-detach --ttl 4h --cost-limit 5
when   launch runs
then   EITHER each launched config carries TTL=4h and CostLimit=5
       OR the command fails before launching, naming the unsupported flag
       and in both cases: no stderr line claims a safeguard that is not on the instances

Assert on the per-row LaunchConfig values that reach provisioning (or the resulting
spawn:ttl / spawn:cost-limit tags), not on the CLI variables — the CLI variables are
correct today; the bug is that they never travel.

Cross-references

• --estimate-only is ignored on the foreground sweep path — --no-detach launches every row instead of estimating #524 — --estimate-only is dropped on the same path, with the same root cause (per-row
  configs built from scratch, dispatch happening before the check). --estimate-only is ignored on the foreground sweep path — --no-detach launches every row instead of estimating #524 is the P0 of the
  pair because it fires on the command a user runs specifically to spend nothing.
• Unknown param-file keys silently become PARAM_* env vars — a mistyped or invented spend control caps nothing #526 — unknown param-file keys silently becoming PARAM_*, which closes off the
  workaround.
• Audit cost accumulation for accuracy — include EBS, public IPv4, etc. across all instance states #51 — cost-accounting accuracy audit. Related in spirit: this issue is about a cap that
  isn't applied, that one about a figure that may be incomplete.

Found while building a heterogeneous GROMACS benchmark sweep for the cost-to-result
project; recorded there as finding 15 in docs/platform-findings.md. A draft matrix in that
project carried cost_limit: 8 under defaults: labelled "the enforced cap" — it was
enforcing nothing, and only grepping cmd/sweep.go revealed it. That is the exact
misconfiguration this issue predicts a user will write.

[scttfrdmn]

Reproduced under Substrate — this is no longer just a source trace.

While writing the #524 regression test (test/e2e/tier0_sweep_estimate_test.go, Tier 0, real
spawn binary against the Substrate emulator), the pre-fix run launched two instances from a
2-row param file invoked with --no-detach --ttl 1h. spawn list -o json reported them as:

instance_id:i-fcae5d60a3ffa2b2  instance_type:c5.large   name:sweep-0  state:running
  ttl:            <-- EMPTY
  idle_timeout:1h
instance_id:i-4fcef13655a12adb  instance_type:c5.xlarge  name:sweep-1  state:running
  ttl:            <-- EMPTY
  idle_timeout:1h

--ttl 1h was passed on the command line. launch_sweep.go:83 printed
Using safeguards: ttl=1h, idle-timeout= on the way past. The instances received no TTL at
all — ttl is empty, and what actually bounds them is the unrelated 1h idle timeout the
zombie guard auto-applies at launch_sweep.go:305-313.

Three things this pins down:

1. The drop is confirmed end to end, not inferred. --ttl does not reach the instances.
2. The safeguards log line prints a value that provably does not arrive — exactly the
   "affirms a safeguard at the instant it is discarded" behaviour in the issue body.
3. The accidental backstop is an idle timeout, not a TTL, and the two are not
   interchangeable. A sweep row running a long compute-bound job is not idle, so it is not
   bounded by idle_timeout:1h — it runs on. A user who passed --ttl 1h and read the
   safeguards line has no bound at all on a busy instance.

Point 3 is the part worth keeping when this is fixed: --ttl silently degrading to an
auto-applied idle timeout is worse than degrading to nothing, because spawn list shows a
timeout in the row and it looks bounded.

Note the param file in that test sets no ttl: under defaults:, which is the sharp case
named in the issue body. The reproduction is free — no AWS account, no spend.

[scttfrdmn]

Fix open as #529 (stacked on #528 / #524, since both live in the sweep launch path).

The pre-fix evidence, captured by the new Tier 0 regression tests against a param file with --ttl 1h --cost-limit 8: every launched instance came up with spawn:ttl absent, no spawn:cost-limit, and spawn:idle-timeout=1h present — the zombie guard's substitute, which never fires on a compute-bound row while still reading as a bound. stderr for that same launch:

   Using safeguards: ttl=1h, idle-timeout=
⚠️  Auto-setting --idle-timeout=1h for all sweep instances

Both lines in one run: the safeguard reported as active, then the fallback applied because it was not.

Fix folds the CLI flags into paramFormat.Defaults right after the param file is parsed — that map is the one seam both orchestration paths read (foreground merges it per row; detached uploads it to S3 for the Lambda). Precedence is most-specific-wins: row params: > CLI flag > file defaults:. cost_limit: gets a real parser case that errors on an unusable value rather than leaving the cap at 0.

Two knock-ons: --estimate-only now prices the TTL you actually passed (--ttl 4h was previously estimated at the 1h fallback — a 4× understatement on the command whose job is to tell you the number before you spend), and the --no-detach guard is per row instead of reading the CLI variable, so a file-provided ttl: satisfies it and unbounded rows are named.

Caveat carried in the PR: the sweep-orchestrator Lambda is not in this repo, so the detached path's half of the fix is not verifiable here.

7 Tier 0 tests, all asserting on EC2 tags rather than parsed values — the flag was parsed, onto a config nobody copied from. All 7 verified failing at da8e0dd and passing on the fix.