Skip to content

fix: status no longer presents an unread TTL as a resolved "none" - #522

Merged
scttfrdmn merged 1 commit into
mainfrom
fix/508-status-ttl-reconciliation
Aug 19, 2026
Merged

fix: status no longer presents an unread TTL as a resolved "none"#522
scttfrdmn merged 1 commit into
mainfrom
fix/508-status-ttl-reconciliation

Conversation

@scttfrdmn

Copy link
Copy Markdown
Contributor

Summary

Three related defects in spawn status/spored status, all from the same field incident (#508):

  1. TTL: none was rendered from data that was never read. When spored couldn't read its own tags (the CRITICAL: launch --iam-policy-file omits the spored baseline policy — TTL, --on-complete and --pre-stop all silently dead (the #406 fix covers 2 of 3 role paths) #502 IAM gap, or any ec2:DescribeTags denial), the load failure was silently swallowed into zero-value config defaults, and status printed TTL: none — instance will not auto-terminate as a definite claim. Added Config.ConfigLoadError (threaded from pkg/provider/ec2.go's tag-load failure) so cmd/spored/main.go's status renderer can print TTL: UNKNOWN — could not read config (<error>) instead when the config was never actually resolved.
  2. The two lifecycle lines were never reconciled. That on-instance "none"/"UNKNOWN" line could sit directly above spawn status's own Termination deadline: <t> line — read from the same tags but with the caller's credentials, which usually succeed even when the instance role can't. Added ttlReconciliationNotice in cmd/status.go: when the on-instance view is unresolved AND a tag-based deadline exists, it prints an explicit "Lifecycle mismatch" notice naming both facts and what they mean together (the deadline exists, nothing on the instance can see it, so nothing on the instance will enforce it) — the sentence the reporter had to assemble by hand.
  3. Started/Elapsed described the CLI/agent invocation's own age, not the instance's. startTime := time.Now().Add(-uptime) was the fallback whenever spawn:launch-time couldn't be read — exactly defect 1's failure mode — so a 7h39m-old instance reported Elapsed: 0s. Added a middle fallback tier: identity.PendingTime, EC2's own record of the instance's launch request time from the instance identity document via IMDS — no IAM permission required, so it survives a tag-read failure. Extracted resolveLaunchTime as a pure function and label the source inline whenever it isn't the authoritative tag.

Also flagged in the issue but not addressed here (out of scope / not reproduced): the observation that the CPU/network panel numbers looked byte-identical across 7 instances — that's a separate, unconfirmed claim about the metrics-sampling path, not the TTL/Elapsed rendering this issue is about; happy to investigate separately if it recurs.

Test plan

Fixes #508

@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 12 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/spored/main.go 42.85% 8 Missing ⚠️
cmd/status.go 90.00% 2 Missing ⚠️
pkg/provider/ec2.go 0.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@scttfrdmn
scttfrdmn force-pushed the fix/508-status-ttl-reconciliation branch 2 times, most recently from fe8d03d to 1bd7de3 Compare August 19, 2026 05:27
Three related defects in `spawn status`/`spored status`:

1. When spored can't read its own tags (e.g. the #502 IAM gap), the
   config load failure was silently swallowed into zero-value
   defaults, and status rendered "TTL: none -- instance will not
   auto-terminate" -- a definite claim from data that was never
   actually read. Added Config.ConfigLoadError, threaded from the
   provider's tag-load failure, so status can render "TTL: UNKNOWN --
   could not read config (<error>)" instead.

2. That on-instance line could sit directly above spawn status's own
   "Termination deadline: <t>" (read from tags with the CALLER's
   credentials) with nothing connecting the two -- an operator had to
   notice the contradiction by hand. spawn status now detects this
   exact combination and prints an explicit "Lifecycle mismatch"
   notice.

3. Started/Elapsed described the status-agent invocation's own age,
   not the instance's, falling back to time.Now() when the
   spawn:launch-time tag couldn't be read -- exactly the failure mode
   in (1) -- so a 7h39m-old instance reported "Elapsed: 0s". Added a
   second fallback tier: EC2's own PendingTime from the instance
   identity document via IMDS, which needs no IAM permission, so the
   instance's real age survives a tag-read failure. The fallback
   source is labelled inline when it isn't the authoritative tag.

Fixes #508
@scttfrdmn
scttfrdmn force-pushed the fix/508-status-ttl-reconciliation branch from 1bd7de3 to 3789535 Compare August 19, 2026 05:38
@scttfrdmn
scttfrdmn merged commit a6cc7a7 into main Aug 19, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant