fix: status no longer presents an unread TTL as a resolved "none" - #522
Merged
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
scttfrdmn
force-pushed
the
fix/508-status-ttl-reconciliation
branch
2 times, most recently
from
August 19, 2026 05:27
fe8d03d to
1bd7de3
Compare
Three related defects in `spawn status`/`spored status`: 1. When spored can't read its own tags (e.g. the #502 IAM gap), the config load failure was silently swallowed into zero-value defaults, and status rendered "TTL: none -- instance will not auto-terminate" -- a definite claim from data that was never actually read. Added Config.ConfigLoadError, threaded from the provider's tag-load failure, so status can render "TTL: UNKNOWN -- could not read config (<error>)" instead. 2. That on-instance line could sit directly above spawn status's own "Termination deadline: <t>" (read from tags with the CALLER's credentials) with nothing connecting the two -- an operator had to notice the contradiction by hand. spawn status now detects this exact combination and prints an explicit "Lifecycle mismatch" notice. 3. Started/Elapsed described the status-agent invocation's own age, not the instance's, falling back to time.Now() when the spawn:launch-time tag couldn't be read -- exactly the failure mode in (1) -- so a 7h39m-old instance reported "Elapsed: 0s". Added a second fallback tier: EC2's own PendingTime from the instance identity document via IMDS, which needs no IAM permission, so the instance's real age survives a tag-read failure. The fallback source is labelled inline when it isn't the authoritative tag. Fixes #508
scttfrdmn
force-pushed
the
fix/508-status-ttl-reconciliation
branch
from
August 19, 2026 05:38
1bd7de3 to
3789535
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three related defects in
spawn status/spored status, all from the same field incident (#508):TTL: nonewas rendered from data that was never read. Whensporedcouldn't read its own tags (the CRITICAL:launch --iam-policy-fileomits the spored baseline policy — TTL, --on-complete and --pre-stop all silently dead (the #406 fix covers 2 of 3 role paths) #502 IAM gap, or anyec2:DescribeTagsdenial), the load failure was silently swallowed into zero-value config defaults, and status printedTTL: none — instance will not auto-terminateas a definite claim. AddedConfig.ConfigLoadError(threaded frompkg/provider/ec2.go's tag-load failure) socmd/spored/main.go's status renderer can printTTL: UNKNOWN — could not read config (<error>)instead when the config was never actually resolved.spawn status's ownTermination deadline: <t>line — read from the same tags but with the caller's credentials, which usually succeed even when the instance role can't. AddedttlReconciliationNoticeincmd/status.go: when the on-instance view is unresolved AND a tag-based deadline exists, it prints an explicit "Lifecycle mismatch" notice naming both facts and what they mean together (the deadline exists, nothing on the instance can see it, so nothing on the instance will enforce it) — the sentence the reporter had to assemble by hand.Started/Elapseddescribed the CLI/agent invocation's own age, not the instance's.startTime := time.Now().Add(-uptime)was the fallback wheneverspawn:launch-timecouldn't be read — exactly defect 1's failure mode — so a 7h39m-old instance reportedElapsed: 0s. Added a middle fallback tier:identity.PendingTime, EC2's own record of the instance's launch request time from the instance identity document via IMDS — no IAM permission required, so it survives a tag-read failure. ExtractedresolveLaunchTimeas a pure function and label the source inline whenever it isn't the authoritative tag.Also flagged in the issue but not addressed here (out of scope / not reproduced): the observation that the CPU/network panel numbers looked byte-identical across 7 instances — that's a separate, unconfirmed claim about the metrics-sampling path, not the TTL/Elapsed rendering this issue is about; happy to investigate separately if it recurs.
Test plan
go build ./...,go vet ./...,gofmt -l .(clean)go test ./cmd/... ./cmd/spored/... ./pkg/provider/... ./pkg/agent/...TestResolveLaunchTime(cmd/spored) — verified the IMDS-fallback case fails against a pre-fix version that skips straight to the agent's own start time, matching the exact status: printsTTL: none — instance will not auto-terminateand aTermination deadlinein the same screen, from two unreconciled sources — and Started/Elapsed describe the CLI invocation, not the instance (Elapsed: 0s at 7h39m) #508 symptom (Elapsed: 0son a 7h-old instance)TestTTLReconciliationNotice(cmd) — five cases covering both unresolved-TTL renderings (noneandUNKNOWN), the past-due variant, and the two "nothing to reconcile" cases (no tag deadline; on-instance already resolved)golangci-lint run ./...— no new findings in touched files[Unreleased]Fixes #508