Skip to content

fix: cleanup no longer counts already-destroyed resources as removable - #521

Merged
scttfrdmn merged 1 commit into
mainfrom
fix/516-cleanup-phantom-resources
Aug 19, 2026
Merged

fix: cleanup no longer counts already-destroyed resources as removable#521
scttfrdmn merged 1 commit into
mainfrom
fix/516-cleanup-phantom-resources

Conversation

@scttfrdmn

Copy link
Copy Markdown
Contributor

Summary

  • spawn cleanup --dry-run reported 9 resources as "would be removed" in an account where all nine had already been destroyed 12+ hours earlier. Two compounding defects, per the issue's own root-cause analysis:
    1. enrichInstanceState's NotFound fallback (pkg/aws/cleanup.go) only fires on a batch DescribeInstances error. EC2 distinguishes two absence shapes: a syntactically-valid-but-never-existed id fails the whole batch with InvalidInstanceID.NotFound (the fallback's actual trigger), but a real id that has since aged out of the API answers with an empty result — no error at all. So State stayed "" for exactly the population cleanup exists to sweep, and the per-id fallback (whose own comment promises a gone instance is "never mistaken for one whose state is merely unknown") never ran.
    2. cmd/cleanup.go's removable/running/address split never consulted State at all — so even a resource the pipeline correctly resolved to "deleted" (volumes, which do 400 on an already-deleted id — the working control in the issue) still landed in removable and in the "N resource(s) would be removed" headline count, contradicting its own displayed deleted state one row away.
  • Fixed both: enrichInstanceStateWith (extracted for testability) now marks any instance id absent from a successful DescribeInstances response "deleted" directly, in the success path — no longer dependent on an error EC2 doesn't raise for aged-out ids. cmd/cleanup.go's split is extracted into splitCleanupResources, which adds an alreadyGone bucket for State == "deleted", excluded from the removable count and reported separately as tag-mapping residue (the Resource Groups Tagging API's index outlives the resources it describes — that staleness is expected, treating it as ground truth was the bug).
  • Also implemented the issue's optional item 4: RemoveResource's instance branch now tolerates InvalidInstanceID.NotFound via the existing ignoreNotFound helper, matching its volume/key-pair/security-group siblings — a real (non-dry-run) sweep of stale residue is a satisfied request, not a doomed Terminate failure.
  • Did not implement item 3 (a distinct "0 resource(s) would be removed" wording distinguishing "clean account" from "stale account") beyond a minimal version — the dry-run message now says Dry run: 0 resource(s) would be removed (N tag mapping(s) are residue for resources that no longer exist). when removable is empty but residue exists.

Test plan

  • go build ./..., go vet ./...
  • go test ./pkg/aws/... ./cmd/...
  • golangci-lint run ./pkg/aws/... ./cmd/... — no new findings in touched files
  • gofmt -l . — clean
  • CHANGELOG.md updated under [Unreleased]

Fixes #516

@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 51.42857% with 17 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/cleanup.go 44.44% 15 Missing ⚠️
pkg/aws/cleanup.go 75.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@scttfrdmn
scttfrdmn force-pushed the fix/516-cleanup-phantom-resources branch from 243ec54 to df901b0 Compare August 19, 2026 05:15
Two compounding defects:

1. enrichInstanceState's NotFound fallback only fires on a batch
   DescribeInstances error, but EC2 answers an aged-out instance id with
   an empty result, not an error -- so State stayed "" for exactly the
   population cleanup exists to sweep. An id absent from a successful
   response is now marked "deleted" directly, in the success path,
   instead of relying on an error EC2 doesn't raise for this case.

2. cmd/cleanup.go's removable/running/address split never consulted
   State at all, so even a resource correctly resolved to "deleted"
   still landed in removable and in the "N resource(s) would be
   removed" count -- contradicting its own displayed state one line
   away. Extracted the split into splitCleanupResources with a new
   alreadyGone bucket, reported separately as tag-mapping residue and
   excluded from the removable count.

Also added ignoreNotFound tolerance to RemoveResource's instance
branch, matching its volume/key-pair/security-group siblings, so a
real sweep of stale residue is a satisfied request rather than a
doomed Terminate failure.

Fixes #516
@scttfrdmn
scttfrdmn force-pushed the fix/516-cleanup-phantom-resources branch from df901b0 to ce28436 Compare August 19, 2026 05:26
@scttfrdmn
scttfrdmn merged commit fa8bb4e into main Aug 19, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant