Clean up the erlang per-arch repos too - #281
Merged
Merged
Conversation
The erlang per-arch repos were left out because DockerChecker.expected_elixir_tags/0 reads those rows to decide which Elixir images to build. It ranks them by version rather than by date, so the newest tag of an old OTP line stays in use however old it gets and a plain age cutoff would delete it. Keeping the erlang tags whose os_version the build matrix currently targets is enough to protect that, and costs 420 of the 71,061 stale erlang tags. The os_versions are re-read per batch so a base image released mid-run pulls the tags built against it back under protection, and the per-chunk re-check applies the same rule. Docker Hub meters the API per account, not per source IP. A pod deleting at 27 requests/minute from one node was pushed to 56 429s in 92 requests by a second node bursting from a different IP, with x-ratelimit-remaining reaching -534 on both. Scoping a repo per node cannot buy throughput.
It existed to point one node at each per-arch repo, which the account budget makes pointless. Nothing passes it, so the per-arch repo list stops being a parameter and the filter that kept a caller from aiming the deleter at a manifest repo goes with it. The dry run no longer claims a scheduled run stops at one batch; it has cleared the whole backlog since #279.
ericmj
marked this pull request as ready for review
July 30, 2026 22:57
`x <> ALL('{}')` is true, so an empty os_version list protected no erlang
tag rather than every one of them. `builds()` is empty on a database
whose base_image_tags have never been populated, and it drops a line
whose regex matches nothing, so the input that means "the matrix could
not be read" was the input that deleted the most. It now holds the
erlang repos back entirely. A tag whose metadata carries no os_version
is spelled out as kept rather than left to `NULL <> ALL(...)`.
The os_versions are pinned for the run like the cutoff. Re-reading them
per batch removed protection as readily as it added it: a base image
released mid-run rotates the os_version it replaces out of the matrix,
and a run lasting days would then delete the erlang tags on it before
the checker had built their replacements.
An elixir request now reserves the erlang image it builds FROM. Expiring
a request only sets its state, leaving any build already enqueued, so a
job could outlive the protection its base image had and fail on a
missing FROM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The erlang per-arch repos were left out of the cleanup because
DockerChecker.expected_elixir_tags/0reads those rows to decide which Elixir images to build. It ranks them by version rather than by date, so the newest tag of an old OTP line stays in use however old it gets and a plain age cutoff would delete it.Keeping the erlang tags whose
os_versionthe build matrix currently targets is enough to protect that, and costs 420 of the 71,061 stale erlang tags. Both sides read the samebuilds(): a build request whoseos_versionis not current is expired rather than built, so a request can never need a base image the cleanup has taken. The os_versions are re-read per batch so a base image released mid-run pulls the tags built against it back under protection, and the per-chunk re-check applies the same rule.Every Elixir image is
FROM hexpm/erlang-${ARCH}:${ERLANG}-${os}-${OS_VERSION}, and an elixir build request does not reserve that base tag. It does not need to, given the above, andrequest_jobs/1already enqueues the erlang build instead of the elixir one when the base is missing.:reposgoes with it. It existed to point one node at each per-arch repo on the theory that Docker Hub metered per source IP; it meters per account, so a second node draws down the same budget and the two together get no more through than one. Nothing passes the option, so the repo list stops being a parameter and the filter that kept a caller from aiming the deleter at a manifest repo goes with it.The dry run no longer claims a scheduled run stops at one batch. It has cleared the whole backlog since #279.