ci: say why the suite is green — and fix the OOM that saying it uncovered - #4
Merged
Merged
Conversation
…e excused The header explained that CI was green BECAUSE twenty tests ran as strict expected failures. That stopped being true when node.py was repaired: there is no expected_red_until_fixed test left, so a failing test here is now simply a failure — something to fix in node.py, not something to write down. The machinery stays, armed and unused, as the tripwire for the next known gap, and the comment now says that rather than describing a state the repository has left. The per-basis reporting step is kept and made unambiguous. It printed pytest's own "no tests collected" line for each basis, which reads like a query that broke; it now prints a count, because the zero IS the claim being made. Comments and one reporting line only. The trigger, the runner, the Python version, the timeout and the install and run steps are untouched, so CI behaviour does not change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…OOM-killed CI caught this on the previous commit's branch: test_an_input_larger_than_the container_is_not_fatal failed on the runner while passing locally and on main. It was not flakiness in the test. It was a real, latent defect that main had been passing by luck. The step streams correctly — it never holds more than one block of an object in memory. That is not sufficient, because the container's memory limit counts the PAGE CACHE its own reads and writes create. Moving a 128 MiB object through a temporary file leaves the kernel holding 128 MiB of cache for it inside a 64 MiB cgroup, and the program's own flat footprint is irrelevant to the OOM killer. Fixed by asking for those pages back every 8 MiB, on all four streaming paths: the input download, the multipart upload body, the line count and the digest. posix_fadvise(POSIX_FADV_DONTNEED), after an fsync on the write side because dirty pages cannot be dropped. It is advice rather than a guarantee and costs nothing where it is unavailable, so it is best-effort throughout. Measured, by running the harness with the ceiling moved: without this change, 128 MiB into 32 MiB -> OOM-killed, exit 137 with this change, 128 MiB into 32 MiB -> passes with this change, 128 MiB into 64 MiB -> passes (the suite's own case) The step's code is otherwise identical between those runs; the only difference is who was holding the bytes. docs/PROTOCOL.md section 3.5 and CLAUDE.md now say this, because "stream and you are safe" is what every author will otherwise conclude from the existing text, and the failure it leads to reports itself as exit 137 with no marker and no explanation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits. The first is the comment-only follow-up to #3 that was asked for; the second is a real defect the first one's CI run uncovered, which is why this PR is not comment-only after all.
1.
ci:the suite is green because it passes, not because twenty tests are excusedSplit out of #3 because the token used there lacked the
workflowscope, and GitHub protects files under.github/workflows/separately.The header said, in bold, that CI was green because twenty tests ran as strict expected failures, and that fixing one would turn the run red as a prompt to reclassify it. After #3 there is no
expected_red_until_fixedtest left, so that sentence pointed at machinery no longer carrying anything — and a reader arriving at a red run would have gone looking for a test to promote instead of a bug to fix.It now says: green because every test passes; the machinery stays, armed and unused, as the tripwire for the next known gap; a failing test is now simply a failure.
The per-basis reporting step piped
pytest --collect-onlystraight to the log, so with nothing to collect it printedno tests collected (129 deselected)three times — which reads like a query that broke rather than an answer of none, and the answer of none is exactly the standing claim the step exists to make. It now prints a count:expected-red on basis_contract: 0. Both that and the--print-labelsline are guarded so pytest's exit code 5 cannot fail the step underbash -eo pipefail.No behaviour change: trigger, runner, Python version, timeout, and the install and run steps are untouched.
2. Release the page cache while streaming
The commit above should have been a no-op. Its CI run came back red, on
test_an_input_larger_than_the_container_is_not_fatal— a test that passes locally and passed onmain.It was not flaky. It was a real defect that
mainhad been passing by luck.node.pystreams correctly: it never holds more than one block of an object in memory. That is necessary and not sufficient. A container's memory limit counts the page cache its own reads and writes create, so moving a 128 MiB object through a temporary file leaves the kernel holding 128 MiB of cache for it inside a 64 MiB cgroup. The step's own flat footprint is irrelevant to the OOM killer, and the ending is exit 137 with no marker and no explanation — which classifies as transient, so the platform reads it as "retry me" for work that can never succeed.Fixed by asking for those pages back every 8 MiB on all four streaming paths — the input download, the multipart upload body, the line count and the digest — with
posix_fadvise(POSIX_FADV_DONTNEED), after anfsyncon the write side because dirty pages cannot be dropped. Best-effort throughout: it is advice to the kernel, and it costs nothing where it is unavailable.Measured by moving the harness's own ceiling:
The step's code is otherwise identical between those runs. The only difference is who was holding the bytes.
docs/PROTOCOL.md§3.5 andCLAUDE.mdnow say so, labelled RECOMMENDATION and marked as measured — because "stream and you are safe" is what every author will otherwise conclude from the existing text, and this is the failure that conclusion leads to.128 passed, 1 skippedlocally on this branch.🤖 Generated with Claude Code