ci: report whether the documentation followed the change, here and on the website - #719
ci: report whether the documentation followed the change, here and on the website#719HuggeK wants to merge 4 commits into
Conversation
ftw.sourceful.energy is a separate repository and it deep-links in here: a README anchor behind each install button, docs/ pages, the raw install.sh URL the front page tells people to pipe into bash. Nothing here knew those links existed, so they rotted. Today the site's four install buttons point at #option-a-raspberry-pi-sd-card-image through #option-d-build-from-source and its closing "Get started" button at #quick-start -- five headings this README has not had for months -- and "Browse drivers" opens a drivers/ tree that stopped holding driver source when it moved to srcfl/device-drivers. .github/check-web-links.sh resolves every one of those links against the checkout: a file has to be tracked by git, not merely present, since drivers/*.lua is fetched into a working copy and 404s on github.com; a #fragment has to match a heading that exists; a browsed directory has to hold tracked files. It resolves them against the base commit as well, so a link that was already broken is reported as already broken instead of being blamed on the pull request that happened to touch the file. A directory that survives a change which empties it -- how the driver move read from in here -- gets its own line. The job warns and writes a job summary. It never fails the merge: the fix belongs in srcfl/ftw-web, often in someone else's hands, and blocking a driver fix here on a website pull request there would cost more than the broken link does. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
|
Status update, because the ground moved under this PR today. The broken links are fixed. srcfl/ftw-web#6 landed and is deployed. All five install/CTA links now resolve, verified against the live site: and each of those headings exists in the README on Two corrections to the description. A verification pass over One more thing this guard would have caught, if it is cheap to include: a link can resolve and still be wrong. Install button 03 pointed at a Left as a draft — it is yours to mark ready. |
A link check only catches the drift that happens to be a URL. The rest of it is
quieter: a capability described in docs/ and nowhere a prospective user reads, a
document that names the file you just rewrote and still describes the old
behaviour, an install command copied onto the front page that this README has
since changed.
So the check now reports four things, and the two new ones only speak when the
change carries a changeset -- this repository's own test for "a user can see
this", which means the check inherits that judgement instead of inventing a
competing one, and `changeset add --empty` opts out of both at once.
- website links into this repository that no longer resolve, still split by
whether this change is what broke them;
- the install command the site tells people to pipe into bash, compared with
the one this README publishes;
- the documents in here that name the paths the change touched, and whether
any of them moved with it -- derived with git grep, so there is no
path-to-document table to leave stale;
- the website's own sections, read out of its index.html, as the question of
which promise the change just altered.
`no-website-change` in the pull request description silences the last one and
records the decision where review can see it. AGENTS.md and CONTRIBUTING.md say
the same thing in the two places a contributor and an agent actually start.
Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
The marker was matched anywhere in the pull request description, so a PR that explains the escape hatch takes it. This one's own description does exactly that, and silenced the section it was describing. Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
A change a user can see is finished in three places: the code, the documentation in here, and the description of FTW that people read before they clone anything. The third one has no reminder attached to it, and the first sign that it was skipped is a website that describes a version of FTW that no longer exists.
This adds
.github/check-docs-follow-change.sh, run as a second job in the existingrepo hygieneworkflow on every PR tomaster, and says the same thing inAGENTS.mdandCONTRIBUTING.md— the two places a contributor and an agent actually start.What it reports
Always:
drivers/*.luais fetched into a working copy and 404s on github.com); a#fragmentmust match a heading that exists;tree/master/<dir>must hold tracked files; a link into a ref this repo does not publish is reported rather than skipped.curl … | bashis a copy of this README's, not something generated from it, so the two are compared verbatim.Only when the change carries a changeset — this repository's own test for "a user can see this", so the check inherits that judgement instead of inventing a competing one, and
changeset add --emptyopts out of both at once:git grep, not a path-to-document table that goes stale the first time a package is renamed. A change toscripts/install.shnamesREADME.md,docs/operations.md,docs/upgrade-from-legacy.mdand all fivedocs/setup-guide/*translations — which is exactly the list you want in front of you before you claim the installer is documented.index.html, as the question of which promise the change just altered:Three design decisions worth arguing with
It never fails the merge. A broken link is a fact, but "does this need describing on the website" is a judgement call about a repository one over — one a contributor without commit rights there cannot act on anyway. A check that blocks on a judgement call gets satisfied rather than read. It emits a one-line
::warning::and writes the full report to the job summary.no-website-changeon a line of its own in the PR description silences report 4. It has to be its own line — bulleted, ticked or followed by a reason is fine — so that a pull request describing the escape hatch does not take it. This description mentions the marker three times and still nudges. Without an escape hatch, an advisory nudge on every user-visible PR becomes background noise, and the first thing that becomes noise stops being read. With one, the decision is recorded where review can see it instead of living in the author's head. The body is read through the environment, never interpolated into the script.Blame is computed, not guessed. Every link is resolved twice: once against the checkout and once against the PR's base commit. A link already broken when the branch started is reported under already broken before this change. Otherwise every PR touching
README.mdwould be told it broke five links it never touched.The rule that catches what existence checks miss
When the drivers moved out,
drivers/survived andBUNDLED_SOURCE.jsonis tracked, so the link kept resolving — a pure existence check sees nothing wrong. Hence a separate report for a browsed directory that a change empties out:That is real output, produced by running the check with #712 as the head and its parent as the base.
Verified against real states of this repository
15 checked, nothing to report, exit 0drivers/as emptied of its 37 tracked drivers## Install on Linux#install-on-linuxto broken by this change, lists the headings the README now hascurllinescripts/install.sh+ aminorchangeset, no docswebsite: skippedThe link half of this is already proven in anger: run against the site as it was this morning, it reported exactly the five anchors that srcfl/ftw-web#6 (merged) had to fix.
No changeset needed for
.github/**and*.md, but one is included since this is a behaviour contributors will meet.🤖 Generated with Claude Code