Skip to content

ci: report whether the documentation followed the change, here and on the website - #719

Closed
HuggeK wants to merge 4 commits into
srcfl:masterfrom
HuggeK:ci/web-link-nudge
Closed

ci: report whether the documentation followed the change, here and on the website#719
HuggeK wants to merge 4 commits into
srcfl:masterfrom
HuggeK:ci/web-link-nudge

Conversation

@HuggeK

@HuggeK HuggeK commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

A change a user can see is finished in three places: the code, the documentation in here, and the description of FTW that people read before they clone anything. The third one has no reminder attached to it, and the first sign that it was skipped is a website that describes a version of FTW that no longer exists.

This adds .github/check-docs-follow-change.sh, run as a second job in the existing repo hygiene workflow on every PR to master, and says the same thing in AGENTS.md and CONTRIBUTING.md — the two places a contributor and an agent actually start.

What it reports

Always:

  1. Website links into this repository that no longer resolve. The target must be tracked by git, not merely present (drivers/*.lua is fetched into a working copy and 404s on github.com); a #fragment must match a heading that exists; tree/master/<dir> must hold tracked files; a link into a ref this repo does not publish is reported rather than skipped.
  2. The install command, which exists twice. The front page's curl … | bash is a copy of this README's, not something generated from it, so the two are compared verbatim.

Only when the change carries a changeset — this repository's own test for "a user can see this", so the check inherits that judgement instead of inventing a competing one, and changeset add --empty opts out of both at once:

  1. The documents in here that describe what you touched, and whether any moved with the change. Found with git grep, not a path-to-document table that goes stale the first time a package is renamed. A change to scripts/install.sh names README.md, docs/operations.md, docs/upgrade-from-legacy.md and all five docs/setup-guide/* translations — which is exactly the list you want in front of you before you claim the installer is documented.
  2. The website's sections, read out of its own index.html, as the question of which promise the change just altered:
the website is where this change becomes visible to anyone who has not cloned the
repository. It describes FTW in these sections today:
  #top             Your home energy, coordinated in the open.
  #start           Start with the software, not a sales call.
  #architecture    One local system across mixed hardware.
  #capabilities    A serious energy runtime, in a repository you can inspect.
  #extend          New hardware should be a driver, not a fork.
  #community       Free software, shared stewardship.
  #organizations   Open code. Commercial accountability.
Three design decisions worth arguing with

It never fails the merge. A broken link is a fact, but "does this need describing on the website" is a judgement call about a repository one over — one a contributor without commit rights there cannot act on anyway. A check that blocks on a judgement call gets satisfied rather than read. It emits a one-line ::warning:: and writes the full report to the job summary.

no-website-change on a line of its own in the PR description silences report 4. It has to be its own line — bulleted, ticked or followed by a reason is fine — so that a pull request describing the escape hatch does not take it. This description mentions the marker three times and still nudges. Without an escape hatch, an advisory nudge on every user-visible PR becomes background noise, and the first thing that becomes noise stops being read. With one, the decision is recorded where review can see it instead of living in the author's head. The body is read through the environment, never interpolated into the script.

Blame is computed, not guessed. Every link is resolved twice: once against the checkout and once against the PR's base commit. A link already broken when the branch started is reported under already broken before this change. Otherwise every PR touching README.md would be told it broke five links it never touched.

The rule that catches what existence checks miss

When the drivers moved out, drivers/ survived and BUNDLED_SOURCE.json is tracked, so the link kept resolving — a pure existence check sees nothing wrong. Hence a separate report for a browsed directory that a change empties out:

website links that resolve, but no longer hold what they send people to see:
  https://github.com/srcfl/ftw/tree/master/drivers
      still resolves, but this change removes 37 tracked file(s) under drivers/

That is real output, produced by running the check with #712 as the head and its parent as the base.

Verified against real states of this repository

Scenario Result
Site as published before srcfl/ftw-web#6 the 5 dead README anchors, split as pre-existing, exit 1
Site as published now (#6 merged) 15 checked, nothing to report, exit 0
Base = parent of #712 flags drivers/ as emptied of its 37 tracked drivers
Scratch commit renaming ## Install on Linux attributes #install-on-linux to broken by this change, lists the headings the README now has
Scratch commit changing the README's curl line reports the two install commands side by side
Scratch commit: scripts/install.sh + a minor changeset, no docs names the 10 documents that describe the installer
Same, with the marker on its own line in the body website: skipped
9 opt-out phrasings, including this PR's own body own-line forms skip, mid-sentence mentions still nudge

The link half of this is already proven in anger: run against the site as it was this morning, it reported exactly the five anchors that srcfl/ftw-web#6 (merged) had to fix.

No changeset needed for .github/** and *.md, but one is included since this is a behaviour contributors will meet.

🤖 Generated with Claude Code

ftw.sourceful.energy is a separate repository and it deep-links in here: a
README anchor behind each install button, docs/ pages, the raw install.sh URL
the front page tells people to pipe into bash. Nothing here knew those links
existed, so they rotted. Today the site's four install buttons point at
#option-a-raspberry-pi-sd-card-image through #option-d-build-from-source and
its closing "Get started" button at #quick-start -- five headings this README
has not had for months -- and "Browse drivers" opens a drivers/ tree that
stopped holding driver source when it moved to srcfl/device-drivers.

.github/check-web-links.sh resolves every one of those links against the
checkout: a file has to be tracked by git, not merely present, since
drivers/*.lua is fetched into a working copy and 404s on github.com; a
#fragment has to match a heading that exists; a browsed directory has to hold
tracked files. It resolves them against the base commit as well, so a link that
was already broken is reported as already broken instead of being blamed on the
pull request that happened to touch the file. A directory that survives a
change which empties it -- how the driver move read from in here -- gets its
own line.

The job warns and writes a job summary. It never fails the merge: the fix
belongs in srcfl/ftw-web, often in someone else's hands, and blocking a driver
fix here on a website pull request there would cost more than the broken link
does.

Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
@frahlg

frahlg commented Jul 30, 2026

Copy link
Copy Markdown
Member

Status update, because the ground moved under this PR today.

The broken links are fixed. srcfl/ftw-web#6 landed and is deployed. All five install/CTA links now resolve, verified against the live site:

#install-on-linux  #install-on-home-assistant  #local-development  #documentation

and each of those headings exists in the README on master. So the outage this PR opened with is over, and what is left here is the part that keeps it from happening again — which is the more valuable half anyway.

Two corrections to the description. A verification pass over ftw-web resolved the live link targets against GitHub's own rendered id="user-content-…" anchors rather than by computing slugs. The published buttons were #option-b-docker-installer and #option-c-home-assistant-os-add-on, not the docker-compose-style names quoted above. The count of five, and every other claim, held up.

One more thing this guard would have caught, if it is cheap to include: a link can resolve and still be wrong. Install button 03 pointed at a ## Install on Home Assistant section that said the add-on "has no published beta or stable app yet" — untrue since ftw-v0.1.0-beta.1 on 2026-07-22. #723 fixes the text. A pure link check will not catch that class of rot, so it is worth being explicit in the docs that this guard checks reachability, not accuracy.

Left as a draft — it is yours to mark ready.

A link check only catches the drift that happens to be a URL. The rest of it is
quieter: a capability described in docs/ and nowhere a prospective user reads, a
document that names the file you just rewrote and still describes the old
behaviour, an install command copied onto the front page that this README has
since changed.

So the check now reports four things, and the two new ones only speak when the
change carries a changeset -- this repository's own test for "a user can see
this", which means the check inherits that judgement instead of inventing a
competing one, and `changeset add --empty` opts out of both at once.

  - website links into this repository that no longer resolve, still split by
    whether this change is what broke them;
  - the install command the site tells people to pipe into bash, compared with
    the one this README publishes;
  - the documents in here that name the paths the change touched, and whether
    any of them moved with it -- derived with git grep, so there is no
    path-to-document table to leave stale;
  - the website's own sections, read out of its index.html, as the question of
    which promise the change just altered.

`no-website-change` in the pull request description silences the last one and
records the decision where review can see it. AGENTS.md and CONTRIBUTING.md say
the same thing in the two places a contributor and an agent actually start.

Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
@HuggeK HuggeK changed the title ci: tell the author when a change breaks a link on the website ci: report whether the documentation followed the change, here and on the website Jul 30, 2026
The marker was matched anywhere in the pull request description, so a PR that
explains the escape hatch takes it. This one's own description does exactly
that, and silenced the section it was describing.

Co-authored-by: HuggeK <48095810+HuggeK@users.noreply.github.com>
@HuggeK HuggeK closed this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants