From 26adba3ce04ce0ea73269bc9821a6785eb94a6b5 Mon Sep 17 00:00:00 2001 From: Fredrik Ahlgren Date: Tue, 28 Jul 2026 20:06:21 +0200 Subject: [PATCH] build: fetch the bundled drivers, and prove a build needs nothing else Groundwork for FTW holding no driver source at all. Nothing is deleted here and the committed files still ship; what changes is that fetching them becomes a first-class build step and CI proves it sufficient. make drivers fetch drivers/ from the pinned commit make drivers-present fail with the remedy when the directory is empty `test` depends on the presence check rather than on the fetch, so the inner loop keeps costing no network. The new drivers-from-pin job is the point. It deletes the committed drivers, confirms the guard fires, fetches from the pin, checks all 37 came back, runs the driver suites against the fetched copies, and asserts they are byte-identical to what was committed. While the files are still in git that is redundant with the drift check. The moment they leave git it is the only thing between a fresh clone and a build that cannot find a driver. The fetch writes to the same path the committed files occupy, so nothing that reads drivers/ needs to know where they came from. The fifteen Go test files that build that path by hand are untouched, and so are the Dockerfile and the release tarballs. Verified locally: rm drivers/*.lua, guard fires, make drivers restores 37 of 37, go test ./internal/drivers/... ./internal/driverinventory/... passes, git diff on drivers/ is empty. Co-Authored-By: Claude Opus 5 Signed-off-by: Fredrik Ahlgren --- .../fetch-bundled-drivers-at-build-time.md | 5 ++ .github/workflows/test.yml | 50 +++++++++++++++++++ Makefile | 25 +++++++++- README.md | 10 +++- 4 files changed, 86 insertions(+), 4 deletions(-) create mode 100644 .changeset/fetch-bundled-drivers-at-build-time.md diff --git a/.changeset/fetch-bundled-drivers-at-build-time.md b/.changeset/fetch-bundled-drivers-at-build-time.md new file mode 100644 index 00000000..7f6822bf --- /dev/null +++ b/.changeset/fetch-bundled-drivers-at-build-time.md @@ -0,0 +1,5 @@ +--- +"ftw": patch +--- + +`make drivers` fetches the bundled snapshot from the pinned device-drivers commit, and CI proves a build works from the pin alone. diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index c0a77f79..9d3b7642 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -164,6 +164,56 @@ jobs: - run: npm ci --prefer-offline --no-audit --no-fund - run: npm test + # Proves the cutover before it happens: throw the committed drivers away, + # fetch them from the pin alone, and run the suites that read them. While + # the files are still committed this is redundant with the drift check. The + # moment they stop being committed it is the only thing standing between a + # fresh clone and a build that cannot find a driver. + drivers-from-pin: + name: drivers build from the pin alone + needs: changes + if: needs.changes.outputs.drivers == 'true' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - uses: actions/setup-go@v7 + with: + go-version: '1.26' + cache-dependency-path: go/go.sum + + - name: Discard the committed snapshot + run: rm -f drivers/*.lua + + - name: The presence guard notices + run: | + if make drivers-present 2>/dev/null; then + echo "::error::drivers-present passed with no drivers on disk" + exit 1 + fi + echo "guard fired as expected" + + - name: Fetch from the pin + run: make drivers + + - name: Every pinned driver came back + run: | + want="$(jq -r '.drivers | length' drivers/BUNDLED_SOURCE.json)" + got="$(ls drivers/*.lua | wc -l | tr -d ' ')" + echo "pinned ${want}, fetched ${got}" + [ "$want" = "$got" ] || { echo "::error::fetched ${got} of ${want}"; exit 1; } + + # The fetched files land at the same path the committed ones did, so + # nothing that reads drivers/ needs to know where they came from. + - name: The driver suites pass against the fetched copies + working-directory: go + run: go test ./internal/drivers/... ./internal/driverinventory/... + + - name: They are byte-for-byte what was committed + run: | + git diff --exit-code -- drivers/ \ + || { echo "::error::fetched snapshot differs from the committed one"; exit 1; } + echo "identical" + drivers: name: drivers needs: changes diff --git a/Makefile b/Makefile index e1a41522..e1e8d87f 100644 --- a/Makefile +++ b/Makefile @@ -13,7 +13,8 @@ .PHONY: help test optimizer-install optimizer-test compose-migration-test container-boundary-test build build-arm64 build-amd64 build-windows-amd64 release \ run-sim dev fmt vet clean e2e ci ci-ui ci-hw-pi docs \ - verify verify-all install-hooks driver-repository-validate driver-versions + verify verify-all install-hooks driver-repository-validate driver-versions \ + drivers drivers-present VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null || echo dev) LDFLAGS := -s -w -X main.Version=$(VERSION) @@ -42,11 +43,31 @@ help: @echo " ci-ui browser smoke against FTW_BASE_URL" @echo " ci-hw-pi deploy candidate to Pi CI slot + browser smoke" @echo " fmt vet Go format + static checks" + @echo " drivers fetch drivers/ from the pinned device-drivers commit" @echo " clean nuke build artifacts" +# ---- Bundled drivers ---- +# +# drivers/ is a snapshot of srcfl/device-drivers at the commit pinned in +# drivers/BUNDLED_SOURCE.json. It is fetched, not authored here. The Go tests +# read it, the container image copies it, and the release tarballs carry it, +# so it has to be populated before any of those run. + +# Fetch the snapshot. Safe to re-run; it writes the same bytes every time. +drivers: + bash scripts/sync-bundled-drivers.sh + +# Cheap enough to run before every test invocation, and it costs no network. +# Fetching here instead would put a remote call in the inner loop. +drivers-present: + @ls drivers/*.lua >/dev/null 2>&1 || { \ + echo "drivers/ has no .lua files. Run 'make drivers' to fetch the" >&2; \ + echo "snapshot pinned in drivers/BUNDLED_SOURCE.json." >&2; \ + exit 1; } + # ---- Testing ---- -test: optimizer/.venv/.installed +test: optimizer/.venv/.installed drivers-present @status=0; \ optimizer/.venv/bin/pytest -q optimizer/tests & py_pid=$$!; \ (cd go && go test ./...) & go_pid=$$!; \ diff --git a/README.md b/README.md index 498a52af..e812b511 100644 --- a/README.md +++ b/README.md @@ -50,8 +50,14 @@ editable source and FTW's default signed driver channel. [`drivers/BUNDLED_SOURCE.json`](drivers/BUNDLED_SOURCE.json). It exists because startup is deliberately local: a gateway boots and runs without the network, so a remote refresh must never block it. Editing a `.lua` file here is not a way to -change a driver — fix it upstream, move the pin, and run -`scripts/sync-bundled-drivers.sh`. CI fails if the two drift. +change a driver — fix it upstream, move the pin, and run `make drivers`. +CI fails if the two drift. + +A fresh clone gets the files from git today. `make drivers` fetches the same +bytes from the pin, and CI proves the two agree on every pull request by +throwing the committed copies away and rebuilding from the pin alone. That is +the groundwork for dropping them from git entirely, so this repository holds no +driver source at all. ## Install on Linux