fix(ui): publish robots.txt at the site root again

[DeepDiver1975]

Closes #100.

Root cause

Not simply "the file was forgotten" — the mechanism for publishing root-level static files was never carried over from docs-ui.

@antora/ui-loader decides where a UI file lands in classifyFile() (node_modules/@antora/ui-loader/lib/load-ui.js): a file is published at the site root only if it matches a glob from the static_files: key of a ui.yml descriptor found in the merged UI file map (bundle files ⊕ ui.supplemental_files). Everything else is classified as an asset and written under ui.output_dir, i.e. assets/.

Evidence:

• There was no ui.yml anywhere in this repo, and the stock antora-ui-default bundle ships none either (74 files, verified in the cached bundle) — so no code path existed that could put any file at the site root.
• The retired docs-ui had src/ui.yml with a 32-entry static_files: list including robots.txt. That descriptor, not the file's location, is what made /robots.txt serve.
• Legacy content confirmed byte-for-byte from docs-ui@master:src/static/robots.txt (60 bytes, trailing newline).

The migration copied docs-ui's branding assets into ui/supplemental/ but dropped its ui.yml, so robots.txt had nowhere to land — and the sitemap that antora-extensions/sitemap-cleanup.js still publishes has had nothing pointing crawlers at it since.

Changes

• ui/supplemental/ui.yml — the missing UI descriptor, listing robots.txt under static_files. Documents that it replaces a bundle ui.yml wholesale, should the stock bundle ever gain one.
• ui/supplemental/robots.txt — legacy content, plus an explicit empty Disallow: — RFC 9309 §4.2.1's way of saying "crawl everything". The legacy file's bare User-agent: * group carried no rule at all.
• test/static-files.test.js — build-output guards in the same style as latest-alias.test.js (assert against public/, skip when unbuilt): robots.txt at the root, its Sitemap: line matching site.url derived from site.yml (so a domain change can't leave it stale), the advertised sitemap.xml actually present in the build, and neither robots.txt nor ui.yml leaking into assets/. That last check covers the exact silent failure mode here — a missing or incomplete descriptor still builds green.

Written test-first: it failed with robots.txt was not published to the site root before the fix.

Verification

Full npm run antora rebuild, then public/ served locally:

robots.txt    200 (text/plain; charset=UTF-8)
sitemap.xml   200
assets/ui.yml 404

npm test: 20/20 pass (17 pre-existing + 3 new). public/assets/{css,js,img,font} unchanged — adding the descriptor does not disturb ui.output_dir.

Out of scope

• docs-ui also listed .nojekyll in static_files. It is unnecessary now: upload-pages-artifact / deploy-pages serves the artifact directly, with no Jekyll pass.
• The llms.txt question from robots.txt missing on doc.owncloud.com (dropped during monorepo migration) #100 — that issue itself suggests splitting it out, and it is not a regression.

🤖 Generated with Claude Code