Skip to content

fix(llms): require a fully-qualified base URL for link rewriting - #1613

Open
neon60 wants to merge 1 commit into
developfrom
fix/llms-base-url-validation
Open

fix(llms): require a fully-qualified base URL for link rewriting#1613
neon60 wants to merge 1 commit into
developfrom
fix/llms-base-url-validation

Conversation

@neon60

@neon60 neon60 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Motivation

_resolve_base_url accepted any truthy candidate, so a scheme-less html_baseurl such as "instinct.docs.amd.com" was emitted verbatim, producing malformed links like "instinct.docs.amd.com/page.html" in llms.txt and llms-full.txt. Validate each candidate with urlparse and use it only when it has both a scheme and a netloc; otherwise warn and fall back to relative links.

Technical Details

  • In src/rocm_docs/llms.py, _resolve_base_url now parses each candidate (rocm_docs_llms_base_url → html_baseurl → READTHEDOCS_CANONICAL_URL) with urllib.parse.urlparse and only accepts it when both scheme and netloc are present (e.g. https://example.com).
  • A candidate that is present but not fully-qualified (e.g. instinct.docs.amd.com) is skipped with a logger.warning, and resolution continues to the next candidate — ultimately falling back to relative links (page.html) rather than emitting a malformed absolute link.
  • No behavior change on Read the Docs: RTD sets READTHEDOCS_CANONICAL_URL to a fully-qualified https:// URL, which passes validation and continues to produce correct absolute links. The fix only affects builds where the resolved base URL lacks a scheme (typically local/non-RTD builds using a scheme-less html_baseurl default).- Added from urllib.parse import urlparse import.

Test Plan

  • Added three unit tests in tests/test_llms.py exercising _resolve_base_url directly:
    • a fully-qualified URL is accepted (and trailing slash stripped),
    • a scheme-less value is rejected, falling back to "" (relative links),
    • a scheme-less higher-priority candidate is skipped in favor of a valid lower-priority one.
  • Ran the full tests/test_llms.py suite to confirm no regression (the existing absolute-URL test uses a valid https:// base and must still pass).
  • End-to-end validation against a real consumer: built the system-acceptance-docs site (which has a scheme-less html_baseurl default) with this patched generator and inspected the generated llms.txt/llms-full.txt.

Test Result

  • Unit tests: 22 passed (19 existing + 3 new) in tests/test_llms.py, no regressions.
  • End-to-end build succeeded and logged the new warning: llms: ignoring base URL 'instinct.docs.amd.com' because it is not a fully-qualified URL (needs a scheme, e.g. https://); internal links will be relative. Generated links changed from the malformed AMD Instinct MI355X to the valid relative AMD Instinct MI355X.

Submission Checklist

_resolve_base_url accepted any truthy candidate, so a scheme-less
html_baseurl such as "instinct.docs.amd.com" was emitted verbatim,
producing malformed links like "instinct.docs.amd.com/page.html" in
llms.txt and llms-full.txt. Validate each candidate with urlparse and use
it only when it has both a scheme and a netloc; otherwise warn and fall
back to relative links.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant