gate: make green mean green -- fail-closed evidence, and one product profile - #2
Closed
ebt55 wants to merge 29 commits into
Closed
gate: make green mean green -- fail-closed evidence, and one product profile#2ebt55 wants to merge 29 commits into
ebt55 wants to merge 29 commits into
Conversation
…S.md
Docs and metadata only -- no converter code touched, no measured number moves.
The repo carried release-grade claims for something never released: pyproject
said 0.2.0/Beta, the README led with a 15/18 gate on an 18-document corpus that
no longer exists, and the holdout that falsifies the headline (0/4 on wild PDFs)
was three sections away from it. Nothing has been published, so the version reset
is free today and expensive after the first release.
- pyproject: 0.2.0 -> 0.1.0a1, Development Status 4-Beta -> 3-Alpha
- README: corpus and holdout numbers now sit in the same table; every figure
reconciled to STATUS.md (16 docs, 13/16 refine lane, 15/16 pages, within2pt
0.510, dy50 0.69pt, 7 pdfium parity regressions -- was 9); the stale SSIM mean
dropped rather than restated, since the file itself says never to headline it;
unrunnable `my_samples` example replaced with `corpus/pdfs`; duplicated
target-renderer section merged; Versions ladder stated (no AGPL wheel will
ever be published)
- FINDINGS: banner line -- v1.1 is a pre-release internal label, no tag
- STATUS: two inline notes where an 18-document figure was being read as current
- SESSIONS.md: created; the working log the protocol requires
- uv.lock: records the pdfium extra, which it did not before -- the same class
of omission that once left the parity gate reporting ModuleNotFoundError
Verified: `uv run python -c importlib.metadata.version('exactdoc')` -> 0.1.0a1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… 15 -> 16) Two separate provisioning failures, both silent. 1. Chromium absent: gen_corpus called subprocess.run([None, ...]) and died on a bare TypeError before writing anything -- including the ReportLab and fpdf2 documents that need no browser at all. Now the tools are probed up front, a missing one skips its documents and prints what to set (CHROME=/SOFFICE=), and the run exits 0 with the skip list plus an explicit warning that an incomplete corpus is not comparable to the recorded baselines. A tool that IS present and fails is the opposite case and now exits 1 with its stderr. 2. l1_word_native was never generated on this machine and nobody could tell: the LibreOffice profile was passed as `file:///` + a RELATIVE path, which soffice resolves against the filesystem root. It exited 1, wrote nothing, and the function returned None without printing. The corpus came back with 15 documents instead of 16 and every figure downstream was quietly computed over a different corpus than the one on record. harness.py never had this bug -- it builds the profile from tempfile.gettempdir(). before: 10 PDFs in testkit/adv (15-document corpus), no diagnostic after: 11 PDFs in testkit/adv (16-document corpus), capability line printed python testkit/gen_corpus.py testkit/adv tests/test_corpus_degradation.py covers both directions by running the generator with the tools patched out; the bare-machine case exits 0 having still produced the two pure-Python documents, and a CHROME pointing at nothing exits 1 instead of raising FileNotFoundError. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gate is this project's real product and its dependencies were folklore: LibreOffice, headless Chromium and the metric-compatible fonts are all required and none of them was declared anywhere. Verified by running it against a bare ubuntu:24.04 container populated from `git archive HEAD` -- a literal fresh clone, with no python installed at all. before: hand-provisioning, undocumented, discovered by reading source after: bash scripts/bootstrap.sh -> 6/6 capabilities OK, exit 0 Two findings from that run, both now encoded: - Executable is not the same as working. Ubuntu`s `chromium-browser` apt package is a snap shim: it installs, sits on PATH, is executable, and exits 1 on every invocation inside a container. So the finders probe with --version rather than test -x. The corpus generator caught this correctly (8 FAILED, exit 1) precisely because of the previous commit -- before it, the run would have produced 8 documents fewer and said nothing. - Playwright`s headless shell needs its system libraries (libatk, libnss...); without them the binary is present, executable, and exits 127. Installed with --with-deps when we are root on apt. Also declares pypdfium2 in testkit/README`s quick start, which omitted it -- the exact omission that once left backend_parity.py reporting ModuleNotFoundError instead of regressions -- and documents the SOFFICE/CHROME overrides in a table rather than a sentence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
scripts/bootstrap.sh is authored on Windows and executed on Linux. With the default autocrlf the shebang arrives as `#!/usr/bin/env bash\r`, the kernel looks for an interpreter literally named bash\r, and the resulting error names the interpreter rather than the line ending. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The goldens are a comparison between two runs on a corpus that is REGENERATED
on every machine, so they are only meaningful against a declared environment --
and they declared none. Each golden now carries {pymupdf, reportlab, fpdf2,
python, platform} and `verify` names the fields that differ instead of letting
environment drift arrive looking exactly like parser breakage.
Which matters, because the drift attributed to the platform is not the
platform. Measured in an ubuntu:24.04 container, changing one variable at a
time:
pymupdf 1.28.0, reportlab 5.0.0, Linux 7/7
pymupdf 1.28.0, reportlab 4.5.1, Linux 7/7
pymupdf 1.26.0, reportlab 5.0.0, Linux 6/7 02_research_paper p2 blocks 7->4
pymupdf 1.24.14, reportlab 5.0.0, Linux 6/7
That is the exact symptom previously recorded as cross-platform drift,
reproduced on one machine by changing only the parser library. The goldens are
pinned to the PyMuPDF version, not to the operating system -- Windows and Linux
agree bit for bit at equal versions.
uv run python testkit/golden_ir.py verify (7/7 both platforms, pymupdf 1.28.0)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ent-neutral Protocol says a re-freeze is a decision with its own reviewable commit and a written justification, never a way to make a failure disappear. This one makes no failure disappear: the goldens verified 7/7 both before and after, on both platforms. Its only purpose is to attach the manifest the previous commit added, and to move the freeze environment to the canonical one (Linux, as gate.yml already declares CI the number of record) now that Linux and Windows are measured to agree bit for bit at equal library versions. Content-neutrality is checked, not asserted: every golden was compared against its committed version with the new `manifest` key stripped. checked 7 goldens content-neutral: every golden gained a manifest and nothing else moved Frozen with pymupdf 1.28.0, reportlab 5.0.0, fpdf2 2.8.7, python 3.12, Linux. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…3 -> 1.00)
The gate failed a correct conversion, and which machine you ran it on decided
whether it did.
page_words tokenises with PyMuPDF's words extractor, which splits on
whitespace. Chinese, Japanese and Korean do not use any, so a "word" is an
entire rendered line -- up to 32 characters here. LibreOffice re-wraps that
line one character earlier than the source and the token matches nothing,
though every character survived. Windows happened to wrap it the same way;
Linux did not. That is the whole of the difference between 13/16 and 12/16 on
the refine lane.
Attribution, not assumption. Two hypotheses were tested:
fonts FALSIFIED. Source PDFs are character-identical across the two
platforms (82 ideographs / 37 Hangul / 33 Kana / 88 Arabic / 71
Hebrew on both) and the render-back carries all of them. Installing
fonts-noto-cjk moved doc_recall by 0.0000.
tokens CONFIRMED. Of 94 source tokens, 16 unmatched -- 11 Hangul, 4 CJK,
1 Kana, and zero Latin, Arabic or Hebrew. Arabic and Hebrew DO use
spaces and never had the problem. Mean unmatched token 9.3 chars
against 4.4 matched.
So runs in scriptio-continua scripts are tokenised per character, with the run
box divided across them. Verified by re-scoring the EXISTING renders -- same
DOCX, same render-back on disk, so only the metric changed:
c4_i18n doc_recall 0.8298 -> 1.0000 within2pt 0.3333 -> 0.4160
other 15 identical to four decimal places, every metric
The prediction written down before running was "c4_i18n moves, nothing else
does". One document moved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… 0 on Linux)
runall.py exited 1 if any document missed a threshold, and three documents have
never cleared them: c3_tables (D3, nested tables flatten), c5_graphics (rasterised
SVG + gradient band) and 04_exec_brief (live-text coverage 0.941 against 0.95).
So the gate returned non-zero on every run it had ever made, on every platform.
That is why the CI step carried continue-on-error -- not because the Linux
thresholds were uncalibrated, but because the check could never pass. A check
that always fails carries exactly as much information as one that always passes,
and this repository already has a chapter on what that costs (STATUS.md §5).
The known-failing set is now recorded per lane in testkit/gate_baseline.json,
measured on the canonical Linux environment, and the run fails on:
- a document failing that the record says should pass
- a NEW metric failing on an already-failing document
- a STALE record: a document that passes while the record says it fails,
because a record that over-permits silently re-admits the regression it
was meant to catch
Recorded and then verified against itself on ubuntu:24.04:
9 known failure(s) in the record, 0 new, 0 stale GATE_EXIT=0
Linux baseline, both lanes, 16 documents:
lane pagematch within2pt livetext dy50med
norefine 13/16 0.349 0.9652 2.20
refine 15/16 0.512 0.9652 0.62
gate passed: 12/16 norefine, 13/16 refine
which agrees with the Windows figures in STATUS.md §1 (0.361/0.510 within2pt,
2.79/0.69 dy50) across different fonts, a different LibreOffice build and a
different Chromium. The harness is portable; only its provisioning was folklore.
Re-record deliberately: GATE_BASELINE=update REFINE=lanes python testkit/runall.py ...
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
continue-on-error is gone from the fidelity lanes. It was there because
runall.py could not pass (previous commit); with the regression baseline
recorded on ubuntu:24.04 it exits 0, so an actual regression will now turn the
build red instead of being uploaded as an artifact nobody opens.
- provisioning is scripts/bootstrap.sh --strict, the same command a
contributor runs, so CI cannot drift away from the documented setup
without going red. --strict makes a missing oracle fatal: an incomplete
corpus silently produces numbers for a different corpus than the one on
record
- dependencies come from uv.lock (--frozen via uv). The goldens are pinned to
the PyMuPDF version -- measured, 1.26 and 1.24 both put 02_research_paper
p2 at 4 blocks where 1.28 puts 7 -- so an unpinned resolve would fail the
golden step for a reason that has nothing to do with this repository
- backend_parity.py runs for the first time in CI. Reporting-only by design:
it is the number M2 is driving to zero, so non-zero is the expected state.
It runs because it once lost the ability to run at all (an undeclared
pypdfium2) and nobody noticed
- tests/test_corpus_degradation.py added to the unit step
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
STATUS §1 now carries the canonical Linux numbers beside the Windows ones,
because two environments agreeing is itself the evidence that the harness is
portable:
lane pagematch within2pt livetext dy50med
norefine 13/16 0.349 0.9652 2.20
refine 15/16 0.512 0.9652 0.62
against Windows 0.361/0.510 and 2.79/0.69 -- different fonts, a different
LibreOffice build, a different Chromium.
Three entries added to §5, the register of measurement mistakes, since each
produced a confident wrong answer during this milestone:
- gating on any failure when three documents had never passed, so the check
could never pass and had to be ignored
- tokenising words on whitespace for a script that has none
- reading a PyMuPDF-version difference as cross-platform golden drift
SESSIONS.md closes M1 with each acceptance box and the command output beside
it, including the one box that stays unchecked: CI has not been observed green
because the branch is not pushed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
backend_geom.py answered "is the geometry the same" (yes: baselines identical
on 4734 of 4734 lines). exp_regroup.py answered "is it the grouping" (for about
half the failing documents, yes). Two documents were explained by neither and
the defect register carried them as unattributed, which is how a plausible
guess ends up being read as a known cause -- this repository has a chapter on
that (STATUS.md §5).
backend_spans.py pairs lines across backends by baseline and x, then diffs span
structure, text, injected space runs, mono flags and style keys. Run on the two
failures with two passing documents as controls, it falsifies every candidate
hypothesis in one pass:
c7_code 03_tech f1 (ok) r1 (ok)
space-run diff 0% 0% 0% 0%
text diff 0% 36% 40% 42%
lines unmatched 16/26 9/73 0/20 0/36
Multi-space synthesis is not it -- space runs agree everywhere. Different text
is not it either: the two PASSING controls have more text difference (40%, 42%)
than the failing c7_code, which has none at all. What c7_code has is 16 of 26
lines that do not pair with their PyMuPDF counterpart on position.
--only narrows backend_parity.py to named documents. The full run converts 16
documents twice and renders both; a hypothesis about one document should cost
that document, not the corpus. It prints a warning that a subset run cannot
report the swap as acceptable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both backend.py and STATUS.md D2 called testkit/golden_ir.py "the
specification". That framing is what kept the port aimed at reproducing another
library's private behaviour bit for bit, and it is wrong for three measured
reasons:
- This backend already refuses to reproduce three PyMuPDF behaviours BECAUSE
THEY ARE BUGS: RTL returned in visual order (renders Arabic backwards),
gradients dropped (white text left invisible on white), Calibri reported as
serif. A specification you deliberately violate is not a specification.
- PyMuPDF's grouping is not stable across its own point releases. Measured
this session: 1.24.14 and 1.26.0 put page 2 of 02_research_paper in 4
blocks, 1.28.0 puts it in 7. A target that moves with a dependency version
cannot be a finish line.
- The golden is regenerated from a corpus that is itself regenerated, so it
describes an environment as much as a parser.
The golden IR is a fast, oracle-free, per-document diff for locating WHERE two
parsers disagree. backend_parity.py decides whether the disagreement matters.
When they conflict, the parity gate wins.
STATUS D2 also gains the attribution for the code-heavy pair, which it had
carried as "explained by neither geometry, grouping nor fonts": PDFium does not
report leading indentation and PyMuPDF synthesises it -- for ` def
__init__(...)` PDFium's first character is `d` at x=93.17 with no space before
it, against PyMuPDF's line starting at x=72.25 with four leading spaces.
docs only, no measured number moves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ode .16 -> .59)
The code-heavy pair was the one part of the defect register with no attributed
cause, and none of the four suspected causes was it. Measured with the new
testkit/backend_spans.py: space runs agree on 100% of lines in every document,
so multi-space synthesis is not it; c7_code has ZERO text differences while the
two passing controls have 40% and 42%, so different text is not it either.
What c7_code had was 16 of its 26 lines failing to pair with their PyMuPDF
counterpart on position at all.
The cause, read off the raw character stream rather than inferred from the
grouping code: for ` def __init__(...)`, PDFium's first character is `d` at
x=93.17 with no space anywhere before it, while PyMuPDF reports the same line
starting at x=72.25 with four leading spaces. PDFium synthesises the spaces a
producer drew by positioning -- but only BETWEEN two characters, because that is
the only place a gap exists to measure. At the start of a line there is nothing
to the left of the first glyph, so the indent is simply gone. The line box then
starts at first ink, the paragraph is written at the wrong x, and every glyph on
the line is displaced by the width of the indent.
Reconstruction measures each line against the leftmost line of its monospace
run -- consecutive mono lines, which is the extent of one listing. The block's
own minimum will not do: a block whose every line is indented would measure zero
indent.
Lines that SHARE a baseline are excluded, and that is not a detail. A
configuration table whose cells are monospace puts three of them on one baseline
at x=61/153/223; read as a listing, the second and third are "indented" by 18
and 32 spaces and dragged back to the margin. Measured cost of that bug before
the exclusion: 03_tech_report_code 0.23 -> 0.03.
full corpus, backend_parity.py --refine 3, ubuntu:24.04
c7_code 0.16 -> 0.59 (pymupdf 0.91)
03_tech_report_code 0.23 -> 0.02 (pymupdf 0.46)
the other 14 unchanged
verdict count 8 regressions, unchanged
03_tech_report_code gets worse while the IR gets more faithful, so its loss is
downstream of the parser, not in it. Landed deliberately with the owner's
decision and that trace is the next task, not a mystery being shipped past.
Also recorded in the code: a local-median pitch estimator for the block splitter
was tried here and REVERTED. It split the code listing correctly and cost
02_research_paper 0.57 -> 0.02, because a local window inside a dense two-column
body finds a pitch small enough to cut paragraphs in half.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…red worse The owner lifted the forbidden-file rule so the code-heavy regression could be traced into infer.py/docxout.py. It never needed to go there -- the cause is in the parser -- and the fix it suggested was then rejected by the gate. Both halves are recorded because the second is the useful one. WHERE 03 LOSES IT. Rendered x for two source lines that both start at x=84.40: PyMuPDF render 90.25 and 90.25 (+5.9, correct and consistent) PDFium render 199.70 and 281.30 (+115 and +197) Two lines sharing a source x landing 80pt apart is a structural failure, not drift: the listing is laid out as flowing prose. PyMuPDF builds that region as role=code, one 504pt column, line_breaks=True, 10 visual lines, 15 runs. PDFium built it as role=table with columns of 3.0pt and 501.0pt and three paragraphs with line_breaks=False. WHY. The backends disagree about what a stroked path's bbox means. FPDFPageObj_GetBounds returns the ink envelope -- the path inflated by its line width in every direction -- where PyMuPDF returns the geometric path: box border, 0.75pt PyMuPDF x=54.00..54.00 (w 0) PDFium x=53.25..54.75 (w 1.50) callout accent, 3pt PyMuPDF x=57.00..57.00 (w 0) PDFium x=54.00..60.00 (w 6.00) infer.py's table detector reads that 1.5pt bar as a column boundary, which is exactly the phantom 3pt first column. _classify already compensated for this for ORIENTATION by reading path points rather than bounds; taking the bbox from the same place makes PDFium build the region identically to PyMuPDF -- role=code, one column, line_breaks=True, 10 lines, 15 runs, an exact structural match. AND THE GATE REJECTED IT. Full corpus, backend_parity.py --refine 3: 8 regressions -> 9. 01_whitepaper_market pages 3/3 -> 3/4 and w 0.31 -> 0.01; 02_research_paper 2/2 -> 2/3; c5_graphics 1/1 -> 1/2; r1_reportlab_report became a regression at 0.57 -> 0.38; c7_code 0.59 -> 0.30 with word_recall 1.00 -> 0.95. Reverted. A stroked box's ink envelope CONTAINS its text while the geometric path is the centreline, so making the bbox faithful makes containment tests fail at the edges, and box detection loses boxes: c7_code goes from two TableEls to one. The convention is not independently correct or incorrect -- it has to match whatever the containment tests were tuned against, and they were tuned against PyMuPDF's. That is the third time this session a demonstrably more faithful IR scored worse, and it is the strongest evidence yet for the plan's RC1: the downstream is tuned to PyMuPDF's shape, including the arbitrary parts of that shape. No code change: parse_pdfium.py is back at the previous commit. Session log only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Run 30455217670 on PR #1, green in 3m21s, every step passing: bootstrap.sh --strict provisioned a real runner corpus 16 documents unit tests purity + corpus degradation golden IR 7/7 documents match lane norefine 12/16 pass · 12 known, 0 new, 0 stale lane refine 13/16 pass · 9 known, 0 new, 0 stale lane comparison norefine 13/16 0.366 0.9652 2.20 refine 15/16 0.529 0.9652 0.68 backend parity 8 regressions, 7 same, 1 better Two things this establishes that the container run could not. The regression baseline is PORTABLE. It was recorded on a local ubuntu:24.04 and transferred to GitHub's runner with 0 new and 0 stale in BOTH lanes -- different LibreOffice build, different Chromium, different font package versions. A baseline that only held on the machine that produced it would have been a liability rather than a gate. The goldens survive a second, independent Linux. 7/7 there, having been frozen in a local container -- which is the cross-platform stability claim this project had been unable to make, and the reason it could not make it was a PyMuPDF version difference rather than the operating system. STATUS §1 now carries CI as the number of record with the container and Windows columns beside it. Three environments agree on every structural number and differ only in the third decimal of within2pt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…578 transformed) Law 15, written because getting this wrong cost a session. _page_paths takes bboxes from FPDFPageObj_GetBounds (page space, ink envelope) while _classify, _rect_pts and the frame-edge decomposition read FPDFPath_GetPathSegment points -- which PDFium reports in OBJECT space, before the path object's transform. Those two families of numbers are not comparable, and an experiment last session replaced the bbox with a raw-points bbox, silently scaling whole pages. The failure mode is invisible at the microscope: two paths eyeballed by hand were identity-matrix paths, which agree perfectly. It is only visible across a corpus. document paths non-id raw ok worst raw miss matrix ok 01_whitepaper_market 41 34 7 588.00pt 41 02_research_paper 13 13 0 420.00pt 13 03_tech_report_code 46 44 2 692.00pt 46 04_exec_brief 18 16 2 590.00pt 17 05_memo 1 1 0 623.20pt 1 c1_whitepaper 43 43 0 758.16pt 43 c2_paper2col 5 5 0 578.00pt 5 c3_tables 342 342 0 1680.00pt 342 c5_graphics 9 9 0 646.50pt 9 c6_long 50 50 0 5438.00pt 50 c7_code 4 4 0 382.00pt 4 c8_toc_links 3 3 0 226.25pt 3 f1_fpdf_brief 12 0 12 0.00pt 12 l1_word_native 11 0 11 0.00pt 11 r1_reportlab_report 14 14 0 377.40pt 14 CORPUS 612 578 34 -- 611 The correspondence is exact: raw points reconstruct GetBounds on 34 paths and there are 34 identity-matrix paths in the corpus. Every Chromium document is 100% non-identity. Worst raw miss is 5438pt on a Letter page. This reproduces the scrutiny's table cell for cell from an independently written probe, so the finding is confirmed rather than taken on trust. A probe bug was found and fixed before the table was trusted: the first run reported a worst miss of exactly 1.00pt on hundreds of paths, a suspiciously round constant. PDFium reports a stroke WIDTH of 1.0 on fill-only objects, and the probe was inflating every filled rect by it. The envelope is now added only for genuinely stroked objects, with the width scaled by the matrix like everything else -- the same class of error as the one being corrected. Probe only. No parser change in this commit; the geometry fix follows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…sions held)
_page_paths now applies FPDFPageObj_GetMatrix to every segment point before the
y-flip, derives the path bbox from those transformed points instead of
FPDFPageObj_GetBounds, and scales the stroke width by the matrix. _classify,
_rect_pts and the frame-edge decomposition consume the same points, so the
mixed-space logic is gone rather than worked around.
Two separate defects, one fix:
1. Segment points are in OBJECT space. 578 of the corpus's 612 path objects
carry a non-identity matrix -- every path on every Chromium document -- and
untransformed points miss the true bounds by up to 5438pt. _classify's
orientation tests have been comparing object-space dx/dy against
page-space w/h in shipped code.
2. GetBounds is the INK envelope, not the geometric path. PyMuPDF reports the
path, and every threshold downstream was tuned against that. A 0.75pt box
border arrived 1.5pt wide, and infer.py's table detector read that bar as a
column: 03_tech_report_code's code listing was built as a two-column table
with a 3.0pt first column and its line breaks discarded.
DrawCmd diff against a worktree of the previous commit -- real code, byte-
identical PDFs. 611 draws before, 611 after, NO shape changes, no path lost:
111 bboxes moved, and every one shrank by at most its own stroke width, which
is what removing an ink envelope should look like. The single exception is
explained: a complex path in 04_exec_brief moved 2.60 against a 2.00 stroke; it
is the line chart's series polyline and its new bbox lands exactly on the data
marker centres (100.0 and 470.0, markers 97.4..102.6 and 467.4..472.6). The
excess is the miter join.
03_tech's borders now match PyMuPDF coordinate for coordinate --
vline x=54.00..54.00 w=0.00 lw=0.75, accent x=57.00..57.00 w=0.00 lw=3.00 --
and the region classifies role=code, rows=1, col_widths=[504.0], leading 11.50,
line_breaks=True, 10 visual lines, 15 runs: identical to PyMuPDF's layout.
backend_parity.py --refine 3 8 regressions, 8 same, 0 better
c7_code 0.59 -> 0.76 (pymupdf 0.91)
03_tech_report_code 0.02 -> 0.05 (pymupdf 0.46)
04_exec_brief 0.34 -> 0.20 -- loses its BETTER verdict
the other 12 pinned exactly
THE TRADE (law 17). 04_exec_brief drops 0.14 and lands level with PyMuPDF (0.20
against 0.22) instead of ahead of it. It is the line-chart document, and the old
number came from an inflated chart bbox -- a figure region larger than the
drawing it contains. A score earned off a geometric error is not worth keeping,
and the document is not a regression either way.
A PREDICTION THAT FAILED. I wrote before running that 03_tech would recover to
>=0.23 because the phantom column was its attributed cause. It moved to 0.05.
The column is provably gone, so the column was A cause and not the dominant one:
matching the structure of one region does not bound the error of the page. That
document's remaining loss is unnamed and belongs to M2.d, not to a guess here.
Invariance: golden IR 7/7, purity 16/16, parity's pymupdf column unchanged --
no shared code touched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…§12.6)
Law §12.6 says never tune a threshold before plotting the two distributions it
is supposed to separate. This repository has already proved once that such a
threshold CANNOT exist for a different decision (table-cell gaps vs
coincidental same-baseline gaps, identical medians), and lost a day finding out
by tuning. block_gaps.py makes the question answerable in minutes.
It labels every consecutive pdfium line pair with PyMuPDF's own answer -- same
block or not -- and plots gap/reference for three candidate references.
Result, on the boundary decision behind c6_long and c8_toc_links:
PER DOCUMENT every reference separates perfectly. CORPUS-WIDE none does,
because each document's clean split sits at a different ratio:
c6_long 0 of 194 wrong at 1.05
c8_toc_links 0 of 16 wrong at 1.05
f1_fpdf_brief 0 of 11 wrong at 1.00
r1_reportlab_report 0 of 23 wrong at 1.00
l1_word_native 0 of 16 wrong at 1.24
CORPUS (685 pairs) best fixed 1.11, still 135 wrong
The decision is well-posed locally; the global constant is what is wrong.
Scoring candidate rules against PyMuPDF's labels:
shipped gap <= median * 1.60 355/685 wrong (52%)
fixed gap <= p20 * 1.60 278/685 (41%)
fixed gap <= p20 * 1.30 178/685 (26%)
fixed gap <= p20 * 1.15 140/685 (20%)
fixed gap <= p20 * 1.05 154/685 (22%)
adaptive per-page Otsu cut 322/682 (47%)
Two findings worth the tool's existence. The shipped rule is wrong more often
than right on this labelled set. And the ADAPTIVE estimator -- the clever
option, the one that suggests itself after the local_pitch dead end -- is worse
than a fixed factor on a better reference. Measuring that cost minutes;
implementing it would have cost a session.
The 52% overstates the parser's real error: this scores the else-branch
condition applied uniformly to every consecutive pair, while _build_blocks_one
has the same-baseline and size-change branches in front of it. It ranks rules;
it does not measure the parser. The gate measures the parser.
Instrument only. No parser change in this commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… -> 6)
First reduction in the parity count.
The pattern, named before the rule (§9.A). A block-boundary diff says pdfium
disagrees with PyMuPDF in exactly one direction and no other: it MERGES where
PyMuPDF splits -- c6_long 72 times out of 201 lines, c8_toc_links 3 out of 17,
zero splits-where-PyMuPDF-merges in either. The context names it: consecutive
paragraphs, list items and TOC entries fused into one block. Body pitch on those
pages is ~15pt, the boundaries are 18.0-23.2pt, and the shipped rule admitted
anything up to median_pitch * 1.6 ~= 24pt.
The reference was the bug, not the factor. The MEDIAN gap is biased upward by
precisely what it should exclude: the between-block gaps are in the sample, and
so are a table's row pitches. The 20th percentile approximates the tightest
recurring pitch on a page, which is what body text sets, and is unmoved by
however many wide gaps sit above it.
Distributions plotted before the threshold was touched (§12.6,
testkit/block_gaps.py, 685 line pairs labelled with PyMuPDF's own answer):
shipped gap <= median * 1.60 355/685 wrong (52%)
fixed gap <= p20 * 1.30 178/685 (26%)
fixed gap <= p20 * 1.15 140/685 (20%) <- this
fixed gap <= p20 * 1.05 154/685 (22%)
adaptive per-page Otsu cut 322/682 (47%)
The plot also says no fixed factor is RIGHT: every document separates cleanly at
its own ratio (1.00 to 1.24 across the corpus) and 140 of 685 stay wrong at the
best compromise. And the adaptive estimator -- the clever option after the
local_pitch dead end -- measured worse than the simple one, before being written.
Block boundaries after the change: c6_long 201 of 201 lines agree with PyMuPDF,
c8_toc_links 17 of 17. Grouping on those two is finished.
backend_parity.py --refine 3 6 regressions, 10 same, 0 better
c6_long 0.21 -> 0.46 still regression (pymupdf 0.76)
c8_toc_links 0.54 -> 0.78 still regression (pymupdf 1.00)
01_whitepaper_market 0.31 -> 0.48 still regression
05_memo 0.49 -> 0.64 LEFT the set, equals pymupdf exactly
c1_whitepaper 0.00 -> 0.12 LEFT the set (pymupdf 0.18)
c7_code 0.76 -> 0.72
c2_paper2col 0.21 -> 0.20
other 8 unchanged
THE TRADE (law 17): c7_code -0.04 and c2_paper2col -0.01, neither changing a
verdict, both inside the comparator's 0.08 band. c7_code stays far above the
0.16 this branch found it at.
PREDICTIONS, three of five wrong. I predicted c6_long >= 0.50 (got 0.46) and
c8_toc_links >= 0.80 (got 0.78) -- right direction, missed the numbers -- and
predicted l1_word_native as the likely new regression, which did not move at
all. The two documents that actually left the set, 05_memo and c1_whitepaper,
I did not predict. A global change to blocking does not respect the document you
had in mind, and that is the lesson worth keeping from this one.
M2.c's acceptance asked that both target documents LEAVE the regression set.
They did not; they improved ~0.25 each and stayed. But their boundaries now
match PyMuPDF exactly, so grouping is exhausted as an explanation for them and
their residual belongs to M2.d's re-attribution, not to another grouping pass.
Invariance: golden IR 7/7, purity 16/16, pymupdf column unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…> 3)
_page_chars has taken y from FPDFText_GetLooseCharBox since the ink box was
found to push every line below its true ascent -- the reason is in a comment
right there. It kept taking x from FPDFText_GetCharBox, the tight ink box. The
bug was half-fixed, and the other half was the single largest remaining cause.
PyMuPDF reports every line of c6_long starting at exactly x=61.500, the pen
origin. PDFium reported the first glyph's INK edge, which moves with whichever
letter begins the line:
'L' 63.194 (+1.694) '1' 62.606 (+1.106) 'R' 62.004 (+0.504)
'm' 61.794 (+0.294) 'T' 61.574 (+0.074) 'w' 61.489 (-0.011)
That is the left side bearing: a different error on every line, so no per-page
correction removes it, and it is present on every document regardless of
structure. Probed before being relied on (law 15): loose.left equals
FPDFText_GetCharOrigin's x to 0.000 on every character sampled, and equals
PyMuPDF's line x0 exactly.
HOW IT WAS FOUND. testkit/residual.py (new) splits placement error into the part
a second pass could remove -- per-page affine in y, per-page constant in x --
and reports the CEILING a perfect anchoring fix could reach. It said every
document's horizontal error was 2-5x PyMuPDF's and did NOT shrink under per-page
correction, while the vertical axis was fine. Meanwhile backend_spans said
c6_long's IR was identical to PyMuPDF's on lines, spans, text, spaces and styles
(0% on all four) with block boundaries 201/201 -- and it still scored 0.46
against 0.76. An identical IR that scores differently means the difference is
inside the instruments' tolerances: sub-point, per-line, horizontal.
backend_parity.py --refine 3 3 regressions, 13 same, 0 better
03_tech_report_code 0.05 -> 0.48 LEFT the set, above pymupdf's 0.46
c6_long 0.46 -> 0.76 LEFT the set, equals pymupdf exactly
c8_toc_links 0.78 -> 1.00 LEFT the set, equals pymupdf exactly
c7_code 0.72 -> 0.82 (pymupdf 0.91, needs 0.02 more)
01_whitepaper_market 0.48 -> 0.53 (pymupdf 0.72)
02_research_paper 0.57 -> 0.57 (pymupdf 0.76)
mean over 16 docs 0.384 -> 0.461 (pymupdf 0.511)
The risk named in advance -- that advance boxes tile, shrinking the gaps the
space synthesis reads -- did not materialise. c8_toc_links' text diff went
12% -> 0%, c6_long stayed 0%, c7_code's span fragmentation improved 73% -> 65%.
Cleaner input, not degraded input.
WHY THE LAST THREE CHANGES LOOKED FLAT. They were not flat; they were masked by
this error underneath them. c6_long could only land exactly on 0.76 because its
blocks were already exactly right (M2.c); 03_tech could only reach 0.48 because
its code box was already classified correctly (M2.b). Each looked disappointing
alone and paid in combination. Worth remembering the next time a correct change
measures flat.
Invariance: golden IR 7/7, purity 16/16, pymupdf column unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… width
Span counts now match PyMuPDF exactly on every corpus document (0% diff, from
65% on c7_code). The gate did not move, and the prediction that it would is
recorded as failed.
TWO DEFECTS, ONE COMMIT, because the first is wrong without the second.
1. _build_lines split a span on `gap > SPAN_GAP_EM` in the same condition as the
style change, and that test ran BEFORE the space-insertion branch -- so a
space-sized gap ended the span instead of becoming a space. Measured on
c7_code: 79 of 79 intra-line span boundaries sat between spans of IDENTICAL
style, every one at a 3.401pt gap, giving 105 spans for 26 lines where
PyMuPDF gives 26. Nothing is left unbounded by dropping it: _build_lines has
already split the LINE at LINE_SPLIT_EM further up.
2. Doing only that produced `def..rerank` -- every space doubled -- which
exposed the real defect. PDFium reports a generated space's box as
degenerate, `x..x` at a single coordinate, so inheriting the previous
character's end gives it 1.70pt where the true advance is 5.10pt. The
remaining 3.401pt surfaces as a phantom gap that is indistinguishable from a
producer positioning words, and a second space gets synthesised on top of the
one already there.
The space is now given one space-advance of width, capped at wherever the
next character starts. The cap is not decoration: running it to the next
character also closes the gap BETWEEN TABLE CELLS, which _build_lines splits
rows on, and that fused cells back into single lines -- 01_whitepaper_market
130 lines -> 105, 03_tech_report_code 73 -> 53.
span count diff c7_code 65% -> 0% 02_research 20% -> 0%
03_tech 13% -> 0% 01_whitepaper 5% -> 0%
c7_code spans 105 -> 26 for 26 lines, exactly PyMuPDF's 1.00 per line
backend_parity.py --refine 3 3 regressions, 13 same, 0 better (unchanged)
PREDICTION FAILED. H1 said c7_code would reach >=0.88 because span fragmentation
was costing placement. It scored 0.82 before and 0.82 after. The hypothesis is
falsified: fragmented and merged runs lay out identically here.
THE TRADE (law 17). Costs r1_reportlab_report 0.03 and changes no verdict; buys
no measured score. Kept because it buys correctness this metric cannot see:
without it the parser emits DOUBLED SPACES in its text -- wrong content, and
invisible to live_text_cov, which strips whitespace. l1_word_native 0.03 -> 0.01
also now equals PyMuPDF exactly. And this branch has twice seen structurally
correct changes measure flat and then pay in combination; that is precedent, not
proof, and is labelled as such.
Invariance: golden IR 7/7, purity 16/16, pymupdf column unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…33% -> 0%)
PDFium synthesises a space at the end of a line, where the producer merely
stopped drawing. It is line-break decoration rather than content, and PyMuPDF
does not report it: measured on 01_whitepaper_market, 25% of lines differed from
PyMuPDF's text by exactly one trailing space -- |Tier.| against |Tier|, |*.|
against |*|, |...Confidential.| against |...Confidential|.
Dropped after any RTL reordering, so "trailing" means the end of the logical
text rather than the right-hand end of the visual row.
text diff 03_tech_report_code 33% -> 0%
01_whitepaper_market 29% -> 12%
02_research_paper 36% -> 22%
backend_parity.py --refine 3 3 regressions, 13 same, 0 better (unchanged)
r1_reportlab_report 0.55 -> 0.58, recovering exactly the 0.03 the previous
commit cost it, so the two together are score-neutral.
Four of the six documents worked on this session -- 03_tech_report_code,
c6_long, c7_code, c8_toc_links -- are now identical to PyMuPDF on every axis
this repository can measure: lines, spans, text, space runs, style keys and
block boundaries, all 0%.
Invariance: golden IR 7/7, purity 16/16, pymupdf column unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Justified text pulls its word gaps wide -- 7.84pt at 9.5pt type on 02_research_paper. A space CHARACTER is already present there, but the stretched remainder still cleared SPACE_GAP_EM, so the synthesis added another on top and the existing guard capped the addition at one. That is exactly how every gap came out as two spaces: mupdf |Speculative.decoding.accelerates.autoregressive.generation| pdfium |Speculative..decoding..accelerates..autoregressive..generation| In proportional text a gap already occupied by a space contributes no additional space. Monospace keeps counting, because there a run length is code indentation and collapsing it once cost 19 unmatched words and 40pt of drift on a listing. text diff 02_research_paper 22% -> 8% 01_whitepaper_market 12% -> 8% space-run diff 02_research_paper 22% -> 7% 01_whitepaper_market 5% -> 1% backend_parity.py --refine 3 3 regressions, 13 same, 0 better THE GATE DID NOT MOVE. Not one number changed -- 02_research_paper 0.57 before and after, 01_whitepaper_market 0.53 before and after. PREDICTION FAILED, AND IT EXPLAINS THE PREVIOUS TWO. I wrote beforehand that this one would move because "doubled spaces displace every word after them on a justified line -- this one moves ink". It does not: in justified text the renderer redistributes inter-word space to fill the measure, so the NUMBER of source spaces has no bearing on where words land. LibreOffice re-justifies to the same width whether the source says one space or two. That is three structurally confirmed, score-flat hypotheses in a row -- span fragmentation, trailing spaces, justified spacing -- and this third one retroactively explains the first two. Text- and span-level differences in this corpus do not reach within2pt, because the renderer normalises exactly those degrees of freedom. Structural convergence of the IR is finished, and now demonstrated finished rather than assumed. THE TRADE (law 17): no score movement, no regression, no verdict change. Kept because one space is the correct content and two is not; live_text_cov strips whitespace and cannot see it, and a user opening the DOCX would. Also recorded and not fitted: `1..Introduction` -> `1.Introduction` runs the opposite way, PyMuPDF emitting two spaces at a wide heading gap where pdfium emits one. A second, rarer pattern with the opposite sign, left alone. Invariance: golden IR 7/7, purity 16/16, pymupdf column unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
c7_code leaves the regression set on PyMuPDF's exact number, and 05_memo
overtakes it.
This fixes a defect I introduced in the body-pitch commit. The page-wide 20th
percentile cured the median's upward bias and inherited the same shape of error
downward: c7_code sets its code listings at an 11.25pt pitch, which drags the
page percentile BELOW the body text's 15.0pt, so body paragraphs split into one
block per line. Each line then became its own justified paragraph, and a
one-line justified paragraph is not stretched to the measure.
Found by the memo's own step-3 tool rather than by guessing. The noise floor
said the harness is exact here, not noisy:
refine pymupdf pdfium gap
0 0.56 0.38 -0.180 (a different configuration, not noise)
1 0.91 0.82 -0.090
2 0.91 0.82 -0.090
3 0.91 0.82 -0.090
3 again 0.91 0.82 -0.090 (bit-identical repeat)
Then per-word attribution: the entire gap was 17 words on two source lines, with
pdfium's horizontal error accumulating -2.67pt per word gap -- one space advance
at that size -- perfectly linearly. Words clearing 2pt under PyMuPDF but not
pdfium: 17. The other way round: 0. The block diff named it: three
`pdfium SPLITS where PyMuPDF merges`, every one at exactly gap=15.0.
Text of one size shares one leading, so the reference now lives with the text
rather than with the page. This is NOT the sliding window that was tried and
reverted: a window has no idea what it is averaging over and cut
02_research_paper's paragraphs in half; a size bucket is a property of the text
itself. Falls back to the page percentile when a size has fewer than 3 samples.
backend_parity.py --refine 3 2 regressions, 13 same, 1 better
c7_code 0.82 -> 0.91 leaves the set, EQUALS pymupdf exactly
05_memo 0.64 -> 0.88 now BETTER than pymupdf's 0.64
all others unchanged
block boundaries c7_code 23/26 -> 26/26; c6_long 201/201; c8_toc_links 17/17
The Decision Memo §4 owner decision box is moot: c7_code needed 0.02 and gained
0.09. No ACCEPTED_SHORTFALL entry is required and the mechanism was not created.
Remaining: 01_whitepaper_market 0.53 vs 0.72, 02_research_paper 0.57 vs 0.76.
Invariance: golden IR 7/7, purity 16/16, pymupdf column unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ffset
residual.py gains --hist and a docstring that says which way its arrows read
(raw -> after-fit, never p50 -> p90 -- the memo asked, and the old wording was
genuinely ambiguous).
A median cannot tell a few lines displaced by a whole leading from every line
off by a fraction of a point. Those need opposite fixes. The histogram can, and
on 02_research_paper it does: unimodal, no mass anywhere near one leading
(13.6pt), so the remaining vertical error is an anchoring difference and not a
wrap or line-count one.
The control makes it conclusive. The two backends produce THE SAME
DISTRIBUTION, DISPLACED:
bucket pymupdf bucket pdfium
+0.0 252 -> +1.5 225
+1.0 35 -> +2.5 47
+3.0 55 -> +4.5 55
Every cluster moved by exactly +1.5pt, and the 55-word cluster appears with an
identical count on both sides. A constant offset, not scatter -- and the reason
a per-page affine fit removes so little of it is that a least-squares line
through a multi-modal distribution sits between the modes.
WHERE IT ENTERS. Baselines are identical on every line (dbase = +0.00). The line
BOXES are not: pdfium's y0 sits 0.57-2.60pt lower, scaling with type size.
margin_t is taken from the topmost line's box top and comes out 63.30 under
PyMuPDF against 64.90 under pdfium -- a 1.6pt page-wide shift, which is the
+1.5 mode.
WHY IT CANNOT SIMPLY BE CONVERGED. The box is font-dependent in both backends,
from different metric sources:
font pymupdf up,down pdfium up,down
Helvetica 1.075, 0.299 0.905, 0.211
Times-Roman 1.053, 0.281 0.891, 0.215
Symbol 1.010, 0.293 1.010, 0.293 <- agree exactly
pdfium IS reading font metrics -- Helvetica and Times differ -- just not the
same ones, and where both fall back to embedded metrics (Symbol) they agree to
three decimals. PyMuPDF's numbers come from its own built-in base-14 table.
Reproducing them means vendoring MuPDF's private font metrics, which §13 forbids
and which this branch has already measured to be version-dependent.
CAUSALITY TESTED, NOT ASSUMED. A temporary labelled experiment scaled the box
toward PyMuPDF's ratios (1.188 above the baseline, 1.417 below):
02_research_paper 0.57 -> 0.64 (pymupdf 0.76)
01_whitepaper_market 0.53 -> 0.54 (pymupdf 0.72)
So the convention is a real cause worth +0.07 on the worst-affected document,
and it is NOT the whole gap. The experiment is reverted: a fitted pair of
constants that does not even close the gap is not something to ship.
No parser change in this commit. Instrument and record only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The planning and review documents all share the form "exactdoc <emdash> <something>.md" and sit in the repo root beside the tracked docs, so each new one arrived untracked and one `git add -A` away from being published. The escalation packet was the third to need its own rule. The tracked docs (README, STATUS, THEORY, FINDINGS) carry no prefix, so the pattern cannot catch them by accident. This only takes effect on branches that have it. On main these files are still loose, which is the state the owner was warned about and has not resolved either way -- track them under docs/ or keep them ignored, but decide once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…k (c)
Ruling law 18 gate 4 failed. The shared pipeline carries no change; infer.py is
back at HEAD. testkit/margin_probe.py and the attribution stay.
GATE 3 PASSED, and convincingly. Anchoring the page origin on
`baseline - (leading - 0.21*size)` -- the writer's own paragraph-top formula,
one expression for every backend, no conditionals, no constant that is not
already exactdoc's -- took per-document backend disagreement in margin_t from up
to 2.97pt to 0.000pt on 14 of 16 documents. The two exceptions were attributed
and are not the convention: c5_graphics is the documented gradient divergence
(PyMuPDF does not report the band, so its topmost element is different content),
and 02_research_paper keeps 0.64pt from a block-membership difference that
changes the median leading.
GATE 4 FAILED, on the document the prediction named.
pymupdf lane, before -> after
c6_long 0.76 -> 0.45 <-- REGRESSION, 4x the comparator band
c4_i18n 0.42 -> 0.39
04_exec_brief 0.22 -> 0.20
f1_fpdf_brief 0.62 -> 0.60
r1_reportlab_report 0.60 -> 0.58
01_whitepaper_market 0.72 -> 0.71
other 10 unchanged
gate lanes refine within2pt 0.529 -> 0.486, dy50 0.68 -> 0.80
pass counts held 12/16 and 13/16, 0 new / 0 stale
Law 18 forbids any pymupdf REGRESSION verdict, so this stops here.
WHY, DIAGNOSED RATHER THAN RETRIED. Under the shipped rule c6_long's origin is
set by a single-line bullet block at box top 65.19; under the anchored rule by a
body paragraph at 62.20, because leading - 0.21*size = 13.5pt exceeds that
text's ink ascent of ~11.3pt. The formula is right -- it returns the top of the
first line's BOX IN THE FLOW MODEL, which is what para_top means. What is wrong
is applying it to the origin alone: _assemble_chunks derives every element's
space_before from the running position, so the origin and the spacing chain
measured against it desynchronise. A baseline-consistent vertical model needs
margin_t, _para_box and the space_before chain moved together, which is a much
larger change than this escalation granted and not something to attempt by
iterating against a gate at the end of a session.
FALLBACK (c) APPLIED. STATUS.md D2 now carries the attribution, the
embedded-metrics control (Symbol agrees to three decimals where both backends
use the same source), the proof of parser-side exhaustion (FPDFFont_GetAscent
returns exactly the loose-box ratios), and the acceptance line: 0 regressions
except 01_whitepaper_market and 02_research_paper.
WHAT THE ESCALATION BOUGHT. The convention is proven to be the cause AND proven
not to be fixable at the origin alone -- a closed line of enquiry rather than an
open one. And 14 of 16 documents reaching 0.000pt origin agreement is the
evidence that a complete baseline-anchored vertical model would work; it is the
partial application that fails.
Invariance: golden IR 7/7, purity 16/16 (parser never touched).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ct is
The repository's docs still described a project blocked on the licence swap by
a 7-regression fine-placement gap. That has not been true for some time and the
docs were the last place it survived.
NEW: ROADMAP.md -- sequence and distance, as the counterpart to STATUS.md's
numbers. It answers the question the other files did not: how far is this from
being finished?
pip install exactdoc under Apache-2.0 three working sessions
"fully working on any PDF" much further, and a different project
That distinction is the point of the file. Shipping a permissive, honest,
well-measured alpha is close; making the converter good on documents it has
never seen is open-ended, and is deliberately off the critical path to a
release. The roadmap also records what has been CLOSED and why, so nobody
spends a session rediscovering it.
STATUS.md
- D2 downgraded from "severity: high (blocks relicensing)" to "low (no longer
blocks relicensing)". It was the single claim most out of date: the parser
is at 2 regressions from 9, with 14 of 16 documents at or above the
incumbent, four exactly equal and two better
- §1 carries the flip's amended acceptance line
- §3 pending-work table rewritten: the flip and the release are now items 1-4,
and the open research (D1 pagination, the baseline-consistent vertical
model, the Google Docs cover band) is honestly below them
README.md
- status banner names the relicence as the next milestone
- headline numbers aligned to the CI run of record (52.9% within-2pt, 0.68pt)
- licence section rewritten: it described the swap as unresolved and quoted
7 regressions. It now states the measured position, the reason the last two
are accepted rather than chased, and that no AGPL wheel will ever ship
- ROADMAP linked first in Documentation
THEORY.md -- dated addition, the two things the port taught that generalise:
- the renderer normalises whatever it is free to normalise, which is why three
structurally-correct changes each moved within2pt by exactly zero
- anchor the PAGE ORIGIN on baselines too, not just paragraphs -- and the
warning that half-converting the vertical chain desynchronises it, learned
by doing exactly that and reverting
testkit/README.md -- documents the six backend-comparison instruments built
during the port, none of which were listed, plus the two habits they encode
(probe a native API's quantity before building on it; a subset run never
decides).
docs only, no measured number moves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…profile
The licence flip is a change to which parser produces every number, and it was
about to be judged by a gate that could pass while none of the following held.
Each was reachable, and most are measured rather than hypothesised:
* `harness.evaluate()` returns {"error": ...} when the render fails and nothing
read the key. A renderer dying on all 16 documents scored zero failures.
* An absent metric hit `if v is None: continue`, so losing within2pt removed
the check rather than failing it -- the same dimension whose absence once hid
a 0.510 -> 0.291 regression.
* The baseline stored the NAMES of failing metrics. 04_exec_brief's live text
was on record as "known failing" at 0.941 and could have fallen to 0.10.
* page_match is a boolean, so a document already over could go from one page to
forty unremarked.
* Nothing compared the run to a manifest. Measured in a bare ubuntu:24.04
container: gen_corpus produced 3 of 16, printed "the corpus is incomplete,
numbers are NOT comparable", exited 0 -- and the gate scored those 3 against
the 16-document baseline and passed.
* REFINE=lanes returned only the refined lane's status, so the control lane
could regress freely.
* backend_parity exited on `regressions == 0` while ROADMAP and STATUS said two
documents were formally accepted. That disagreement was settled by marking
the CI step continue-on-error, which retired the one gate the whole
relicensing effort was aimed at.
What lands:
exactdoc/options.py One immutable PRODUCT profile. The API refined 0 times,
the CLI 2, and the published numbers came from a CI lane
that ran 3; a measurement describing no shipping
configuration is a coincidence. `backend` is a real
option now, so the instruments stop assigning over
convert.parse_pdf -- an injection that had become a no-op
setting an attribute nobody reads.
testkit/gate.py The decision, pure over measured results, so it can be
mutation-tested without a corpus or an oracle. Three
separate questions: regression (against the recorded
number), absolute (--absolute, and it fails today, on the
record), stale (a shortfall that now passes).
gate_baseline.json Schema 2: every gated metric of every document in both
lanes, numerically, plus the environment and the defect
ID each shortfall answers to. Recorded on Linux with
LibreOffice 24.2.7.2, and it reproduces the documented
numbers -- product 15/16 pages, within2pt 0.5118, dy50
0.62pt; raw 13/16, 0.3486, 2.20pt.
corpus_manifest.* The exact 16 documents, generator, dialect, page count.
No content hash: both generators embed timestamps, so a
hash would fail every run and be deleted within a week.
parity_policy.json The acceptance rule as data. 0 regressions, 11 same, 1
better, 2 expected divergences, 2 accepted shortfalls
bounded by recorded numeric floors. continue-on-error
comes off the step.
evidence.py One artifact every published number traces to. Its merge
skips None sections: the standalone step meant to fill in
the environment was passing the empty template's
`parity: None` over the verdict just recorded, and a green
run ended having forgotten its own parity result.
test_gate_mutations 30 tests, 106 assertions, about a second, no corpus
needed. One per false-green path above.
Two items closed by measuring instead of building:
* Superscript. parse_pdfium hardcodes False and the plan was to implement it.
backend_superscript.py measures the level that matters: the writer never sees
the parser's flag -- dialect and infer recover it from geometry -- and all 16
documents agree at the layout level. No code written.
* scripts/bootstrap.sh discovers Chromium and writes scripts/env.sh, which
nothing sources: every CI step is its own shell. CI only ever worked because
the GitHub runner image ships /usr/bin/google-chrome. _paths.py reads the
record itself, and the bare container now generates all 16.
Also: D10 in STATUS for the two rasterisation shortfalls the baseline needed an
ID for, attributed by measurement (figure-excluded coverage 0.988 / 0.978 /
0.966) and honest that rasterisation is the dominant cause on all three and the
whole cause on none. The dialect table summed to 17 for a 16-document corpus.
dy_p50 gets a proportional tolerance: it is the one gated metric that is not a
fraction, running 0.04pt to 101pt, and one absolute slack cannot serve both ends.
The full CI-equivalent sequence is verified green on the canonical Linux
environment. Phase 1 is unchanged and now stated honestly in STATUS section 3:
`fitz` is on the default runtime path well past the parser, so the "mechanical"
licence flip is not mechanical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Owner
Author
|
Superseded by #4. PR #4 has been retargeted onto Closing as a draft slice that has been folded into the combined |
ebt55
added a commit
that referenced
this pull request
Jul 30, 2026
INT-00 from the Google-Docs-first build plan. The pre-transition graph, every merge-base and commit count, and the live CI state for PRs #1-#4, captured before anything was mutated so the whole transition is reversible by inspection. Three things the verification found, none of which was in the plan or in the repository: - the plan's own graph draws PR #2 as based on the execution-plan branch. It is based on `main`. Branch ancestry is as drawn; the pull request's declared base is not. The retarget still yields the intended seven commits, verified by merge-base before it was requested and by GitHub after. - the parity gate fails in FIVE places, not two: the two unwaived dy_p50 regressions plus three `c4_i18n` below-floor dimensions. PR #4's body said two, and ROADMAP.md still says the gate fails on exactly the two unwaived regressions and nothing else. The verdict tally and the failure count are different things, and conflating them is how the claim survived. - those three `c4_i18n` floors are labelled `ratified` and are stale. They were recorded before the font set was pinned, and `c4_i18n` is the CJK + Arabic + Hebrew document -- precisely the one a font environment moves. They are to be remeasured, not waived. No force-push, no branch deleted, no public SHA rewritten: all six branches still resolve to their pre-transition commits, and PR #1 was merged with a real merge commit (two parents) so 0cd7d11 remains an ancestor of main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — review work, not ready to merge. Stack 1 of 3.
Makes the fidelity gate mean something, so that the licence swap can be judged by
it. This is PR 1 of the plan's three-PR sequence; PR 2 (backend seam) and PR 3
(gate hardening) are stacked on top.
Why
The swap changes which parser produces every published number, and it was about
to be judged by a gate that could pass while none of the following held. Each was
reachable; most are measured rather than hypothesised.
harness.evaluate()returns{"error": ...}when the render fails and nothingread the key. A renderer dying on all 16 documents scored zero failures.
if v is None: continue, so losingwithin2ptremoved thecheck rather than failing it — the same dimension whose absence once hid a
0.510 → 0.291 regression.
04_exec_brief's livetext was on record as "known failing" at 0.941 and could have fallen to 0.10.
page_matchis a boolean, so a document already over could go from one page toforty unremarked.
ubuntu:24.04container: the generator produced 3 of 16 documents, printed "the corpus is
incomplete, numbers are NOT comparable", exited 0 — and the gate scored those 3
against a 16-document baseline and reported a pass.
REFINE=lanesreturned only the refined lane's status, so the control lanecould regress freely.
backend_parity.pyexited onregressions == 0while ROADMAP and STATUS saidtwo documents were formally accepted. That disagreement was settled by marking
the CI step
continue-on-error, which retired the one gate the entirerelicensing effort was aimed at.
What lands
exactdoc/options.pyPRODUCTprofile. The API refined 0 times, the CLI 2, and the published numbers came from a CI lane that ran 3 — a measurement describing no shipping configuration.backendbecomes a real option, so the instruments stop assigning overconvert.parse_pdftestkit/gate.py--absolute), stalegate_baseline.jsoncorpus_manifest.*parity_policy.jsonevidence.jsontests/test_gate_mutations.pyClosed by measuring instead of building
parse_pdfiumhardcodessuperscript=Falseandthe roadmap queued implementing it. The writer never sees the parser's flag —
dialectandinferrecover it from geometry — and all 16 documents agree atthe layout level.
bootstrap.shwrote oracle paths to a file nothing sourced. Every CI stepis its own shell. CI only ever worked because the GitHub runner image ships
/usr/bin/google-chrome, which is provisioning by accident.Evidence
Canonical
ubuntu:24.04, LibreOffice 24.2.7.2, Liberation metric fonts.Reproduces the recorded baseline: product lane 15/16 page match, within-2pt
0.5118, dy₅₀ 0.62pt; raw 13/16, 0.3486, 2.20pt.
GitHub Actions has not yet run these commits.
🤖 Generated with Claude Code