Skip to content

The two ends of one report disagreed about the same files - #28

Merged
melbinjp merged 1 commit into
mainfrom
coverage-denominator
Aug 19, 2026
Merged

The two ends of one report disagreed about the same files#28
melbinjp merged 1 commit into
mainfrom
coverage-denominator

Conversation

@melbinjp

Copy link
Copy Markdown
Owner

The header asks likeliest_docs_directory whether anything unread is NAMED like documentation, and prints either a widen suggestion or "none of them is named like a documentation tree". The verdict at the bottom then called every one of those files "documentation file(s)" and divided by them.

On zhukunpenglinyutong/desktop-cc-gui the same run says, forty lines apart:

   5812 documentation file(s) elsewhere in the tree were NOT read
     openspec/ 5271, src-tauri/ 251, .agents/ 232, dev-guidelines/ 45, ...
     none of them is named like a documentation tree
...
This judged 63 of 5875 documentation file(s). 5812 were never read, so this
verdict covers 1% of the documentation in this project.

5,271 of those are an OpenSpec change-proposal tree. Nobody would want them checked against HEAD, the tool has already said as much, and then it reports 1% coverage of "the documentation in this project".

Measured, because one repository is not a defect

39 sweep captures, 24 of which print a coverage line:

coverage range 1% to 99%, median 27.5%
under 10% 7 of 24
a docs-named directory found unread 4 of 24

So in twenty of twenty-four cases the tool divided by a denominator it had itself just judged to contain no documentation tree.

The fourteen commonest biggest-unread directories, and exactly one is documentation:

skills/ 2   src/ 2   .agents/ 2   scripts/ 1   ts/ 1   datafusion/ 1   crates/ 1
libs/ 1   benchmarks/ 1   examples/ 1   docs.feldera.com/ 1   tools/ 1
mobile/ 1   mcpjam-inspector/ 1

Per-package READMEs and agent instruction files, which find_docs excludes on purpose and whose docstring argues why.

What changes and what does not

The percentage stays. A narrow pass must never read as a clean one, which is the entire reason this line exists - the README promises "a clean report over two files in a project with three hundred cannot be mistaken for a clean report over three hundred" and printing it forty lines from the verdict is how it got mistaken in the first place.

What goes is the assertion about what those files are:

- This judged 63 of 5875 documentation file(s). 5812 were never read, so this
- verdict covers 1% of the documentation in this project.
+ This judged 63 of 5875 documentation file(s) in the tree, 1%. The 5812 it did
+ not read are outside the default scope and listed above; none of them is in a
+ directory named like documentation.

And the other branch, which is the one the change exists for - when something unread really is a documentation tree, the last line says so rather than making the reader scroll:

This judged 5 of 219 documentation file(s) in the tree, 2%. The 214 it did not
read include `docs.feldera.com/`, which is named like a documentation tree, so
this verdict is narrower than it looks.

report_coverage now returns the directory it computed instead of throwing it away, and the verdict carries the same judgement the header already made.

209 tests. Two rewritten to the new wording, one new for the docs-tree branch, and the new one also asserts the header still prints the actionable --docs form, so the two ends are checked together.

The header asks `likeliest_docs_directory` whether anything unread is NAMED like
documentation, and prints either a widen suggestion or "none of them is named
like a documentation tree". The verdict then called every one of those files
"documentation file(s)" and divided by them.

On `zhukunpenglinyutong/desktop-cc-gui` the same run says, forty lines apart,
that none of the 5,812 unread files looks like documentation and that the
verdict "covers 1% of the documentation in this project". 5,271 of them are an
OpenSpec change-proposal tree, which nobody would want checked against HEAD.

Measured over 39 sweep captures, 24 with a coverage line: 1% to 99%, median
27.5%, seven under 10%. A directory named like documentation was found unread in
4 of the 24 - so in twenty cases the tool divided by a denominator it had itself
just judged to hold no documentation. One of the fourteen commonest
biggest-unread directories is documentation; the rest are skills/, src/,
.agents/, crates/, libs/, ts/, tools/, mobile/, examples/, benchmarks/.

The percentage STAYS. A narrow pass must never read as a clean one and that is
the entire reason this line exists. What goes is the assertion about what those
files are. And when something unread IS a documentation tree the verdict now
names it, instead of leaving the reader who only reads the last line to scroll
up for it.

209 tests. Two rewritten to the new wording and one new, covering the branch the
change exists for.
@melbinjp
melbinjp merged commit 26d3725 into main Aug 19, 2026
20 checks passed
@melbinjp
melbinjp deleted the coverage-denominator branch August 19, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant