Add 4 construct rows for characteristics with no coverage id - #465
Merged
Conversation
conformance_coverage.rs's mapping (machine-consuming PanGloss repo) found 4 CharacteristicKind values -- rewrite direction (LeftToRightRewrite/ RightToLeftRewrite), phonological subrule-level MPR/POS gating (SubruleGating), and multi-CharacterDefinitionTable stratification (MultiTable) -- with no constructs.txt row distinctly tagging them, so no fixture could ever be written to exercise them by name. The existing "RewriteRule Iterative/Simultaneous" rows tag application-order, not directionality; "MPR features/groups" is morphological MprGroup territory, not phonological subrule gating; the two existing CharacterDefinitionTable rows are about segmentation/pattern constructs within one table, not about multiple tables disagreeing across strata. Add one row per characteristic so each can be evidenced independently.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## conformance-framework #465 +/- ##
======================================================
Coverage 73.34% 73.34%
======================================================
Files 443 443
Lines 37214 37214
Branches 5110 5110
======================================================
Hits 27294 27294
Misses 8798 8798
Partials 1122 1122 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
johnml1135
added a commit
to sillsdev/PanGloss
that referenced
this pull request
Jul 27, 2026
… citation liveness G8 -- widen the conformance cross-check from the 6 Proven kinds to all 20. `supported_kinds()` now returns every `CharacteristicKind`. A new `EvidenceRequirement` + `evidence_requirement_for(Disposition)` encode the honest per-disposition rule from ADR 0001: Proven/ConfigPredicate/ConfirmOnly need a passing analysis fixture (with `disposition` now carried on every `CoverageReportRow` so a build-breaking flip can stage Proven first), while FailClosed needs a REFUSAL witness instead -- a refused construct can never have a passing analysis fixture, so demanding one would be incoherent. `supported_coverage_report`/`supported_uncovered` take the two evidence sets as separate parameters so they cannot cross-contaminate. This fixed a real overclaim, not just a scoping gap. `build_ledger` previously graded every row -- FailClosed included -- against the passing-fixture set alone. Because MprGroupOverwrite (FailClosed) and MprGroupAppend (ConfirmOnly) share the construct id "MPR features/groups", MprGroupOverwrite could report Covered purely because its SIBLING's passing fixture tagged the same shared id, without the refusal ever being exercised. Pinned by `fail_closed_row_is_covered_via_refusal_witness_regardless_of_passing_set`. Computed flip-blocker list (still ADVISORY -- the flip is a separate step): 20 rows, 16 Covered, 4 Uncovered, 0 Unmappable. Proven (hard error on flip): LeftToRightRewrite, SubruleGating ConfigPredicate/ConfirmOnly (stageable): RightToLeftRewrite, MultiTable FailClosed: none -- MprGroupOverwrite has its witness Those 4 are exactly the newly-mapped G9 characteristics: the mapping now exists, but no fixture yet tags the new construct ids. G9 -- align the two construct vocabularies. Added the 4 missing rows to machine/conformance/constructs.txt (rewrite direction L-to-R and R-to-L, subrule-level required/excluded POS-or-MPR gating, multiple character-definition tables) and mapped them in `construct_ids_for`, taking Unmappable to zero. Upstream PR: sillsdev/machine#465. Submodule pointer bumped to that branch tip (4560e9e2, not yet merged upstream). The 12 orphan constructs.txt rows get no invented characteristics -- they are documented one-by-one with a reason in a new ORPHAN_CONSTRUCT_ROWS const (output-action primitives, template slots, boundary markers, guesser, disjunctive allomorph free variation, stem names, syntactic feature agreement, alpha-variables, compounding/rule constraint variants, and tracing, which is an engine debug feature with no capability basis). Golden regenerated from the test's own computation, never hand-edited. NEW -- citation liveness (tests/coverage_citation_liveness.rs, mine, on review). G8 makes a FailClosed row's Covered verdict depend ENTIRELY on a hand-written citation string, and nothing verified that string pointed at anything real. A citation naming a deleted file or a renamed #[test] would report Covered forever -- exactly the overclaim ADR 0001 exists to forbid. Three gates now check that every ledger citation's `tests/<file>.rs` paths exist, that every `::<test_fn>` reference resolves to a real `fn`, and (strictest, for the one disposition where the citation IS the evidence) that a FailClosed row's refusal witness resolves to an actual `#[test]` in one of its own cited files. Proven non-vacuous both ways before commit: renaming the cited fn fails `every_ledger_citation_names_a_test_fn_that_exists` (1 of 30 references); renaming the cited file fails both the file gate and the FailClosed gate. Each gate also asserts it scanned a non-zero number of citations, so a future citation-format change fails loudly instead of going silently vacuous. It does NOT check that a cited test proves what its `note` claims -- that stays a human review obligation, stated as such. Also: dropped a real-word/language-noun reference from conformance-staging/edge-cases/template-category-sharing (grammar.xml comment, STAGING.md prose, words.yaml note); the remaining prose references in that fixture are queued for the part-B delanguaging sweep. And corrected a doc line in coverage_ledger.rs that still described `supported_kinds` as the Proven-only subset, which G8 makes false. Verification: cargo test -p pg-foma 328 passed / 0 failed / 13 ignored, plus every integration binary green; cargo clippy -p pg-foma --all-targets clean on every touched file; openspec validate --all --strict 27 passed; run-conformance.sh --skip-build at the 15+1 baseline; `pangloss coverage` reports 0 Unmappable. The cross-check is deliberately NOT yet build-breaking. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
johnml1135
added a commit
to sillsdev/PanGloss
that referenced
this pull request
Jul 27, 2026
…whole class
The 4 rows blocking the build-breaking flip were not missing evidence -- their
evidence was being thrown away by a string mismatch.
`conformance_coverage::construct_ids_for` maps each CharacteristicKind to
`constructs.txt` row-id strings and credits coverage by byte-for-byte set
match. Three staged fixtures wrote the CHARACTERISTIC NAME instead:
subrule-morphosyntactic-gating "SubruleGating"
-> "RewriteSubruleDef gating: required/excluded POS or MPR at the subrule level"
right-to-left-bounded-quantifier-rewrite "RightToLeftRewrite"
-> "RewriteRule direction (Dir): right-to-left"
bistratal-overlapping-segment-representation "MultiTable"
-> "CharacterDefinitionTable: more than one table, one per stratum"
None matched a row id, so those tags contributed exactly ZERO coverage --
silently. The fixtures looked right, nothing failed, and the rows just sat
Uncovered. `constructs.txt`'s own header explains how: the harness treats an
unknown `exercises:` value as "a soft warning... never a hard error."
LeftToRightRewrite had no tag anywhere. Now tagged on
mpr-gated-exception's `menulik` parse, chosen because BOTH its rewrite rules
genuinely fire (nasal assimilation then obstruent deletion, per the fixture's
own pre-existing derivation note) rather than being present-but-unused, and
because `load_rewrite_rule` gives a rule with no `multipleApplicationOrder`
attribute `Dir::LeftToRight` by default. Verified against the fixture's own
documented derivation, not assumed.
Only tag strings changed -- no signature, no `parses:`, no ground truth. Each
fixture's STAGING.md gained a "Coverage-tag correction" section; a stale line
in subrule-morphosyntactic-gating's STAGING.md saying the row had not landed
yet is corrected (it landed in sillsdev/machine#465).
NEW GATE, the actual deliverable: tests/exercises_tag_liveness.rs asserts every
`exercises:` tag (per-word and per-parse) across BOTH discovery roots is a
literal constructs.txt line, reports every offender at once with fixture + word
+ signature + offending string, and asserts it scanned a non-zero number of
tags so a format change fails loudly instead of going vacuous. It reuses the
existing shared `pg_conformance_fixtures::discover` rather than walking either
root a second time. Proven non-vacuous: reverting one tag to "MultiTable" fails
with the exact fixture/word/tag named; reverting the corruption passes again.
It deliberately does NOT check that a tag is the RIGHT tag -- that a fixture
genuinely exercises what it claims stays a human-authoring obligation, stated
as such.
Cross-check moves 20 rows / 16 Covered / 4 Uncovered / 0 Unmappable
-> 20 / 20 Covered / 0 Uncovered / 0 Unmappable. Independently re-run and
confirmed here, not taken on report.
NOT flipping the cross-check to build-breaking in this commit -- see the
shared-construct-id analysis landing next for why that is still one step away.
Verification: cargo test -p pg-foma green including the three coverage gates;
cargo clippy -p pg-foma --all-targets clean on the new file; run-conformance.sh
--skip-build at the 15+1 baseline; openspec validate --all --strict 27 passed.
One known workspace failure, pg-cli's trace_render golden, is from concurrent
in-flight tracing work in other crates and is not touched or caused by this
change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
johnml1135
added a commit
to sillsdev/PanGloss
that referenced
this pull request
Jul 27, 2026
…aking (the finish line)
tasks.md 6.2 / design.md D7 step 7: "This is the finish line, not a follow-on
cleanup step." Done.
The gate now asserts zero Uncovered AND zero Unmappable rows across all 20
CharacteristicKinds, each graded against its own EvidenceRequirement --
PassingFixture for Proven/ConfigPredicate/ConfirmOnly, RefusalWitness for
FailClosed (nothing compiles for a refused construct, so demanding a passing
analysis fixture for one would be incoherent).
Four fixes had to land first. I did not flip when the count first read 20/20,
because a green build-breaking gate that can silently start lying is worse than
an advisory report -- the green light is what gets cited:
G9 Unmappable had to reach zero; 4 constructs.txt rows did not exist
(sillsdev/machine#465)
G8 the check had to cover all 20 rows, not the 6 Proven ones, and had to
stop grading FailClosed rows against the passing-fixture set -- which
let MprGroupOverwrite report Covered off a sibling's shared construct
id with the refusal never exercised
tags exercises_tag_liveness.rs -- three fixtures tagged CHARACTERISTIC NAMES
where a row id is required, so their evidence counted for nothing; an
unknown tag is now a hard error, not constructs.txt's documented "soft
warning"
ids structural_witness_gate.rs -- four row ids are each mapped by two
characteristics, so the finer could report Covered on the coarser
sibling's evidence; three now have a mechanized grammar-shape witness
and the fourth pair is excluded by derivation
Plus coverage_citation_liveness.rs, which stops the RefusalWitness citations --
the one place a Covered verdict rests on a hand-written string -- from becoming
dangling pointers.
Failure messages are per-disposition rather than one undifferentiated count, so
a failure says what KIND of evidence is missing and therefore what would fix it,
and each names the two likely causes (a fixture whose words started failing, or
a tag that is not a literal row id). The full report still prints on every run.
Non-vacuity, proven not assumed: the entry assertion now requires the report to
enumerate EVERY CharacteristicKind, so a short report cannot make the gate pass
trivially. And sabotaging one Proven row's tag produces exactly:
COVERAGE REGRESSION (Proven): [SubruleGating] are admission-filtered
unconditionally yet have no passing conformance fixture tagging their
construct id.
Reverted, verified clean by git diff, and all four coverage gates re-run green
(1 + 3 + 1 + 2 passed).
Stated in the code, not just here -- what this gate does NOT assert: that a
fixture tags the RIGHT construct (a human-authoring risk no string or shape
check closes in general); that Covered means Admit (ten rows are
ConfigPredicate, three ConfirmOnly -- D1 keeps ConfirmOnly -> Admit a separate
optional track); and that every CONFIGURATION inside a covered row is closed.
Row-level coverage and configuration-level completeness are different claims and
D7 requires both -- the open splits are recorded in
docs/conformance/circumfix-structural-composite-census.md,
needs-decision-resolutions.md, and multitable-shared-representation-design.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
johnml1135
added a commit
to sillsdev/PanGloss
that referenced
this pull request
Jul 27, 2026
….1/6.0/6.2 done Audit 3a gains a "Resolved 2026-07-25 (later the same day)" block superseding the older entries below it, for the four gaps that landed plus the overclaim found while building G3. tasks.md corrections, all of these were stale rather than open: 2.1/2.2 constructs.txt rows -> sillsdev/machine#465, submodule pointer bumped. Recorded honestly that the PR is OPEN, not merged, so the pointer references a branch tip and must be re-pointed on merge. 3.1 G8's widening, noting it fixed a real overclaim (MprGroupOverwrite Covered off a sibling's shared id) rather than only widening scope. 6.0 structural witness gate, incl. the honest detail that narrowing the circumfix predicate to allomorph-0-only still passed, so census C1's preemption does not affect qualification today. 6.2 THE FLIP, with the sabotage output that proves it bites. Still open and unchanged: 6.1/6.3 (plan_interaction_coverage's own report and flip) and 6.4 (D7's definition of done, which needs the configuration-level splits closed too, not just row-level coverage). openspec validate --all --strict: 27 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pg-foma::conformance_coveragecross-check maps everyCharacteristicKindto one or moreconstructs.txtidentifiers. Four characteristics had no corresponding row at all, so no fixture could ever be written to exercise them by name: rewrite-rule directionality (left-to-right / right-to-left), phonological subrule-level MPR/POS gating, and stratification across more than oneCharacterDefinitionTable.Detail
RewriteRule direction (Dir): left-to-right/RewriteRule direction (Dir): right-to-left— the existingRewriteRule Iterative/RewriteRule Simultaneousrows tag application order, not directionality; no row named direction as its own phenomenon.RewriteSubruleDef gating: required/excluded POS or MPR at the subrule level— the existingMPR features/groupsrow is morphologicalMprGroupterritory (MprGroupAppend/MprGroupOverwrite); this is the distinct phonological-subrule gate.CharacterDefinitionTable: more than one table, one per stratum— the two existingCharacterDefinitionTablerows are about segmentation/pattern constructs within one table, not about multiple tables potentially disagreeing across strata.Test plan
🤖 Generated with Claude Code
This change is