R.2-R.5: fingerprint, its specification, and the collapsed store - #6
Merged
Conversation
R.2 (fingerprint + corpus), R.3 (critic), R.4 (store schema), R.5 (migrations
and the FTS5 guard), plus the fixes for the two blockers R.3 raised.
The critic returned FAIL and it was right to
R.3 did not accept R.2. Both blockers were proved rather than asserted, and
both were fixed before this landed.
BLOCKER 1 -- the algorithm existed only in Go. The critic re-implemented the
plan's four-clause spec text in Python and got 55e27b07... where the committed
golden is 13c60ccf... . NormalizeMatch applies nine-plus operations documented
nowhere: a 193-entry reserved-word list, identifier-preservation rules, comment
syntaxes, string-delimiter and escape rules, a number grammar. That breaks S6
one level up -- a second producer implementing from the written spec emits
different digests, silently, forever, which is precisely what "one fingerprint
algorithm, defined once" exists to prevent. It also left R.16 with no honest
way to go green, since its oracle must be built "from scratch, not by importing
fingerprint.go".
Ruling: amend the specification, do not weaken the algorithm. The extra rules
earn their keep -- abstracting a namespace qualifier collapses Ns::Helper(v)
and Other::Helper(v) onto one digest. internal/record/FINGERPRINT-SPEC.md is
now the authoritative definition of anvil-fp/v1. It is IN-TREE, not in plan/,
because plan/ is gitignored and a second producer working from a clone has to
be able to read it. fingerprint_spec_test.go asserts the document's
reserved-word list and every algorithm constant match the code, so the spec
cannot drift silently -- verified by perturbing the document and watching the
guard fire.
BLOCKER 2 -- DAST route templating was unimplemented. The spec defines
route_template as derived, with numeric/UUID/hash segments replaced by a
placeholder; CanonicalRouteTemplate did none of it, and all three DAST fixtures
arrived pre-templated so the corpus could not see the gap. Two producers seeing
one defect at /api/users/12345/orders would emit three digests. This is worse
than the SAST case because the DAST tier is what earns "verified fixed" under
S7, and a reproduction that cannot be matched to its prior finding cannot prove
a fix.
Ruling: area 40 owns the fingerprint, so area 40 canonicalizes. Area D emits
whatever route it observed. Pushing it to D would leave two areas each believing
the other did it. CanonicalRouteTemplate now templates all-digit, UUID, long-hex
and long-alphanumeric segments onto one frozen <VAR> token, and normalises
already-templated segments ({id}, :id) onto the same token so the producer's
choice of syntax cannot fork the digest. Thresholds are documented and justified
against the stated asymmetry: over-templating merges distinct routes and loses a
finding, under-templating is recoverable.
Blocker 1 is closed by evidence, not by assertion
An independent agent, forbidden from opening fingerprint.go, wrote a Python
oracle from FINGERPRINT-SPEC.md alone and reproduced all 8 committed digests and
42/42 mutations on the first run, with no iteration. That is exactly the test
R.16 will have to pass.
Its honest caveat is recorded as Appendix Z: MATCH means the spec is sufficient
for what the corpus exercises, and the corpus exercises a narrow slice. Six
under-determined points are listed, of which Z4 matters most -- the ordinal
grouping key is not exercised at all, because every SAST fixture supplies a
pre-computed ordinal. An independent implementation could get that key wrong and
still pass every current fixture. Recording it costs nothing now and would be
expensive to rediscover as a digest divergence.
Two judgment calls R.2 made, and which side it changed
- Ns::Helper: changed the IMPLEMENTATION. The left operand of :: is a namespace
or type in every language that has the operator, never a local, so abstracting
it contradicted the spec's own "replace LOCAL identifiers". Deliberately not
extended to . or ->, whose left operand is usually a receiver bound to a local;
a lexer cannot tell a package qualifier from a receiver, and guessing wrong
there forks the digest of unchanged code, which is the worse failure.
- Ordinals: changed the TEST, whose `want` contradicted its own comment eight
lines below and would have put two candidates on ordinal 0 in one group.
Digest changes, stated explicitly: only dast-01 and dast-03 moved, both solely
because route templating normalised {id}/{path} onto <VAR>. The other five
goldens are byte-identical to their pre-fix values.
R.4/R.5: one handoff table carrying all thirteen dispositions and area O's
consumption_class, per rulings G9/G10. Forward-only numbered migrations with a
checksummed ledger. The FTS5 startup guard creates a real virtual table rather
than trusting a version number -- kept even though FTS5 is now positively
verified on v1.56.0, because a future bump could drop it silently.
Evidence: gofmt, go vet, go build all clean; go test -count=1 ./... green across
cmd/anvil, internal/record, internal/store. go test -race cannot run on this
Windows host (cgo.exe exit 2, no C toolchain) -- pre-existing and host-wide;
CI runs it on Linux.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
R.2(fingerprint + corpus),R.3(critic),R.4(store schema),R.5(migrations + FTS5 guard) — plus fixes for the two blockersR.3raised.The critic returned FAIL, and it was right to
R.3did not acceptR.2. Both blockers were proved rather than asserted, and both are fixed here.Blocker 1 — the algorithm existed only in Go
The critic re-implemented the plan's four-clause spec text in Python and got
55e27b07…where the committed golden is13c60ccf….NormalizeMatchapplies nine-plus operations documented nowhere: a 193-entry reserved-word list, identifier-preservation rules, comment syntaxes, string-delimiter and escape rules, a number grammar.That breaks S6 one level up — a second producer implementing from the written spec emits different digests, silently, forever, which is precisely what "one fingerprint algorithm, defined once" exists to prevent. It also left
R.16with no honest way to go green, since its oracle must be built "from scratch, not by importing fingerprint.go".Ruling: amend the specification, don't weaken the algorithm. The extra rules earn their keep — abstracting a namespace qualifier collapses
Ns::Helper(v)andOther::Helper(v)onto one digest.internal/record/FINGERPRINT-SPEC.mdis now the authoritative definition ofanvil-fp/v1, in-tree rather than inplan/, becauseplan/is gitignored and a second producer working from a clone has to be able to read it.fingerprint_spec_test.goasserts the document's word list and every constant match the code — verified by perturbing the document and watching the guard fire.Blocker 2 — DAST route templating was unimplemented
The spec defines
route_templateas derived, with numeric/UUID segments replaced by a placeholder;CanonicalRouteTemplatedid none of it, and all three DAST fixtures arrived pre-templated so the corpus couldn't see the gap. Two producers seeing one defect at/api/users/12345/orderswould emit three digests. Worse than the SAST case — the DAST tier is what earns "verified fixed" under S7, and a reproduction that can't be matched to its prior finding cannot prove a fix.Ruling: area 40 owns the fingerprint, so area 40 canonicalizes. Pushing it to area D would leave two areas each believing the other did it. Already-templated segments (
{id},:id) normalise onto the same<VAR>token, so the producer's choice of syntax can't fork the digest.Blocker 1 is closed by evidence, not assertion
An independent agent, forbidden from opening
fingerprint.go, wrote a Python oracle from the spec alone and reproduced all 8 digests and 42/42 mutations on the first run, with no iteration. That is exactly the testR.16will have to pass.Its honest caveat is recorded as Appendix Z: MATCH means the spec is sufficient for what the corpus exercises, and the corpus exercises a narrow slice. Six under-determined points are listed. Z4 matters most — the ordinal grouping key isn't exercised at all, because every SAST fixture supplies a pre-computed ordinal, so an independent implementation could get that key wrong and still pass every fixture.
Two judgment calls, and which side R.2 changed
Ns::Helper— changed the implementation. The left operand of::is a namespace or type in every language that has the operator, never a local. Deliberately not extended to.or->, whose left operand is usually a receiver bound to a local: a lexer can't tell a package qualifier from a receiver, and guessing wrong there forks the digest of unchanged code, which is the worse failure.wantcontradicted its own comment eight lines below and would have put two candidates on ordinal 0 in one group.Digest changes, stated explicitly
Only
dast-01anddast-03moved, both solely because route templating normalised{id}/{path}onto<VAR>. The other five goldens are byte-identical to their pre-fix values — the algorithm was documented, not altered.R.4 / R.5
One
handofftable carrying all thirteen dispositions and area O'sconsumption_class, per rulings G9/G10. Forward-only numbered migrations with a checksummed ledger. The FTS5 startup guard creates a real virtual table rather than trusting a version number — kept even though FTS5 is now positively verified on v1.56.0, because a future bump could drop it silently.Testing
go test -racecannot run on this Windows host (cgo.exeexit 2, no C toolchain) — pre-existing and host-wide. CI runs it on Linux.