Report whether a typed source is actually verifying anything - #16
Merged
Conversation
Setting a source's type to STRIPE or GITHUB does not switch verification on. The provider's own signing secret has to be set on the source, and until it is the source accepts anything. Measured against a real project: a GITHUB source with no secret accepted an unsigned request *and* one carrying `sha256=deadbeef`, and produced events from both. With the secret set, an unsigned request came back VERIFICATION_FAILED with no event, and a genuine signed Stripe delivery passed. `setup` says the secret is a manual step, once, in its own output. The README stated the verification as a flat benefit, and nothing mentioned it again. doctor now reports it, and the interesting part is what it can and cannot know. A source with a secret configured is byte-identical to one without over the API — `config.auth` is absent and `verification` is null either way, confirmed against a source whose secret was definitely set. So there is nothing to read on the source itself. What the API does expose is `verified` on each inbound request, so the check reads observed traffic: * some requests unverified -> fail, and say the secret is missing or mismatched * all verified -> confirm, naming the count it saw * no requests at all -> say "unconfirmed", not "fine" The third case is the one worth being careful about. With no traffic there is genuinely no signal, and claiming the source is fine would be the one answer certain to be wrong. Plain WEBHOOK sources are left alone: they verify with the project's own signing secret, which the adapter already checks, so the warning would be noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both follow-ups from the live-testing pass.
The finding
Setting a source's type to
STRIPEorGITHUBdoes not switch verification on. The provider's own signing secret must be set on the source, and until it is the source accepts anything. All four quadrants measured against a real project:sha256=deadbeefVERIFICATION_FAILED, 0 eventsverified: true→ agent runsetupmentions the manual step once in its own output. The README stated the verification as a flat benefit, and nothing raised it again.What doctor can and cannot know
The interesting constraint: a source with a secret configured is byte-identical to one without, over the API.
config.authis absent andverificationisnulleither way — confirmed against a source whose secret was definitely set (screenshot +verified: trueon its requests). So there is nothing to read on the source itself.What the API does expose is
verifiedon each inbound request. So the check reads observed traffic:That third case is the one worth care. With no traffic there is genuinely no signal, and reporting the source as healthy would be the one answer certain to be wrong. It passes rather than fails, because unknowable is not broken.
Plain
WEBHOOKsources are left alone — they verify with the project's own signing secret, which the adapter already checks, so the warning would be noise.Also
README corrected in both places it made the claim: the comparison table now says "once the provider's secret is set on the source", and the How-it-works section spells out that
setupcannot set it and points atdoctor.Verified
313 tests, ruff clean. Both guards mutation-checked: treating unverified traffic as fine, and nagging plain WEBHOOK sources, each fail the suite.
Ships in the wheel (
cli.py,api.py) — patch release when convenient.