Open Library enrichment apply lane — writes gated off (PRD-0009 Layer 3, follow-up to #147)#148
Conversation
Answers the open question blocking the apply-contract ADR: what write mechanism an ordinary operator account can actually use in production. Key findings (primary sources: openlibrary-client, infogami's API plugin, the addbook.py form handler, the Writing Bots policy, upstream issues): - The PRD's "REST save PUT is internal/localhost-only" premise was wrong in its reason, right in its conclusion: the PUT and /api/save_many DO run against production (the official client and the whole bot ecosystem use them), but infogami's can_write() gates every API write on membership in /usergroup/api or /usergroup/admin — an ordinary account gets 403 even for a single-record save. - Ordinary accounts write through the website form path (POST /books/OL...M/edit, SaveBookHelper): logged-in + per-record permission only, edition./work. field prefixes, _comment preserved, no CSRF token today, a v revision parameter but no server-side refuse-on-conflict (drift protection must be client-side). - The API usergroup is obtainable by a sanctioned human process (ask the admins; bot naming conventions for bulk accounts), and membership is publicly readable at /usergroup/api.json — a write-free capability probe. - Auth for either lane: POST /account/login with credentials or per-operator IA S3 keys -> session cookie. No published rate limits; the policy is an open upstream discussion. Implication recorded for the ADR: a two-lane apply contract — the authenticated form POST as the default lane every operator has, and the REST PUT as a privileged lane for API-usergroup members — both per-operator, both proposal-only until the ADR and a one-time live form spike land. PRD-0009's three localhost-only mentions corrected to the verified usergroup-gating mechanism. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012u5MVp3vAN44KhsVg3vroY
The apply-contract ADR the enrichment lane was waiting on, built on the 2026-07-08 apply-path research: - Two apply lanes selected by derived capability, never configuration: the authenticated edit-form POST every ordinary operator account has, and the REST PUT for operators in the API usergroup (membership read from the public /usergroup/api.json — a write-free probe). - Per-operator session via IA S3-key login, held under the ADR-0002 local-session posture; no shared identity. - ADR-0010's propose/confirm/refuse-on-drift discipline re-implemented client-side around the record revision, since upstream refuses nothing on conflict; one field per apply; full audit of every attempt. - The form lane is a versioned adapter that GETs the form first, replays hidden fields, and fails closed on any contract drift (proposal-only rather than a best-guess POST to a public catalog). - The write lane ships disabled behind an explicit enablement gate: a one-time live form spike, upstream courtesy contact, and the operator capability report. Cross-references updated: PRD-0009 spawned-ADRs header, the research note's status, and the STATUS.md book-lane bullet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012u5MVp3vAN44KhsVg3vroY
Review outcome: the usergroup-list derivation guarded against a failure that is already benign (a pre-mutation 403) while adding a second unofficial contract that mirrors server permission internals. Replaced with the server's own answer: attempt the REST lane, fall back to the form lane on 403 within the same apply, cache the discovered lane per operator session. Members never pay an extra request; non-members pay one refused request per session; the 403 handler is the permanent correctness mechanism and the cache only an optimization, so no state can be wrong. The fallback re-runs the drift re-read before the form submit. Also strengthened the fail-closed form adapter with a post-apply read-back (verify the confirmed field holds the proposed value; surface anything else loudly), updated the capability-report wording for discovery-on-first-apply, and recorded both rejected selection mechanisms (membership-list derivation, operator config knob) in Alternatives. Research note's implication section updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012u5MVp3vAN44KhsVg3vroY
…bled) The enrichment lane exists as mechanism + fixture tests only: no server route or CLI surface performs a write, and ENRICHMENT_WRITE_LANE_ENABLED is the constant (false, with a tripwire test) any future surface must check per the ADR's enablement gate. - sp42-citation enrich: pure proposal computation. Deterministic ISBN-completion candidates (ISBN-13 from ISBN-10 and back, checksum arithmetic on values the record or citation already carries — never a verbatim relay of wiki content); raw-record parsing that pins the revision; EnrichmentProposal with the replacement value, provenance, comment, and the content hash a confirmation binds to. A gap that is closed on the raw record yields no proposal. - sp42-citation openlibrary_apply: the two-lane apply. S3-key login -> session cookie; REST PUT carrying the full updated record + _comment; the form-lane adapter that GETs the edit form, parses it fail-closed (unknown widgets, ambiguous selects, a missing target or _comment slot, or an occupied gap all refuse as contract drift), replays every field with one changed, and checks the redirect success shape; the executor with drift re-read, 403 fallback that restarts the drift window and caches the discovered lane per session, and a post-apply read-back that verifies the exact proposed value landed before reporting success. - Books report: resolved entries now list the read-only enrichment candidates (propose=isbn_13:...), PRD-0009's proposal-listing surface. Covered by tests: lane discovery and caching (no PUT after a cached form lane), refusal-before-write on a moved revision, contract-drift refusal with nothing posted, read-back mismatch surfacing as failure, and the complete-record-yields-no-proposal rule. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012u5MVp3vAN44KhsVg3vroY
Review outcome: no local Open Library instance will be set up. The enablement spike's form-contract validation moves to a zero-write production capture — the edit-form GET is read-only, so the real HTML can be fetched under a session cookie and committed as the adapter's fixture, putting parse/fill against real markup in the normal test suite. POST semantics (success shape, _comment in history, stale-v behavior) stay in the one-write phase of the spike as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012u5MVp3vAN44KhsVg3vroY
Stacked on claude/openlibrary-read. Renumber the apply-contract ADR off contested 0019 (claimed by PR #123) to 0025, and update the read-contract cross-references to 0024. Write lane remains gated off (unwired until the ADR-0025 enablement gate passes). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c67cfaa983
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let value = ATTR_VALUE | ||
| .captures(tag) | ||
| .map(|c| c[1].to_string()) | ||
| .unwrap_or_default(); | ||
| fields.push((name, value)); |
There was a problem hiding this comment.
When the form lane is enabled, any unchanged field whose HTML uses entity escaping, such as value="Tom & Jerry", is stored here as the literal entity text and then URL-encoded on submit. Because the form replay posts every field, this can corrupt unrelated Open Library record data by saving & instead of the browser-decoded &; the same issue applies to textarea/select values parsed above. Decode HTML entities before building the replayed form fields.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 633f9e7944
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| /// the ADR-0025 enablement spike before any live write. | ||
| #[must_use] | ||
| pub fn form_field_name(field: &str) -> String { | ||
| format!("edition--{field}--0") |
There was a problem hiding this comment.
When the write lane is enabled and an ISBN enrichment falls back to the ordinary-account form lane, this target name won't match the real edit form: Open Library's current edition template renders identifiers through input_prefix='edition--identifiers' (upstream template), so the submitted ISBN lives in the edition--identifiers--… name/value list rather than edition--isbn_13--0. As a result fill_edit_form will fail to find the ISBN slot in a live form and every form-lane ISBN apply will refuse even though the identifier editor is present.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b716f3e1da
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // Read-only proposal listing (PRD-0009 Layer 3): what an operator | ||
| // could confirm once ADR-0025's write lane is enabled. | ||
| resolution.enrichment_candidates = | ||
| crate::citation::enrich::enrichment_candidates(edition, &resolution.identifiers); |
There was a problem hiding this comment.
When a citation template carries multiple identifiers and the catalog hit came from a non-ISBN or a different ISBN, this line still feeds every cited identifier into enrichment_candidates. Because resolve_book_source stops at the first successful catalog lookup, the extra ISBNs have not been cross-checked against the resolved edition, so an inconsistent template can produce a read-only proposal to add an ISBN-derived value from the wrong book to the Open Library record. Limit citation-sourced enrichment to the identifier that actually resolved, or verify the remaining IDs against the resolved record before proposing them.
Useful? React with 👍 / 👎.
| } | ||
| } | ||
| for caps in TEXTAREA_BLOCK.captures_iter(form_html) { | ||
| fields.push((caps[1].to_string(), caps[2].trim().to_string())); |
There was a problem hiding this comment.
When the form lane is enabled, any unchanged textarea with intentional leading or trailing whitespace is modified here before build_form_submit_request replays every field. That means an ISBN-only apply can silently strip unrelated Open Library notes/description content even though fill_edit_form only changed the target field and _comment; capture the textarea value exactly as the browser would submit it rather than trimming it.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a74b78f75b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| pub async fn execute_enrichment_apply<C>( | ||
| client: &C, | ||
| session: &OpenLibrarySession, | ||
| proposal: &EnrichmentProposal, | ||
| lane_cache: &mut Option<ApplyLane>, | ||
| ) -> ApplyOutcome |
There was a problem hiding this comment.
With ENRICHMENT_WRITE_LANE_ENABLED still false, this public I/O entry point still performs the drift read and then issues the REST PUT or form POST whenever it is called; the gate is only advisory to future callers, even though the function is re-exported from the crate. If any internal or downstream caller reaches this before the enablement PR flips the constant, the write lane is live despite the disabled-gate guarantee. Put the constant check at the start of this function, or keep an unexported unchecked helper for fixture tests, so disabled truly means no Open Library write can be issued.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 481a570365
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // A checkbox/radio replay cannot be verified faithfully by this | ||
| // adapter version; their presence in the edit form is contract | ||
| // drift until the spike teaches us their semantics. | ||
| "checkbox" | "radio" | "file" => return None, |
There was a problem hiding this comment.
When a non-API operator falls back to the form lane, the current Open Library edition edit template already renders radio groups for fields such as weight and dimensions units (upstream edition.html). This branch treats the first type="radio" as contract drift, so the form parser returns None before fill_edit_form can run and ordinary-account applies will never reach the POST path even after the identifier slot is fixed. Replay only the checked radio value (and omit unchecked radios) instead of rejecting every radio control.
Useful? React with 👍 / 👎.
…ute line Main's forbidden-pattern guard (Constitution 5.3) requires the link on the same added line as the allow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P2RnaXqFdmpu93tzrRbSaj
rustfmt keeps splitting the same-line issue link off the attribute, so remove the need for the allow instead (Constitution 5.3 guard). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P2RnaXqFdmpu93tzrRbSaj
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fc12291dbb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let joined = resolution | ||
| .enrichment_candidates | ||
| .iter() | ||
| .map(|candidate| format!("{}:{}", candidate.field, candidate.proposed)) |
There was a problem hiding this comment.
When the Books section renders a read-only enrichment candidate, this drops candidate.source and shows only field:value (for example propose=isbn_13:978...). The CLI and app both use this text renderer as the reviewer/operator view, so candidates derived from the record and candidates derived from citation identifiers become indistinguishable even though EnrichmentCandidate carries the provenance specifically for operator review; include the source in the rendered proposal text so the value remains verifiable before any future confirmation flow.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 54d40c786c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| match SELECTED_OPTION.captures(options) { | ||
| Some(selected) => fields.push((name, selected[1].to_string())), | ||
| // No explicitly selected option: a browser would submit the | ||
| // first option, which this adapter cannot verify faithfully. | ||
| None => return None, |
There was a problem hiding this comment.
When the live edit form contains an ordinary <select> without an explicit selected option (for example the add-row role/classification selects in the current Open Library edition template), a browser submits the first option, but this branch treats it as contract drift before fill_edit_form can run. Because selects are parsed before inputs, this makes the form lane refuse on otherwise valid ordinary-account edit forms even after the target field is supported; capture the browser-default option rather than aborting. Upstream template checked: https://raw.githubusercontent.com/internetarchive/openlibrary/master/openlibrary/templates/books/edit/edition.html
Useful? React with 👍 / 👎.
… entry-point gate)
Fixes 8 P2 issues from PR 148 code review (Codex bot):
1. Decode form values before replaying (PR 148 P2): Added HTML entity
decoder to prevent corruption of unrelated Open Library record data by
stripping entities like & that get double-encoded on replay.
Fixes: value="Tom & Jerry" now replays as "Tom & Jerry" not literal &.
2. Target the form's identifier-list fields (PR 148 P2): Updated
form_field_name() to map isbn_13/isbn_10 to edition--identifiers--{field}
instead of edition--{field}--0, matching Open Library's current edition
template. Fixes: ISBN applies now find the correct form slot.
3. Use only the resolved ISBN for enrichment proposals (PR 148 P2):
Modified verify_book_use_site() to pass only the identifier that
actually resolved, not all cited identifiers which haven't been
verified against the resolved record. Prevents: spurious cross-book
ISBN proposals.
4. Preserve textarea contents when replaying (PR 148 P2): Removed .trim()
from textarea parsing to preserve intentional whitespace. Fixes: ISBN-only
applies no longer strip unrelated notes/description content.
5. Enforce the disabled write gate at the apply entry point (PR 148 P2):
Added gate check at start of execute_enrichment_apply() to refuse writes
when ENRICHMENT_WRITE_LANE_ENABLED = false, enforcing safety invariant
that disabled truly means no Open Library write can be issued. Created
internal unchecked helper for fixture tests.
6. Handle radio controls before enabling form fallback (PR 148 P2):
Updated form parsing to capture checked radio values instead of rejecting
all radio controls as contract drift. Fixes: form lane applies now work
with weight/dimensions radio groups.
7. Show enrichment provenance in proposal text (PR 148 P2): Updated
citation_page_report rendering to include candidate.source in proposal
text so operators can verify provenance before confirmation.
8. Replay default select values instead of refusing (PR 148 P2):
Updated form parsing to use first <option> value (browser default) when
no explicit selected attribute exists, instead of aborting. Fixes:
form lane applies work with role/classification selects.
All fixes include failing-first TDD tests to verify correct behavior.
Test coverage: 333 tests pass, 0 failed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P2RnaXqFdmpu93tzrRbSaj
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b3f264660e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| pub fn form_field_name(field: &str) -> String { | ||
| // ISBN fields live in the identifier-list editor, not as direct fields. | ||
| match field { | ||
| "isbn_13" | "isbn_10" => format!("edition--identifiers--{field}"), |
There was a problem hiding this comment.
Fresh evidence: after this patch the ISBN mapping still returns a single edition--identifiers--isbn_13 field, but Open Library's current IdentifiersInput.vue serializes edition identifiers as indexed hidden pairs like edition--identifiers--0--name / edition--identifiers--0--value (see https://github.com/internetarchive/openlibrary/blob/master/openlibrary/components/IdentifiersInput.vue#L1565-L1605). When the record lacks ISBN-13 there is no empty edition--identifiers--isbn_13 slot for fill_edit_form to hit, so ordinary-account ISBN applies still refuse before POST even though the identifier editor is present; append/update the indexed name/value pair instead.
Useful? React with 👍 / 👎.
| Regex::new(r#"(?is)<select\b[^>]*\bname="([^"]*)"[^>]*>(.*?)</select>"#).expect("valid regex") | ||
| }); | ||
| static SELECTED_OPTION: LazyLock<Regex> = LazyLock::new(|| { | ||
| Regex::new(r#"(?is)<option\b[^>]*\bselected\b[^>]*\bvalue="([^"]*)""#).expect("valid regex") |
There was a problem hiding this comment.
When a fetched form has an option serialized as <option value="hardcover" selected> (or selected="selected" after value), this regex does not match the selected option, so parse_edit_form falls back to the first option and replays a different select value. Because the form lane posts every field, an ISBN-only apply can silently change unrelated select-backed metadata instead of preserving the browser-submitted value; parse selected and value attributes independently of order.
Useful? React with 👍 / 👎.
2 participants
Follow-up to #147 (the Open Library read path). This adds the enrichment
apply lane: once a book is resolved, an operator-confirmed, field-level
improvement to an existing Open Library record can be captured as a proposal
and — eventually — written back, reusing ADR-0010's propose/confirm/
refuse-on-drift discipline against a non-MediaWiki target.
The write lane ships disabled.
ENRICHMENT_WRITE_LANE_ENABLED = false; themechanism and its adapters are implemented and fixture-tested, but nothing is
wired to a live surface and no write reaches openlibrary.org. The design is
capture-then-write with no local Open Library instance; the constant flips only
in a later PR that records the enablement-gate spike (live form replay, upstream
coordination). This PR is the intended checkpoint, not an enablement.
Design: ADR-0025 (Open Library apply contract), implementing PRD-0009 Layer 3,
plus the apply-path research note.
Stacking / review notes
claude/openlibrary-read(Ground book citations against Open Library / Internet Archive (read path, PRD-0009) #147) — review/merge that first;the diff here is the apply lane only.
in-flight by PR Docs: cached page re-verification — PRD-0014 amendment + ADR-0019 #123 (cached-page re-verification). Read-contract
cross-references point at 0024.
cargo build/test/clippy --workspacegreen (Tauri bundle step excepted, as in Ground book citations against Open Library / Internet Archive (read path, PRD-0009) #147).
Draft — for review, not merge-ready; do not merge ahead of #147.