[claude] Detect and recover when OpaqueChanges become applicable

[myieye]

[Claude, autonomous]

Problem

Since #80, a change with an unknown $type is stored and synced onward as an OpaqueChange, but SnapshotWorker skips applying it. Snapshots and projected tables are then silently missing its effect, and:

• When the app later registers the type, nothing notices. The stored JSON would now deserialize concretely, but snapshots are never re-folded, so the stale state persists forever.
• Some types will never become known (a different app writing to the same project). That's a legitimate steady state, but a client currently can't even answer "is my materialized state incomplete?", which matters for consumers that write materialized state into other systems (FwHeadless → FieldWorks).
• If the first change to an entity is opaque, no snapshot exists, so later known changes to that entity are skipped too.

Recovery itself is the easy part: the commit log is complete everywhere (opaque JSON round-trips verbatim), so RegenerateSnapshots() already repairs state once the type is registered. What's missing is detection, and ideally a cheaper replay scope than "everything".

One idea: an UnappliedChangeTypes table

One aggregate row per unknown discriminator, written at commit ingest (CrdtRepository.AddCommits, same transaction): TypeName (PK), EarliestCommitId, ChangeCount.

• Detection = intersect TypeNames with registered discriminators at project open. Non-empty: replay from the predecessor of the earliest affected commit (the existing DeleteStaleSnapshots + UpdateSnapshots path), then delete the recovered rows.
• Never-known types never intersect; their rows sit inert and double as a cheap "this project has changes I can't apply" signal for UI or sync guards.
• Existing DBs backfill with one json_extract(Change, '$."$type"') scan over ChangeEntities.

Ingest is deliberate: the converter also runs on read paths, and the SnapshotWorker skip sites also fire for ordinary out-of-order arrivals, so ingest is the only place that sees each commit exactly once.

Alternatives

Persist nothing; fingerprint the registered type set. On growth, scan history for the new types; regenerate if found.

• Pro: zero bookkeeping, fully retroactive.
• Con: a scan every release that adds a type; "am I incomplete right now?" also costs a scan; the registered set can differ per host (e.g. conditional AddRemoteResourceEntity), so set comparison needs care.

Per-skip rows (CommitId, ChangeIndex, TypeName, EntityId).

• Pro: exact.
• Con: unbounded in the never-known steady state; no benefit over the aggregate, since replay is scoped by earliest commit, not per change; only records skips after it ships, so it needs the backfill anyway.

Regenerate whenever the registered type set changes.

• Pro: no detection code.
• Con: full replay after nearly every release; RegenerateSnapshots is currently unlocked, non-transactional, and slow on big projects, so everyone pays for a rare event.

Do nothing; rely on the manual regenerate troubleshooting action.

• Pro: zero code.
• Con: staleness is silent, so nobody knows to press the button; FwHeadless would sync stale state indefinitely.

Notes

• Whichever way this goes, it should land in the same release as Tolerate unknown IChange $type via PeekThenConcreteChangeConverter #80 or close behind: builds that skip changes without recording anything leave stale snapshots that later detection can't tell from a clean DB without the backfill scan.
• Auto-regeneration should wait until RegenerateSnapshots is locked, transactional, and fast enough. Detection plus a visible warning is already useful on its own.

Context: sillsdev/languageforge-lexbox#2482 (review discussion that prompted this).