Skip to content

feat: add a runtime parse boundary and six persisted-document namespaces - #10

Merged
ernysans merged 1 commit into
mainfrom
ernysans-core-node-schemas
Aug 22, 2026
Merged

feat: add a runtime parse boundary and six persisted-document namespaces#10
ernysans merged 1 commit into
mainfrom
ernysans-core-node-schemas

Conversation

@ernysans

Copy link
Copy Markdown
Member

Why

This package shipped types only. A TypeScript type is erased before any stored data is read, so it constrains what a caller writes and never what arrives — and the gap gets closed with a cast:

const amount = Number(data['amount'] ?? 0);

A cast is worse than an unchecked read. It does not merely skip the check, it suppresses the diagnostic that would have reported the mistake, so everything downstream is correctly typed against a value that was never verified. Number('abc') is NaN, NaN loses every comparison it is subsequently used in, and nothing throws.

This PR supplies the missing parse boundary. It does not invent a type system — the domain types already existed and are good.

What

Every namespace now exports a zod schema co-located with its Interface, plus parse and safeParse. There is no branch that yields a typed value which was never validated: ParseSuccess has no issues, ParseFailure has no data.

New shared module src/interface/schema.tsParseResult / ParseError / parseResult / parseOrThrow, the AssertSchemaOutput compile-time proof, and field builders (finiteNumber, nonNegativeNumber, counter, documentId, nonEmptyString, token, epochSeconds, epochMillis, timestampLike, auditTimestamp, openValue, requiredKey).

Schemas added to Account, Block, EventData, MessagingEvent, Post, Price, BaseFirestore, MessageQueue, BasePlaceData, PlaceData.

Six new namespaces for persisted shapes with no declared document type: Idempotency, Ledger, Reservation, Entitlement, Capacity, MessageUsage.

Decisions worth reviewing

Unknown keys are preserved — not dropped, not rejected. BaseFirestore declares [x: string]: any, so a strict schema would contradict the published type and reject documents predating it; a stripping schema (zod's default) silently deletes stored fields on a read-modify-write. z.looseObject is the only option that loses nothing. Tested per namespace.

Timestamps are validated structurally and passed through by reference. Validating a Timestamp with z.object({seconds, nanoseconds}) rebuilds it into a plain object and destroys the class instance — writing that back breaks any TTL policy keyed on the field, silently. z.custom validates without reconstructing. This is the runtime resolution of the fields typed any: their honest type needs the server SDK's FieldValue, which must not enter every consumer's dependency closure, so the constraint is enforced where it can actually be checked. A write payload carrying a server sentinel is deliberately rejected, with a documented widening escape (Schema.extend({created: openValue()})).

epochSeconds and epochMillis have deliberately disjoint ranges. Capacity.expiresAt is seconds; Reservation.expiresAt is milliseconds. Same field name, two units, three orders of magnitude apart — a field name is not a unit. Neither value can now validate as the other.

Capacity enforces that expiresAt and expires denote the same instant whenever both are present (expires.seconds === Math.trunc(expiresAt)). Both failure modes are silent: a TTL derived with the wrong unit either deletes a live hold early or never deletes it at all. Capacity.ObjectSchema is exported for callers validating a hold whose TTL is not yet derived.

| null policy. With strictNullChecks: false these annotations are unenforced by our own compiler but are emitted into the .d.ts and enforced by a consumer compiling strictly. So the rule is asserted at runtime instead: a field annotated | null accepts null; a field annotated only ? rejects it, because ? and | null are different claims and a parse must not return a value the declared type says cannot occur. .nullish() is used nowhere. test/interface/schema.test.ts drives null into every key of all 14 schemas and asserts the accepting set equals an explicit inventory — and does the same for required-vs-optional. Audit/event timestamps are any-typed and therefore decided explicitly: they reject null, since an explicitly null timestamp is not a time and reading it as epoch zero sorts it first and expires it immediately.

Gates

Gate Result
npm run build exit 0
npm test 903 passed, 18 files, exit 0 (was 480)
npm run typecheck exit 0
npm run lint exit 0
git status --porcelain -uall -- lib/ empty after rebuild
.github/scripts/check-private-markers.sh exit 0

lib/ is regenerated and committed in the same commit. The eight new modules land as untracked output, which is precisely the case git diff --exit-code would have missed — the -uall form catches them.

Evidence

20 of 20 mutations detected. Each loosens one control, runs the suite, and is reverted; the substitution itself is verified before the result is trusted, so a no-op mutation cannot report a false green.

Mutation Failures
Price.amount money → any 3
Ledger.amount money → any 7
Capacity drop cross-field expiry check 4
Capacity.expiresAt seconds → unbounded 1
Reservation.expiresAt millis → unbounded 3
Entitlement.type enum → string 2
Idempotency.state enum → string 2
MessageUsage.period closed grammar → any string 1
Account.status enum → string 3
Post.type enum → string 1
Block.type enum → string 4
EventData.amount money → any 4
MessagingEvent.user.id required → optional 3
place drop latitude bounds 5
queue.pending counter → any 10
base_db.created audit timestamp → open value 12
Price.uid drop nullable 8
model barrel drop one export * 4
epochMillis overlap the seconds range 4
looseObjectobject (silently strip unknown keys) 2

And the asymmetry between the two gates, measured on this branch. Retyping Ledger.Interface.consumed?: number to string while leaving the schema alone: npm test reports 903 passed, exit 0; npm run typecheck reports exit 2, naming the divergence. That is why both are required, and why the compile-time proof is worth having.

Things the compile-time proof caught during development

Both were silent divergences that would have shipped a type contradicting its own schema:

  1. Block.value — required, but zod infers a z.union key as optional in this repo, because strictNullChecks: false collapses zod's internal "optional" | undefined marker to "optional". parse() would have returned a type claiming value may be absent when at runtime it never is. .nonoptional() and .pipe() do not help; both re-declare the same marker. Resolved with a requiredKey helper built on z.custom, whose marker is an optional property and therefore unaffected.
  2. Idempotency.Response.body — same class of bug via z.nullable().

Deliberately left out — please confirm

Ledger.service is a validated string, not an enum. The set of metered surfaces is owned by the metering service, not by this package. A partial copy here would reject legitimate records, and an allow-list that is minimal rather than complete is a capability regression wearing a validation costume. Some plausible members also describe billable surfaces not implied by any type this package already publishes, which is business detail rather than shape. Callers needing strict membership should check the parsed value against their own enum.

By contrast, Idempotency.State (in_progress/completed/failed) and Entitlement.Status (processing/complete/failed) are declared: both are generic lifecycle vocabularies carrying no business specifics. Entitlement.type and Capacity.type reuse the already-public Price.Type rather than declaring a parallel enum, so the two cannot drift.

No thresholds, rate limits or pricing rules are encoded anywhere. All example values are obviously synthetic.

Found but not fixed

  • MessagingEvent.Status is declared but unreferenced — the enum exists and is documented as the lifecycle status of a messaging event, but MessagingEvent.Interface has no status field. Adding one would be additive and safe, but it is outside this PR's scope and I would be guessing at its optionality. Unknown-key preservation means a stored status round-trips untouched today; there is a test asserting that.
  • tests.instructions.md §3 says test/ "is not currently linted". Measured otherwise: eslint.config.js scopes only its custom rules block to src/**/*.ts, while eslint.configs.recommended and the typescript-eslint presets apply to everything not in ignores — and test/ is not ignored. npm run build failed on a @typescript-eslint/array-type violation in a new test file during this work. So no-explicit-any and no-unused-vars are on for tests even though they are off for src. Worth correcting in that doc.

Not addressed

strictNullChecks / noImplicitAny remain off; flipping them is its own change. The | null decisions above are made explicit and runtime-asserted specifically so that follow-up is boring.

This package shipped types only. A TypeScript type is erased before any
stored data is read, so it constrains what a caller writes and never what
arrives, and the gap was being closed with casts and coercion. A cast is
worse than an unchecked read: it suppresses the diagnostic that would have
reported the mistake, and `Number(x ?? 0)` turns a malformed value into a
`NaN` that loses every comparison it is subsequently used in without
anything throwing.

Adds, per namespace, a zod schema co-located with its `Interface`, plus
`parse` and `safeParse` helpers that turn `unknown` into the interface or
a typed failure. There is no branch that yields a typed value which was
never validated.

New shared module `src/interface/schema.ts`:
- ParseResult / ParseError / parseResult / parseOrThrow
- AssertSchemaOutput, a type-level proof that each schema produces its
  declared interface, so divergence is a compile error
- field builders: finiteNumber, nonNegativeNumber, counter, documentId,
  nonEmptyString, token, epochSeconds, epochMillis, timestampLike,
  auditTimestamp, openValue, requiredKey

Schemas added to Account, Block, EventData, MessagingEvent, Post, Price,
BaseFirestore, MessageQueue, BasePlaceData and PlaceData.

Six new namespaces for persisted shapes that had no declared document
type: Idempotency, Ledger, Reservation, Entitlement, Capacity and
MessageUsage.

Notable decisions, all documented in the source:
- Unknown keys are preserved, not dropped and not rejected. That matches
  the `[x: string]: any` index signature on BaseFirestore; a stripping
  schema would delete stored fields on a read-modify-write and a strict
  one would reject documents predating the schema.
- Timestamps are validated structurally and passed through by reference,
  so a live Timestamp keeps its prototype and any TTL policy keyed on it
  still works. This is the runtime resolution of the fields typed `any`.
- epochSeconds and epochMillis have deliberately disjoint ranges.
  Capacity.expiresAt is seconds and Reservation.expiresAt is
  milliseconds; a field name is not a unit.
- Capacity enforces that expiresAt and expires denote the same instant
  whenever both are present.
- `| null` is accepted only where an interface declares it, and rejected
  where the interface says only `?`. Asserted at runtime because
  strictNullChecks is off and the compiler cannot check it.

Ledger.service is a validated string rather than an enum: the set of
metered surfaces is owned by the metering service, and a partial copy
here would reject legitimate records.

903 tests, all gates green, and lib/ is regenerated in this commit.
@ernysans
ernysans merged commit f40b78e into main Aug 22, 2026
2 checks passed
@ernysans
ernysans deleted the ernysans-core-node-schemas branch August 22, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant