fix(schema): accept null for fields Firestore stores as null

.github/instructions/serialized-models.instructions.md

@@ -33,10 +33,40 @@ it is a wrong shape replicated into every consumer.
   (`Account.Interface`). Enums sit beside it in the same namespace.
 - **Fields are optional (`?`) by default** for stored entities. Firestore documents are sparse and
   partially populated; a required field in the type is a promise the datastore does not keep.
-- **`?` and `| null` mean different things.** Use `?` for "may not be present" and `| null` for
-  "explicitly absent and must survive a JSON round-trip" — `undefined` keys are dropped by
-  `JSON.stringify`, `null` keys are not. `place.ts` uses `| null` deliberately; match the
-  surrounding convention rather than mixing.
+- **A stored optional field must be declared `?: T | null` and validated with `.nullish()`.**
+  This is not a style preference, it is what the datastore does. Firestore stores an absent
+  optional field as an **explicit `null`** under common write patterns, so `?` alone describes a
+  shape that stored documents do not have. A schema built from `.optional()` rejects `null`, which
+  means it rejects the very documents it exists to validate — measured at 25/25 stored `account`
+  documents and 12/12 stored `price` documents before this was fixed.
+  - Declare `.nullish()` on the schema field **and** `| null` on the interface property, together.
+    They are one change. A schema that accepts `null` while the interface promises it cannot occur
+    is the runtime-versus-declaration mismatch that no amount of type checking can see.
+  - The inventory in `test/interface/schema.test.ts` (`nullRejecting`) enforces this. It is
+    deliberately a **reject-list**, so it shrinks toward empty and a blanket loosening turns it
+    red — an accept-list would silently grow instead.
+- **`?` and `| null` mean different things, and a stored field is usually both.** `?` is "the key
+  may not be present"; `| null` is "the key is present and explicitly empty, and that must survive
+  a JSON round-trip" — `undefined` keys are dropped by `JSON.stringify`, `null` keys are not. Both
+  occur in stored data, which is why `.nullish()` rather than either alone is the default there.
+  Keep `null` in the parse output rather than folding it to `undefined`: a read-modify-write
+  through a folding schema deletes the stored field, and `x === undefined` and `'key' in obj` give
+  different answers for the two.
+- **Two exemptions, and only these two.** Both are inventoried in the test above:
+  - A **required** field never accepts `null`. A required field carrying `null` is exactly the
+    load-bearing absence the requirement exists to stop.
+  - An **instant-valued** field — anything validated by `auditTimestamp()` or `timestampLike()` —
+    stays `.optional()` and keeps rejecting `null`. An explicitly null timestamp is not a time, and
+    reading one as epoch zero sorts it first and expires it immediately. No stored null was
+    observed in any of these fields, so this exemption costs nothing today; if one is ever
+    observed, the fix is a documented decision about what a null instant means, **not** a blanket
+    loosening.
+- **Validating an inbound payload is a different job from reading a stored document.** No schema in
+  this package currently validates an HTTP body, a callable `data` argument or a webhook payload —
+  every `parse`/`safeParse` here is a stored-document boundary. If one is ever added, `.optional()`
+  is correct for it, because a JSON body genuinely omits a key rather than nulling it, and
+  `.nullish()` there would weaken untrusted-input validation. **Do not reuse a stored-document
+  schema for an inbound payload; declare a separate one.**
 - **Constrain string enumerations with an enum or a union.** Where a raw stored value must be
   tolerated, use `T | string` — **never `T | any`**, which collapses to `any` and silently stops
   discriminating while still reading as though it constrains something.

lib/interface/base_db.d.ts

@@ -18,13 +18,13 @@ export interface BaseFirestore {
     /**
      * Firestore document identifier, typically the auto-generated document ID.
      */
-    id?: string;
+    id?: string | null;
     /**
      * Backup status flag for long-term historical storage.
      * When `true`, the document has been successfully backed up to the
      * historical long-term database. When `false`, the backup is still pending.
      */
-    backup?: boolean;
+    backup?: boolean | null;
     /**
      * Server-side timestamp recorded when the document was first created.
      */
@@ -61,16 +61,30 @@ export interface BaseFirestore {
  * at the boundary instead, where it can actually be checked. See
  * {@link auditTimestamp} for the shapes accepted, what is rejected, and how to
  * widen a schema for a write payload.
+ *
+ * {@link BaseFirestore.id} and {@link BaseFirestore.backup} are `.nullish()`,
+ * because a stored document writes an unset optional field as an explicit
+ * `null` rather than omitting it, and a field that accepted only `undefined`
+ * would reject documents that are otherwise entirely well formed.
+ *
+ * The three timestamps are deliberately **not** `.nullish()` and keep rejecting
+ * `null`. An explicitly null timestamp is not a time: read as one it becomes
+ * epoch zero, which sorts first and — for {@link BaseFirestore.expiry}, a TTL —
+ * expires the document immediately. No stored null was observed in any of the
+ * three, so the exemption costs nothing today. If one is ever observed, the fix
+ * is a decision about what a null instant means, recorded here, rather than a
+ * blanket loosening. The `nullRejecting` inventory in
+ * `test/interface/schema.test.ts` is what holds that line.
  */
 export declare const baseFirestoreShape: {
     /**
      * See {@link BaseFirestore.id}.
      */
-    id: z.ZodOptional<z.ZodString>;
+    id: z.ZodOptional<z.ZodNullable<z.ZodString>>;
     /**
      * See {@link BaseFirestore.backup}.
      */
-    backup: z.ZodOptional<z.ZodBoolean>;
+    backup: z.ZodOptional<z.ZodNullable<z.ZodBoolean>>;
     /**
      * See {@link BaseFirestore.created}.
      */
@@ -100,8 +114,8 @@ export declare const baseFirestoreShape: {
  * schema instead.
  */
 export declare const BaseFirestoreSchema: z.ZodObject<{
-    id: z.ZodOptional<z.ZodString>;
-    backup: z.ZodOptional<z.ZodBoolean>;
+    id: z.ZodOptional<z.ZodNullable<z.ZodString>>;
+    backup: z.ZodOptional<z.ZodNullable<z.ZodBoolean>>;
     created: z.ZodOptional<z.ZodType<string | number | import("./schema.js").TimestampLike | Date, unknown, z.core.$ZodTypeInternals<string | number | import("./schema.js").TimestampLike | Date, unknown>>>;
     updated: z.ZodOptional<z.ZodType<string | number | import("./schema.js").TimestampLike | Date, unknown, z.core.$ZodTypeInternals<string | number | import("./schema.js").TimestampLike | Date, unknown>>>;
     expiry: z.ZodOptional<z.ZodType<string | number | import("./schema.js").TimestampLike | Date, unknown, z.core.$ZodTypeInternals<string | number | import("./schema.js").TimestampLike | Date, unknown>>>;

lib/interface/base_db.js

@@ -21,16 +21,30 @@ import { auditTimestamp, documentId } from './schema.js';
  * at the boundary instead, where it can actually be checked. See
  * {@link auditTimestamp} for the shapes accepted, what is rejected, and how to
  * widen a schema for a write payload.
+ *
+ * {@link BaseFirestore.id} and {@link BaseFirestore.backup} are `.nullish()`,
+ * because a stored document writes an unset optional field as an explicit
+ * `null` rather than omitting it, and a field that accepted only `undefined`
+ * would reject documents that are otherwise entirely well formed.
+ *
+ * The three timestamps are deliberately **not** `.nullish()` and keep rejecting
+ * `null`. An explicitly null timestamp is not a time: read as one it becomes
+ * epoch zero, which sorts first and — for {@link BaseFirestore.expiry}, a TTL —
+ * expires the document immediately. No stored null was observed in any of the
+ * three, so the exemption costs nothing today. If one is ever observed, the fix
+ * is a decision about what a null instant means, recorded here, rather than a
+ * blanket loosening. The `nullRejecting` inventory in
+ * `test/interface/schema.test.ts` is what holds that line.
  */
 export const baseFirestoreShape = {
     /**
      * See {@link BaseFirestore.id}.
      */
-    id: documentId().optional(),
+    id: documentId().nullish(),
     /**
      * See {@link BaseFirestore.backup}.
      */
-    backup: z.boolean().optional(),
+    backup: z.boolean().nullish(),
     /**
      * See {@link BaseFirestore.created}.
      */