fix(driver-memory): render the pipeline dump's RegExp operand instead of dropping it to {} (#7853) (#7869)

`AnalyticsResult.sql` from `generateSqlFromPipeline()` dumped each mingo stage
with a bare `JSON.stringify`. A `RegExp` has no own enumerable properties, so
every pattern operand rendered as `{}` — `{"name":{"$regex":{}}}` — and the one
field an author debugging an in-memory chart is looking for was the one the
dump dropped.

Measured across the twelve operators this face declares: exactly three carry a
pattern (`$contains`, `$icontains`, `$notContains` inside `$not`) and all three
were affected. They now render the pattern's own literal syntax, `/source/flags`
— chosen over the mongo-shaped `{$regex, $options}` because the `RegExp` sits AT
the `$regex` key, so a value replacer producing the pair renders the doubled
`{"$regex":{"$regex":"et","$options":""}}`, a shape no mongo query has, and
flattening it would make the dump disagree with the pipeline it dumps.

No executed behaviour changes: the dump is explicitly not SQL, and `query()`'s
rows, `generateSql()`'s SQL, and the other nine operators' dumps are unchanged.

The existing dump check asserted structure (`$not` wraps a `$regex`) and passed
throughout — it is strengthened here to assert the pattern TEXT for each
affected operator, since `{"$regex":{}}` is non-empty and satisfies any
presence-only assertion.


Claude-Session: https://claude.ai/code/session_01UzMdsWW2aZwGCEhfDLo1sT

Co-authored-by: Claude <noreply@anthropic.com>

Author: huangyiirene

.changeset/analytics-pipeline-dump-regexp.md

@@ -0,0 +1,44 @@
+---
+"@objectstack/driver-memory": patch
+---
+
+fix(driver-memory): the analytics pipeline dump shows its RegExp pattern instead of `{}` (#7853)
+
+`MemoryAnalyticsService.query()` returns `AnalyticsResult.sql` — a stage-by-stage
+dump of the mingo pipeline it actually executed, and the only thing an author
+debugging an in-memory chart is given. It dumped each stage with a bare
+`JSON.stringify`, and a `RegExp` has **no own enumerable properties**, so every
+pattern operand rendered as `{}`:
+
+```
+-- MongoDB Aggregation Pipeline on table: deal
+/* Stage 1: $match */ {"name":{"$regex":{}}}
+```
+
+The `$match` stage was reported as constraining `name` by an empty object. The
+one field the reader came for is the one the dump dropped. Measured across the
+twelve operators this face declares, exactly three carry a pattern and all three
+were affected: `$contains`, `$icontains`, and `$notContains` (nested inside
+`$not`). The same three now render:
+
+```
+/* Stage 1: $match */ {"name":{"$regex":"/et/"}}
+/* Stage 1: $match */ {"name":{"$regex":"/[Bb][Ee][Tt]/"}}
+/* Stage 1: $match */ {"name":{"$not":{"$regex":"/et/"}}}
+```
+
+**No executed behaviour changes.** This dump is explicitly not SQL — its own
+header says `-- MongoDB Aggregation Pipeline on table: …` — so it is a
+transparency surface, not a runnable one, and the rows `query()` returns and the
+SQL `generateSql()` emits are byte-identical before and after. The other nine
+operators' dumps are unchanged.
+
+**Why the pattern's own literal syntax** (`/source/flags`) and not the
+mongo-shaped `{"$regex":"…","$options":"…"}` the rest of the dump speaks: the
+`RegExp` sits AT the `$regex` key, so a value replacer producing the mongo pair
+renders the doubled `{"$regex":{"$regex":"et","$options":""}}` — a shape no mongo
+query has. Flattening it to the real spelling would mean rewriting the parent
+object, making the dump disagree with the pipeline it claims to dump, since what
+mingo executes at that key is a JS `RegExp`. The literal form is also the only
+one-token rendering that keeps the FLAGS, which matter here: `$icontains`' fold
+lives in the pattern source (#6520) while `$contains` is case-exact (#7723).

packages/drivers/driver-memory/src/memory-analytics.ts

@@ -480,6 +480,62 @@ function sizeDistinctSet(values: readonly unknown[]): number {
   return new Set(values.filter((v) => v !== null && v !== undefined)).size;
 }
 
+/**
+ * [#7853] A `JSON.stringify` replacer that renders a `RegExp` operand instead of
+ * dropping it — the one value type the pipeline dump carries that
+ * `JSON.stringify` erases.
+ *
+ * ## What was lost
+ *
+ * A `RegExp` has no own enumerable properties, so `JSON.stringify` renders it as
+ * `{}`. Three operators put one into the `$match` stage — `contains` and
+ * `icontains` as `{$regex: …}`, `notContains` as `{$not: {$regex: …}}` (measured:
+ * those three and no others, out of the twelve this face declares) — so
+ * `{name: {$contains: 'Industries'}}` dumped as
+ *
+ * ```
+ * /* Stage 1: $match *\/ {"name":{"$regex":{}}}
+ * ```
+ *
+ * The one field an author debugging a chart is looking for is the one the dump
+ * dropped. This is lost information on a transparency surface, not the #5333
+ * class: the dump is explicitly NOT SQL (its header says so) and `{}` reads as
+ * "something is missing here" rather than as a working predicate, which is why
+ * it is graded below #7117 rather than beside it.
+ *
+ * ## Why the pattern's own literal syntax and not `{"$regex":"…","$options":"…"}`
+ *
+ * The mongo-shaped form is what the rest of the dump speaks, and it was the
+ * first candidate. It cannot be reached from a value replacer, and the reason is
+ * structural rather than cosmetic: the `RegExp` sits AT the `$regex` key, so
+ * replacing it with `{$regex, $options}` renders the doubled
+ * `{"name":{"$regex":{"$regex":"Industries","$options":""}}}` — a shape no mongo
+ * query has. Flattening it into the real mongo spelling means rewriting the
+ * PARENT object, which would make the dump disagree with the pipeline it claims
+ * to be dumping: what mingo executes is a JS `RegExp` object at that key, not a
+ * source/options pair. Trading a degenerate rendering for a plausible-but-wrong
+ * one is the #5333 direction, and this card is explicitly not that.
+ *
+ * So the value is rendered as the JS literal it is, `/source/flags`, which is
+ * also the only one-token form that keeps the FLAGS. Flags are not decoration
+ * here: `$icontains`' fold lives in the pattern SOURCE (#6520) while `$contains`
+ * is case-EXACT (#7723, #4706 Q2 = A), so a rendering that dropped `i` would
+ * recreate a smaller copy of this same information loss on the one axis those
+ * two operators differ.
+ *
+ * ## What it deliberately does not touch
+ *
+ * Every other value on this path already renders faithfully, measured rather
+ * than assumed: a `Date` comparand is canonicalized to an ISO string by
+ * {@link MemoryAnalyticsService.comparandsFor} before it reaches here, and
+ * `toJSON` runs BEFORE a replacer in any case, so dates are unchanged. A
+ * `BigInt` comparand does throw — but out of mingo's own `Query.compile` during
+ * EXECUTION, before this dump is ever built, so no replacer here reaches it.
+ */
+function pipelineDumpReplacer(_key: string, value: unknown): unknown {
+  return value instanceof RegExp ? `/${value.source}/${value.flags}` : value;
+}
+
 /**
  * Configuration for MemoryAnalyticsService
  */
@@ -1280,9 +1336,13 @@ export class MemoryAnalyticsService implements IAnalyticsService {
   private generateSqlFromPipeline(table: string, pipeline: Record<string, any>[]): string {
     // Simplified SQL generation for debugging
     // This is a basic representation of the aggregation pipeline
+    //
+    // [#7853] The replacer is what keeps a `RegExp` operand from rendering as
+    // `{}` — see {@link pipelineDumpReplacer} for why the pattern's own literal
+    // syntax and not the mongo-shaped `{$regex, $options}`.
     const stages = pipeline.map((stage, idx) => {
       const op = Object.keys(stage)[0];
-      return `/* Stage ${idx + 1}: ${op} */ ${JSON.stringify(stage[op])}`;
+      return `/* Stage ${idx + 1}: ${op} */ ${JSON.stringify(stage[op], pipelineDumpReplacer)}`;
     }).join('\n');
 
     return `-- MongoDB Aggregation Pipeline on table: ${table}\n${stages}`;

packages/drivers/driver-memory/src/memory-driver-filter-logic-conformance.test.ts

@@ -696,17 +696,63 @@ describe('[#5374] operator semantics — the analytics face against the live que
    * absent and the rows happen to line up", and the emitted `$match` is the
    * artifact the issue actually diagnosed.
    */
-  it('the emitted $match wraps the negation around a pattern instead of a bare scalar', async () => {
+  const matchStageOf = async (where: FilterCondition): Promise<string> => {
     const { sql } = await service.query({
       cube: COMPARAND_TABLE,
       measures: [`${COMPARAND_TABLE}.count`],
-      where: { name: { $notContains: 'et' } } as FilterCondition,
+      where,
     });
-    const matchStage = /\/\* Stage 1: \$match \*\/ (.*)/.exec(sql ?? '')?.[1] ?? '';
-    // `JSON.stringify` renders a RegExp as `{}`, so assert on the STRUCTURE the
-    // pipeline carries rather than on that rendering.
+    return /\/\* Stage 1: \$match \*\/ (.*)/.exec(sql ?? '')?.[1] ?? '';
+  };
+
+  it('the emitted $match wraps the negation around a pattern instead of a bare scalar', async () => {
+    const matchStage = await matchStageOf({ name: { $notContains: 'et' } } as FilterCondition);
     expect(matchStage).not.toBe('{"name":{"$not":"et"}}');
     expect(matchStage).toContain('"$not"');
     expect(matchStage).toContain('"$regex"');
   });
+
+  /**
+   * [#7853] The dump's CONTENT for the pattern operators, not merely its shape.
+   *
+   * The assertion above pins the structure `$not` wraps, and it passed for as
+   * long as the pattern itself was missing: `JSON.stringify` renders a `RegExp`
+   * as `{}` (no own enumerable properties), so `{name: {$contains: 'et'}}`
+   * dumped as `{"name":{"$regex":{}}}` — non-empty, structurally correct, and
+   * silent about the one field an author reading this dump came for. That is
+   * why the cases below assert the pattern TEXT: a `toBeDefined()` or a
+   * `toContain('$regex')` cannot tell the two states apart.
+   *
+   * The rendering is the pattern's own literal syntax, `/source/flags`. No
+   * operator this face declares carries a RegExp FLAG today — measured:
+   * `$icontains`' fold is compiled into the pattern source (#6520) and
+   * `$contains` is case-exact (#7723) — so the flags segment renders empty
+   * here; it is in the form because a fold that ever moved into a flag would
+   * otherwise vanish exactly the way the source did.
+   */
+  const PATTERN_DUMP_CASES: Array<[string, FilterCondition, string]> = [
+    // The pattern, plainly, for the operator the issue measured.
+    ['$contains', { name: { $contains: 'et' } } as FilterCondition, '{"name":{"$regex":"/et/"}}'],
+    // The ASCII fold is IN the source, so the dump shows the folded character
+    // classes rather than an `i` — this is what #6520 compiled, made visible.
+    ['$icontains', { name: { $icontains: 'BET' } } as FilterCondition, '{"name":{"$regex":"/[Bb][Ee][Tt]/"}}'],
+    // The negation still wraps a pattern, and now the pattern is legible.
+    ['$notContains', { name: { $notContains: 'et' } } as FilterCondition, '{"name":{"$not":{"$regex":"/et/"}}}'],
+    // A comparand carrying a regex metacharacter shows its ESCAPE. `a.p` is a
+    // literal here, not "any character between a and p" (#5567's direction), and
+    // the dump is the only place an author can see which of the two ran.
+    ['$contains with a metacharacter', { name: { $contains: 'a.p' } } as FilterCondition, '{"name":{"$regex":"/a\\\\.p/"}}'],
+  ];
+
+  for (const [label, where, expected] of PATTERN_DUMP_CASES) {
+    it(`the pipeline dump renders ${label}'s pattern instead of dropping it to {}`, async () => {
+      const matchStage = await matchStageOf(where);
+      expect(
+        matchStage,
+        `${label}: the dump lost its RegExp operand — this is the \`{"$regex":{}}\` state, ` +
+          'which is non-empty and passes every assertion that only checks for presence',
+      ).not.toContain('{}');
+      expect(matchStage).toBe(expected);
+    });
+  }
 });