fix(cli): reserve stdout for the --json payload across the bootSchemaStack family (#6217) (#6524)

`--json` has exactly one audience — a program — and every subcommand that
boots a kernel was handing that program a stream it could not parse.
`ObjectLogger` routes `debug`/`info`/`warn` to stdout and only `error`/`fatal`
to stderr, so `os migrate recorded-by --json` produced "~60 INFO lines +
payload + 2 shutdown lines" on stdout while stderr stayed empty. The only
recourse was a "find the last lone `{` and its matching `}`" extractor — #4873
was forced to write one to assert its own payload — and that heuristic
silently picks the wrong text as soon as a log line looks like JSON.

This takes route 1 from the issue (redirect to stderr), not route 2 (drop the
kernel to `logLevel: 'silent'`): route 2 throws the operator's diagnostics
away, warnings included, and could not quiet the one line that never goes
through the logger at all (`console.log` in `loadArtifactBundle`:
`[StandaloneStack] no compiled artifact …`). Route 3 (make the logger default
to stderr in `packages/core`) would change `os serve` / `os dev` output for
every existing user and is a maintainer call — untouched here.

The fix lands on the family's shared boot seam: `bootSchemaStack` gains a
REQUIRED `jsonOutput` option. Before the boot can print its first byte it
takes over `process.stdout.write` and forwards everything the kernel and its
plugins write to stderr — nothing is discarded — while the payload goes out
through `writeStdoutDirect` on the real stdout. `shutdown()` gives stdout back
only after the kernel is fully down, so the two shutdown lines cannot land
under the payload either. A failed boot deliberately keeps the reservation:
the command's next act on that path is to emit its error payload, and a
half-started kernel can still log. The option is required so a family member
added later has to decide at compile time instead of inheriting the bug.

All nine members covered: `os migrate plan` / `apply` / `resume` /
`recorded-by` / `summary-nulls` / `value-shapes` / `files-to-references`,
`os migrate meta --stored`, and `os meta resync`. Human-mode output is
unchanged (verified: stdout identical, stderr still empty).

Tests:
- `packages/cli/test/json-stdout-purity.e2e.test.ts` — the shared family
  expectation. Discovers the members from source (calls `bootSchemaStack` AND
  declares a `--json` flag), reconciles that set against the driven list, then
  runs each as a real child process asserting a bare `JSON.parse(stdout)`, no
  logger record anywhere on stdout, and the boot diagnostics still present on
  stderr (so a regression toward silencing goes red too). A new member that is
  not driven goes red.
- `packages/cli/test/migrate-exit-code.e2e.test.ts` — the `jsonPayload()`
  heuristic extractor #4873 wrote under duress is deleted; it is now a bare
  `JSON.parse(stdout)`.
- `packages/cli/src/utils/json-stdout.test.ts` — the mechanism at unit level.

Reverse verification (predicted red, and red): with the reservation disabled,
27 of the family e2e's 28 cases fail — the one that stays green is the
member-inventory case, which asserts a set rather than behaviour — and the
exit-code pin fails with `SyntaxError: Unexpected token 'S', "[Standalone"...`,
exactly the error recorded in the issue.

Fixes #6217


Claude-Session: https://claude.ai/code/session_017uFVNMmTxLpmfQYiuKM1Yx

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-project-manager

.changeset/lucky-pears-arrive.md

@@ -0,0 +1,25 @@
+---
+'@objectstack/cli': patch
+---
+
+fix(cli): `--json` now owns stdout — kernel boot logs move to stderr (#6217)
+
+Every `os migrate` / `os meta` subcommand that boots a kernel wrote its
+machine-readable payload into a stream it shared with ~60 INFO lines. The
+kernel logger routes `debug`/`info`/`warn` to stdout and only `error`/`fatal`
+to stderr, so `os migrate recorded-by --json | jq .` failed with `parse error:
+Invalid numeric literal` while stderr sat completely empty — a `--json` flag
+whose only audience is a program, handing that program something it cannot
+parse.
+
+With this change, a `--json` run reserves stdout for its payload: everything
+the kernel and its plugins write goes to **stderr** instead, including the
+`[StandaloneStack] no compiled artifact …` notice that never went through the
+logger at all. `JSON.parse(<entire stdout>)` now succeeds with no heuristic
+extraction, and no diagnostic is lost — every line an operator used to see is
+still printed, on the stream diagnostics belong on.
+
+Covers the whole family that shares the boot seam: `os migrate plan` / `apply`
+/ `resume` / `recorded-by` / `summary-nulls` / `value-shapes` /
+`files-to-references`, `os migrate meta --stored`, and `os meta resync`.
+Human-mode runs are unchanged.

packages/cli/src/commands/meta/resync.ts

@@ -83,7 +83,7 @@ export default class MetaResync extends Command {
 
     let stack;
     try {
-      stack = await bootSchemaStack({ databaseUrl: flags['database-url'] });
+      stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'] });
     } catch (error: any) {
       if (flags.json) await emitJson({ error: error.message }, 0, { compact: true });
       else printError(error.message || String(error));

packages/cli/src/commands/migrate/apply.ts

@@ -126,7 +126,7 @@ export default class MigrateApply extends Command {
     try {
       // `deferSchemaDdl` is what makes the prompt below meaningful: without it
       // the boot has already created tables and added columns by this point.
-      stack = await bootSchemaStack({ databaseUrl: flags['database-url'], deferSchemaDdl: true });
+      stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], deferSchemaDdl: true });
     } catch (error: any) {
       if (flags.json) { await emitJson({ error: error.message }, 0, { compact: true }); this.exit(1); }
       printError(error.message || String(error));

packages/cli/src/commands/migrate/files-to-references.ts

@@ -161,6 +161,7 @@ export default class MigrateFilesToReferences extends Command {
     let stack;
     try {
       stack = await bootSchemaStack({
+        jsonOutput: flags.json,
         databaseUrl: flags['database-url'],
         extraPlugins: await buildDataMigrationPlugins({ storage: true }),
       });

packages/cli/src/commands/migrate/meta.stored-flow-resolution.integration.test.ts

@@ -94,6 +94,7 @@ describe('os migrate meta --stored — the protocol resolves the engine itself (
 
   it('rewrites a pre-17 flow row with NO canonicalizeFlow passed by the command', async () => {
     const stack = await bootSchemaStack({
+      jsonOutput: false,
       databaseUrl: `file:${dbFile}`,
       projectRoot: dir,
       extraPlugins: await buildDataMigrationPlugins({ automation: true }),
@@ -163,6 +164,7 @@ describe('os migrate meta --stored — the protocol resolves the engine itself (
     // verbatim and leave the row `pending` forever; `saveMetaItem` now
     // canonicalizes flow bodies before its schema gate.
     const stack = await bootSchemaStack({
+      jsonOutput: false,
       databaseUrl: `file:${dbFile}`,
       projectRoot: dir,
       extraPlugins: await buildDataMigrationPlugins({ automation: true }),
@@ -223,6 +225,7 @@ describe('os migrate meta --stored — the protocol resolves the engine itself (
     // The honest negative: the coverage comes from the engine being present,
     // not from the report defaulting to optimistic.
     const stack = await bootSchemaStack({
+      jsonOutput: false,
       databaseUrl: `file:${dbFile}`,
       projectRoot: dir,
       extraPlugins: await buildDataMigrationPlugins(),

packages/cli/src/commands/migrate/meta.ts

@@ -518,6 +518,7 @@ export default class MigrateMeta extends Command {
       // nothing. No storage adapter: unlike the file migration, nothing here
       // reads bytes.
       stack = await bootSchemaStack({
+        jsonOutput: flags.json,
         ...(flags['database-url'] ? { databaseUrl: flags['database-url'] } : {}),
         extraPlugins: await buildDataMigrationPlugins({ automation: true }),
       });

packages/cli/src/commands/migrate/plan.ts

@@ -77,7 +77,7 @@ export default class MigratePlan extends Command {
 
     let stack;
     try {
-      stack = await bootSchemaStack({ databaseUrl: flags['database-url'], deferSchemaDdl: true });
+      stack = await bootSchemaStack({ jsonOutput: flags.json, databaseUrl: flags['database-url'], deferSchemaDdl: true });
     } catch (error: any) {
       if (flags.json) { await emitJson({ error: error.message }, 0, { compact: true }); this.exit(1); }
       printError(error.message || String(error));

packages/cli/src/commands/migrate/recorded-by.ts

@@ -95,6 +95,7 @@ export default class MigrateRecordedBy extends Command {
     let stack;
     try {
       stack = await bootSchemaStack({
+        jsonOutput: flags.json,
         databaseUrl: flags['database-url'],
         extraPlugins: await buildDataMigrationPlugins(),
       });

packages/cli/src/commands/migrate/resume.ts

@@ -105,6 +105,7 @@ export default class MigrateResume extends Command {
     let stack;
     try {
       stack = await bootSchemaStack({
+        jsonOutput: flags.json,
         databaseUrl: flags['database-url'],
         extraPlugins: await buildDataMigrationPlugins(),
       });

packages/cli/src/commands/migrate/summary-nulls.ts

@@ -167,6 +167,7 @@ export default class MigrateSummaryNulls extends Command {
     let stack;
     try {
       stack = await bootSchemaStack({
+        jsonOutput: flags.json,
         databaseUrl: flags['database-url'],
         extraPlugins: await buildDataMigrationPlugins(),
       });