fix(metadata-protocol): derive the reference graph from the type schemas (#9190) (#9324)

`findReferencesToMeta` — the admin "Used by" panel behind
`GET /api/v1/meta/:type/:name/references` — was driven by a hand-curated
table of 7 target types and 40 dotted paths. Measured against the schemas
it claimed to describe, 34 of the 40 named properties no metadata type
declares, leaving 5 of its 7 target keys answering `{ references: [] }`
unconditionally while appearing covered. The panel's empty state reads
"Nothing in the metadata graph points at this item. Safe to delete."

Coverage is now derived at boot from `DEFAULT_METADATA_TYPE_REGISTRY` and
each type's schema, in the shape #7894 used for the URL-spelling map, so a
newly declared type arrives covered. The unit of derivation is a PROPERTY,
not a path: recursive containers (app navigation) make an exhaustive path
list unbounded, and the walk reports where the name was actually found.

No wire change: response shape, status codes and error envelope untouched.


Claude-Session: https://claude.ai/code/session_01NTKPDRoynY8i3HmdSFUxFj

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-zhuang

.changeset/reference-paths-derivation.md

@@ -0,0 +1,39 @@
+---
+'@objectstack/metadata-protocol': minor
+---
+
+Derive the metadata reference graph from the type schemas instead of curating it by hand
+
+`GET /api/v1/meta/:type/:name/references` — the admin "Used by" panel, rendered
+immediately before a rename or a delete — was driven by a hand-written table of
+seven target types and forty dotted paths. Measured against the schemas it was
+supposed to describe, **34 of those 40 paths named properties no metadata type
+declares**: `app.navItems[]` / `app.tabs[]` (the schema declares `navigation`
+and `areas`), `agent.tools[]` (removed in `@objectstack/spec` 17),
+`permission.objects[].name` (a name-keyed record, not an array),
+`object.fields{}.referenceTo` (the field property is `reference`),
+`dashboard.widgets[].view`, `page.viewName`, and every path the table listed for
+`flow`. Five of its seven target types therefore answered `{ references: [] }`
+unconditionally, on every deployment, while appearing to be covered — and an
+empty panel reads as "nothing depends on this, safe to delete".
+
+Coverage is now derived at boot from `DEFAULT_METADATA_TYPE_REGISTRY` and each
+type's Zod schema, so a newly declared metadata type arrives covered instead of
+waiting for someone to remember it. Seventeen target types now resolve real
+reference sites, including `permission`-to-object grants (through the record
+key, which the old path grammar could not express), `translation`, `dataset`,
+`action`, `report`, `doc` and `datasource`, plus flow-node references such as
+`subflow`. References nested inside recursive containers — a view named from a
+third-level app navigation group — are found at any depth, which no finite path
+list could do.
+
+No wire change: the response shape, status codes and error envelope are
+untouched. The `path` and `kind` values now describe where the reference was
+actually found rather than which table row matched.
+
+Two gaps are deliberately declared rather than papered over: `external_catalog`
+resolves no schema, so its references are not computable and it is named in the
+derivation's `unwalkableSourceTypes` (pinned by a test, so the set cannot grow
+silently), and reference properties whose name does not spell their target —
+`FieldSchema.reference` is the one carried — need a producer-side annotation to
+become derivable.

packages/metadata-protocol/src/protocol.read-seam-empty-accumulator.test.ts

@@ -210,12 +210,23 @@ describe('[#8896] searchAll — an object that could not be READ is not an objec
 
 describe('[#8896] findReferencesToMeta — a source type that could not be READ is not a source type with no references', () => {
     /**
-     * `view` has three matchers (`dashboard`, `app`, `page`), so a single
-     * failing source type leaves the other two answering — which is exactly the
-     * pre-fix trap: a SHORT list that looks complete. `page` carries a real
-     * reference to `my_view`, so the healthy half is observable.
+     * `view` is reachable from four source types (`app`, `object`, `page`,
+     * `view`), so a single failing source type leaves the others answering —
+     * which is exactly the pre-fix trap: a SHORT list that looks complete.
+     * `page` carries a real reference to `my_view`, so the healthy half is
+     * observable.
+     *
+     * [#9190] The fixture used to spell that reference `page.viewName`, which
+     * `PageSchema` does not declare — it agreed with the hand-curated path
+     * table, and the table was wrong. The real site is `view`, reached through
+     * a `dataSource`, and the derived walk finds it wherever the document puts
+     * it rather than at one memorised path.
      */
-    const pageReferencingTheView = { name: 'home_page', label: 'Home', viewName: 'my_view' };
+    const pageReferencingTheView = {
+        name: 'home_page',
+        label: 'Home',
+        slots: { header: { dataSource: { view: 'my_view' } } },
+    };
 
     function engineWhereTypeFails(failingType: string | null, error?: unknown) {
         const typeReads: string[] = [];
@@ -239,18 +250,24 @@ describe('[#8896] findReferencesToMeta — a source type that could not be READ
         const result = await protocol.findReferencesToMeta({ type: 'view', name: 'my_view' });
 
         expect(result.references).toEqual([
-            { type: 'page', name: 'home_page', label: 'Home', path: 'viewName', kind: 'page' },
+            {
+                type: 'page',
+                name: 'home_page',
+                label: 'Home',
+                path: 'slots.header.dataSource.view',
+                kind: 'page view',
+            },
         ]);
-        // All three source types were really consulted — this is what makes
-        // "one of them failed" a meaningful condition below.
-        expect(typeReads).toContain('dashboard');
+        // Every source type that can name a view was really consulted — this is
+        // what makes "one of them failed" a meaningful condition below.
         expect(typeReads).toContain('app');
+        expect(typeReads).toContain('object');
         expect(typeReads).toContain('page');
     });
 
     it('a source type whose read FAILS fails the whole scan, envelope intact', async () => {
         const injected = connectionDropped();
-        const { engine, typeReads } = engineWhereTypeFails('dashboard', injected);
+        const { engine, typeReads } = engineWhereTypeFails('app', injected);
         const protocol = new ObjectStackProtocolImplementation(engine as never);
 
         const caught = await rejection(
@@ -267,7 +284,7 @@ describe('[#8896] findReferencesToMeta — a source type that could not be READ
         expect(ErrorCode.safeParse(caught.code).success).toBe(true);
         // The driver's own error is not lost — it rides as `cause`.
         expect(caught.cause).toBe(injected);
-        expect(typeReads).toContain('dashboard');
+        expect(typeReads).toContain('app');
         // Pre-fix this resolved `{ references: [ …the page hit… ] }` — one real
         // reference presented as the complete dependency list, which an admin
         // reads as "safe to delete".
@@ -286,7 +303,7 @@ describe('[#8896] findReferencesToMeta — a source type that could not be READ
         expect(result.references).toEqual([]);
     });
 
-    it('a target type absent from REFERENCE_PATHS still returns an empty list without reading anything', async () => {
+    it('a target type with no derived reference site still returns an empty list without reading anything', async () => {
         const { engine, typeReads } = engineWhereTypeFails(null);
         const protocol = new ObjectStackProtocolImplementation(engine as never);
 
@@ -300,15 +317,21 @@ describe('[#8896] findReferencesToMeta — a source type that could not be READ
         // The benign discrimination lives in `getMetaItems`, one layer down —
         // this seam inherits it rather than repeating it, and this pin is what
         // proves the inheritance still holds through the removed `catch`.
-        const { engine, typeReads } = engineWhereTypeFails('dashboard', tableNotProvisioned('sys_metadata'));
+        const { engine, typeReads } = engineWhereTypeFails('app', tableNotProvisioned('sys_metadata'));
         const protocol = new ObjectStackProtocolImplementation(engine as never);
 
         const result = await protocol.findReferencesToMeta({ type: 'view', name: 'my_view' });
 
         expect(result.references).toEqual([
-            { type: 'page', name: 'home_page', label: 'Home', path: 'viewName', kind: 'page' },
+            {
+                type: 'page',
+                name: 'home_page',
+                label: 'Home',
+                path: 'slots.header.dataSource.view',
+                kind: 'page view',
+            },
         ]);
         // Proof the benign branch was actually EXERCISED.
-        expect(typeReads).toContain('dashboard');
+        expect(typeReads).toContain('app');
     });
 });

packages/metadata-protocol/src/protocol.read-verb-canonical-fold.test.ts

@@ -122,6 +122,12 @@ function makeStubEngine() {
             getPackage: () => undefined,
             registerItem: () => {},
             registerObject: () => {},
+            // [#9190] `getMetaItems({ type: 'app' })` decorates each app with
+            // its contributed nav groups. The double gained this the day a
+            // reference scan first read `app` — identity is the right stub,
+            // because the fold under test is about the TYPE KEY a read used,
+            // not about nav contribution.
+            applyNavContributions: (app: unknown) => app,
         },
     };
     return { engine, tables, reads, items };
@@ -434,37 +440,49 @@ describe('#9157 — findReferencesToMeta', () => {
     });
 
     it('CONTROL: a manifest-PRESENT plural still resolves its real dependents', async () => {
+        // [#9190] The fixture moved from `dashboard.widgets[].view` to
+        // `app.navigation[].viewName`, and the move is the point rather than a
+        // detail: `DashboardSchema` declares no `view` property anywhere, so
+        // the old fixture agreed with the old hand-curated path table and BOTH
+        // described a document the platform cannot store. `AppSchema.navigation`
+        // is real, and the site the walk uses for it is derived from that
+        // schema.
         const { engine, items } = makeStubEngine();
-        items.dashboard = [{ name: 'sales_dash', label: 'Sales', widgets: [{ id: 'w1', view: 'all_leads' }] }];
+        items.app = [{ name: 'sales_app', label: 'Sales', navigation: [{ viewName: 'all_leads' }] }];
         const p = new ObjectStackProtocolImplementation(engine);
 
         const res = await p.findReferencesToMeta({ type: PRESENT_PLURAL, name: 'all_leads' });
 
         expect(res.references).toEqual([
-            { type: 'dashboard', name: 'sales_dash', label: 'Sales', path: 'widgets[].view', kind: 'dashboard widget' },
+            { type: 'app', name: 'sales_app', label: 'Sales', path: 'navigation[].viewName', kind: 'app viewName' },
         ]);
     });
 
-    it('the manifest-ABSENT class is NOT closed here, and that is stated rather than implied', async () => {
-        // ⚠️ Honest scope pin. The card's `translations` example claims this verb
-        // answers `{ references: [] }` for a manifest-absent type — true, and the
-        // fold does not change it: every `REFERENCE_PATHS` key (`object`, `view`,
-        // `tool`, `skill`, `flow`, `dashboard`, `page`) is manifest-PRESENT, so
-        // `translation` has no registry entry either. This method's own doc calls
-        // an unregistered target a legitimate no-hit rather than an error.
+    it('[#9190] the manifest-ABSENT residue #9157 pinned here is CLOSED, and both spellings reach the same real hits', async () => {
+        // ⚠️ This pin has MOVED, deliberately. #9157 asserted that
+        // `translation` answers `{ references: [] }` whichever spelling you use
+        // — true then, because the hand-curated table had no `translation` key
+        // and this method's doc called that a legitimate no-hit. #9190 closed
+        // it the way the ruling required: by DERIVATION, not by adding a key.
+        // `DocSchema.translations` is a real, schema-declared reference site, so
+        // the walk finds it without anyone having written `translation` down.
         //
-        // Closing it is a `REFERENCE_PATHS` COVERAGE question, not a spelling
-        // one, and it is a different card. Asserted so a reader cannot over-read
-        // this PR's claim, and so the day `translation` gains a matcher this
-        // test goes red and asks to be re-read.
-        const { engine } = makeStubEngine();
+        // What #9157 owns is UNCHANGED and is what this test still proves: the
+        // two spellings fold to one answer. What changed is that the answer is
+        // no longer vacuously empty, so the test can prove the fold on a
+        // non-trivial result — which is a stronger assertion than the empty one
+        // it replaces.
+        const { engine, items } = makeStubEngine();
+        items.doc = [{ name: 'intro', label: 'Intro', translations: { greeting: { title: 'Hallo' } } }];
         const p = new ObjectStackProtocolImplementation(engine);
 
         const viaPlural = await p.findReferencesToMeta({ type: OVERLAY_ABSENT_PLURAL, name: 'greeting' });
         const viaCanonical = await p.findReferencesToMeta({ type: OVERLAY_ABSENT_TYPE, name: 'greeting' });
 
-        expect(viaPlural.references).toEqual([]);
-        expect(viaCanonical.references).toEqual([]);
+        expect(viaCanonical.references).toEqual([
+            { type: 'doc', name: 'intro', label: 'Intro', path: 'translations{key}', kind: 'doc translations' },
+        ]);
+        expect(viaPlural.references).toEqual(viaCanonical.references);
     });
 
     it('CONTROL: a spelling that reaches for no declared type is served, not refused', async () => {