mdcode: generate BigQuery property-graph DDL from the Semantic Model IR - #24
Merged
libei merged 2 commits intoAug 2, 2026
Merged
Conversation
Adds the first destination for the Semantic Model IR: a generator that emits
a single `CREATE OR REPLACE PROPERTY GRAPH` statement over the entities'
existing base tables, with model-level metrics rendered as inline MEASURE(...)
properties.
- src/libts/semantic/bigquery.ts — maps the IR to graph DDL:
- NODE TABLES from entities (KEY from the primary key; fields as properties),
EDGE TABLES from relationships (association table for M:N, source base
table for a direct FK), with explicit element KEYs.
- Measures: each model-level metric is placed on the single entity its
aggregate references (a cross-table aggregate cannot be one MEASURE, so it
is skipped + warned). A BigQuery measure may only aggregate an EXPOSED
PROPERTY, so each metric is lowered into a derived operand property plus a
MEASURE over it; identical operands are exposed once and reused.
- Only SUM/AVG/COUNT/MIN/MAX are emitted (the complete set BigQuery accepts in
a measure); anything else is skipped + warned rather than emitted as DDL
BigQuery rejects. COUNT(*) lowers to COUNT over the key property.
- AI-first metadata (label, temporal-dimension role, description, and the
aiContext instructions/synonyms/examples) is folded into OPTIONS(description)
— the only metadata sink the graph DDL exposes — on the graph, element
tables, properties, and measures.
- dataSource is consumed as the loader's normalized, fully-qualified string;
an under-qualified ref is completed from options and warned if still short.
- Structural drops (keyless node, dangling edge, unplaceable metric) are
reported in `warnings`, never silently emitted as invalid DDL.
- tests/libts/semantic/bigquery.test.ts — behavior spec over hand-built IR with
inline goldens; the two chain/M:N goldens are the exact DDL a live BigQuery
instance accepted (GRAPH_EXPAND + AGG and a GQL MATCH), plus an invariant that
no emitted MEASURE ever wraps a shape BigQuery rejects.
- tests/libts/semantic/bigquery.e2e.test.ts + *.bigquery.golden.sql — file -> IR
-> DDL golden suite over a corpus of real-shaped fixtures; regenerate with
UPDATE_GOLDENS=1.
Second in the capability-layered series after the IR + loader; depends on that
PR and has no downstream dependencies of its own.
Owner
Author
|
cc @dmitrylychagin @amirhormati — PR2 of the semantic-model upstreaming series (BigQuery property-graph DDL generator). Stacked on PR1 ( |
Move the generator's behavior coverage onto readable, textual pairs: a `<fixture>.yaml` semantic model next to a committed `<fixture>.bigquery.golden.sql` showing the exact DDL and warnings. Adds three fixtures to the e2e golden corpus: - measure_lowering: every metric-to-MEASURE lowering shape (bare column, inline expression, shared operand reused once, COUNT(DISTINCT), an operand whose name collides with the metric, and a qualifier preserved inside a string literal). - metric_skips: each unplaceable metric (spans multiple tables, ratio, unsupported aggregate, name collision) skipped and reported, with the one valid measure and its edge still emitted. - keyless_dimension: a keyless node skipped and its edge omitted. bigquery.test.ts now holds only what the open AI-first format cannot express (it is a subset of the IR): an M:N association edge (hand-built IR, checked against a committed golden file), the IR-contract cases the loader never produces (COUNT(*) with a declared home; declared entities disagreeing with the expression), degenerate inputs, and the structural MEASURE-shape invariant guard run over the loaded fixtures. The two inline VERIFIED_*_DDL strings are gone (now golden files). No generator changes; behavior coverage is unchanged.
libei
marked this pull request as ready for review
August 2, 2026 06:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second in the capability-layered series merging the semantic-model tooling from the
libei/knowledge-catalogfork back into upstream. Stacked on GoogleCloudPlatform#258 (Semantic Model IR + loader); it has no downstream dependencies of its own.What this PR adds
The first destination for the Semantic Model IR: a generator that emits a single
CREATE OR REPLACE PROPERTY GRAPHstatement over the entities' existing base tables, with model-level metrics rendered as inlineMEASURE(...)properties.src/libts/semantic/bigquery.ts— maps the IR to BigQuery graph DDL:COUNT(*)lowers to a count over the key property.SUM/AVG/COUNT/MIN/MAXare emitted (the complete set BigQuery accepts inside a measure); any other aggregate is skipped + warned rather than emitted as DDL BigQuery would reject.aiContextinstructions/synonyms/examples) is folded intoOPTIONS(description=...)— the only metadata sink the graph DDL exposes — on the graph, element tables, properties, and measures.dataSourceas the loader's normalized, fully-qualified string; completes an under-qualified ref from options and warns if it is still short.warnings, never silently emitted as invalid DDL.tests/libts/semantic/bigquery.test.ts— behavior spec over hand-built IR with inline goldens. The chain and M:N goldens are the exact DDL a live BigQuery instance accepted (validated withGRAPH_EXPAND+AGGand a GQLMATCH), plus a parsed-DDL invariant that no emitted MEASURE ever wraps a shape BigQuery rejects.tests/libts/semantic/bigquery.e2e.test.ts+*.bigquery.golden.sql— a file → IR → DDL golden suite over a corpus of real-shaped fixtures; the goldens are the reviewable "big picture" (open a.yamlnext to its.bigquery.golden.sql). Regenerate withUPDATE_GOLDENS=1.Scope
Property-graph DDL generation only. The CLI push path and Knowledge Catalog emit/pull follow in later PRs in the series.
Testing
npx tsc --noEmit— cleanbun test tests/libts/semantic/— 109 pass / 0 fail