Skip to content

Issue #756: honor inlined entity references on schema create-by-upload#1007

Open
bjagg wants to merge 3 commits into
LIF-Initiative:mainfrom
bjagg:fix/issue-756-honor-references
Open

Issue #756: honor inlined entity references on schema create-by-upload#1007
bjagg wants to merge 3 commits into
LIF-Initiative:mainfrom
bjagg:fix/issue-756-honor-references

Conversation

@bjagg

@bjagg bjagg commented Jun 23, 2026

Copy link
Copy Markdown
Contributor
Description of Change

Problem. MDR's schema generator (schema_generation_service.add_ref) encodes a reference to another entity as an inlined object under a property key carrying a Ref infix — Ref<Child> for a plain reference, <relationship>Ref<Child> when the association has a relationship — rather than emitting an OpenAPI $ref. The create-by-upload reader only looked for $ref, so re-uploading an MDR-exported schema silently dropped every reference, and the entity-creation pass then materialized each reference as a bogus child entity.

Solution.

  • Add parse_reference_key / is_inlined_reference helpers to detect the inlined-reference convention alongside genuine $ref. is_inlined_reference requires a PascalCase entity name after the marker so an embedded child merely named Reference is not misread.
  • Rewrite create_reference_associations_for_children to handle both $ref and inlined references: resolve the referenced and parent entities (UniqueName from embedded metadata, falling back to the name parsed from the key — entities are created in the first pass with UniqueName defaulting to their schema-key name), and create the Placement="Reference" EntityAssociation with the correct relationship. It now receives the entity's own key name for robust parent resolution and treats a reference as a leaf (no spurious recursion into its narrowed fields).
  • Skip references in create_entity_and_children_if_needed so they are no longer created as embedded child entities.

Side effects / limitations. Detection keys off the Ref infix per the issue report. That is inherently ambiguous for an entity legitimately named Ref<Something>; an in-code NOTE documents the preferred two-sided fix — have the generator stamp an explicit marker (e.g. x-lif-reference-to: <UniqueName>) and key off that instead of string-parsing the property name. Resolution assumes the uploaded schema was generated with entity metadata (the realistic export case); the key-name fallback covers the no-metadata case.

How reviewers should test. uv run pytest test/components/lif/mdr_services/test_schema_upload_service.py — covers key parsing, reference/child discrimination, relationship extraction, plain-reference (no relationship), and the embedded-child non-match. The post-pass test asserts a Reference-placement association is created with the resolved parent/child ids. End-to-end validation against a live MDR-generated schema is still worth doing before relying on this in production (see Additional Notes).

Related Issues

Closes #756

Type of Change
  • Bug fix (non-breaking change which fixes an issue)
Project Area(s) Affected
  • components/
  • test/ or e2e/

Checklist
  • commit message follows commit guidelines (see commitlint.config.mjs)
  • tests are included (unit and/or integration tests)
  • code passes linting checks (uv run ruff check)
  • code passes formatting checks (uv run ruff format)
  • code passes type checking (uv run ty check) — no new diagnostics introduced
  • pre-commit hooks have been run successfully
Testing
  • Automated tests added/updated
Additional Notes

🤖 Generated with Claude Code

…reate-by-upload

MDR's schema generator inlines entity references as objects under a "Ref"-infix
property key ("Ref<Child>" / "<relationship>Ref<Child>", see
schema_generation_service.add_ref) rather than emitting an OpenAPI "$ref". The
upload reader only looked for "$ref", so every reference in an MDR-exported
schema was silently dropped on re-upload — and the create pass then materialized
each reference as a bogus child entity.

Detect inlined references (is_inlined_reference / parse_reference_key) alongside
genuine "$ref", resolve the referenced and parent entities (UniqueName from
embedded metadata, falling back to the name parsed from the key), and create the
Reference-placement EntityAssociation with the correct relationship. Skip
references in the entity-creation pass so they are no longer created as child
entities. Pass the entity's own key name into the post-pass for robust parent
resolution.

Detection keys off the "Ref" infix per the issue report; an in-code NOTE records
the preferred two-sided fix (an explicit generator-stamped marker) since the
infix is ambiguous for entities legitimately named "Ref<Something>". Add unit
tests for the helpers and the reference post-pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ound-trip test

Drives the real pipeline against a live Postgres DB: seed a source model with a
Person --issuedBy(Reference)--> Organization association, run
generate_openapi_schema, then round-trip the result back through
create_data_model_from_openapi_schema. Asserts the inlined
"issuedByRefOrganization" reference is recreated as a Reference-placement
association (parent->child, relationship preserved) and is not materialized as a
child entity. Fails against the pre-fix reader (the reference degrades to an
Embedded association with no relationship).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bjagg

bjagg commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Added the end-to-end round-trip test flagged in the description (test/bases/lif/mdr_restapi/test_schema_roundtrip.py, commit b14043f). It drives the real pipeline against a live Postgres DB: seed a source model with a Person --issuedBy(Reference)--> Organization association → generate_openapi_schemacreate_data_model_from_openapi_schema, then asserts the inlined issuedByRefOrganization reference is recreated as a Reference-placement association (parent→child, relationship preserved) and not materialized as a child entity.

Verified it fails against pre-fix main: there the reference degrades to an Embedded association with no relationship. Full mdr_services suite still green; ruff/format clean.

@bjagg bjagg requested a review from cbeach47 June 23, 2026 18:40
Address review nit: 'prop_name[:idx] or None if idx > 0 else None' relied on
operator precedence and the 'or None' was dead code when idx > 0. Replace with
the equivalent, clearer 'prop_name[:idx] if idx > 0 else None' plus a comment.
No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bjagg

bjagg commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@cbeach47 — ready for review. Honors MDR's inlined Ref<Child> references on create-by-upload (unit tests + a live-Postgres generate→upload round-trip). Note: trivial conflict with #1006 in schema_upload_service.py — easiest to merge #1006 first, then this rebases cleanly (keep this PR's version).

@bjagg

bjagg commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

👋 @dereck-symmetry — pairs with #1006 (@cbeach47 also on it).

What it is: MDR create-by-upload (#756) now honors MDR's inlined Ref<Child> references — its schema generator inlines references instead of emitting a standard $ref, and the uploader used to silently drop them (and materialize bogus child entities).

Where to look: components/lif/mdr_services/schema_upload_service.py (ref detection + the association post-pass) + a live-Postgres generate→upload round-trip test.

Note: merge #1006 first (they conflict trivially; keep this PR's version). Same review approach as #1006. #1026 tracks the durable follow-up (an explicit export marker instead of the Ref-infix heuristic).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

Bug: MDR schema creation by upload not honoring references

1 participant