Skip to content

Commit ba23d97

Browse files
fix: enforce uniqueItems on generated array fields (#59)
* fix: enforce uniqueItems on generated array fields datamodel-code-generator drops `uniqueItems`, so generated list fields accept duplicate entries in violation of the schema. Three UCP array properties declare `uniqueItems: true` at 2026-04-08: context.eligibility, card_payment_instrument.brands, and identity_linking.required_claims. Extend postprocess_models.py to collect array property names declared with `uniqueItems` and inject a `field_validator(mode="after")` into each generated class that declares a matching list field. The check uses equality (`item in seen`) so it holds for both hashable (str) and unhashable (model) items. Mirrors the data-driven, idempotent approach used for minProperties (#55). Two generated models carry the affected list fields and gain the validator: Context.eligibility and Constraints.brands. required_claims has no generated typed field (ScopePolicy is extra="allow" free-form), so it is not enforceable and is skipped. Add UniqueItemsInjectorTest (scan walks nested properties and ignores non-arrays; injection targets only matching list fields, is idempotent, and enforces uniqueness when exec'd) and UniqueItemsSemanticTest (Constraints rejects duplicate brands, accepts unique/None). * chore: bump version to 0.4.4 and regenerate models --------- Co-authored-by: damaz91 <federico.damato91@gmail.com>
1 parent bbf715b commit ba23d97

6 files changed

Lines changed: 351 additions & 40 deletions

File tree

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,8 @@ from ucp_sdk.models.schemas.shopping.checkout import Checkout
7171
checkout = Checkout.model_validate(checkout_data)
7272

7373
# Access typed fields
74-
print(checkout.status) # "incomplete" | "ready_for_complete" | ...
75-
print(checkout.currency) # ISO 4217 currency code
74+
print(checkout.status) # "incomplete" | "ready_for_complete" | ...
75+
print(checkout.currency) # ISO 4217 currency code
7676
for item in checkout.line_items:
7777
print(f"{item.item.title}: {item.quantity}")
7878
```

postprocess_models.py

Lines changed: 181 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -14,39 +14,46 @@
1414

1515
"""Post-generation fixes for constraints datamodel-code-generator ignores.
1616
17-
``minProperties`` on an object schema WITH declared properties is dropped by
18-
the generator (issue #49): every field is optional, so an empty instance
19-
passes validation in violation of the schema. (``minProperties`` on a
20-
free-form object property is already handled natively — the generator maps it
21-
to ``Field(min_length=...)`` on the dict field.)
22-
23-
This script scans the preprocessed schemas for root-level ``minProperties``
24-
constraints and injects a ``model_validator(mode="after")`` into the matching
25-
generated classes. JSON Schema counts the keys present on the object, so the
26-
validator counts provided fields (``model_fields_set``) unioned with extra
27-
keys (``model_extra``) — an explicit null is a present key, and unknown keys
28-
on ``extra="allow"`` models count too.
29-
30-
``contains`` / ``minContains`` / ``maxContains`` on an array schema is likewise
31-
dropped by the generator: ``totals.json`` requires *exactly one* ``subtotal``
32-
*and exactly one* ``total`` entry, but the generated ``Totals`` is a bare
33-
``list[Total]`` alias, so an empty array (or one missing either required entry,
34-
or with duplicates) validates in violation of the schema. An array root is
35-
emitted as a ``TypeAliasType`` wrapping ``Annotated[list[...], ...]`` rather
36-
than a ``BaseModel`` subclass, so ``model_validator`` cannot apply; this script
37-
instead injects a module-level counting function, threaded into the alias
38-
metadata as a ``pydantic.AfterValidator``. Every predicate is derived from
39-
``contains.properties.<field>.const`` — nothing is hard-coded — and one function
40-
enforces *all* of a schema's contains bounds.
41-
42-
The pristine (pre-preprocessing) schemas are read for this: ``totals.json``
43-
carries its two containment rules as two ``allOf`` branches, and
44-
``preprocess_schemas.py`` merges ``allOf`` into the root, where a JSON node can
45-
hold only one ``contains`` — so the second (``total``) would be lost if the
46-
preprocessed output were scanned. generate_models.sh snapshots the originals to
47-
``ucp/raw_schemas`` before preprocessing for exactly this reason. The bound is
48-
applied to the base model and to its generated request variants (linked by file
49-
stem), and travels wherever the alias is reused as a field type.
17+
Three constraint families are handled:
18+
19+
* ``minProperties`` on an object schema WITH declared properties is dropped by
20+
the generator (issue #49): every field is optional, so an empty instance
21+
passes validation in violation of the schema. (``minProperties`` on a
22+
free-form object property is already handled natively — the generator maps it
23+
to ``Field(min_length=...)`` on the dict field.) The script scans the
24+
preprocessed schemas for root-level ``minProperties`` constraints and injects
25+
a ``model_validator(mode="after")`` into the matching generated classes.
26+
JSON Schema counts the keys present on the object, so the validator counts
27+
provided fields (``model_fields_set``) unioned with extra keys
28+
(``model_extra``) — an explicit null is a present key, and unknown keys on
29+
``extra="allow"`` models count too.
30+
31+
* ``contains`` / ``minContains`` / ``maxContains`` on an array schema is likewise
32+
dropped by the generator: ``totals.json`` requires *exactly one* ``subtotal``
33+
*and exactly one* ``total`` entry, but the generated ``Totals`` is a bare
34+
``list[Total]`` alias, so an empty array (or one missing either required entry,
35+
or with duplicates) validates in violation of the schema. An array root is
36+
emitted as a ``TypeAliasType`` wrapping ``Annotated[list[...], ...]`` rather
37+
than a ``BaseModel`` subclass, so ``model_validator`` cannot apply; this script
38+
instead injects a module-level counting function, threaded into the alias
39+
metadata as a ``pydantic.AfterValidator``. Every predicate is derived from
40+
``contains.properties.<field>.const`` — nothing is hard-coded — and one function
41+
enforces *all* of a schema's contains bounds.
42+
43+
The pristine (pre-preprocessing) schemas are read for this: ``totals.json``
44+
carries its two containment rules as two ``allOf`` branches, and
45+
``preprocess_schemas.py`` merges ``allOf`` into the root, where a JSON node can
46+
hold only one ``contains`` — so the second (``total``) would be lost if the
47+
preprocessed output were scanned. generate_models.sh snapshots the originals to
48+
``ucp/raw_schemas`` before preprocessing for exactly this reason. The bound is
49+
applied to the base model and to its generated request variants (linked by file
50+
stem), and travels wherever the alias is reused as a field type.
51+
52+
* ``uniqueItems`` on an array is dropped entirely by the generator, so a list
53+
field accepts duplicate entries in violation of the schema. The script
54+
collects the names of array properties declared with ``uniqueItems`` and
55+
injects a ``field_validator(mode="after")`` into each generated class that
56+
declares a matching list field.
5057
5158
Runs from generate_models.sh between generation and formatting; idempotent.
5259
"""
@@ -79,6 +86,24 @@ def {marker}(self):
7986
return self
8087
'''
8188

89+
_UNIQUE_MARKER = "_enforce_unique_items"
90+
91+
_UNIQUE_VALIDATOR_TEMPLATE = '''
92+
@field_validator("{field}", mode="after")
93+
def {marker}_{field}(cls, value): # noqa: N805
94+
"""JSON Schema uniqueItems: reject duplicate entries."""
95+
if value is None:
96+
return value
97+
seen = []
98+
for item in value:
99+
if item in seen:
100+
raise ValueError(
101+
"Items must be unique (schema uniqueItems=true)"
102+
)
103+
seen.append(item)
104+
return value
105+
'''
106+
82107

83108
def find_root_min_properties(schema_dir):
84109
"""Map schema title -> minProperties for root-level object constraints."""
@@ -321,6 +346,103 @@ def inject_array_contains(source, alias_name, groups):
321346
return _ensure_pydantic_import(out, "AfterValidator")
322347

323348

349+
def _iter_nodes(root):
350+
"""Yield every dict/list node in a JSON tree (cycle-safe)."""
351+
stack = [root]
352+
seen = {id(root)}
353+
while stack:
354+
cur = stack.pop()
355+
yield cur
356+
if isinstance(cur, dict):
357+
children = cur.values()
358+
elif isinstance(cur, list):
359+
children = cur
360+
else:
361+
children = ()
362+
for child in children:
363+
if isinstance(child, (dict, list)) and id(child) not in seen:
364+
seen.add(id(child))
365+
stack.append(child)
366+
367+
368+
def find_unique_items_fields(schema_dir):
369+
"""Collect property names whose array value carries ``uniqueItems``.
370+
371+
Walks every schema (root and nested) for object properties declared as an
372+
array with ``uniqueItems: true``. Returns the set of property names so the
373+
injector can locate the matching generated list fields by name.
374+
"""
375+
fields = set()
376+
for path in sorted(Path(schema_dir).rglob("*.json")):
377+
try:
378+
schema = json.loads(path.read_text(encoding="utf-8"))
379+
except (OSError, json.JSONDecodeError):
380+
continue
381+
if not isinstance(schema, dict):
382+
continue
383+
for node in _iter_nodes(schema):
384+
if not isinstance(node, dict):
385+
continue
386+
props = node.get("properties")
387+
if not isinstance(props, dict):
388+
continue
389+
for name, prop in props.items():
390+
if (
391+
isinstance(prop, dict)
392+
and prop.get("uniqueItems") is True
393+
and (prop.get("type") == "array" or "items" in prop)
394+
):
395+
fields.add(name)
396+
return fields
397+
398+
399+
def inject_unique_items(source, unique_fields):
400+
"""Inject uniqueness validators for list fields declared ``uniqueItems``.
401+
402+
Scans each generated class for list-typed fields whose name is in
403+
``unique_fields`` and appends a ``field_validator`` to the class body.
404+
"""
405+
if not unique_fields:
406+
return source
407+
class_re = re.compile(r"^class \w+\(", re.M)
408+
matches = list(class_re.finditer(source))
409+
if not matches:
410+
return source
411+
new_source = source
412+
patched = False
413+
# Process from the last class to the first so earlier insert offsets
414+
# (computed against the original source) stay valid as text is appended.
415+
for match in reversed(matches):
416+
body_start = match.end()
417+
tail = re.compile(r"^\S", re.M)
418+
end_match = tail.search(source, body_start)
419+
body_end = end_match.start() if end_match else len(source)
420+
body = source[body_start:body_end]
421+
targets = []
422+
for field_match in re.finditer(
423+
r"^ (\w+): [^\n]*\blist\[", body, re.M
424+
):
425+
field = field_match.group(1)
426+
marker = f"def {_UNIQUE_MARKER}_{field}("
427+
if field in unique_fields and marker not in body:
428+
targets.append(field)
429+
if not targets:
430+
continue
431+
methods = "".join(
432+
_UNIQUE_VALIDATOR_TEMPLATE.format(
433+
marker=_UNIQUE_MARKER, field=field
434+
)
435+
for field in targets
436+
)
437+
prefix = new_source[:body_end].rstrip("\n")
438+
suffix = new_source[body_end:]
439+
new_source = prefix + methods + ("\n" + suffix if suffix else "")
440+
patched = True
441+
if patched:
442+
new_source = _ensure_pydantic_import(new_source, "field_validator")
443+
return new_source
444+
445+
324446
def _patch_min_properties():
325447
"""Inject minProperties validators; return (patched_count, exit_code)."""
326448
constraints = find_root_min_properties(SCHEMA_DIR)
@@ -425,13 +547,37 @@ def _patch_array_contains():
425547
return patched, 0
426548

427549

550+
def _patch_unique_items():
551+
"""Inject uniqueItems validators; return (patched_count, exit_code)."""
552+
unique_fields = find_unique_items_fields(SCHEMA_DIR)
553+
if not unique_fields:
554+
sys.stdout.write("postprocess: no uniqueItems constraints found\n")
555+
return 0, 0
556+
unique_patched = 0
557+
touched = []
558+
for path in sorted(OUTPUT_DIR.rglob("*.py")):
559+
source = path.read_text(encoding="utf-8")
560+
updated = inject_unique_items(source, unique_fields)
561+
if updated != source:
562+
path.write_text(updated, encoding="utf-8")
563+
unique_patched += 1
564+
touched.append(path)
565+
sys.stdout.write(
566+
f" uniqueItems fields {sorted(unique_fields)} -> "
567+
f"{unique_patched} module(s) patched"
568+
f" ({', '.join(str(t) for t in touched) or 'none'})\n"
569+
)
570+
return unique_patched, 0
571+
572+
428573
def main():
429574
"""Main entry point to scan schemas and patch generated models."""
430575
patched_mp, rc_mp = _patch_min_properties()
431576
patched_ac, rc_ac = _patch_array_contains()
432-
total = patched_mp + patched_ac
577+
patched_ui, rc_ui = _patch_unique_items()
578+
total = patched_mp + patched_ac + patched_ui
433579
sys.stdout.write(f"postprocess: {total} module(s) patched\n")
434-
return rc_mp or rc_ac
580+
return rc_mp or rc_ac or rc_ui
435581

436582

437583
if __name__ == "__main__":

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[project]
22
name = "ucp-sdk"
3-
version = "0.4.3"
3+
version = "0.4.4"
44
description = "UCP Python SDK"
55
readme = "README.md"
66
license = {file = "LICENSE"}

src/ucp_sdk/models/schemas/shopping/types/card_payment_instrument.py

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020

2121
from typing import Literal
2222

23-
from pydantic import AnyUrl, BaseModel, ConfigDict, Field
23+
from pydantic import AnyUrl, BaseModel, ConfigDict, Field, field_validator
2424

2525
from .available_payment_instrument import AvailablePaymentInstrument
2626
from .payment_instrument import PaymentInstrument
@@ -69,6 +69,20 @@ class Constraints(BaseModel):
6969
Limit to specific card brands (e.g., ['visa', 'mastercard', 'amex']).
7070
"""
7171

72+
@field_validator("brands", mode="after")
73+
def _enforce_unique_items_brands(cls, value): # noqa: N805
74+
"""JSON Schema uniqueItems: reject duplicate entries."""
75+
if value is None:
76+
return value
77+
seen = []
78+
for item in value:
79+
if item in seen:
80+
raise ValueError(
81+
"Items must be unique (schema uniqueItems=true)"
82+
)
83+
seen.append(item)
84+
return value
85+
7286

7387
class AvailableCardPaymentInstrument(AvailablePaymentInstrument):
7488
"""

src/ucp_sdk/models/schemas/shopping/types/context.py

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818

1919
from __future__ import annotations
2020

21-
from pydantic import BaseModel, ConfigDict
21+
from pydantic import BaseModel, ConfigDict, field_validator
2222

2323
from . import reverse_domain_name
2424

@@ -59,3 +59,17 @@ class Context(BaseModel):
5959
"""
6060
Buyer claims about eligible benefits such as loyalty membership, payment instrument perks, and similar. Recognized claims MAY inform the Business response (e.g., member-only product availability, adjusted pricing in catalog, provisional discounts at cart or checkout). Businesses MUST ignore unrecognized values without error. Values MUST use reverse-domain naming (e.g., 'com.example.loyalty_gold', 'org.school.student') and MUST be non-identifying.
6161
"""
62+
63+
@field_validator("eligibility", mode="after")
64+
def _enforce_unique_items_eligibility(cls, value): # noqa: N805
65+
"""JSON Schema uniqueItems: reject duplicate entries."""
66+
if value is None:
67+
return value
68+
seen = []
69+
for item in value:
70+
if item in seen:
71+
raise ValueError(
72+
"Items must be unique (schema uniqueItems=true)"
73+
)
74+
seen.append(item)
75+
return value

0 commit comments

Comments
 (0)