Description
The validator loads input with yaml.safe_load, which silently keeps the last value when a YAML or JSON mapping contains duplicate keys. Schema and semantic validation therefore run against an altered document and may report success after user-authored data has been discarded.
Reproduction
version: 0.2.0.dev0
semantic_model:
- name: sales
datasets:
- name: orders
source: staging.orders
source: production.ordersuv run validation/validate.py duplicate.yaml
echo $?
Actual result:
Validation PASSED: duplicate.yaml
0
The validator only sees production.orders. The same behavior occurs for duplicate top-level keys, model keys, entire collections, quoted or explicitly tagged equivalent keys, and JSON object keys. A valid second occurrence can also hide an invalid first value.
Expected behavior
Reject explicit duplicate mapping keys as invalid input, identify the duplicate key and source locations, and exit non-zero. Legal YAML merge-key overrides should remain supported.
Suggested implementation
Use a custom loader derived from yaml.SafeLoader that detects repeated explicit keys before delegating mapping construction to SafeLoader. Add regression coverage for nested duplicates, JSON input, aliases, merge overrides, and valid mappings.
Description
The validator loads input with
yaml.safe_load, which silently keeps the last value when a YAML or JSON mapping contains duplicate keys. Schema and semantic validation therefore run against an altered document and may report success after user-authored data has been discarded.Reproduction
Actual result:
The validator only sees
production.orders. The same behavior occurs for duplicate top-level keys, model keys, entire collections, quoted or explicitly tagged equivalent keys, and JSON object keys. A valid second occurrence can also hide an invalid first value.Expected behavior
Reject explicit duplicate mapping keys as invalid input, identify the duplicate key and source locations, and exit non-zero. Legal YAML merge-key overrides should remain supported.
Suggested implementation
Use a custom loader derived from
yaml.SafeLoaderthat detects repeated explicit keys before delegating mapping construction toSafeLoader. Add regression coverage for nested duplicates, JSON input, aliases, merge overrides, and valid mappings.