Skip to content

Fix #906: execute viral propagation only at combination points (main, DuckDB-only)#908

Draft
javihern98 wants to merge 6 commits into
mainfrom
cr-906-main
Draft

Fix #906: execute viral propagation only at combination points (main, DuckDB-only)#908
javihern98 wants to merge 6 commits into
mainfrom
cr-906-main

Conversation

@javihern98

Copy link
Copy Markdown
Contributor

Summary

DuckDB-only adaptation for main of the 1.9.X change in #907 (issue #906). Viral attribute propagation was executed (and required) on essentially every dataset operator; per the VTL 2.2 Attribute propagation rule it must run only when input data points are combined, and otherwise copy viral attributes unchanged.

  • Combination points (rule executed, rule required): an operation over two or more datasets (dataset–dataset binary, joins, if/case with ≥2 dataset branches); an aggregation/analytic group-by/partition; the hierarchy roll-up.
  • Everywhere else (viral copied unchanged, no rule required): unary ops, dataset⊕scalar, calc/keep/drop/rename/unpivot, cast, time shifts, nvl, set operators, check_datapoint/check_hierarchy.

The SemanticError 1-3-3-6 moves from a global per-statement check into the combining operators' validate methods (still fires during semantic_analysis()), flagging only the viral attributes actually combined. The row-preserving DuckDB SQL emissions become plain copies (vp_dataset_wide_sql removed); nvl copies its primary operand (COALESCE-filled).

main is DuckDB-only, so there is no pandas evaluate/apply_row_preserving path (this is the difference from the 1.9.X PR #907).

Fixes #906.

Checklist

  • Code quality checks pass (ruff format, ruff check, mypy)
  • Tests pass (pytest) — full suite: 5056 passed, 11 skipped
  • Documentation updated (error 1-3-3-6 description; error_messages.rst regenerates from messages.py)

Impact / Risk

Notes

DuckDB-only adaptation of the 1.9.X change (cr-906 / PR #907). Viral
propagation runs and is required only where input data points are combined
(>=2 datasets, aggregation/analytic group-by, hierarchy roll-up); everywhere
else viral attributes are copied unchanged. Adds require_rules/
combined_viral_components, moves the 1-3-3-6 check from the global per-statement
site into the combining operators' validate methods, and makes the row-preserving
DuckDB emissions copy (removing vp_dataset_wide_sql). nvl copies its primary
operand. Supersedes #897's dataset-wide check_datapoint viral aggregate (a
row-preserving validation now copies per datapoint).

Fixes #906.
…ropagation, joins, keep, parsing, validation) (cr-906-main)

# -- check_datapoint validation operator --

_DPR = """

# -- Rule execution on row-preserving dataset-level operators (issue #877) --

_AGG_RULE = "define viral propagation VP (variable VAt_1) is aggregate max end viral propagation;"
# -- Rule execution on row-preserving dataset-level operators (issue #877) --

_AGG_RULE = "define viral propagation VP (variable VAt_1) is aggregate max end viral propagation;"
_ENUM_RULE = (
{"name": "VAt_1", "type": "Number", "role": "Viral Attribute", "nullable": True},
],
}
_GP_STR_DS = {
],
}

_ENUM_PAIR_RULE = """
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Viral propagation applied on non-combining operators (aggregate rule collapses viral column on abs)

1 participant