You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
created_dag_version_id (airflow-core/src/airflow/models/dagrun.py:301-304) has existed as a column since AIP-65 DAG versioning (#42913/#43735). #49097 added the ORM relationship (created_dag_version) on top of it, with a docstring restating what the name already communicated:
"The id of the dag version column that was in effect at dag run creation time."
and the paired relationship docstring:
"The dag version that was active when the dag run was created, if available."
Documented or not, the name itself was always the contract — a write-once historical fact, not a live pointer. For a bundle-versioned (pinned) run, the version recorded at creation was — by design — the version that run would run forever. Per a conversation with the original author of AIP-65 DAG versioning: bundle-versioned runs were always meant to execute the version pinned at creation; the way to get "run with the latest code" behavior was to not use a versioned bundle at all (unpinned/local bundles always resolve to the latest DagVersion — see DBDagBag._version_from_dag_run, airflow-core/src/airflow/models/dagbag.py:210-216). There was never supposed to be a way to move a pinned run off the version it was created with.
What changed
Starting with #52177 (introduces the run_on_latest_version flag) and #54984 ("Run verify_integrity when cleared to run on latest"), and extended by #59764 and #65835/#66901, created_dag_version_id began being mutated after creation:
clear_task_instances(..., run_on_latest_version=True) — airflow-core/src/airflow/models/taskinstance.py:474 and :497
An open PR, #71425, is currently extending this same mutation to running/queued DagRuns.
None of these renamed or re-documented the field. It still says "recorded at creation" while several code paths now treat it as "the version this run should currently be considered pinned to" — two different, and now conflated, meanings living in one column.
Why this matters
The mismatch between the documented contract and actual behavior has already produced concrete defects downstream, because other code correctly relied on the original contract and nothing flagged that the contract had changed underneath it:
Both are symptoms, not the root cause. The root cause is that one field is now asked to represent two different things — an immutable historical fact, and a mutable "current pinned version" pointer — with no way for a reader to tell which one they're getting, or whether the run has been through a partial version bump that leaves it internally inconsistent.
What needs deciding
This is a design question as much as a naming one:
Does "run on latest version" belong on bundle-versioned/pinned runs at all? The original design intent was that pinning is absolute, and "latest" is achieved by not pinning. If the answer is no, the mutation added since Run verify_integrity when cleared to run on latest #54984 should be reconsidered/reverted for the pinned case.
If the capability is intentionally kept, created_dag_version_id should stop being overloaded: introduce a genuinely separate, clearly-named mutable field for "the version this pinned run is currently expected to run at" (e.g. pinned_dag_version_id or current_dag_version_id), keep created_dag_version_id strictly immutable as documented, and audit every consumer (_version_from_dag_run, get_dag_for_run_or_latest_version, DagRun.dag_versions, _get_new_task_ids, and anything else reading this field) to use the correct one.
Either way, the current state — one field, two incompatible meanings, no rename, no updated docs — should be resolved before more functionality (e.g. #71425) is built on top of it.
Drafted-by: Claude Code (Sonnet 5) (no human review before posting)
Background
created_dag_version_id(airflow-core/src/airflow/models/dagrun.py:301-304) has existed as a column since AIP-65 DAG versioning (#42913/#43735). #49097 added the ORM relationship (created_dag_version) on top of it, with a docstring restating what the name already communicated:and the paired relationship docstring:
Documented or not, the name itself was always the contract — a write-once historical fact, not a live pointer. For a bundle-versioned (pinned) run, the version recorded at creation was — by design — the version that run would run forever. Per a conversation with the original author of AIP-65 DAG versioning: bundle-versioned runs were always meant to execute the version pinned at creation; the way to get "run with the latest code" behavior was to not use a versioned bundle at all (unpinned/local bundles always resolve to the latest
DagVersion— seeDBDagBag._version_from_dag_run,airflow-core/src/airflow/models/dagbag.py:210-216). There was never supposed to be a way to move a pinned run off the version it was created with.What changed
Starting with #52177 (introduces the
run_on_latest_versionflag) and #54984 ("Run verify_integrity when cleared to run on latest"), and extended by #59764 and #65835/#66901,created_dag_version_idbegan being mutated after creation:_update_dagrun_to_latest_version—airflow-core/src/airflow/models/taskinstance.py:344clear_task_instances(..., run_on_latest_version=True)—airflow-core/src/airflow/models/taskinstance.py:474and:497An open PR, #71425, is currently extending this same mutation to running/queued DagRuns.
None of these renamed or re-documented the field. It still says "recorded at creation" while several code paths now treat it as "the version this run should currently be considered pinned to" — two different, and now conflated, meanings living in one column.
Why this matters
The mismatch between the documented contract and actual behavior has already produced concrete defects downstream, because other code correctly relied on the original contract and nothing flagged that the contract had changed underneath it:
DagRun.dag_versionssilently drops versions still in use by uncleared task instances after a partial "run on latest version" clear.only_newclear (_get_new_task_ids) can report zero new tasks when there genuinely are some, for the same reason.Both are symptoms, not the root cause. The root cause is that one field is now asked to represent two different things — an immutable historical fact, and a mutable "current pinned version" pointer — with no way for a reader to tell which one they're getting, or whether the run has been through a partial version bump that leaves it internally inconsistent.
What needs deciding
This is a design question as much as a naming one:
created_dag_version_idshould stop being overloaded: introduce a genuinely separate, clearly-named mutable field for "the version this pinned run is currently expected to run at" (e.g.pinned_dag_version_idorcurrent_dag_version_id), keepcreated_dag_version_idstrictly immutable as documented, and audit every consumer (_version_from_dag_run,get_dag_for_run_or_latest_version,DagRun.dag_versions,_get_new_task_ids, and anything else reading this field) to use the correct one.Either way, the current state — one field, two incompatible meanings, no rename, no updated docs — should be resolved before more functionality (e.g. #71425) is built on top of it.
Drafted-by: Claude Code (Sonnet 5) (no human review before posting)