Skip to content

DagRun.dag_versions silently drops versions after a partial "run on latest version" clear #71454

Description

@dstandish

Problem

DagRun.dag_versions (airflow-core/src/airflow/models/dagrun.py:515-530) has a shortcut for bundle-versioned (pinned) runs:

@property
def dag_versions(self) -> list[DagVersion]:
    """Return the DAG versions associated with the TIs of this DagRun."""
    # when the dag is in a versioned bundle, we keep the dag version fixed
    if self.bundle_version:
        return [self.created_dag_version] if self.created_dag_version is not None else []
    ...

This assumes a pinned run's task instances can only ever be on the single version recorded in created_dag_version_id. That was true when created_dag_version_id was immutable (set once at DagRun creation, per #49097). It is no longer true: clear_task_instances(..., run_on_latest_version=True) (added in #54984, extended in #65835/#66901) bumps created_dag_version_id to the latest version, but only bumps the dag_version_id of the cleared task instances — uncleared task instances explicitly keep their old version (see the comment at airflow-core/src/airflow/models/taskinstance.py:477: "Only cleared TIs get latest dag_version_id above; do not rewrite others.").

Repro

  1. Create a pinned (bundle-versioned) DagRun with tasks A and B, both created against version V1.
  2. Deploy a new version V2 (no task additions/removals needed).
  3. Clear only task A with run_on_latest_version=True. Now: dag_run.created_dag_version_id == V2, task A's TI is on V2, task B's TI is still on V1.
  4. Read dag_run.dag_versions (or view the Grid UI, which reads it directly at airflow-core/src/airflow/api_fastapi/core_api/routes/ui/grid.py:353).

Expected

[V1, V2] — the run genuinely has task instances on both versions.

Actual

[V2] — task B's actual version (V1) is silently dropped, because the property trusts created_dag_version_id alone instead of the real per-TI versions once bundle-version pinning is set.

Impact

The Grid UI's "Dag versions" indicator for a run can show a single version chip while some of that run's tasks are still genuinely executing under an older version — misleading when debugging "what code actually ran here."

Suggested fix

Don't special-case bundle-versioned runs in dag_versions; compute it the same way as unpinned runs (union of _ti_dag_versions/_tih_dag_versions), or explicitly detect and represent the mixed-version case.

Context

This is one concrete consequence of created_dag_version_id's contract having drifted from "the version recorded at DagRun creation" (its documented meaning) to "the version this run should currently run at" (its behavior since #54984). See the companion issue tracking that broader contract problem: #71453


Drafted-by: Claude Code (Sonnet 5) (no human review before posting)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions