Enable Delta Change Data Feed globally for all spellbook tables

[shahil-bhikha]

Add delta.enableChangeDataFeed = true to the create_table_properties macro to enable CDC for all Delta tables created by dbt, both in production and CI.

This applies globally at CREATE TABLE time via the WITH clause for all models using file_format = 'delta' (100+ models across all sub-projects).

Property is rendered via dune_properties() macro into valid Trino SQL and applies to prod, dev, and CI equally. CDC is required for downstream datashare replication pipelines.

[cursor]

PR Summary

Medium Risk
Wide blast radius across every Delta table on recreate, but limited to a standard table property with no query or auth changes.

Overview
Adds delta.enableChangeDataFeed = true to the shared create_table_properties macro so every Delta table built through trino__create_table_as gets CDC enabled at CREATE OR REPLACE time in the Trino WITH clause (via dune_properties()).

This is a global default for prod, dev, and CI—not per-model config—so it affects all spellbook Delta table / incremental models on the next full table rebuild, supporting downstream datashare replication that depends on change data feed.

^{Reviewed by Cursor Bugbot for commit 9ab5751. Configure here.}

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[shahil-bhikha]

I have read the CLA Document and I hereby sign the CLA

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: CDC on temporary staging tables
  • I passed the temporary flag into create_table_properties and now only enable delta.enableChangeDataFeed when the table is not temporary.

Or push these changes by commenting:

@cursor push b4936268e8

Preview (b4936268e8)

diff --git a/dbt_macros/dune/adapters.sql b/dbt_macros/dune/adapters.sql
--- a/dbt_macros/dune/adapters.sql
+++ b/dbt_macros/dune/adapters.sql
@@ -4,7 +4,7 @@
     {%- do _properties.update({'partitioned_by': "ARRAY['" + (config.get('partition_by') | join("', '") )  + "']"}) -%}
   {%- endif -%}
   create or replace table {{ relation }}
-    {{ create_table_properties(_properties, relation) }}
+    {{ create_table_properties(_properties, relation, temporary) }}
   as (
     {{ sql }}
   );
@@ -45,14 +45,16 @@
   {%- endif -%}
 {%- endmacro -%}
 
-{% macro create_table_properties(_properties, relation) %}
+{% macro create_table_properties(_properties, relation, temporary=false) %}
   {%- if not (target.name == 'ci' and target.database == 'dune') -%}
     {%- set modified_identifier = relation.identifier | replace("__dbt_tmp", "") -%}
     {%- set unique_location = modified_identifier ~ '_' ~ time_salted_md5_prefix() -%}
     {%- set location= 's3a://%s/%s/%s' % (s3_bucket(), relation.schema, unique_location) -%}
     {%- do _properties.update({'location': "'" + location + "'"}) -%}
   {%- endif -%}
-  {%- do _properties.update({'delta.enableChangeDataFeed': 'true'}) -%}
+  {%- if not temporary -%}
+    {%- do _properties.update({'delta.enableChangeDataFeed': 'true'}) -%}
+  {%- endif -%}
   {%- if target.name == 'ci' -%}
     {%- do _properties.update({'extra_properties': "map_from_entries(ARRAY[ROW('dune.public', 'true')])"}) -%}
   {%- endif -%}

_{You can send follow-ups to the cloud agent here.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 9ab5751. Configure here.}

[cursor]

CDC on temporary staging tables

Medium Severity

Change Data Feed is now set for every create_table_properties call, including short-lived __dbt_tmp staging tables built during incremental and table materializations. Those relations are created through trino__create_table_as with temporary=true, but that flag is never passed into create_table_properties, so CDC can be enabled on tables meant only for load-time swaps rather than datashare replication targets.

 

^{Reviewed by Cursor Bugbot for commit 9ab5751. Configure here.}