Skip to content

Latest commit

 

History

21 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

dbt_cortex_agent

A dbt package that adds cortex_agent and cortex_skill materializations for creating and managing Snowflake Cortex Agents and Agent Skills directly from dbt — the same way the dbt_semantic_view package manages Semantic Views.

Define agents and skills as dbt models, wire tools to other dbt models (semantic views, Cortex Search services) with ref() / source(), and let dbt build deploy everything in dependency order — fully integrated into your DAG, lineage, and orchestration.

Currently, as Cortex Agents are only available on the Snowflake adapter, this package is only available to be used on the Snowflake adapter.


At a glance

  • Materializations: cortex_agent, cortex_skill
  • Warehouse: Snowflake (Cortex Agents)
  • dbt compatibility: dbt 1.5+
  • Underlying DDL: CREATE AGENT / PUT 'file://...' @stage

Full SQL API coverage. The default mode wraps your model body in FROM SPECIFICATION $$ ... $$, so the entire agent specification grammar is available with no package change. For total control of every clause, switch on raw_ddl and the package becomes a pure pass-through to Snowflake SQL.


Installation

From dbt Hub (recommended)

Add the package to your project's packages.yml:

packages:
  - package: Matts52/dbt_cortex_agent
    version: 1.0.0

Then install:

dbt deps

From GitHub

Alternatively, install directly from GitHub:

packages:
  - git: "https://github.com/Matts52/dbt-cortex-agent.git"
    revision: 1.0.0
dbt deps

Usage

1. Specification mode (default)

The body of the model is the agent specification YAML. The package wraps it in FROM SPECIFICATION $$ ... $$ and emits the optional COMMENT and PROFILE clauses from config.

models/sales_agent.sql:

{{
  config(
    materialized = 'cortex_agent',
    comment      = 'Sales analytics assistant',
    profile      = {
      'display_name': 'Sales Assistant',
      'avatar': 'sales-icon.png',
      'color': 'blue'
    }
  )
}}
models:
  orchestration: claude-4-sonnet
orchestration:
  budget:
    seconds: 30
    tokens: 16000
instructions:
  response: "Respond in a friendly but concise manner."
  orchestration: "Use Analyst for revenue questions; use Search for policy questions."
  sample_questions:
    - question: "What was our revenue last quarter?"
tools:
  - tool_spec:
      type: "cortex_analyst_text_to_sql"
      name: "Analyst1"
      description: "Converts natural language to SQL for financial analysis."
  - tool_spec:
      type: "cortex_search"
      name: "Search1"
      description: "Searches company policy and documentation."
tool_resources:
  Analyst1:
    semantic_view: "{{ ref('sales_semantic_view') }}"
    execution_environment:
      type: "warehouse"
      warehouse: "MY_WAREHOUSE"
  Search1:
    name: "{{ source('cortex', 'policy_search_service') }}"
    max_results: 5
    filter:
      "@eq":
        region: "North America"
    title_column: "title"
    id_column: "doc_id"

This compiles to roughly:

create or replace agent MY_DB.MY_SCHEMA.SALES_AGENT
comment = 'Sales analytics assistant'
profile = '{"display_name": "Sales Assistant", "avatar": "sales-icon.png", "color": "blue"}'
from specification
$$
models:
  orchestration: claude-4-sonnet
...
$$

Tip — wiring tools to your DAG. Because the body is rendered through Jinja, you can use {{ ref(...) }} and {{ source(...) }} inside tool_resources to point a tool at a semantic view or Cortex Search service managed elsewhere in your project. This makes the agent a proper downstream node in your lineage graph.

Enabling web search

Add web_search_tool = true to the config block to give the agent access to Snowflake's built-in web search capability:

{{
  config(
    materialized    = 'cortex_agent',
    web_search_tool = true
  )
}}
models:
  orchestration: claude-4-sonnet
instructions:
  response: "Be concise."
  orchestration: "Use web search to answer questions about current events."

This injects a tool_spec entry into the agent specification YAML:

create or replace agent MY_DB.MY_SCHEMA.MY_AGENT
from specification
$$
...
tools:
  - tool_spec:
      type: "web_search"
      name: "web_search"
$$

Omitting the config key (or setting it to false) produces no tools entry.

Note: If your spec already contains a tools: block (e.g. for cortex_search or cortex_analyst_text_to_sql), set web_search_tool = false and add the entry directly in your spec's tools: list to avoid a duplicate key.

2. Raw DDL mode (raw_ddl=true)

The body is everything that follows CREATE OR REPLACE AGENT <name> — a direct pass-through to Snowflake. Use this when you want to control the exact clause ordering or adopt new CREATE AGENT syntax before the package models it.

Note: The comment, profile, and web_search_tool configs are silently ignored in raw DDL mode — a compile-time warning is emitted if any of them are set alongside raw_ddl=true. Add web search support directly in the spec's tools: list:

tools:
  - tool_spec:
      type: "web_search"
      name: "web_search"

models/raw_agent.sql:

{{ config(materialized='cortex_agent', raw_ddl=true) }}
comment = 'Fully hand-written DDL'
profile = '{"display_name": "Raw Agent"}'
from specification
$$
models:
  orchestration: claude-4-sonnet
instructions:
  response: "Be concise."
$$

3. Versioned publish / canary rollout (versioning=true)

By default, every dbt run issues CREATE OR REPLACE AGENT — the new spec is live the instant the run finishes, with no version history and no rollback path.

Set versioning=true to opt into named-version DDL instead. The package tracks whether the agent already exists and issues the appropriate DDL:

  • First run (agent absent): CREATE AGENT IF NOT EXISTS ... ADD VERSION '<name>' FROM SPECIFICATION $$...$$
  • Subsequent runs (agent present): ALTER AGENT ... ADD VERSION '<name>' FROM SPECIFICATION $$...$$
  • Promotion (when set_default=true, the default): ALTER AGENT ... SET DEFAULT_VERSION = '<name>'

Minimal example

{{
  config(
    materialized = 'cortex_agent',
    versioning   = true
  )
}}
models:
  orchestration: claude-4-sonnet
instructions:
  response: "Be concise."

Each run auto-generates a version name from run_started_at in the format v_YYYYMMDD_HHMMSS. All models versioned in the same dbt run share one version name, so rolling back a full run is as simple as flipping the default pointer back to the previous timestamp name.

Staging a canary version

Deploy a new spec without affecting live traffic by setting set_default=false:

{{
  config(
    materialized = 'cortex_agent',
    versioning   = true,
    version_name = 'v_canary',
    set_default  = false          -- create version but keep existing default live
  )
}}

After validating the canary out of band, promote it:

dbt run --select my_agent --vars '{"version_name": "v_canary", "set_default": true}'

Rollback

Flip DEFAULT VERSION back to any prior version name by re-running with an explicit version_name and set_default=true:

dbt run --select my_agent --vars '{"version_name": "v_20250101_120000", "set_default": true}'

Note: versioning=true is incompatible with raw_ddl=true. If both are set, a compile-time warning is emitted and the materialization falls back to CREATE OR REPLACE behavior. Use specification mode (raw_ddl=false) to enable versioning.


Skills (cortex_skill materialization)

Agent skills are modular packages of instructions (and optional scripts) that give agents repeatable, task-specific capabilities. Snowflake stores them as files on a named stage — there is no CREATE SKILL SQL statement.

The cortex_skill materialization uploads a skill's SKILL.md file to a Snowflake stage automatically during dbt build, before any agent that depends on it is created.

Defining a skill

Skill files live in a directory alongside the .sql model — same name as the model, no extension. The directory must contain at least SKILL.md and may include any companion scripts (e.g. .py files for the code execution tool). The .sql model is config only:

models/
  skills/
    forecaster_skill.sql       ← dbt model (config only)
    forecaster_skill/          ← skill directory (all files uploaded to stage)
      SKILL.md                 ← required
      forecaster.py            ← optional companion scripts

models/skills/forecaster_skill.sql:

{{
  config(
    materialized = 'cortex_skill',
    stage        = '@my_db.my_schema.skill_stage'
  )
}}

At runtime the materialization runs:

CREATE STAGE IF NOT EXISTS my_db.my_schema.skill_stage;

PUT 'file:///absolute/path/to/forecaster_skill/*'
    @my_db.my_schema.skill_stage/skills/forecaster_skill/
AUTO_COMPRESS = FALSE
OVERWRITE = TRUE;

Every file in the directory is uploaded in a single PUT. Filenames on the stage exactly match the local filenames — no suffix is appended.

Wiring a skill to an agent

Use the cortex_skill_path() macro with ref() to wire a skill to an agent. This both registers the DAG dependency (so the skill file is deployed before the agent is created) and derives the correct stage path from the skill model's stage config automatically — no hard-coded paths or variables needed:

models/my_agent.sql:

{{
  config(materialized = 'cortex_agent')
}}
models:
  orchestration: claude-4-sonnet
instructions:
  response: "Be concise."
  orchestration: "Use the forecaster skill to answer forecasting questions."
skills:
  - name: forecaster
    source:
      type: STAGE
      path: "{{ dbt_cortex_agent.cortex_skill_path(ref('forecaster_skill')) }}"

dbt build will deploy forecaster_skill first (writing SKILL.md to the stage), then create or replace the agent with the resolved path in the spec.

cortex_skill configuration reference

Config Required Type Description
stage Yes string Fully-qualified stage path, e.g. @my_db.my_schema.skill_stage.

Standard dbt configs (database, schema, alias, tags, pre_hook, post_hook, …) work as usual. The model alias becomes the skill folder name on the stage.

Notes.

  • The stage is created automatically with CREATE STAGE IF NOT EXISTS if it does not already exist.
  • The SKILL.md content must not contain $$ (used as the SQL dollar-quote delimiter internally).
  • To remove a deployed skill file, run REMOVE @<stage>/skills/<name>/SKILL.md in Snowflake directly.


cortex_mcp_server — External MCP servers

The cortex_mcp_server materialization creates a Snowflake External MCP Server object (CREATE EXTERNAL MCP SERVER) from a config-only dbt model. Once created, the MCP server can be wired into a cortex_agent model with ref() so that the DAG enforces correct build order.

Bootstrap: create the API integration first

An External MCP Server references a Snowflake API INTEGRATION object that authenticates Snowflake's outbound calls to the MCP endpoint. API integrations are account-level objects that require ACCOUNTADMIN (or CREATE INTEGRATION) privilege to create — they cannot be created by a typical dbt service account during dbt build.

Run the create_mcp_api_integration operation once per MCP endpoint before dbt build, using an admin-privileged role:

# Dynamic Client Registration (recommended for DCR-capable providers, e.g. Atlassian):
dbt run-operation create_mcp_api_integration --args '{
  integration_name: jira_mcp_api_integration,
  allowed_prefixes: ["https://mcp.atlassian.com"],
  auth_type: OAUTH_DYNAMIC_CLIENT,
  oauth_resource_url: "https://mcp.atlassian.com/v1/mcp"
}'

# OAuth2 client credentials (for providers without DCR):
dbt run-operation create_mcp_api_integration --args '{
  integration_name: my_mcp_api_integration,
  allowed_prefixes: ["https://api.example.com/mcp"],
  auth_type: OAUTH2,
  oauth_client_id: "abc123",
  oauth_client_secret: "s3cr3t",
  oauth_token_endpoint: "https://api.example.com/oauth/token",
  oauth_authorization_endpoint: "https://api.example.com/oauth/authorize"
}'

Use dry_run=true to preview the DDL without executing it:

dbt run-operation create_mcp_api_integration --args '{
  integration_name: jira_mcp_api_integration,
  allowed_prefixes: ["https://mcp.atlassian.com"],
  auth_type: OAUTH_DYNAMIC_CLIENT,
  oauth_resource_url: "https://mcp.atlassian.com/v1/mcp",
  dry_run: true
}'

If the API integration does not exist when dbt build runs, the materialization will fail immediately with a clear error message that names the missing integration and shows the bootstrap command to run.

Defining an MCP server model

The model body is empty — all parameters are supplied via config():

models/atlassian_mcp_server.sql:

{{
  config(
    materialized    = 'cortex_mcp_server',
    display_name    = 'Atlassian (Jira & Confluence)',
    url             = 'https://mcp.atlassian.com/v1/mcp',
    api_integration = 'jira_mcp_api_integration'
  )
}}

Wiring an MCP server to an agent

Use the cortex_mcp_server_name() macro with ref() to wire the server into an agent model body. This both registers the DAG dependency (the agent will not be created until the MCP server object exists) and derives the correct database.schema.name automatically:

models/my_agent.sql:

{{
  config(materialized = 'cortex_agent')
}}
models:
  orchestration: claude-4-sonnet
instructions:
  response: "Be concise."
  orchestration: "Use the Atlassian MCP server for Jira and Confluence questions."
mcp_servers:
  - server_spec:
      name: "{{ dbt_cortex_agent.cortex_mcp_server_name(ref('atlassian_mcp_server')) }}"

cortex_mcp_server configuration reference

Config Required Type Description
display_name Yes string Human-readable label shown in Snowflake.
url Yes string MCP server endpoint URL.
api_integration Yes string Name of the pre-existing Snowflake API integration object.

create_mcp_api_integration operation reference

Parameter Required Type Description
integration_name Yes string Snowflake object name for the API integration.
allowed_prefixes Yes list[string] Base URL(s) of the MCP server, matched as a prefix.
auth_type No (default OAUTH_DYNAMIC_CLIENT) string OAUTH_DYNAMIC_CLIENT or OAUTH2.
oauth_resource_url Yes (OAUTH_DYNAMIC_CLIENT) string MCP server URL used for DCR.
oauth_client_id Yes (OAUTH2) string OAuth2 client ID.
oauth_client_secret Yes (OAUTH2) string OAuth2 client secret.
oauth_token_endpoint Yes (OAUTH2) string OAuth2 token endpoint URL.
oauth_authorization_endpoint Yes (OAUTH2) string OAuth2 authorization endpoint URL.
oauth_client_auth_method No (OAUTH2 only) string CLIENT_SECRET_BASIC or CLIENT_SECRET_POST.
oauth_discovery_url No (OAUTH2 only) string OIDC discovery URL.
oauth_refresh_token_validity No (OAUTH2 only) int Refresh token validity in seconds.
enabled No (default true) bool Whether the integration is enabled.
if_not_exists No (default false) bool Use IF NOT EXISTS instead of OR REPLACE.
dry_run No (default false) bool Log DDL without executing.
comment No string Optional COMMENT clause.

Privilege note. create_mcp_api_integration requires ACCOUNTADMIN or the CREATE INTEGRATION account-level privilege. This is a one-time admin operation; normal dbt runs do not need elevated privileges once the integration exists.


cortex_agent configuration reference

Config Mode Type Description
comment specification string Sets the agent-level COMMENT clause. Single quotes are escaped automatically.
profile specification dict or string Sets the PROFILE clause. A dict is serialized to JSON for you (display_name, avatar, color); a string is used verbatim.
web_search_tool specification bool (default false) When true, injects a tool_spec entry for web search into the agent specification YAML, enabling live web search for the agent. If your spec already has a tools: block, add the entry there directly instead.
raw_ddl both bool (default false) When true, the model body is treated as raw DDL appended after CREATE OR REPLACE AGENT <name>, and comment / profile / web_search_tool configs are ignored (a compile-time warning is emitted if any of these are set).
versioning specification bool (default false) Master switch for named-version mode. When true, uses ADD VERSION DDL instead of CREATE OR REPLACE. Incompatible with raw_ddl=true (a warning is emitted and the run falls back to CREATE OR REPLACE).
version_name specification (versioning=true) string Name of the version to create. When omitted, auto-generated as v_YYYYMMDD_HHMMSS from run_started_at — deterministic within a run and safe as a Snowflake identifier.
set_default specification (versioning=true) bool (default true) When true, flips the agent's DEFAULT VERSION to this version after creating it. Set false to create a staging/canary version without affecting live traffic.

Standard dbt configs (database, schema, alias, tags, pre_hook, post_hook, grants, enabled, …) all work as usual. The agent is created in the model's target database/schema with the model's alias as its name.


How it works

  • cortex_skill materialization (macros/materializations/cortex_skill.sql) — runs pre-hooks, creates the stage if needed, issues PUT 'file://dir/*' @stage/skills/<name>/ to upload all skill files, runs post-hooks, and returns the relation.
  • cortex_agent materialization (macros/materializations/cortex_agent.sql) — sets the query tag, runs pre-hooks, issues a single CREATE OR REPLACE AGENT statement, runs post-hooks, and returns the relation.
  • DDL builder (macros/relations/cortex_agent/create.sql) — constructs the statement for both specification and raw modes.
  • Drop / rename (macros/relations/cortex_agent/{drop,rename}.sql) — provide drop agent if exists and alter agent ... rename to DDL.

Every run issues CREATE OR REPLACE AGENT, which is idempotent and atomic, so re-running a model simply replaces the agent in place.


Limitations & notes

  • Relation type. Snowflake Agents are not yet a first-class dbt relation type, so the node is tracked internally as a view for graph/lineage purposes only. dbt never issues CREATE VIEW for it — the materialization only ever runs CREATE OR REPLACE AGENT.
  • persist_docs is not supported. Use the inline comment config (or COMMENT clause in raw mode) instead. This mirrors the dbt_semantic_view package's behavior for the same underlying reason.
  • Name collisions. CREATE OR REPLACE AGENT fails if a non-agent object of the same name already exists in the schema. Choose a name/alias that does not collide with an existing table or view.
  • Privileges. The executing role needs the privileges to create agents (e.g. CREATE AGENT on the schema) and to reference any semantic views or Cortex Search services named in tool_resources. See the Cortex Agents docs.

Integration tests

A runnable integration-test project lives in integration_tests/. See integration_tests/README.md for setup (Snowflake env vars, dbt deps, dbt build).


References

License

MIT License. See LICENSE.

About

Materialize Snowflake Cortex Agents from dbt

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages