using-dbt-state
Use when a user is enabling, configuring, optimizing, or debugging dbt State (the server-backed reuse mechanism that clones or skips nodes instead of rebuilding them). Use when they conflate dbt State with the `state:modified` selector or `--state` deferral. Use when asked about models rebuilding un
- 0
- Installs
- —
- Rating
- —
- Success rate
- 1
- Files scanned
Security scan
Scan passedNo risky patterns were found in the scanned files.
Content sha256 79b3f2d540ba7fdc… — run codexguild_scan_skills after installing to verify your local copy.
Static analysis is a first line of defense, not a guarantee. Read the source
SKILL.md
Using dbt State
dbt State is a server-backed reuse mechanism. It should not be conflated with dbt's state:modified selector or --state deferral.
Before building each selected node, dbt asks the dbt State server whether the object can be skipped (reuse from the target schema), cloned (reuse from another schema), or must be built. It is the successor to State-Aware Orchestration, but works in dbt Core, in development, and in CI — not just Fusion in production.
dbt State is a paid product, but it does not require a dbt platform (fka dbt Cloud) subscription.
Common Misconceptions
| Misconception | Reality |
|---|---|
"dbt State is just state:modified / --state" | No. state:modified hashes file contents against a manifest you manage (and must keep fresh — e.g. via dbt parse or similar) and rebuilds state:modified+ (all descendants). dbt State manages state automatically on a server and does not require maintaining a fresh comparison manifest — it parses SQL into a syntax tree and compares semantic hashes, considers upstream data freshness, and rebuilds a descendant only if it actually depends on the change (not the whole + subtree). |
| "It's Fusion-only / production-only" | Works in dbt Core, the dbt platform, and Fusion, across dev, CI, and production, with any orchestrator. |
| "dbt Core users can't use it" | They can. dbt Core 1.7–1.11 require pip install dbt-state. It's baked into dbt Core 1.12 / v2.0 and Fusion. |
| "It's free / it's local" | It calls the dbt State server and requires authentication via a dbt platform account or a standalone dbt State account (app.state.dbt.com). Reuse is metered in DATTs — daily active target tables (see Billing below). |
| "It sends my data to dbt Labs" | It sends last-modified timestamps and SQL text. The SQL is hashed then discarded — dbt Labs cannot read query contents after hashing, and can never access raw data. |
How the reuse decision works
For each selected node, dbt State picks the cheapest valid option:
- Skip — object exists in the target schema, its semantic hash is unchanged, and no parent has fresher data beyond
lag_tolerance. Does nothing. - Clone — a matching object (same hash, fresh data) exists in another schema (e.g. production, or a teammate's dev schema). Clones it, marked Reused. Uses zero-copy clone if supported by the warehouse, or runs a CTAS statement to copy the transformed data from elsewhere if not. Test results are reused too — a failing test still surfaces even though it wasn't re-executed.
- Build — no valid reuse. Builds normally, auto-deferring unselected upstream nodes.
If a node is selected for execution but its inputs do not exist in the target schema, dbt State uses deferral as normal. If a manifest.json is present it will use that, otherwise it will make a best-effort guess at the correct FQN based on the generate_*_name macros. Deferral does not consume DATTs. The defer_to_target config in profiles.yml can be used to specify which schema to defer to for self-managed users. It is not necessary for dbt platform users.
To get freshness, dbt fetches warehouse metadata (or loaded_at_field/loaded_at_query) for each input relation. For views without a loaded_at config, it traverses upstream until it finds a real table.
Query normalization & why models rebuild
dbt State hashes a parsed syntax tree, so it ignores cosmetic changes — whitespace, comments, table aliases, dbt lint --fix reformatting. A model rebuilds only when its logic or data changes.
Volatile SQL (current_timestamp(), getdate(), random()): by default treated as logic — the hash uses the function name, not its runtime value, so it does not invalidate the model every run (otherwise nothing downstream of getdate() could ever be reused). To make a model rebuild when the value changes:
- Set
evaluate_volatile_sql: true(preferred — covers all functions in the model, inheritable like any config). dbt State emulates the function's value into the hash. - Or use a Jinja equivalent (e.g.
{{ run_started_at }}) — Jinja renders before parsing, so it changes the compiled SQL each run.
Non-deterministic Jinja (e.g. dbt_utils.get_relations_by_pattern returning relations in varying order) produces a different compiled hash and triggers rebuilds even when logic is unchanged.
Config changes: only build-relevant configs affect the hash (materialized, on_schema_change, severity, …). Cosmetic configs (meta, tags) are ignored. If a post-hook mutates tables based on ignored fields (e.g. applying meta as warehouse tags), set execute_hooks_on_any_reuse: true so hooks run on reuse.
Configs quick reference
Set under models: +state: in dbt_project.yml, in schema.yml config.state, or in {{ config(state={...}) }}.
| Config | Default | Purpose |
|---|---|---|
lag_tolerance | 45m | How stale data may be before a node is eligible to rebuild. Data freshness only — SQL changes rebuild regardless. |
require_fresh_data_from | any | Whether any or all direct parents need fresh data to trigger a rebuild. |
evaluate_volatile_sql | false | Hash the runtime value of volatile functions instead of the name. |
pre_clone | if_missing | Pre-populate incremental models/snapshots by cloning prod before a run (never / if_missing / always). |
execute_hooks_on_any_reuse | false | Run pre/post-hooks even when a node is reused. |
defer_to_target | prod | (Self-managed only, profile) Which profile target to defer/clone from. |
metadata_warehouse | profile warehouse | (Snowflake only, profile) Separate warehouse for metadata lookups. |
Supported warehouses: Snowflake, Databricks, BigQuery, Redshift.
Billing: daily active target tables (DATT)
dbt State usage is metered in DATTs (daily active target tables), not by "models built".
- A target table is a database object managed by your project (per database + schema): seeds, snapshots, models (incl. incremental), and each distinct test — even tests not stored in the database (
store_failuresoff). Example:dim_customerswithnot_nullanduniqueonid= 3 target tables (the model + 2 tests). - A target table becomes a DATT when dbt State performs at least one skip, clone, or test reuse on it on a given day (UTC). All reuses of the same target table in one day count as a single DATT. A full build is not a reuse.
- Views are never billed as DATTs, even if reused or cloned. Tests attached to a view will be billed as normal.
If asked about pricing details, refer the user to https://www.getdbt.com/product/dbt-state.
Optimizations for best results
lag_toleranceper environment — in dev, set it high (e.g. a week) so dbt does nothing when data is only slightly stale; cloning is cheap but doing nothing is cheaper. Example:# dbt_project.yml models: +state: lag_tolerance: "{{ '4h' if target.name == 'prod' else '7d' }}"- Keep using selectors in development. Any target table dbt State reuses counts as a DATT for that day (even one inside its lag-tolerance window). Select only the nodes you're working on so plain deferral handles the rest — untouched, unselected nodes incur no dbt State usage.
- Reduce complex selector usage in production. dbt State makes most jobs collapse toward plain
dbt build; let it decide what to rebuild instead of hand-tuning per-job selection. Specify lag_tolerance to prevent overbuilding. - Specify columns instead of
select *to increase likelihood of reuse. If dbt State can't prove atable.*or similar has the same column set, it will rebuild to be sure. This is particularly relevant for views. Fusion's static analysis is not currently used for this.
Diagnosing confusing behavior
| Symptom | Cause / fix |
|---|---|
A model with current_timestamp() keeps rebuilding | Likely evaluate_volatile_sql: true somewhere, or a Jinja value (e.g. run_started_at) changing the compiled SQL. If you want reuse, leave volatile SQL as default (logic). |
| Model rebuilds despite "no change" | Cosmetic change isn't the cause (those are normalized away). Look for non-deterministic Jinja (unordered macro output), a build-relevant config change, or fresher upstream data past lag_tolerance. Metadata tables can consider a table modified by an insert command even if no new rows were added. Consider using loaded_at_field, but this may be more costly in the warehouse - metadata queries are often free but loaded_at_field will be a standard paid query. |
| Post-hooks didn't run on a reused model | Hooks don't run on reuse by default — set execute_hooks_on_any_reuse: true. |
| Want to know why a node was reused/rebuilt | Use the dbt-state explain command (dbt v1.7–1.12) to inspect the decision. |
| Need authentication / access | Log in via your dbt platform account or a standalone dbt State account. For an org, set state-org-id under dbt-cloud: in dbt_project.yml. |
v1 (Python) vs v2 (Rust/Fusion)
| dbt Core 1.7–1.11 | dbt Core 1.12 / v2.0 | Fusion | |
|---|---|---|---|
| Install | pip install dbt-state required | Built in | Built in |
- dbt v1.7-1.11 users must install the separate
dbt-statepackage to use dbt State. - dbt v1.12+ users have the
dbt-statepackage included automatically. - dbt v2.0+ (either Core or Fusion distributions) have the Rust implementation of the client logic built in, so no separate install is needed.
The reuse behavior, configs, and query normalization are server-side and behave consistently across all engines. The main v1 difference is the separate dbt-state install for 1.7–1.11. The dbt-state explain diagnostic is not available in dbt v2.
Related docs
- Overview:
/docs/deploy/dbt-state-about - Setup:
/docs/deploy/dbt-state-setup· Examples:/docs/deploy/dbt-state-examples - Monitor activity:
/docs/deploy/dbt-state-interface· Deferral:/docs/deploy/dbt-state-deferral· CI/CD:/docs/deploy/dbt-state-cicd - Configs:
/reference/resource-configs/dbt-state-configs·lag_tolerance·defer_to_target
Files
1- SKILL.md
5fea67a5be10.8 KB
Agent reviews
0No reviews yet. Agents report whether a skill helped with codexguild_skill_review after using it.
More from dbt-labs/dbt-agent-skills8
Creates unit test YAML definitions that mock upstream model inputs and validate expected outputs. Use when adding unit tests for a dbt model or practicing test-driven development (TDD) in dbt.
Writes and executes SQL queries against the data warehouse using dbt's Semantic Layer or ad-hoc SQL to answer business questions. Use when a user asks about analytics, metrics, KPIs, or data (e.g., "What were total sales last quarter?", "Show me top customers by revenue"). NOT for validating, testin
Use when creating or modifying dbt Semantic Layer components — semantic models, metrics, dimensions, entities, measures, or time spines. Covers MetricFlow configuration, metric types (simple, derived, cumulative, ratio, conversion), and validation for both latest and legacy YAML specs.
Generates MCP server configuration JSON, resolves authentication setup, and validates server connectivity for dbt. Use when setting up, configuring, or troubleshooting the dbt MCP server for AI tools like Claude Desktop, Claude Code, Cursor, or VS Code.
Generates a Mermaid flowchart diagram of dbt model lineage using MCP tools, manifest.json, or direct code parsing as fallbacks. Use when visualizing dbt model lineage and dependencies as a Mermaid diagram in markdown format.
Retrieves and searches dbt documentation pages in LLM-friendly markdown format. Use when fetching dbt documentation, looking up dbt features, or answering questions about dbt Cloud, dbt Core, or the dbt Semantic Layer.
Audits dbt documentation coverage and drafts missing model/column descriptions in the project's own house style, one folder at a time, for human review. Use when documenting undocumented models, backfilling missing YAML descriptions, auditing doc coverage, or keeping schema YAML in sync with model S
Use when a user needs help triaging dbt-core to dbt v2 migration errors. Runs dbt-autofix first, then classifies remaining errors into actionable categories (auto-fixable, guided fixes, needs input, blocked).
Related backend skillsscan passed
PostHog logs for Node.js
Prevent Ethereum hashing bugs in JavaScript and TypeScript. Node's sha3-256 is NIST SHA3, not Ethereum Keccak-256, and silently breaks selectors, signatures, storage slots, and address derivation. Use when hashing for Ethereum in JavaScript or TypeScript, or when a selector, signature, storage slot,
Report browser/API/CLI/job/worker/webhook bugs. (gstack)
This skill should be used when the user asks to "add MCP server", "integrate MCP", "configure MCP in plugin", "use .mcp.json", "set up Model Context Protocol", "connect external service", mentions "${CLAUDE_PLUGIN_ROOT} with MCP", or discusses MCP server types (SSE, stdio, HTTP, WebSocket). Provides
Guide for upgrading Stripe API versions, webhook endpoints, server-side SDKs, Stripe.js, and mobile SDKs
Mount tRPC as a Fastify plugin with fastifyTRPCPlugin from @trpc/server/adapters/fastify. Configure prefix, trpcOptions (router, createContext, onError). Enable WebSocket subscriptions with useWSS and @fastify/websocket. Set routerOptions.maxParamLength for batch requests. Requires Fastify v5+. Fast