Migrating from dbt
Most teams adopting Rocky have a dbt project today, so the day-one question is “how much rewriting?” The answer: little to none. Run rocky import-dbt against your existing repo, get a Rocky project on disk in seconds, and adopt the trust primitives (typed compile, contracts, column-level lineage, branches, cost) incrementally.
The wedge in five steps:
- Run
rocky import-dbt. Jinja{{ ref() }}and{{ source() }}resolve to bare references; configs become TOML sidecars; the importer writesMIGRATION-NOTES.mdlisting anything that didn’t translate. - Run
rocky compile. First time through, expect real diagnostics:E011on type mismatches,P002onSELECT *blast radius,P001on dialect-portability issues. Each one is something dbt Core couldn’t catch. - Add contracts on the boundary models.
[contract] required_columns = […],protected_columns = […]. From here, the column rename that quietly breaks 47 downstream models becomes anE010in CI before it ships. - Adopt
rocky lineage-diffin PR review. Per-changed-column downstream blast radius. Drops into a PR comment. This is the moment your team stops reviewing changes blind. - Turn on
rocky preview cost. Per-PR cost projection: catch expensive plans before they ship instead of explaining them after.
Prerequisites
Section titled “Prerequisites”Before starting, make sure you have:
- Rocky installed – see Installation
- An existing dbt project with models in a
models/directory - Access to your warehouse credentials (Databricks host, HTTP path, token)
Rocky does not require dbt to be installed. The importer reads .sql files directly and parses Jinja expressions with its own regex-based extractor.
Walkthrough: end-to-end against a tiny dbt project
Section titled “Walkthrough: end-to-end against a tiny dbt project”This walks the full path against a real, runnable example, mirroring the POC at examples/playground/pocs/06-developer-experience/03-import-dbt-validate/. Every command and snippet here was captured from that POC running against the current rocky build.
Setup: the input dbt project
Section titled “Setup: the input dbt project”The POC ships a minimal dbt project with two models, one source, and a schema.yml of generic tests:
dbt_project/├── dbt_project.yml # name: ecommerce, profile: ecommerce, +materialized: table└── models/ ├── sources.yml # source 'raw' / table 'orders' ├── schema.yml # generic tests: unique, not_null, accepted_values, relationships, dbt_utils.accepted_range ├── stg_orders.sql # {{ config(materialized='view') }} + {{ source('raw', 'orders') }} └── fct_revenue.sql # {{ config(materialized='table') }} + {{ ref('stg_orders') }}stg_orders.sql:
{{ config(materialized='view') }}
SELECT order_id, customer_id, amount, LOWER(status) AS statusFROM {{ source('raw', 'orders') }}WHERE status != 'cancelled'fct_revenue.sql:
{{ config(materialized='table') }}
SELECT customer_id, SUM(amount) AS total_revenue, COUNT(*) AS order_countFROM {{ ref('stg_orders') }}GROUP BY customer_idThere is no profiles.yml in the POC and no compiled target/manifest.json, so this exercises the regex-based importer with no warehouse credentials.
Run the importer
Section titled “Run the importer”rocky import-dbt \ --dbt-project dbt_project \ --output-dir imported \ --no-manifest \ --overwriteOutput (table mode):
dbt Migration Report====================
Project: ecommerceMethod: regex
Models: 2 total 2 imported successfully (view: 1, full_refresh: 1)
Sources: 1 tables from 1 sources 1 mapped to Rocky
Tests: 8 total 8 converted to contracts (1 of them composite / multi-column)
Next Steps: 1. rocky compile 2. rocky ai explain --all --save 3. rocky testOutput: 2 models translated, 0 seeds copied → imported rocky.toml → imported/rocky.toml MIGRATION-NOTES.md → imported/MIGRATION-NOTES.mdWhat gets emitted
Section titled “What gets emitted”The importer produces a self-contained Rocky repo on disk. The full layout:
imported/├── MIGRATION-NOTES.md├── rocky.toml└── models/ ├── _defaults.toml ├── stg_orders.sql ├── stg_orders.toml ├── fct_revenue.sql └── fct_revenue.tomlimported/rocky.toml (the importer wrote a DuckDB stub because no profiles.yml was found):
# rocky.toml — generated by `rocky import-dbt`# Connection fields use ${VAR} env-var substitution. Set the env vars# listed in MIGRATION-NOTES.md before running `rocky run`.# Default per-model target: catalog=warehouse, schema=main (see models/_defaults.toml).
[adapter]type = "duckdb"path = "warehouse.duckdb"
[pipeline.default]type = "transformation"models = "models/**"
[pipeline.default.target]adapter = "default"imported/models/_defaults.toml (directory-level target defaults):
[target]catalog = "warehouse"schema = "main"imported/models/stg_orders.sql (Jinja resolved to bare references):
SELECT order_id, customer_id, amount, LOWER(status) AS statusFROM raw.ordersWHERE status != 'cancelled'imported/models/stg_orders.toml (the view materialization maps to a native Rocky view strategy; the model description becomes intent, and the schema.yml generic tests become [[tests]] blocks):
name = "stg_orders"intent = "Staged orders, one row per order, with a normalised status."
[strategy]type = "view"
[target]catalog = "warehouse"schema = "main"table = "stg_orders"
[[sources]]catalog = "warehouse"schema = "raw"table = "orders"
[[tests]]type = "unique"column = "order_id"
[[tests]]type = "not_null"column = "order_id"
[[tests]]type = "not_null"column = "customer_id"
[[tests]]type = "accepted_values"values = ["pending", "completed", "shipped"]column = "status"imported/models/fct_revenue.toml:
name = "fct_revenue"intent = "Per-customer revenue rollup."
[strategy]type = "full_refresh"
[target]catalog = "warehouse"schema = "main"table = "fct_revenue"
[[tests]]type = "unique"column = "customer_id"
[[tests]]type = "not_null"column = "customer_id"
[[tests]]type = "relationships"to_table = "warehouse.main.stg_orders"to_column = "customer_id"column = "customer_id"
[[tests]]type = "in_range"min = "0"column = "total_revenue"imported/MIGRATION-NOTES.md is the canonical record of what didn’t translate: counts of skipped tests and macros, required env vars per adapter, and the explicit “Known limitations” list. Read it first.
Verify the emitted repo loads
Section titled “Verify the emitted repo loads”The cheapest end-to-end check is to compile against the new repo:
cd importedrocky compile --models models ✓ stg_orders (4 columns) ✓ fct_revenue (3 columns) Compiled: 2 models, 0 errors, 0 warningsValidate the generated rocky.toml:
rocky -c rocky.toml validate ok Config syntax valid (v2 format) ok adapter.default: duckdb (local) ok pipeline.default: transformation / models='models/**' ok 2 transformation models loaded ok DAG valid (2 nodes, no cycles)
Validation complete.A clean rocky compile + rocky validate is the success criterion. The POC’s run.sh stops here and then runs rocky validate-migration as an orthogonal cross-check that every dbt model has a matching Rocky model.
Running the emitted repo against real data
Section titled “Running the emitted repo against real data”rocky -c rocky.toml plan followed by rocky apply <plan-id> will work once source data exists in the warehouse — the one precondition the importer cannot supply for you. The dbt project references {{ source('raw', 'orders') }}; the importer translates that to FROM raw.orders but does not create or populate the source. Load the source rows into the configured warehouse (warehouse.duckdb for the DuckDB stub, or your real Databricks/Snowflake target) before invoking rocky apply.
What translates cleanly today, and what doesn’t
Section titled “What translates cleanly today, and what doesn’t”What the importer translates cleanly:
{{ ref('model') }}→ bare table reference + sidecardepends_on{{ source('s', 't') }}→ fully qualified reference + sidecar[[sources]]{{ config(materialized='table' \| 'incremental' \| 'view') }}→ sidecar[strategy]block (viewmaps to a native Rockyviewstrategy){{ config(unique_key=...) }}→mergestrategy withunique_keyarray{{ config(alias='name') }}→ the sidecar[target].table— the output relation — so the data lands in the aliased table rather than one named after the node (dropping this silently mis-routes data){{ config(materialized='microbatch') }}→ a Rockymerge(default) ortime_intervalstrategy, selected by--microbatch-as <merge|time_interval>.mergereuses the dbtunique_keyfor an idempotent key-upsert (dbt microbatch’s partition-replace becomes a key upsert);time_intervalmaps the batch onto Rocky’s partition-window model. Either way the choice is recorded inMIGRATION-NOTES.mdso you can review it.{{ config(merge_update_columns=[...]) }}→ themergestrategy’supdate_columns{{ var('name') }}/{{ var('name', default) }}→ an@var(name)/@var(name, default)run-variable marker left in the emitted SQL, resolved at run time byrocky run --var name=value(see Handle unsupported Jinja)- dbt tags (node- and folder-level) → the sidecar
[tags]block (<tag> = "true") {{ this }}→ the model’s own fully-qualifiedcatalog.schema.tableis_incremental()branches → stripped (Rocky derives the watermark filter from[strategy])- dbt generic tests (
unique,not_null,accepted_values,relationships) are translated column-by-column to[[tests]]blocks — including the configured forms that carryseverity:(awarnbecomes a Rocky warning, not a hard error) andwhere:(a row filter) (see Generic test mapping below) - model-level
dbt_utils.unique_combination_of_columns→ a Rockycompositeuniqueness[[tests]]block over the same column tuple (the columns come from the test config, so no model schema is needed) - Top-level
dbt_project.yml, used to detect project name and seeds path <dbt_project>/seeds/→ copied verbatim into<out>/seeds/profiles.ymladapter type → mapped to a Rocky[adapter]block (DuckDB / Databricks / Snowflake / BigQuery), or a DuckDB stub when absent or unrecognised. The parser resolves YAML anchors/aliases (&anchor/*alias) and{{ env_var('VAR', 'default') }}in thetypefield, so a profile that templates its adapter type now detects the right warehouse instead of silently falling back to the DuckDB stub.
By design, the importer does not translate the following. Rocky has no Jinja runtime, so these need a manual pass. Each item is detected and listed under “Known limitations” in MIGRATION-NOTES.md, with # TODO: dbt-jinja-not-translated comments above any leftover Jinja in emitted SQL:
- dbt tests with no native Rocky equivalent. Beyond the canonical four, the importer now converts several
dbt_utils/dbt_expectationstests to native Rocky assertions —unique_combination_of_columns,accepted_range/expect_column_values_to_be_between(→in_range),expect_column_values_to_match_regex(→regex_match),expect_column_values_to_be_in_set(→accepted_values), anddbt_utils.expression_is_true(→expression); see Generic test mapping. Everything still outside that set — otherdbt_utils.*/dbt_expectations.*tests, project-defined generics, and other model-level tests — is surfaced as a structuredUnsupportedTestwarning per occurrence and not stubbed in the emitted TOML. Rewrite those as a Rockyexpressiontest or a quality-pipeline check. - Singular tests in
tests/(custom SQL): copy and rewrite manually. - dbt macros and
dbt_packages/. Rocky has no Jinja runtime, so macro bodies do not expand. {% for %}/{% set %}outsideis_incremental()on the no-manifest path: refused — the model is listed as a failure rather than half-rendered into broken SQL (the loop/assignment body would survive exactly once). Re-run afterdbt compile(the manifest path resolves them) or rewrite the model. A{% if %}is still emitted verbatim with a TODO marker — its body applies unconditionally, so review it. ({{ var() }}is not in this list: it converts to an@var()run-variable marker — see above.)- Unmapped
materializedvalues (dynamic_table,seed): flattened tofull_refreshand listed inMIGRATION-NOTES.md. (materialized_viewmaps natively to Rocky’smaterialized_viewstrategy.) - Adapters Rocky does not natively support (e.g. Postgres, Redshift): the generated repo stubs DuckDB so the project still loads. Replace the
[adapter]block once a Rocky adapter for the warehouse exists, or pass--target-adapter <kind>to skip detection. - Custom Jinja macros emitting SQL (e.g.
{{ generate_schema_name() }}, dynamicUNION ALLmacros): surfaced as failed models with the macro name in the reason. - Python dbt models (
.pyfiles): not SQL; rewrite manually. - Snapshots, MetricFlow metrics + semantic models, and exposures are not translated, but are now detected and counted —
constructs_droppedin the JSON output plus aDroppedConstructwarning each — so a migration is never silently lossy. - dbt model contracts (
contract: {enforced: true}, columndata_typedeclarations, andconstraints) are not carried over to Rocky’s contract model on import. They are detected and reported rather than dropped on the floor: each emits a warning and increments acontracts_droppedcounter in the JSON output andMIGRATION-NOTES.md, so you know which models had a contract to re-author. See Column-level contracts for writing the Rocky equivalent.
1. Import the dbt Project
Section titled “1. Import the dbt Project”Run rocky import-dbt pointing at your dbt project directory:
rocky import-dbt --dbt-project ./my-dbt-project --output-dir ./rocky-modelsThis scans my-dbt-project/models/ for .sql files and produces Rocky sidecar files in ./rocky-models/:
rocky-models/├── stg_orders.sql├── stg_orders.toml├── stg_customers.sql├── stg_customers.toml├── fct_orders.sql├── fct_orders.toml├── dim_customers.sql└── dim_customers.tomlWhat the importer converts
Section titled “What the importer converts”The importer handles these dbt patterns:
| dbt Pattern | Rocky Conversion |
|---|---|
{{ ref('model_name') }} |
Bare table reference (model_name) + depends_on in TOML |
{{ source('source_name', 'table') }} |
Fully qualified table reference (source_name.table) |
{{ config(materialized='incremental', unique_key='id') }} |
[strategy] section in TOML |
{{ this }} |
Target table reference from [target] in TOML |
schema.yml column tests (unique, not_null, accepted_values, relationships) |
[[tests]] blocks in the model sidecar TOML (see Section 9 below) |
JSON output
Section titled “JSON output”For programmatic use, request JSON via the global -o json flag:
rocky -o json import-dbt --dbt-project ./my-dbt-project --output-dir ./rocky-models{ "version": "<rocky-version>", "command": "import-dbt", "imported": 42, "warnings": 3, "failed": 2, "imported_models": ["stg_orders", "stg_customers", "fct_orders", "..."], "warning_details": [ { "model": "stg_payments", "category": "UnsupportedTest", "message": "dbt_expectations.expect_column_pair_values_A_to_be_greater_than_B has no native equivalent", "suggestion": "rewrite as a Rocky expression test or a quality-pipeline check" } ], "failed_details": [ { "name": "complex_macro_model", "reason": "unsupported Jinja: custom macro {{ generate_schema_name() }}" } ]}Manifest Fast Path
Section titled “Manifest Fast Path”If your dbt project has a compiled manifest (target/manifest.json), Rocky uses it automatically for a more accurate import. All Jinja is pre-resolved in the compiled SQL.
To force or skip the manifest:
--manifest path/to/manifest.json: explicit manifest path--no-manifest: skip manifest, use regex-based import
2. Review the Imported Models
Section titled “2. Review the Imported Models”After import, review each generated model pair. Here is what a typical conversion looks like.
Before (dbt)
Section titled “Before (dbt)”-- models/stg_orders.sql{{ config(materialized='incremental', unique_key='order_id') }}
SELECT order_id, customer_id, order_date, total_amount, _fivetran_syncedFROM {{ source('shopify', 'orders') }}
{% if is_incremental() %}WHERE _fivetran_synced > (SELECT MAX(_fivetran_synced) FROM {{ this }}){% endif %}After (Rocky)
Section titled “After (Rocky)”stg_orders.sql:
SELECT order_id, customer_id, order_date, total_amount, _fivetran_syncedFROM shopify.ordersstg_orders.toml:
name = "stg_orders"
[strategy]type = "merge"unique_key = ["order_id"]
[target]catalog = "warehouse"schema = "main"table = "stg_orders"
[[sources]]catalog = "warehouse"schema = "shopify"table = "orders"Notice that the {{ config() }} block became [strategy] and {{ source() }} became a fully qualified reference. A config(unique_key=...) with no explicit incremental_strategy maps to Rocky’s merge strategy keyed on unique_key (not a bare incremental block). The [[sources]] block and its qualified coordinates come from the sources.yml definition for shopify.orders; without a matching sources.yml entry the importer emits a warning and no [[sources]] block. The is_incremental() guard and {{ this }} are gone: Rocky derives the merge logic from [strategy] and the target table from [target].
3. Handle Unsupported Jinja
Section titled “3. Handle Unsupported Jinja”The importer cannot convert all Jinja patterns. It produces warnings and failures for cases it cannot handle automatically.
Common warnings
Section titled “Common warnings”| Pattern | Importer Behavior | Manual Fix |
|---|---|---|
{{ var('some_var') }} |
Converted to an @var(some_var) run-variable marker in the emitted SQL (not a warning) |
Pass the value at run time with rocky run --var some_var=value, or give the marker an inline default: @var(some_var, fallback). A marker with neither a --var binding nor a default fails to compile. |
{% if target.name == 'prod' %} |
Emitted verbatim with a # TODO marker — the body applies unconditionally, so review it |
Remove environment branching or use separate rocky.toml files per environment |
{% set ... %} variable assignments |
Refused — the model is listed as a failure rather than half-rendered | Inline the value or refactor the query |
Common failures
Section titled “Common failures”| Pattern | Reason | Manual Fix |
|---|---|---|
Custom Jinja macros ({{ generate_schema_name() }}) |
Rocky cannot interpret custom macros | Rewrite the SQL without the macro |
{% for ... %} loops generating SQL |
Dynamic SQL generation not supported | Write out the SQL explicitly or use a CTE |
{% macro ... %} definitions |
Rocky uses pure SQL, not macros | Convert shared logic to CTEs or separate models |
Python dbt models (.py files) |
Not SQL | Rewrite in SQL |
For each failed model, check the error message and rewrite the SQL manually. Most Jinja macros exist to work around SQL limitations that Rocky handles differently (incremental logic, schema naming, environment branching).
generate_surrogate_key
Section titled “generate_surrogate_key”{{ dbt_utils.generate_surrogate_key([...]) }} is a common one. The importer does not auto-convert it: it surfaces as an UnsupportedMacro warning and the call is replaced with a /* TODO: unsupported macro */ marker in the emitted SQL. Rewrite it by hand as a [[surrogate_key]] block in the model sidecar. The block names the output column and the input columns; rocky run injects the hash column at materialization time:
[[surrogate_key]]name = "order_key"columns = ["order_id", "customer_id"]Drop the {{ ... }} expression from the model SQL and let the sidecar add the column. Rocky’s hash matches what dbt-utils produces on the same warehouse: each input is cast to text, NULL-coalesced to the same _dbt_utils_surrogate_key_null_ sentinel, joined with a - separator, and MD5-hashed. The expression is dialect-correct on DuckDB, Databricks, Snowflake, and BigQuery, so the hash values are identical to the dbt output for the matching warehouse.
4. Configure rocky.toml
Section titled “4. Configure rocky.toml”Create a rocky.toml in your project root. Rocky uses named adapters plus named pipelines. A replication pipeline needs two adapter roles: a data adapter that reads and writes table bytes (Databricks, Snowflake) and a discovery adapter that enumerates the source schemas to replicate (Fivetran, Airbyte). Databricks is data-only, so the replication source names it for data movement and points [source.discovery] at a discovery-capable adapter. If you were using dbt with Databricks fed by Fivetran, your settings map directly:
[adapter.prod]type = "databricks"host = "${DATABRICKS_HOST}"http_path = "${DATABRICKS_HTTP_PATH}"token = "${DATABRICKS_TOKEN}"
[adapter.fivetran]type = "fivetran"kind = "discovery"api_key = "${FIVETRAN_API_KEY}"api_secret = "${FIVETRAN_API_SECRET}"destination_id = "${FIVETRAN_DESTINATION_ID}"
[pipeline.bronze]type = "replication"strategy = "incremental"timestamp_column = "_fivetran_synced"
[pipeline.bronze.source]adapter = "prod"catalog = "raw_catalog"
[pipeline.bronze.source.discovery]adapter = "fivetran"
[pipeline.bronze.source.schema_pattern]prefix = ""separator = "__"components = ["source"]
[pipeline.bronze.target]adapter = "prod"catalog_template = "warehouse"schema_template = "staging"
[pipeline.bronze.execution]concurrency = 8
[state]backend = "local"Set the environment variables:
export DATABRICKS_HOST="your-workspace.cloud.databricks.com"export DATABRICKS_HTTP_PATH="/sql/1.0/warehouses/abc123"export DATABRICKS_TOKEN="dapi..."export FIVETRAN_API_KEY="..."export FIVETRAN_API_SECRET="..."export FIVETRAN_DESTINATION_ID="..."Mapping dbt config to Rocky
Section titled “Mapping dbt config to Rocky”dbt (profiles.yml / dbt_project.yml) |
Rocky (rocky.toml) |
|---|---|
host |
[adapter.prod] host |
http_path |
[adapter.prod] http_path |
token |
[adapter.prod] token |
catalog |
[pipeline.<name>.target] catalog_template |
schema |
[pipeline.<name>.target] schema_template |
threads |
[pipeline.<name>.execution] concurrency |
Folder-level config (+materialized, +schema)
Section titled “Folder-level config (+materialized, +schema)”dbt’s dbt_project.yml sets per-directory defaults like marts: +materialized: table and +schema: marts. Rocky’s equivalent is a config group: define the shared routing and strategy once in models/groups/<name>.toml, then have each member model opt in with group = "<name>" in its sidecar. The mapping is direct:
| dbt folder-level | Rocky config group (models/groups/<name>.toml) |
|---|---|
+materialized: table (or incremental, etc.) |
[strategy] block |
+schema: marts |
schema_template = "marts" (a literal is a template with no placeholders) |
schema_template = "mart_{region}"
[strategy]type = "merge"unique_key = ["id"]
[tags]domain = "finance"group = "daily_marts"
[args]region = "emea"The groups differ from dbt’s folder defaults in one way. dbt’s folder defaults apply automatically to every model in the directory; a Rocky config group applies only to models that name it with group = "<name>". Precedence is per-model sidecar over group over models/_defaults.toml, so a member can still override anything the group sets.
Rocky adds a knob dbt has no equivalent for. Set enforce = true on a group and a member model that locally pins a field the group controls (its target schema or its strategy) fails to load instead of silently diverging. Enforced groups are Rocky-only: they turn the group from an overridable default into a governance guarantee that the whole fan-out routes and materializes the same way.
5. Compile the Imported Models
Section titled “5. Compile the Imported Models”Run the compiler to type-check all imported models:
rocky compile --models ./rocky-modelsThe compiler will:
- Resolve
depends_onreferences into a DAG - Type-check column references across model boundaries
- Report type mismatches, contract violations, or missing dependencies
✓ stg_orders (5 columns) ✓ stg_customers (4 columns) ✓ fct_orders (7 columns) ✓ dim_customers (6 columns) Compiled: 4 models, 0 errors, 0 warningsA bare table reference the importer left unresolved (a name matching no model in the project) is not a compile error — Rocky classifies it as an external reference: it shows up in lineage but creates no DAG dependency (see Using Rocky with dbt Packages). The diagnostics you will hit after import come from the type checker and contracts:
- Missing depends_on: The importer may miss dependencies that were implicit in dbt (e.g., via
{{ ref() }}in a macro). Add them to the model’s TOML so the reference resolves to a project model instead of being treated as external. - Type mismatches (
E011): Rocky infers types from upstream models. If a column feeds an incompatible context, the compiler reports it. - Contract violations (
E010–E013): a missing required column, a wrong type, a nullability violation, or a removed protected column fails compilation against a.contract.toml(see Section 9).
6. Run Tests Locally
Section titled “6. Run Tests Locally”Once compilation passes, run local tests using DuckDB:
rocky test --models ./rocky-modelsTesting 4 models...
All 4 models passed
Result: 4 passed, 0 failedTests execute each model’s SQL against DuckDB in dependency order. This catches SQL syntax errors and runtime issues without needing a warehouse connection.
7. Validate the Migration
Section titled “7. Validate the Migration”Compare the dbt and Rocky outputs side by side:
rocky validate-migration --dbt-project ~/my-dbt-projectThis compiles both projects and compares schemas, column types, and optionally sample data.
8. Compare Output with dbt
Section titled “8. Compare Output with dbt”Before switching production traffic, run both tools side by side and compare outputs.
Preview Rocky’s SQL
Section titled “Preview Rocky’s SQL”rocky plan --filter tenant=acmeThis shows the SQL Rocky will generate for each model. Compare it against dbt compile output for the same models.
For a connection-free side-by-side, rocky emit-sql --models ./rocky-models renders the compiled SQL for every transformation model without a warehouse, the same shape dbt compile writes to target/. It also doubles as the exit door: the SQL it produces is plain runnable SQL you keep if you ever step away from the engine. See No lock-in for the full walkthrough.
Run on a test catalog
Section titled “Run on a test catalog”Add a test pipeline to your rocky.toml that points at a sandbox catalog and reuses the same adapters (the prod data adapter and the fivetran discovery adapter defined above):
[pipeline.bronze_test]type = "replication"strategy = "full_refresh"
[pipeline.bronze_test.source]adapter = "prod"
[pipeline.bronze_test.source.discovery]adapter = "fivetran"
[pipeline.bronze_test.source.schema_pattern]prefix = ""separator = "__"components = ["source"]
[pipeline.bronze_test.target]adapter = "prod"catalog_template = "test_warehouse"schema_template = "staging"Run the test pipeline:
plan_id=$(rocky plan --pipeline bronze_test --filter tenant=acme --output json | jq -r .plan_id)rocky apply "$plan_id"Then compare row counts, column types, and data values between the dbt-generated tables and Rocky-generated tables.
9. Convert dbt Tests to Rocky Tests and Contracts
Section titled “9. Convert dbt Tests to Rocky Tests and Contracts”rocky import-dbt translates two kinds of dbt tests onto Rocky sidecars:
- The four canonical column-level generic tests (
unique,not_null,accepted_values,relationships), plus a handful of commondbt_utils/dbt_expectationstests with a native Rocky equivalent, are emitted as[[tests]]blocks on each model sidecar. See Generic test mapping for the full list. - Unit tests from
manifest.unit_tests(dbt 1.8+) are emitted as[[test]]blocks on the matching model sidecar. Manifest-only; the regex path does not see unit tests.
Anything else (column-level type and nullability contracts, project-defined generics, singular tests) still needs a manual step.
Generic test mapping
Section titled “Generic test mapping”For these dbt tests in schema.yml (dbt 1.7+ also accepts data_tests:, which the importer reads as a synonym for tests:):
models: - name: fct_orders columns: - name: order_id tests: - unique - not_null - name: status tests: - accepted_values: values: ['completed', 'pending', 'cancelled'] - name: customer_id tests: - relationships: to: ref('dim_customers') field: customer_idThe importer emits [[tests]] blocks directly into models/fct_orders.toml:
[[tests]]type = "unique"column = "order_id"
[[tests]]type = "not_null"column = "order_id"
[[tests]]type = "accepted_values"values = ["completed", "pending", "cancelled"]column = "status"
[[tests]]type = "relationships"to_table = "warehouse.main.dim_customers"to_column = "customer_id"column = "customer_id"relationships.to: ref('m') resolves to the fully-qualified Rocky table via the importer’s name → (catalog, schema) lookup over the imported model set; cross-project refs fall back to the importer defaults. These tests run as part of rocky test against the materialised tables.
| dbt Test | Rocky [[tests]] |
|---|---|
not_null |
type = "not_null" + column |
unique |
type = "unique" + column |
accepted_values |
type = "accepted_values" + values = [...] + column |
relationships |
type = "relationships" + to_table + to_column + column |
dbt_utils.unique_combination_of_columns (model-level) |
type = "composite" + kind = "unique" + columns = [...] |
dbt_utils.accepted_range, dbt_expectations.expect_column_values_to_be_between |
type = "in_range" + min / max (at least one) + column (numeric bounds only) |
dbt_expectations.expect_column_values_to_match_regex |
type = "regex_match" + pattern + column |
dbt_expectations.expect_column_values_to_be_in_set |
type = "accepted_values" + values = [...] + column |
dbt_utils.expression_is_true |
type = "expression" + expression |
dbt_utils / dbt_expectations tests beyond the ones in the table above — plus project-defined generics and other model-level tests — are surfaced as an UnsupportedTest warning with the model, column, and test name. Rewrite those as a Rocky expression test or a quality-pipeline check; the importer does not stub them in the emitted TOML.
Consolidating repeated tests into named definitions
Section titled “Consolidating repeated tests into named definitions”The importer writes one inline [[tests]] block per column, so a not_null you apply across twelve models lands as twelve identical blocks. To get dbt’s generic-test parity (define a test once, apply it by name), define each test once in models/test_definitions.toml and reference it from each sidecar with [[use_test]]. This is a post-import authoring step, not a conversion the importer performs.
[positive_amount]type = "expression"expression = "amount > 0"
[known_status]type = "accepted_values"values = ["pending", "shipped", "delivered"][[use_test]]name = "positive_amount"column = "total_amount"
[[use_test]]name = "known_status"column = "status"test_definitions.toml is a table of named entries ([name], not an array of [[...]] blocks); each entry is the test type plus its parameters. A [[use_test]] reference binds the named test to a column at the use site and may override the column, severity, or row filter. An unknown name is a hard error at load. These resolve into the same [[tests]] the importer would emit inline, so they run under rocky test --declarative against the configured warehouse the same way.
dbt unit tests (manifest path)
Section titled “dbt unit tests (manifest path)”If your dbt project has compiled to a manifest.json and declares unit_tests: blocks (dbt 1.8+), rocky import-dbt --manifest target/manifest.json walks manifest.unit_tests and emits each entry as a [[test]] block in the matching model’s sidecar TOML. ref('upstream_model') / source('s', 't') wrappers on given.input are stripped to bare references.
unit_tests: - name: stamps_status_when_completed model: fct_orders given: - input: ref('stg_orders') rows: - { order_id: 1, status: 'completed' } expect: format: dict rows: - { order_id: 1, status: 'completed' }# Rocky: models/fct_orders.toml — emitted by `rocky import-dbt`[[test]]name = "stamps_status_when_completed"
[[test.given]]ref = "stg_orders"
[[test.given.rows]]order_id = 1status = "completed"
[test.expect]ordered = false
[[test.expect.rows]]order_id = 1status = "completed"The importer also surfaces three new counters on the --output json payload and in MIGRATION-NOTES.md (unit_tests_found, unit_tests_converted, unit_tests_skipped), plus two warning variants:
OrphanUnitTest: the unit test targets a model the importer didn’t pick up. Skipped and counted as skipped.UnsupportedUnitTestFormat:expect.format = "csv"or"sql", fixture references, or any other shape Rocky’sUnitTestDefdoesn’t yet model. Skipped.
CSV / SQL fixtures and overrides: blocks are deferred until Rocky’s runtime test runner grows the matching surface. Emitted given/expect [[test]] blocks now execute under rocky test as of engine-v1.52.0: the runner seeds a fresh in-memory DuckDB with each given fixture, materializes the model against it, and compares the output to expect (a multiset comparison by default, positional when expect.ordered is set).
Column-level contracts (manual)
Section titled “Column-level contracts (manual)”If you want compile-time guarantees on column types and nullability, beyond the row-level test runtime, add a .contract.toml alongside the model. Contracts are not autogenerated from dbt; write them for the models that need the extra rigour:
[[columns]]name = "order_id"type = "Int64"nullable = false
[[columns]]name = "customer_id"type = "Int64"nullable = false
[[columns]]name = "total_amount"type = "Decimal"nullable = false
[rules]required = ["order_id", "customer_id", "total_amount"]protected = ["order_id"]Compile with contracts
Section titled “Compile with contracts”rocky compile --models ./rocky-models --contracts ./contractsThe compiler validates that every model satisfies its contract at compile time. If a model’s output does not match the contract (missing column, wrong type, removed protected column), compilation fails.
10. Add Intent Descriptions
Section titled “10. Add Intent Descriptions”Rocky’s AI layer uses intent descriptions to understand what each model does. Adding intent to your migrated models enables ai-sync (automatic schema change propagation) and ai-test (test generation).
Generate intent for all models at once:
export ANTHROPIC_API_KEY="sk-ant-..."rocky ai-explain --all --save --models ./rocky-modelsThis reads each model’s SQL, generates a plain-English description, and saves it to the TOML config:
# stg_orders.toml (after ai-explain --save)name = "stg_orders"intent = "Stage raw Shopify orders with order_id, customer, date, and amount columns"depends_on = []
[strategy]type = "incremental"timestamp_column = "_fivetran_synced"
[target]catalog = "warehouse"schema = "staging"table = "stg_orders"11. Incremental Adoption Strategy
Section titled “11. Incremental Adoption Strategy”You do not need to migrate everything at once. Here is a recommended phased approach:
Phase 1: Import and compile
Section titled “Phase 1: Import and compile”- Run
rocky import-dbtto convert all models - Fix compilation errors
- Add contracts for critical models
- Run
rocky ciin your CI pipeline alongside dbt
Phase 2: Test parity
Section titled “Phase 2: Test parity”- Run
rocky testlocally to validate SQL execution - Compare Rocky output against dbt output on a test catalog
- Add
rocky compileas a required check on PRs
Phase 3: Production cutover (per model group)
Section titled “Phase 3: Production cutover (per model group)”- Start with leaf models (no downstream dependents)
- Switch their execution from dbt to Rocky
- Monitor output parity for 1-2 weeks
- Move upstream to the next layer
Phase 4: Full migration
Section titled “Phase 4: Full migration”- Migrate all models to Rocky
- Remove dbt from CI/CD
- Set up Dagster integration for orchestration
Running dbt and Rocky side by side
Section titled “Running dbt and Rocky side by side”During migration, you can run both tools on the same project by keeping the dbt models/ directory and the Rocky rocky-models/ directory separate. Your CI pipeline can run both:
# GitHub Actions examplesteps: - name: dbt compile run: dbt compile
- name: Rocky compile run: rocky compile --models ./rocky-models --contracts ./contracts
- name: Rocky test run: rocky test --models ./rocky-modelsOnce Rocky covers all models, remove the dbt steps.
Keeping dbt packages without converting them
Section titled “Keeping dbt packages without converting them”You don’t need to convert everything. dbt packages like fivetran/facebook_ads or fivetran/stripe produce tables in your warehouse that Rocky can reference directly as external sources. Rocky’s resolver automatically classifies schema-qualified table references (dbt_fivetran.stg_facebook_ads__ad_history) as external: they appear in lineage but do not create DAG dependencies.
This lets you keep vendor-maintained staging packages in dbt and write your custom analytics in Rocky. See Using Rocky with dbt Packages for the full walkthrough.
Troubleshooting
Section titled “Troubleshooting”“model not found” after import
Section titled ““model not found” after import”The importer names models after the SQL file’s stem (e.g., stg_orders.sql becomes stg_orders). If your dbt project uses custom model names via {{ config(alias='...') }}, the depends_on references may not match. Check each TOML file’s name field and update depends_on references accordingly.
Incremental models do not pick up the right watermark
Section titled “Incremental models do not pick up the right watermark”Rocky uses the timestamp_column from the [strategy] section, not Jinja logic. Make sure the column name matches what your data actually contains (e.g., _fivetran_synced, updated_at).
Environment-specific logic
Section titled “Environment-specific logic”dbt uses {{ target.name }} for environment branching. Rocky does not have environment-specific SQL. Use separate rocky.toml files per environment instead:
rocky compile --config pipeline.prod.toml --models ./rocky-modelsrocky compile --config pipeline.dev.toml --models ./rocky-modelsMacros that generate SQL dynamically
Section titled “Macros that generate SQL dynamically”If your dbt project relies on macros that generate SQL (e.g., a union_all macro that combines tables), rewrite the SQL explicitly. In most cases, a CTE with UNION ALL is clearer and more maintainable:
WITH all_orders AS ( SELECT * FROM raw_catalog.us_west_shopify.orders UNION ALL SELECT * FROM raw_catalog.eu_central_shopify.orders)SELECT order_id, customer_id, total_amountFROM all_orders