What is dbt? The complete guide for data engineers

What is dbt?

dbt is an open-source command-line tool (and optional cloud platform) that lets data teams write SQL transformations as version-controlled code. Instead of ad-hoc SQL queries or fragile stored procedures, you define models — plain .sql files that dbt compiles and runs against your data warehouse.

dbt is SQL-first. Every transformation is a .sql file with optional Jinja templating. dbt generates the DDL/DML, manages dependencies between models, and materializes results as tables or views in your warehouse.

Critically, dbt follows the ELT pattern (Extract, Load, Transform), not ETL. It does not move data — tools like Fivetran, Airbyte, or Stitch handle ingestion. dbt only transforms data already in your warehouse.

Why does dbt matter?

• Version control for SQL — every model lives in Git, reviewed like code
• Built-in testing — assert uniqueness, nulls, referential integrity on any column
• Auto-generated documentation — lineage graphs, column descriptions, a data catalog
• Modular models — staging → intermediate → marts layering replaces monolithic views
• Macros + Jinja — DRY up repeated SQL across hundreds of models
• Incremental materialization — process only changed rows on enterprise-scale tables

How does dbt work?

A dbt project is a folder of .sql model files plus a dbt_project.yml. When you run dbt run, dbt:

1. Parses every model and builds a dependency DAG from ref() calls
2. Compiles Jinja → pure SQL targeted at your warehouse
3. Wraps each model in CREATE TABLE AS / MERGE / INSERT INTO DDL
4. Executes models in dependency order — parents first, then children
5. Stores run history + freshness metadata for observability

The result: your warehouse fills up with clean, tested, documented tables and views — built repeatably from SQL that lives in Git.

dbt Core vs dbt Cloud

Choose dbt Core if you already have an orchestrator (Airflow, Dagster, Prefect) and want zero vendor lock-in. Choose dbt Cloud when you need the hosted IDE, scheduler, and team features without building them yourself.

ELT vs ETL — why the order matters

The old ETL pattern (Extract → Transform → Load) transformed data in a separate compute layer (Informatica, Talend, Spark) before loading the transformed result into the warehouse. The new ELT pattern (Extract → Load → Transform) loads raw data first, then transforms inside the warehouse with SQL.

ELT works in 2026 because cloud warehouses (Snowflake, BigQuery, Databricks) are cheap and fast enough to do the transformation work. The benefits:

• Raw data is always preserved — you can re-transform when business logic changes
• Transformation is SQL, not Python or Java — every analyst can read it
• Compute is decoupled from ingestion — scale them independently
• Version control + testing + lineage become tractable (the dbt pattern)

dbt is the canonical ELT tool. Its existence is what made ELT the default architecture for the modern data stack.

Common mistakes (and what to do instead)

• Putting business logic in sources instead of models — sources should be a thin contract layer over raw tables. Logic lives downstream in models.
• Not using ref() — hardcoding table names breaks lineage — every cross-model reference should go through {% raw %}{{ ref('model_name') }}{% endraw %}.
• Skipping tests on critical models — every mart should have not_null + unique + relationships tests on its key columns.
• Monolithic models — split into staging (rename/cast) → intermediate (joins/business logic) → marts (analytics-ready).
• Using dbt as an ingestion tool — dbt only transforms. Use Airbyte/Fivetran/Stitch for ingestion.

Who is dbt for?

dbt is built for analytics engineers and data engineers who want to apply software-engineering practices to SQL: version control, code review, testing, CI/CD, modular code, documentation.

If you write SQL daily and your data lives in a cloud warehouse, dbt is almost certainly the right tool. Teams that benefit most:

• Startups loading raw events from Segment / Rudderstack and modeling them for product analytics
• Marketing analytics teams modeling Salesforce + HubSpot + ad-platform data into a unified customer view
• Finance teams building close-of-books reporting on top of ERP ingestion
• ML teams building feature pipelines that need testable, lineage-tracked transformations