A data engineer designs the systems that move, store, and transform data so it's usable by analysts, ML models, and AI agents. The 2026 path is 6 phases over ~6 months of focused practice: SQL + Python foundations → data modeling → batch pipelines → platform engineering → streaming → AI data systems. Each phase pairs a skill on /curriculum with a portfolio project on /projects.
Why this roadmap
The bar for data engineering has risen sharply over the last 18 months. Every company that rushed to ship AI in 2024 is now sitting on a pile of unreliable data infrastructure. They need engineers who can build and maintain production-grade pipelines, not engineers who can name every tool in the modern data stack.
Each phase below pairs:
- One foundational skill — taught hands-on with code and exercises
- One portfolio project — a production-style artifact that maps onto an interview story
Phases are designed to be sequential. You can skip ahead if you have prior experience, but the projects in later phases assume the patterns from earlier ones.
Foundations · SQL + Python
Window functions, CTEs, joins, and aggregations. Python for I/O, idempotent scripts, and basic API ingestion. By the end of this phase you can write any analytical query and ship a small Python ingestion job.
Data modeling
Dimensional modeling, star schema, slowly-changing dimensions, and dbt's ref() + source() discipline. The model layer is where data engineering separates from analytics — get it right and downstream consumers stop firefighting.
Batch pipelines
Orchestration with Airflow, S3 + Snowflake as the canonical batch substrate, and the discipline of idempotent, restartable tasks. This is the phase where you stop running scripts manually and start running platforms.
This is the first half of the Core DE path.
Phases 1–3 + capstone project = job-ready as a junior/mid data engineer. The full path adds interview prep + a mock-system-design module.
Data platform
Lakehouse table formats (Iceberg, Delta, Hudi), CI/CD pipelines that run dbt tests on every PR, and the data-quality patterns that get you out of the on-call rotation. Platform engineering is the skill that turns mid-level data engineers into staff.
Streaming systems
Event-time semantics, watermarks, exactly-once processing, and the hard truth that most "real-time" requirements are 30-second freshness in disguise. Learn streaming for the cases where it's actually warranted; don't reach for it by default.
AI data systems
Training-data engineering, retrieval pipelines, feature stores, and the eval harnesses that make AI systems reliable in production. This is the differentiator phase — the work that separates a 2024 data engineer from a 2026 AI data engineer.
Capstone: ship a real RAG system.
End-to-end build with chunking, embeddings, hybrid search, reranking, eval harness, and cost-attribution dashboard. Mentor-reviewed.
How long does the full roadmap take?
| Pace | Hours/week | Time to complete |
|---|---|---|
| Casual | 5–8 | 9–12 months |
| Standard | 10–15 | 5–7 months |
| Intensive | 20+ | 3–4 months |
You don't have to finish every phase before applying for jobs. Most readers finish Phases 1–3, ship the capstone project, and land a junior/mid data engineer role. Phases 4–6 then accumulate during the first 18 months on the job.
Which career path fits each phase?
The 5 career paths in our curriculum branch from this roadmap at different points:
- Phases 1–4 → Core Data Engineer (the canonical path)
- Phases 1–3 + dbt deep-dive → Analytics / Product DE
- Phases 1–5 + IaC focus → Data Platform Engineer
- Phases 1–6 → AI Data Engineer
- Phases 4–6 + serving + safety → AI Platform Engineer
The full progression from a non-engineer background to AI Platform Engineer is ~24 months of focused work. The first job lands somewhere between months 5 and 9.
Frequently asked questions
Start shipping.
Three steps from a guide to a job-ready portfolio. Pick one and start now — the rest will follow.
Take the skill
Self-paced module with code, exercises, and a deliverable. Free preview, paid completion.
Start S0X · Sql Mastery →Ship the project
Production-grade build with starter kit + mentor code review. The artifact that gets you interviews.
Open P0X · Ecommerce Data Warehouse →Pick a career path
The full progression — skills + projects + interview prep — for the role you actually want.
See paths →