AWS, GCP, and Azure essentials — storage, compute, IAM, and the FinOps loop every interview tests.
Every modern data team operates on a cloud — and every interview starts with 'tell me how this pipeline would run on AWS.' The candidates who land mid-level data engineering roles can read a cloud bill, design a least-privilege IAM role, and explain when serverless beats a managed warehouse. The ones who can't get screened out in 20 minutes.
The shape, the storage layer, the pipeline, the bill
The five layers every cloud data platform has — ingestion, storage, orchestration, transformation, serving — and why this shape stays constant from a 3-person startup to Netflix. The vocabulary every later module sharpens.
S3 / GCS / ADLS as the durability layer underneath every cloud warehouse and lake — bucket partitioning, prefix limits, request-rate caps, lifecycle policies, and the tier (Standard / IA / Glacier) decisions that decide whether a 50 TB lake costs $1K or $9K a month.
Source → schedule → transform → destination — the four-step shape every orchestrated pipeline collapses to. EventBridge / Cloud Scheduler / Airflow as the trigger layer, plus the idempotency + retry rules that decide whether your Tuesday DAG survives Saturday's network blip.
Compute-seconds + GB-months + request-counts as the three line items every cloud bill reduces to — read the bill, you can predict it; predict it, you can cut it. The forensics that turn a $300K Snowflake surprise into a $90K reality.
Storage at scale, warehouse vs lake vs lakehouse, ELT, serverless
Partitioning strategies that survive 100 M objects, prefix entropy to dodge S3 request-rate limits, multipart uploads, S3 Select for skinny scans, and the tier-transition policies that drop a $9K/month lake to $2K without losing query performance.
Snowflake vs BigQuery vs Redshift Serverless on storage format, query engine, and cost shape — when a lakehouse (Iceberg + Trino) beats a managed warehouse, and when the operational cost of running it eats the savings.
Why ELT (load raw → transform in-warehouse) won in cloud and ETL (transform in-flight) still owns streaming and PII-filtering paths. Glue / Dataflow / Fabric trade-offs, the cost shape of each pattern, and the contract decisions that decide which one your platform standardizes on.
Lambda + Cloud Functions for event-driven ingestion, Fargate / Cloud Run for containerized ETL, and the 15-minute / cold-start / cost-per-invocation cliffs that decide when serverless wins vs when you should pay for an EC2 box.
Breach patterns, the cloud-first case, FinOps loop, AI layer
Over-permissive IAM, leaked keys in CI, unencrypted EBS volumes, public buckets — the four patterns behind every cloud-data breach post-mortem from the last three years, plus least-privilege role design, KMS envelope encryption, and VPC endpoints for warehouse traffic.
Elasticity, managed services, and cloud-native-first tooling as the three structural reasons even cost-sensitive teams move data workloads off-prem — and the two scenarios (sub-millisecond latency, sovereign data) where on-prem still wins.
Cost attribution per pipeline (per-DAG tagging, per-query labels), warehouse autoscaling schedules, query-level cost optimization (filter pushdown, partition pruning), and the FinOps loop — measure → allocate → optimize — that closes the gap between cloud bills and engineering decisions.
Bedrock / Vertex AI / Azure AI Foundry as the managed-LLM layer, the data-engineering work that feeds them (vector stores, retrieval pipelines, evaluation harnesses), and the cost-attribution model that prevents the $40K AI-agent surprise bill.
WHAT GOES WRONG
Cloud fundamentals for data engineers covers the storage, compute, security, and cost-management primitives that every modern data platform runs on. The curriculum walks through S3 / GCS / ADLS object storage, the warehouse-vs-lake-vs-lakehouse decision, ELT pipeline mechanics on AWS / GCP / Azure, least-privilege IAM, and the FinOps loop that turns a $300K surprise bill into a predictable cost line.
Every modern data team operates on a cloud, and the interview screens reflect it — hiring panels at Stripe, Datadog, and every cloud-native data org expect candidates to read the bill, design IAM roles, and explain when serverless beats a managed warehouse. Cloud fundamentals isn't a specialist skill; it's the shared vocabulary the rest of data engineering builds on.
AWS has the broadest service portfolio and the most data-engineering job postings. GCP leads in analytics with BigQuery's serverless query engine. Both are strong for data engineering — the choice usually comes down to existing organizational investment, not capability gap.
Cloud provides elastic scaling, managed services, and pay-per-use pricing — the structural reasons even cost-sensitive teams move off-prem. On-premise still wins for sub-millisecond latency requirements and sovereign-data workloads. Most modern data teams are cloud-first with selective hybrid.
Single-cloud simplifies operations, IAM, and cross-service data transfer cost. Multi-cloud prevents vendor lock-in and enables best-of-breed services (e.g., BigQuery for analytics, AWS for everything else). Most companies are single-cloud primary with selective multi-cloud for specific services.
Cloud fundamentals is the *floor* skill for a mid-level data engineering offer. Hiring panels at Stripe, Datadog, Snowflake, and every cloud-native data team test for it on day one — not because they need a specialist, but because nothing else on a resume reads as credible without it. The engineer who can read the AWS bill is the one who gets trusted with the platform.
Yes — it's the floor screen. Every mid-level data engineering interview tests whether you can read a cloud bill, design an IAM role, and explain when serverless beats a managed warehouse. You can't fake it past a 30-minute screen.
AWS has the broadest job market and most data-engineering tooling. GCP excels in analytics with BigQuery. Azure is strongest in Microsoft-heavy enterprises. Start with whichever your target employers use — concepts transfer between providers within a few weeks of hands-on practice.
Core concepts across all three providers take 3–4 weeks of focused study. Real fluency with one provider's data services — enough to design and operate pipelines without surprises — typically takes 2–3 months of hands-on usage.
AWS, GCP, and Azure all offer free tiers with enough capacity for the storage, compute, and IAM work in this curriculum. Set a $1/month billing alert as your first task — that habit alone separates engineers who get trusted with platforms from those who don't.
AWS Data Analytics Specialty, GCP Professional Data Engineer, and Azure Data Engineer Associate are the most relevant. Practical experience matters more than certifications, but they validate baseline knowledge — and the prep work itself is high-quality cloud fundamentals practice.
A certification gets your resume past the first filter. The interview still tests whether you can read the bill, design IAM, and explain trade-offs. Treat certifications as proof of effort, not proof of skill — pair them with a portfolio project that exercises the concepts.
FinOps applies financial accountability to cloud spending — measure cost per pipeline, allocate it to the team that owns it, optimize what's most expensive. For data engineers, it means tagging warehouses by team, attributing query cost, and making cost a first-class design consideration alongside latency and reliability.