Timeline
10-12 hours
PRO · part 01 free previewPlatform trackP27
Three envs.
Two clouds.
One Terraform spine.
Provision dev / staging / prod across AWS and GCP entirely via Terraform — VPC + IAM, BigQuery + Redshift warehouses with WLM and tiered S3/GCS lifecycle, a 5-role RBAC matrix locked down by permission boundaries and SCPs, KMS-encrypted Secrets Manager with 30-day rotation, CloudTrail audit, and AWS Budgets + a Grafana FinOps anomaly dashboard.
The “walk me through dev/staging/prod for a data platform on AWS+GCP” question — asked at any company running multi-cloud analytics (Snowflake, Databricks, Stripe, Airbnb).
By the end you will have shipped
- A 19-module Terraform codebase with S3 + DynamoDB remote state + locking
- Three Terraform workspaces (dev / staging / prod) with isolated state and CI plan/apply gates
- BigQuery + Redshift warehouses with partitioning, clustering, WLM queues, and tiered S3/GCS lifecycle policies
- A 5-role IAM matrix (platform_admin · data_engineer · data_analyst · ml_engineer · read_only) with permission boundaries + SCP guardrails
- AWS Secrets Manager with 30-day Lambda rotation + GCP Workload Identity Federation
- AWS Budgets + cost allocation tag policy + a Grafana cost-anomaly dashboard + 20-point production hardening checklist
PREREQComfortable with the cloud console (AWS / GCP) and the Terraform basics. We recommend the Cloud Fundamentals path first if Terraform / VPC / IAM are new — it covers the primitives this project composes into a real platform.
platform.{dev,staging,prod} · 2 clouds wired
plan/apply gateTerraform
Workspaces
AWS layer
GCP layer
modules/ · 19 modulesnetworking · iam · kms · …
s3 + dynamodb backend
.github/workflows/
plan / apply gates
one repo · state-locked
workspace: devsingle-AZ · auto-apply
workspace: stagingmulti-AZ · review-gated
workspace: prod3-AZ · approval-gated
isolated state per env
aws_vpc · 3 AZ
iam · 5-role matrix+ permission boundary
s3 · tiered lifecycle
redshift · WLM queues
kms + secrets-mgr
11 modules · per env
google_compute_network
bigquery · medallionpartition + cluster keys
iam + WIFkeyless OIDC auth
gcs · autoclass + repl
cloud-monitoring sink
8 modules · per env
# 5-role RBAC matrix
platform_admin · data_engineer · data_analyst
ml_engineer · read_only
permission_boundary · SCP guardrails
→ no privilege escalation past boundary
● FinOps spine
aws_budgets · 50/80/100/forecast alerts
tag policy · project · team · cost_center
Grafana cost-anomaly dashboard
→ 20-pt prod hardening checklist
3 envs × 2 clouds
isolated state
19
Terraform modules
5
IAM roles + SCPs
Why platform foundations, why now
The data team that ships first is the one with a sane base layer.
Multi-cloud is the default for analytics teams above ~50 engineers — Snowflake on AWS, BigQuery on GCP, often both. The senior+ data engineer who can stand up dev/staging/prod with proper RBAC, secrets rotation, audit, and budget guardrails is the one who unblocks every downstream pipeline + ML workflow.
Multi-cloud is the steady state
Every modern analytics team has at least two clouds — usually because Snowflake is on AWS and the ML stack is on GCP. The platform layer has to span both without forking workflows.
IaC drift is the #1 platform risk
Click-ops in three environments turns into 'why is staging slow?' six months later. Workspaces + remote state with locking + plan/apply gates is the bar for senior platform roles.
Cloud-native IAM is the senior bar
Permission boundaries + SCPs + Workload Identity Federation is what reviewers look for. Hand-edited bucket policies are not — they're the audit finding nobody wants.
FinOps owns the bill, not finance
Budgets, cost allocation tags, anomaly alerts, and tag policies are how engineering teams stay accountable for spend. This is now a senior-DE responsibility, not a corp-finance one.
Curriculum · 4 parts · 10-12 hours
Part 01 is free. The rest unlocks with PRO.
Try the first ~3 hours — wire the Terraform skeleton, stand up dev/staging/prod workspaces with S3 + DynamoDB locking, and ship a multi-AZ VPC + 5-role IAM scaffold. If it clicks, upgrade to unlock the warehouse layer, the RBAC + secrets + audit lock-down, and the FinOps + production hardening sprint.
P27 · 10-12 hours · 4 parts
Free preview PRO required
Part 01 is free — no card required. Get the Terraform spine running before paying.
M01
✓Foundation — multi-env IaC + VPC + IAM scaffold
Set up Terraform 1.6 with AWS + GCP providers pinned. S3 + DynamoDB remote state with workspace-per-env. Multi-AZ VPC with public/private subnets, NAT, IGW. The 5-role IAM scaffold (platform_admin · data_engineer · data_analyst · ml_engineer · read_only) and a GitHub Actions plan/apply gate.
M02
⊘Warehouses + storage — BigQuery, Redshift, S3/GCS lifecycle
BigQuery datasets in a medallion shape (raw → staging → analytics → ml_features) with partitioned + clustered tables. Redshift cluster (env-tiered node types) with WLM queue config. S3 + GCS tiered lifecycle (Standard → IA → Glacier → DeepArchive). Lake Formation + Athena permission scaffold.
M03
⊘Security + access — RBAC matrix, secrets, audit
5-role IAM matrix locked down with permission boundaries. AWS Secrets Manager with 30-day Lambda rotation. KMS keys with rotation. CloudTrail + AWS Config + Service Control Policies (deny CloudTrail disable, enforce S3 encryption, block public buckets). GCP Workload Identity Federation for keyless OIDC auth.
M04
⊘Operations — FinOps + monitoring + production hardening
AWS Budgets with multi-stage SNS alerts (50/80/100/forecasted). Org tag policy + activated cost allocation tags. Grafana FinOps cost-anomaly dashboard YAML. CloudWatch alarms with PagerDuty wiring. Cross-region backup + RTO/RPO targets. 20-point production hardening checklist.
3 parts locked · Unlock all PRO content for $29/mo
Upgrade to PRO →Backed by curriculum
Cloud Data Infrastructure & FinOps
The Cloud Fundamentals path covers the underlying primitives — this project shows you how to compose them into a real multi-cloud platform.
The build, in 3 phases
Three sprints. Three checkpoints. One production-grade platform.
Each phase ends with a tagged commit, a runnable terraform plan, and an artifact a senior reviewer would actually accept.
01~3h
Lay the IaC spine
Terraform skeleton + remote state with locking + multi-AZ VPC + 5-role IAM scaffold. GitHub Actions plan/apply gate enforced. dev/staging/prod workspaces with isolated state.
- ✓S3 + DynamoDB backend with state locking
- ✓Multi-AZ VPC across 3 AZs (public/private + NAT/IGW)
- ✓5-role IAM scaffold + GitHub Actions plan/apply pipeline
02~3h
Stand up the warehouses
BigQuery medallion datasets with partitioned + clustered tables. Redshift cluster with WLM queues. S3/GCS tiered lifecycle. Lake Formation + Athena permission scaffold.
- ✓BigQuery datasets (raw/staging/analytics/ml_features) with partition + cluster keys
- ✓Redshift cluster + WLM JSON config (etl/reporting/adhoc/default)
- ✓S3 + GCS tiered lifecycle (Standard → IA → Glacier → DeepArchive)
03~5h
Lock it down + watch the spend
5-role RBAC matrix with permission boundaries + SCPs. Secrets Manager with rotation. KMS + CloudTrail + Config. Budgets + cost tag policy + Grafana FinOps dashboard. 20-point production hardening checklist.
- ✓RBAC matrix + permission boundaries + SCPs (deny CloudTrail disable, public buckets)
- ✓Secrets Manager 30-day rotation + Workload Identity Federation
- ✓AWS Budgets + cost allocation tags + Grafana cost-anomaly dashboard
Project setup · 10 minutes
Validate locally. Apply only with your own cloud accounts.
The starter kit ships terraform-validate-clean across all 19 modules so you can review, fmt-check, and security-lint the codebase on your laptop — without spinning up any billable resources until you opt in.
What lives in the repo
Everything you need to walk all 4 parts on your laptop, plus the sample audit + cost CSVs that simulate CloudTrail and AWS Cost Explorer exports for offline analysis.
- modules/ — 19 Terraform modules across networking / iam / bigquery / redshift / kms / cloudtrail / budgets
- environments/ — dev/staging/prod tfvars (.example only — never commit credentials)
- audit-logs/ — sample CloudTrail + BigQuery audit-log CSVs for offline access-pattern analysis
- cost-reports/ — sample AWS Cost Explorer + GCP Billing CSVs for FinOps dashboard work
- .github/workflows/ — CI scaffolding with plan/apply gates and security-lint job
- Makefile — validate · fmt-check · security (checkov + tfsec) · plan / apply / destroy targets
Download · Starter Kit
Multi-Cloud Platform Foundation Starter Kit
Pre-validated Terraform codebase (19 modules across AWS + GCP), sample audit + cost CSVs, Makefile with checkov/tfsec hooks, and a CI workflow. Everything terraform-validate-clean out of the box. Skip the boilerplate, start on Part 01.
~/projects/data-access-control — zsh
1. Unzip and enter the starter kit
$ unzip data-access-control-starter.zip
$ cd data-access-control-starter
2. Configure your inputs (NEVER commit the non-.example versions)
$ cp terraform.tfvars.example terraform.tfvars
$ cp backend.tf.example backend.tf
$ cp environments/dev.tfvars.example environments/dev.tfvars
3. Validate locally — no cloud credentials required
$ terraform init -backend=false
$ make validate
$ make fmt-check
4. Security lint with checkov + tfsec (still no credentials)
$ make security
5. (Optional) Plan against your own AWS + GCP accounts
$ export AWS_PROFILE=... GOOGLE_APPLICATION_CREDENTIALS=...
$ terraform workspace select dev
$ terraform plan -var-file=environments/dev.tfvars
19
Terraform modules
5
IAM roles
2,500
audit-log rows
1,400
cost-export rows
What changes vs a single-account sandbox
The same platform — built for the real org chart.
Tutorials ship against a single AWS account + a single GCP project. Production runs across an AWS Organization, multiple GCP folders, real rotation cadences, and budget caps that actually pause spend. Here’s the diff, with the Terraform resources you reach for.
Tutorial versionWhat you ship in 10-12 hrs
×
Account scope
Single AWS account + single GCP project
×
IAM principals
Roles defined per env, no org-wide guardrail
×
Secrets
Secrets Manager with Lambda rotation, KMS-encrypted
×
Audit
CloudTrail + AWS Config rule structure shown
×
Cost controls
Per-account Budgets + SNS email alerts
×
DR
Cross-region backup config + RTO/RPO targets documented
Production versionParts 02-04 + ops
✓
Account scope
aws_organizations_organizational_unit per env + GCP folders with Workload Identity Federation✓
IAM principals
aws_iam_policy with permission boundary + aws_organizations_policy SCP guardrails (deny CloudTrail disable, enforce s3 encryption)✓
Secrets
aws_secretsmanager_secret_rotation with automatically_after_days = 30 + KMS key rotation enabled org-wide✓
Audit
CloudTrail +
aws_config_rule + a SIEM/Snowflake destination — actual rule libraries (CIS, PCI) attached, not just the scaffold✓
Cost controls
aws_budgets_budget + aws_budgets_budget_action with hard caps that pause non-prod, plus tag policy enforcement at the org level✓
DR
Restore drills run quarterly — cross-region backups +
aws_backup_plan with documented RTO ≤ 4h / RPO ≤ 1h for prodPRO benefit · code review
Real review from senior platform engineers who’ve owned this stack.
Submit your Terraform PR, get line-by-line feedback within 48 hours from someone who has actually run dev/staging/prod across AWS + GCP. The kind of review that's quietly worth thousands of dollars in time-to-staff.
CR
4 reviews / month
Submit a Terraform PR, a refactor proposal, or a full repo. Reviewer is matched to your domain — multi-cloud platform / RBAC / FinOps for this project. Async, comments inline, average turnaround 31 hours.
31h
avg turnaround
9.2/10
helpfulness
94%
return next month
OH
2 office hours / month
Live 30-min sessions with a senior platform engineer. Architecture questions, whiteboard a permission-boundary design, mock a system-design interview on multi-cloud platform layout. Group sessions also available.
30 min
per session
2 / mo
included
+ group
unlimited
What PRO unlocks
One subscription. 15+ projects, all curriculum, code review.
PRO is built for senior+ engineers who want production-grade builds and feedback loops — not more tutorials.
What you getFREEPROEXPERT
Projects
Production-grade builds
2
15+
8
Curriculum modules
All 7 tracks
Phase 1 only
All
All + bonus
Code review credits
Senior engineer review
0
4 / month
Unlimited
Career path access
5 paths × full plans
1 path
All 5
All 5 + 1:1
Certificate
Verifiable on LinkedIn
—
Yes
Yes + portfolio review
Community
Discord + office hours
Read-only
Full + 2/mo
Full + 4/mo
$29/mo
billed monthly · cancel anytime
or annual
$249/yr save 28%
Who this is for
Pick this if you own the platform, not just one pipeline on it.
PE
Platform engineers
You're the one tagged on every 'why is staging different from prod' Slack thread. This gives you a clean Terraform spine with isolated state, plan/apply gates, and a 5-role RBAC scaffold to point reviewers at.
DE
Senior data engineers leveling up
You ship dbt + Airflow daily and the next interview loop is asking you to whiteboard the platform underneath. After this you can defend a multi-cloud + multi-env design from first principles.
SR
Cloud SREs / DevOps crossing into data
You know Terraform + IAM cold but the data warehouse layer is unfamiliar. This bridges the gap — BigQuery + Redshift + WLM, S3/GCS lifecycle, and the cost-attribution patterns data teams actually want.
SC
Security-adjacent data engineers
You're the DE who keeps getting pulled into compliance reviews. This is the project that lets you point at a permission-boundary + SCP + Workload Identity Federation setup the security team will sign off on.
Related curriculum
Going deeper? Three tracks back this project.
The platform spine is what this project ships. These three curriculums let you go deeper on the layers it touches — policy + contracts depth, observability beyond CloudTrail, and the system-design framing senior+ interviews actually ask about.
FAQ
Quick answers.
No — and that's the most important thing to know upfront. This is cloud-native IAM RBAC at the platform layer (AWS IAM + GCP IAM + permission boundaries + SCPs). For data-layer fine-grained policies — Open Policy Agent, BigQuery row-level security, dynamic data masking, ABAC — see the Governance & Data Contracts curriculum (/learn/governance) and the Data Governance Contracts project (P25). They pair well: this gives you the platform, that gives you the policies on top of the data.
P26 is depth on Snowflake FinOps — ACCOUNT_USAGE forensics, warehouse right-sizing, dbt cost models, anomaly alerts. This project (P27) is breadth on the platform foundation that supports any FinOps work — multi-cloud Terraform, RBAC, secrets, audit, plus a budgets + tag policy + Grafana FinOps dashboard. P26 is the cost-engineering specialist project; P27 is the platform-engineering generalist project.
No, not for the bulk of the work. The starter kit is terraform-validate-clean across all 19 modules, so you can run init -backend=false, validate, fmt-check, and security-lint on your laptop. You only need real credentials if you want to terraform plan / apply against your own accounts — and the audit log + cost-export sample CSVs let you do the FinOps dashboard work offline either way.
Parts 02-04 (warehouses + RBAC/secrets/audit + FinOps/hardening), the starter kit ZIP, sample audit + cost CSVs, plus 4 code-review credits + 2 office-hours sessions per month and access to all 15+ PRO projects + curriculum across all 7 tracks. Cancel anytime.
Especially the system-design rounds. After this you can whiteboard dev/staging/prod across AWS + GCP, defend a permission-boundary + SCP guardrail design, talk through Workload Identity Federation as the alternative to long-lived service-account keys, and reason about budget caps as a real cost control vs. just a notification.
/learn/cloud-fundamentals teaches the primitives in isolation — what a VPC is, how IAM policies attach, what a partitioned table is. This project composes those primitives into a working multi-cloud platform with three environments, a 5-role RBAC matrix, and FinOps controls wired end-to-end. Curriculum is for fluency; this project is for reps. Most learners do the curriculum first.
Ready to lay the spine?
Start with Part 01 — free, no card. About 3 hours. By the end you'll have Terraform with remote state + locking, dev/staging/prod workspaces with isolated state, a multi-AZ VPC, the 5-role IAM scaffold, and a GitHub Actions plan/apply gate.
P27 · Multi-Cloud Platform Foundation · PRO · part 01 freeUpgrade to PRO →