Your First Event Pipeline
Define what an event actually *is* — fact vs. command vs. message — and walk a single event end-to-end through a producer, broker, and consumer before any schema decisions are on the table.
Phases
3
StreamingPhase 1 freeFull access in Professional
Event-Driven Design
Event schema design, naming standards, evolution contracts, and governance patterns.
Streaming systems live or die on event design. Schemas that evolve safely, names every team understands, and contracts the CI enforces — that's the difference between a platform that scales to 100 services and one that breaks every Friday deploy.
What you'll do
What you'll be able to do.
- Design event schemas with required fields, idempotency keys, and partition decisions that survive production
- Implement schema evolution with backward and forward compatibility enforced in CI
- Build producer-side validation pipelines with DLQs, replay, and the outbox pattern
- Establish event architecture governance with catalogs, PII tagging, and ownership
Phase roadmap.
Phase 1PRO REQUIRED
Event Fundamentals
First pipeline and event basics
1.1
✓Your First Event Pipeline
Define what an event actually *is* — fact vs. command vs. message — and walk a single event end-to-end through a producer, broker, and consumer before any schema decisions are on the table.
1.2
✓Event Fundamentals
Anatomy of a production-safe event: required fields (event_id, occurred_at, event_type, version), idempotency keys, ordering guarantees, and the partitioning decisions that lock in for the life of the topic.
Phase 2PRO REQUIRED
Schema & Standards
Event schema design, naming, and evolution
2.1
⊘Event Schema Design
JSON Schema and Avro structural rules that catch nulls, enums, and type drift at the producer — including the additionalProperties: false decision and how to model nested entities without producing fragile schemas.
2.2
⊘Naming & Modeling Standards
Naming conventions ({domain}.{entity}.{action}), past-tense verb rules, key vs. value placement, and the modeling tradeoffs (fat events vs. thin events, embedded vs. referenced entities) that decide replayability.
2.3
⊘Schema Evolution & Contracts
Backward / forward / full compatibility modes in Confluent Schema Registry, breaking-change taxonomy, contract tests in CI, and a versioning strategy (v1/v2 topics vs. in-place evolution) that keeps consumers alive.
Phase 3PRO REQUIRED
Reliability & Governance
Validation, architecture patterns, and governance
3.1
⊘Event Validation & Reliability
Validate at the producer, not the consumer: schema-registry enforcement, dead-letter queues, poison-pill handling, exactly-once boundaries, and the dual-write problem solved with the outbox pattern.
3.2
⊘Event Architecture Patterns
Choose between event notification, event-carried state transfer, event sourcing, and CQRS based on consumer count, query patterns, and audit needs — and avoid event-sourcing the simple service.
3.3
⊘Event Architecture Governance
Build the event catalog (Confluent Stream Catalog / Backstage / DataHub) on top of the schema registry, with PII tagging, ownership, change approval workflows, and discoverability across teams.
What you'll ship
What you'll build.
- A versioned event-schema repo with backward-compatibility CI gates and a v1→v2 migration playbook
- A producer-side validation pipeline that rejects malformed events at the source — with DLQ + replay tooling
- A naming + modeling style guide every team consumes, with linter rules and a worked order.placed reference event
- An event catalog wired to Schema Registry, surfacing PII tags, owners, downstream consumers, and SLA per topic
Definition
What is Event-Driven Design?
Event-driven design is the practice of defining how systems communicate through events — including schema structure, naming conventions, evolution contracts, and governance. Well-designed events are the foundation of reliable streaming systems at companies like Uber, Shopify, and Netflix, where thousands of services exchange millions of events per second.
Production context
Why this matters in production.
Poorly designed events cause cascading failures across microservices. At Shopify, event schema evolution without proper contracts broke hundreds of downstream consumers. Production event design requires backward compatibility, validation, and governance that prevents breaking changes from reaching production.
Use cases
Common use cases.
- Designing event schemas with proper structure, naming, and versioning
- Implementing schema evolution with backward and forward compatibility
- Building event validation pipelines that catch malformed events before processing
- Establishing naming conventions and modeling standards across engineering teams
- Creating event catalogs and documentation for cross-team discovery
- Governing event architecture with approval workflows and contract enforcement
Compare
Event-Driven vs alternatives.
Event-DrivenvsAPI Design
Event design handles asynchronous communication between decoupled systems. API design handles synchronous request-response patterns. Events enable loose coupling; APIs enable tight integration.
Event-DrivenvsData Contracts
Event design focuses on event structure and evolution. Data contracts are broader, covering any data interface between teams. Event schemas are a specific type of data contract.
Event-DrivenvsSchema Registry
Event design defines the principles and patterns for events. Schema registry (like Confluent Schema Registry) is the tool that enforces schema compatibility. Design comes first; registry enforces it.
Why this matters
Why this skill matters.
This is the skill that separates streaming engineers who ship from streaming engineers who page on-call. Staff and senior streaming engineers at Uber, Shopify, Netflix, and Confluent are paid for exactly this — making event-driven systems evolvable instead of brittle.
FAQ
Common questions about Event-Driven.
Event-Driven DesignStart Phase 1