Event-Driven Architecture
Events decouple producers from consumers, enable asynchronous workflows, and give you a replayable history. They also introduce eventual consistency, debugging complexity, and schema-evolution overhead.
Deep dive
Event types
- Domain events: something happened (
OrderPlaced). Past tense. Owned by the producer. - Commands: a request (
PlaceOrder). Imperative. Targeted at one consumer. - Notification events: lightweight signal, consumers fetch the data ("CDC pattern").
Patterns
- Outbox: write event to DB in same transaction as state change; relay publishes to broker. Solves dual-write.
- Event sourcing: state IS the event log. Powerful but expensive — only for domains that genuinely need full history.
- Saga: choreographed (event chain) or orchestrated (central coordinator). Use orchestration when the flow needs to be visible/maintained centrally.
Schema evolution
Use a schema registry (Avro/Protobuf). Forward + backward compatibility rules in CI. Breaking changes = new event version, not a mutation.
Real-world example
From productionOrder service published events directly from the request handler. When the broker had a blip, orders committed but events never published → downstream inventory desync. Fix: outbox pattern with a CDC connector (Debezium). Five-minute incident, zero data loss; previous pattern had silent inconsistency for hours.
Interview questions
2 senior-levelQ1What's the dual-write problem and how do you solve it?▾
When code writes to two systems (DB + broker) in sequence, a crash between them causes inconsistency. Solutions: outbox pattern (write event to same DB transaction, separate relay publishes), or change-data-capture (read from DB WAL). Both make the broker write a consequence of a single durable commit.
Q2Choreography vs orchestration?▾
Choreography (services react to events) is loosely coupled but hard to reason about end-to-end. Orchestration (central state machine) is easier to monitor and modify but introduces a coordinator. Use orchestration when the business cares about the flow as a first-class entity.
Common mistakes
Treating Kafka as a database (long retention + tombstones + compaction is fragile).
No schema registry — every consumer parses defensively.
Synchronous orchestration code disguised as 'events'.
Trade-offs
Eventual consistency simplifies producers but pushes work to consumers and UX.
Event sourcing gives audit but makes evolution and queries harder.