Skip to content
All notes
6 min read[Architecture][Data]

Designing the event bus before it's cool

Every system we build ends up with an event bus in the middle. Here's why — and the expensive contract-less mistakes we made before we understood it.

Almost every system we build settles into the same shape. A few sources of truth on one side, a few things that need to know about them on the other, and a single clear channel carrying the messages between them. We call that channel the event bus.

It's not an architectural fashion. It's a maintenance decision. When every part of the system can only learn about the rest of the business through one well-defined channel, you can change a component without re-plumbing everything around it. This is especially important in the integrated systems we build — platforms where a Core backend, Data pipelines, and third-party connectors all have to stay in sync without becoming tightly coupled to each other.

What an event bus actually is

Strip away the jargon: an event bus is a channel through which one part of a system announces that something happened, and other parts listen for the announcements they care about. The sender doesn't know who's listening. The listeners don't know — and don't need to know — how the event was produced.

In practice this means: when an order is placed in your Core platform, the Core system publishes an order.placed event to the bus. Your inventory system, fulfilment system, notifications system, and analytics pipeline each subscribe to that event and do their own work. None of them call the Core platform directly. If you later add a loyalty points system, it subscribes to the same event — no changes required upstream.

What it buys you

Three things, concretely:

  • Teams can build against the bus instead of against each other's internal details.The contract is the event schema, not someone else's database model. Subscribers can be written, tested, and deployed independently.
  • New consumers attach to existing events instead of asking for new integrations.Adding a new downstream system is a subscription, not a negotiation. This is what makes our Integrations layer scale: each new tool connects to the bus, not to every other tool.
  • Replay. If a consumer fails, it can catch up from where it stopped, not from the beginning of time. In systems like ourpayments reconciliation platform — which processes ~90,000 transactions a day — this is not optional. A consumer that can't replay is a consumer that loses data on every incident.

What we got wrong first

Our first buses were contract-less. Events had whatever shape the sender felt like that week, and consumers broke silently whenever a field disappeared or got renamed. Apayment.settled event that used to carry amount now carriedtotal_amount. The consumer kept running — consuming zero-value records — until someone noticed the reconciliation numbers were wrong.

Now every event has three things before it ships:

  • A versioned schema — the event's shape is documented and reviewed before the first publish.
  • A schema check at the edge — the bus rejects malformed events before they propagate. Producers get the error immediately; downstream consumers never see the bad data.
  • A dead-letter queue — anything the bus couldn't parse or route lands in a quarantine queue for investigation. Nothing silently disappears.

When to introduce the bus

The answer is earlier than you think, and the reason is cost. Introducing a bus into a greenfield system takes a few days of design work. Introducing one after six months of point-to-point integrations takes weeks of careful surgery — and you'll miss some connections the first time.

We've done both. The surgery version is manageable but expensive. The early version just becomes part of how the system is shaped, and nobody thinks twice about it. It's the same logic we apply in ourspec-first pricing approach: the cost of doing it right at the start is always lower than the cost of finding out you needed to do it right later.

Signs you already need one

If any of these describe your system, you're operating without a bus and paying the price for it:

  • Deploying one service requires coordinating with two or three other teams.
  • Adding a new downstream consumer means modifying the upstream service.
  • You've had at least one incident where a consumer processed stale or duplicate data.
  • Your data pipeline reads directly from the production database because there's no other way to get events.
The bus is not the point. The point is that your business keeps working while the system under it changes.

If you're planning a system, plan for the channel before you plan for the endpoints. It's cheaper to add a bus early and regret it than to bolt one on after your components are talking directly to each other.

Once a system is hardened and in production, the bus is also your first line of observability — every message in flight is a data point. Read more about how we approach the production phase inOperations is where systems are actually built.

[ Commission a system ]

Tell us about the system you need

Write to us about what your business runs on today, what it should run on tomorrow, and where it hurts. We'll reply with a plan and a price.

1.You describe the job it has to do

2.We reply within one business day with a plan and a price

3.We design, build, and operate it