The outbox pattern

The problem it solves

An order is placed. Two things must happen: a row is written to the database, and an event is published so that the warehouse, the email service and the analytics pipeline all learn about it. Two systems, two writes, and no transaction spanning them.

Write the database first and then publish, and there is a window — microseconds usually, minutes during a broker outage — in which the row exists and the event does not. The process dies in that window, or the broker is down and the publish fails after the transaction has already committed, and now an order exists that nothing downstream will ever hear about. No warehouse pick, no confirmation email, no revenue in the report. Nothing errored in a way anybody will see; the order looks fine in the database.

Reverse the order and the failure inverts: publish first, then write, and a crash between them leaves an event for an order that does not exist. Downstream services build state on it, ship goods for it, and reconcile against a database that has never heard of it.

This is the dual-write problem, and it cannot be retried away, because whichever write you retry, the failure window is around the other one.

The mechanism

The outbox pattern removes the second write entirely. In the same transaction that inserts the order, insert a row into an outbox table describing the event. One transaction, one database, full atomicity: either both rows exist or neither does. There is no window, because there is no second system involved at commit time.

A separate relay then reads unpublished outbox rows and publishes them to the broker, marking them sent. The relay can fail, crash, retry, or run behind, and none of it costs correctness — the events are durably recorded in the database, so the worst case is that they are late. Late is a completely different problem from gone.

Two ways to build the relay. Polling: a query for unsent rows on a short interval, publish, mark. Simple, easy to debug, adds load proportional to the poll frequency. Change data capture: tail the database’s replication log (Debezium and friends) and publish what appears. No polling load, no application code in the path, and it works for writes made by anything that touches the database — at the cost of running and understanding a CDC pipeline.

Either way the relay is at-least-once: it can publish and die before marking. Consumers must therefore be idempotent, which they had to be anyway — see idempotency.

What the simulation shows

Orders are created continuously; the broker fails for a window; processes crash at a small rate; database writes fail occasionally.

Run the default, write the row then publish, and watch the panel counting data that will never be consistent again. It rises steadily from the crash rate alone, and then jumps during the broker outage: every order created while the broker is down is an order with no event, permanently, because the transaction already committed and nothing will retry the publish.

Now publish first. The counter still climbs, and it is now counting phantom events — events for orders that were never written. Downstream services will act on those. Inverting the order does not fix the problem, it swaps which kind of corruption you get, and seeing both counters on the same axis is the fastest way to understand that there is no ordering of two writes that works.

Then the outbox. Permanent inconsistency goes to zero. The broker outage does not disappear — the outbox-depth panel shows it clearly, filling for the whole window — but it is now a queue depth rather than data loss. When the broker returns, the relay drains the backlog and the events flow. Late, not gone.

The numbers worth carrying

Exposure without the pattern = crash-or-failure rate × writes. At 400 orders a second with a 0.2% failure window, that is roughly one lost event every second and a half — and a broker outage converts the entire outage duration into lost events at the full order rate.

Relay lag = poll interval ÷ 2 on average, plus publish time. A 200 ms poll adds around 100 ms of mean latency to every event, which is usually irrelevant and occasionally is not; CDC removes it.

Outbox growth during an outage = event rate × outage duration. 400 a second for eight minutes is roughly 200,000 rows, which is nothing for a database and worth having a retention job for anyway, because an outbox table that is never pruned becomes the largest table in the schema.

Where it breaks down

Ordering. The outbox gives you the database’s commit order, and a relay publishing in id order preserves it. But if the relay publishes in parallel for throughput, or if the broker partitions by key, per-entity ordering needs the same care as anywhere else — see queues and ordering.

Table growth. Prune published rows, or use a partitioned table you can drop. This is the most common operational complaint about the pattern and the easiest to prevent.

CDC is a system. Debezium plus Kafka Connect is real infrastructure with its own failure modes, and running it to avoid a polling loop is sometimes the wrong trade for a small service.

It does not give you exactly-once. The relay is at-least-once by construction. Consumers must deduplicate.

The event and the row can drift semantically. If the outbox row is written by different code than the order row, someone will eventually change one and not the other. Generating the event from the same domain object, in the same unit of work, is worth the discipline.

What people get wrong

“Publish inside the transaction.” You cannot — the broker is not in the transaction, and a successful publish followed by a rolled-back transaction is a phantom event. Some frameworks make this look possible by deferring the publish to after-commit, which is the db-then-publish failure with extra confidence.

“Retry the publish.” With what? After the process dies, the intent to publish existed only in memory. The outbox exists precisely to make that intent durable.

“Use two-phase commit.” XA across a database and a broker technically works and is rarely worth it: poor performance, poor support, and blocking behaviour on coordinator failure. The industry moved to the outbox because it uses one transactional resource and gets the same guarantee.

“It only matters at scale.” It matters at any scale where losing one order matters. A 0.1% loss rate on a thousand orders a day is one lost order a day, forever.

In production

Debezium is the standard CDC implementation and its documentation describes the outbox pattern directly, including a recommended table shape (aggregate_type, aggregate_id, type, payload). Most frameworks now ship an implementation — MassTransit, Rails’ after_commit plus a jobs table, Spring’s transactional event listeners paired with an outbox table.

The mirror pattern is worth naming too: the inbox, where a consumer records the message id it is about to process in the same transaction as the effect, which is exactly the idempotency constraint. Outbox on the way out, inbox on the way in, and the dual-write problem is closed at both ends.

The follow-up questions

“You write a row and publish an event. What if the publish fails?” — The whole page. Name the outbox before being led there.

“Why not two-phase commit?” — Performance, support, and coordinator-failure blocking. The outbox needs one transactional resource.

“How does the relay work and what if it dies?” — Polling or CDC, at-least-once, consumers deduplicate. Say what happens when it publishes and dies before marking.

“How do you keep the outbox table from growing forever?” — Prune or partition. It sounds trivial and it is the operational failure people actually hit.