The transactional outbox pattern

The problem

Consider a food delivery app that promises delivery in ten minutes. The Order Service asks Inventory to reserve the items and Delivery to assign an agent. Both services publish completion events through a message broker such as Kafka, SQS, or RabbitMQ. The Order Service waits for both events before confirming the order.

For Inventory, completing its part requires two writes:

1. Save the reservation in its database.
2. Publish an InventoryReserved event to the broker.

Because the database and broker are separate systems, a local database transaction cannot cover both writes. One can succeed while the other fails, leaving the services with inconsistent state. This is called a dual-write problem.

What if we publish inside the database transaction?

In one of my projects, I put the database update and the broker publish call inside the same database transaction, assuming both operations would succeed or fail together.

For Inventory, that would look like this:

BEGIN;

INSERT INTO reservations (order_id, item_id, quantity)
VALUES (:order_id, :item_id, :quantity);

broker.publish('InventoryReserved', { order_id: :order_id });

COMMIT;

This looks like it solves the problem, but it is a classic anti-pattern because the database transaction has no control over the broker. Once the broker accepts the event, rolling back the reservation cannot take it back, which leaves room for the following failures.

Failure scenarios with this transaction

Scenario A: Publishing succeeds, but the database commit fails

After the broker acknowledges InventoryReserved, Inventory tries to commit the database transaction. If the commit fails and the transaction rolls back, the event is still available to the Order Service even though the reservation was never committed.

Scenario B: Publishing times out after the broker accepts the event

The broker accepts InventoryReserved, but its acknowledgement never reaches Inventory. When the publish call times out, the application rolls back the reservation even though the broker already has the event.

Since Inventory cannot tell whether the publish succeeded, retrying can send another copy of the event. The Order Service can use the event ID to avoid processing it twice, but that still does not fix the missing reservation.

Scenario C: Inventory crashes after publishing but before committing

If Inventory crashes after the broker accepts the event but before COMMIT, the database rolls back the open transaction once it detects the lost connection. The broker can still deliver the event to the Order Service, leaving it with a confirmation for a reservation that was rolled back.

Wrapping the calls in a try/catch cannot prevent this because the error handler cannot run after the process has crashed.

Why the local transaction is not enough

Besides these failure cases, waiting for the broker’s acknowledgement delays the database commit and keeps any acquired locks held until the transaction ends.

A local transaction cannot make these two writes atomic because it only controls the database. Coordinating the database and broker would require a distributed transaction supported by both systems. The outbox avoids that requirement by keeping both writes in one database.

The transactional outbox pattern

The outbox pattern separates saving the reservation from publishing its event. Inventory is responsible for saving the reservation and recording the event in an outbox table. Publishing that event to the external broker becomes the sole responsibility of a separate background worker called the outbox relay.

The reservation and outbox entry are written in the same atomic transaction within Inventory’s database, so either both are committed or both are rolled back. Once they are committed, the relay reads the outbox entry and publishes the event. Inventory can finish its database transaction without waiting for the broker’s acknowledgement.

The database’s write-ahead log supports its own crash recovery. The outbox records work still to be delivered. They have different responsibilities.

A relay can crash after publishing but before marking a row as sent. It will publish that row again after recovery, so the Order Service uses an idempotency key to recognise and skip an event it has already processed. An outbox closes the gap between a database update and a durable intent to publish, but does not eliminate duplicate delivery.

Example: Publishing an inventory completion event

sequenceDiagram
    autonumber
    participant I as Inventory Service
    participant DB as Inventory Database
    participant R as Outbox Relay
    participant B as Message Broker
    participant O as Order Service

    I->>DB: Begin transaction
    I->>DB: Reserve inventory
    I->>DB: Insert InventoryReserved event E1 into outbox
    I->>DB: Commit reservation and E1
    DB-->>I: Committed
    Note over I,DB: Inventory can crash here<br/>without losing the outgoing event

    R->>DB: Read committed, unsent outbox rows
    DB-->>R: Event E1
    R->>B: Publish E1
    B-->>R: Publication accepted
    R->>DB: Mark E1 as SENT

    B->>O: Deliver E1
    O->>O: Commit order update and completion record
    O->>B: Acknowledge E1

The relay reads only committed rows. If the reservation transaction rolls back, the outbox insert rolls back too. The relay has nothing to publish for that failed reservation.

Recovering a relay after a crash

If the relay crashes before publishing, it can read the unsent row and try again. If it crashes after the broker accepts the event but before saving SENT, the next attempt can publish a duplicate.

Duplicate delivery is an expected part of recovery, so keep the same idempotency key when retrying. The receiving service uses that key to recognise work it has already committed and skip processing it again. This lets the relay safely retry unsent rows and mark them as SENT only after the broker acknowledges publication.