Reliable Event Publishing With the Outbox Pattern in .NET

A modern Australian e-commerce platform built on ASP.NET Core can chew through thousands of orders during a Melbourne peak trading hour, and every confirmed order is supposed to trigger a downstream integration event. The order lives in SQL Server, the event is supposed to fly out over Azure Service Bus, and somewhere between those two destinations things inevitably drift. A network blip on the NBN, a throttled Service Bus namespace, or a pod recycling mid-deploy can leave a confirmed purchase with no corresponding message in the broker, and the warehouse, the loyalty engine, and the analytics warehouse all quietly disagree about what was actually sold.

The classic symptom is a row in the database that nobody downstream has heard about, and developers in Sydney and Brisbane meetups have spent years trading war stories about reconciling these gaps with cron jobs. The outbox pattern solves the dual-write dilemma by storing the intent to publish a message inside the same database transaction that mutates the aggregate, then handing the actual transmission off to a separate process that reads from that store.

The Dual Write Problem and How Outbox Solves It

When a service needs to update its own state and notify the rest of the world at the same time, it almost never has a tool that does both atomically. SQL Server guarantees atomicity across rows in a single database, but it has no idea what is happening inside Azure Service Bus, RabbitMQ, or Kafka. Developers reach for a TransactionScope, wrap the database commit and the message send inside it, and convince themselves that this is two-phase. It is not. The broker call is not enlisted in the database transaction, so a failure between the commit and the send leaves the system inconsistent, and a failure between the send and the commit sends a message about state that was never persisted.

The outbox pattern sidesteps this trap by collapsing the two writes into one. Inside the aggregate's transaction, alongside the change to the order row, the domain code appends a row to an OutboxMessages table describing what should be published. The database transaction commits, and from that moment the business state and the pending notification are guaranteed to move together. A separate worker process polls the table, ships each row to the broker, and only then marks the row as dispatched. If the worker crashes, the row is still there on the next pass. If the broker is down, the row waits patiently. The contract between the two sides becomes at-least-once delivery rather than a hopeful guess.

Anatomy of an Outbox Implementation in C#

A clean implementation has three moving parts. First, the aggregate raises domain events during its command handlers, just like in any well-modelled DDD codebase. Second, an IUnitOfWork or SaveChangesAsync interceptor captures those events, serialises them to JSON, and inserts an OutboxMessage row for each one within the same DbContext.SaveChanges call. Third, a hosted background service, often an IHostedService registered through HostBuilder.ConfigureServices, periodically queries the outbox table, publishes the payloads to the chosen broker, and updates a status column once the broker confirms receipt.

The interesting design choices sit in the seams between those parts. How are events serialised so that schema evolution does not break older rows? How long is the polling interval, and should it be a polling loop, a change feed, or a SqlTableDependency trigger? How are partial failures batched so that one poison message does not stall every other order sitting behind it? The pattern itself is small, but the production-grade version of it deserves attention to each of those seams, which is exactly the kind of deep dive that the annual C# conference sessions tend to land on.

Comparing Messaging Strategies

Teams often weigh three approaches before settling on the outbox. The table below captures the trade-offs in plain terms so the choice is easier to defend in an architecture review.

The outbox sits in the sweet spot for most .NET teams because it does not require a DTC coordinator, fits naturally on top of Entity Framework Core, and degrades gracefully when the broker is unavailable. Teams running on Azure tend to pair it with Service Bus sessions or Kafka topics, while shops with stricter data residency requirements, common in healthcare and finance across Australia, often keep the relay inside the same region as the primary database.

Modelling the Outbox Table and Entity Framework

The OutboxMessages table is intentionally dull. A typical schema carries an identifier, an aggregate type and identifier, a message type name, a JSON payload, a creation timestamp, a status flag, and a dispatched timestamp. The status flag distinguishes pending rows from completed ones and makes the relay query trivial. Adding an AttemptCount and a NextAttemptAt column turns the relay into a polite retry engine that respects exponential backoff without flooding the broker during a partial outage.

In Entity Framework Core, the outbox entity can be configured with ToTable("OutboxMessages") and a global query filter that hides dispatched rows by default, which keeps the change tracker lean. The interceptor that fills the table usually hooks the SavingChangesAsync event, walks every tracked entity, asks each one for its pending IDomainEvent collection through a small interface like IHasDomainEvents, and inserts an outbox row per event before the save actually executes. The aggregate never calls the broker, which keeps domain code blissfully unaware of messaging concerns and makes the pattern easy to retrofit into existing services that were built before message-driven architecture became fashionable.

Capturing Domain Events Inside a Transaction

The discipline that makes the pattern work is that domain events are raised inside the aggregate method, captured by a property on the entity, and only flushed to the outbox by the unit of work. A common implementation uses a base class like AggregateRoot with a private List<IDomainEvent> _events and a RaiseDomainEvent method. The command handler invokes order.Confirm(), which mutates state and raises OrderConfirmed, then calls await _unitOfWork.SaveChangesAsync(). The interceptor reads _events from the aggregate, clears the list, and writes them as outbox rows.

This shape has a nice side effect. Because events are persisted before the commit completes, they become part of the audit trail and can be replayed into a new environment during a disaster recovery exercise, which is a frequent requirement for APRA-regulated workloads. A Sydney fintech rebuilding its payments service after a regional outage can repopulate downstream systems simply by walking the outbox table from a point-in-time snapshot, which beats the alternative of squinting at logs and hoping nobody noticed the gap.

Building a Background Relay for At-Least-Once Delivery

The relay is a hosted service that runs a loop, picks up a batch of pending outbox rows ordered by creation time, and publishes them through an IMessagePublisher abstraction. After a successful publish it updates the row's status, and after a failure it bumps the attempt counter and schedules the next try. The loop runs forever, sleeps for a short interval when there is nothing to do, and uses a SemaphoreSlim or a CancellationTokenSource to shut down cleanly when the host stops. Most teams wrap the relay in its own deployment slot so that scaling the relay does not scale the API.

Concurrency deserves a real answer. Two relay pods competing on the same outbox table can publish the same message twice, which is acceptable as long as consumers are idempotent. If the team wants stronger ordering, an OUTPUT clause or a SELECT ... WITH (UPDLOCK, ROWLOCK) hint can serialise the claim so only one process owns a batch at a time. On Azure SQL, this is usually cheaper than reaching for Service Bus sessions, and it plays nicely with elastic pools common in Australian SMB deployments where the same SQL tier hosts dozens of small services.

Idempotency, Retries and Observability

At-least-once delivery means consumers will occasionally see the same event twice, and the contract must allow for that. Every payload should carry a stable identifier, usually the outbox row's primary key, and consumers should treat it as a deduplication key. A simple ProcessedMessages table, a Redis SETNX, or a Cosmos Upsert with a deterministic id all work, and the choice usually follows whatever the consumer already runs.

Retries need a ceiling. An outbox row that has failed fifty times probably points to a bug, a malformed schema, or a downstream system that has changed its contract, and the relay should mark it as poison and move on so the rest of the queue keeps flowing. Observability then becomes a matter of logging the move to poison state, raising a metric on attempt counts, and wiring a dashboard that shows the depth of the outbox table. A backlog that grows for more than a minute or two is a signal, and a backlog that grows for an hour is an incident. Australian teams that operate across AEDT and AWST timezones find that a 24-hour rolling dashboard is far more useful than a real-time chart, because the slow drift is what catches people out during the morning hand-over.