DEV Community

Cover image for Transactional Outbox in Go: Reliable Event Publishing Without Distributed Transactions
Serif COLAKEL
Serif COLAKEL

Posted on

Transactional Outbox in Go: Reliable Event Publishing Without Distributed Transactions

Your API creates an order and commits it to PostgreSQL.

The next step is publishing an OrderCreated event to Kafka so that inventory, notifications, and analytics services can react.

You write something like this:

func (s *OrderService) CreateOrder(
    ctx context.Context,
    order Order,
) error {
    if err := s.repo.Create(ctx, order); err != nil {
        return err
    }

    return s.publisher.Publish(ctx, OrderCreated{
        OrderID: order.ID,
    })
}
Enter fullscreen mode Exit fullscreen mode

It looks straightforward.

But there is a failure window.

What happens if PostgreSQL commits the order successfully and Kafka is unavailable?

The order exists, but no event is published.

What if Kafka accepts the event, but your application crashes before recording that it was published?

Your application might publish the same event again after restarting.

These are not unusual edge cases. They are consequences of coordinating independent systems without a shared transaction.

The transactional outbox pattern addresses this problem by storing the business change and the event to be published in the same database transaction.

Instead of trying to atomically commit to PostgreSQL and Kafka, we make the database transaction the source of truth and publish events asynchronously.

In this article, we'll build a PostgreSQL-backed transactional outbox in Go, implement concurrent workers, handle retries and leases, and explore the delivery guarantees that matter in production.


1. The Dual-Write Problem

A dual write happens when an application needs to update two independent systems as part of one logical operation.

For example:

HTTP Request
     |
     v
Create Order in PostgreSQL
     |
     v
Publish OrderCreated to Kafka
Enter fullscreen mode Exit fullscreen mode

There are two possible execution orders.

Database first

1. PostgreSQL commit succeeds
2. Kafka publish fails
3. Application returns an error
Enter fullscreen mode Exit fullscreen mode

The order exists, but downstream services never receive the event.

Retrying the entire HTTP request might create another order unless the endpoint is idempotent.

Message broker first

1. Kafka publish succeeds
2. PostgreSQL commit fails
3. Application returns an error
Enter fullscreen mode Exit fullscreen mode

Now downstream services have received an event for an order that does not exist.

Neither order solves the problem.

A database transaction cannot automatically include a Kafka publish. They are independent systems with independent failure and commit semantics.

Distributed transactions and two-phase commit are possible in some environments, but they introduce operational and architectural complexity.

The transactional outbox pattern takes a different approach.

2. How the Transactional Outbox Works

The central idea is simple:

Store the business change and the event in the same database transaction.

Instead of publishing directly to the broker, insert an event into an outbox_events table.

                 HTTP Request
                      |
                      v
               PostgreSQL TX
                      |
              +-------+-------+
              |               |
              v               v
          Create Order    Insert Event
              |               |
              +-------+-------+
                      |
                    COMMIT
                      |
                      v
                 Outbox Worker
                      |
                      v
                 Message Broker
                      |
                      v
              Downstream Consumers
Enter fullscreen mode Exit fullscreen mode

If the transaction commits, both the order and the outbox event exist.

If the transaction rolls back, neither exists.

This eliminates the inconsistency between the business write and the durable record of the event that must be published.

It does not make publishing exactly once. We'll discuss that distinction later.

3. Designing the Outbox Table

Let's start with a PostgreSQL schema.

CREATE TABLE outbox_events (
    id BIGSERIAL PRIMARY KEY,

    event_id UUID NOT NULL UNIQUE,

    aggregate_type TEXT NOT NULL,
    aggregate_id TEXT NOT NULL,
    event_type TEXT NOT NULL,

    payload JSONB NOT NULL,

    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),

    published_at TIMESTAMPTZ,

    attempts INTEGER NOT NULL DEFAULT 0,
    next_attempt_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),

    claim_token UUID,
    lease_until TIMESTAMPTZ,

    last_error TEXT
);

CREATE INDEX idx_outbox_pending
    ON outbox_events (next_attempt_at, id)
    WHERE published_at IS NULL;
Enter fullscreen mode Exit fullscreen mode

Let's look at the important fields.

Field Purpose
event_id Stable identifier used for deduplication
aggregate_type Business entity type
aggregate_id Entity identifier
event_type Event name
payload Serialized event data
published_at Publication completion timestamp
attempts Number of publication attempts
next_attempt_at Earliest time the event can be retried
claim_token Identifies the worker currently holding the lease
lease_until When another worker can reclaim the event
last_error Most recent publication error

Why do we need both claim_token and lease_until?

A worker can crash after claiming an event. Without a lease, the event might remain stuck forever.

A lease makes the event available again after a bounded period.

The claim token prevents an old worker from updating a row after another worker has reclaimed it.

This is a form of ownership fencing for database updates. It does not prevent an expired worker from publishing to the broker, so consumers still need deduplication.

4. Writing the Business Change and Event Atomically

The outbox only works if the business write and event insertion share the same transaction.

Let's define an event:

package outbox

import (
    "context"
    "encoding/json"
    "fmt"
    "time"

    "github.com/google/uuid"
)

type Event struct {
    ID            uuid.UUID
    AggregateType string
    AggregateID   string
    Type          string
    Payload       json.RawMessage
    CreatedAt     time.Time
}
Enter fullscreen mode Exit fullscreen mode

A repository method can insert the event using the same sql.Tx as the business operation.

func Insert(
    ctx context.Context,
    tx *sql.Tx,
    event Event,
) error {
    const query = `
        INSERT INTO outbox_events (
            event_id,
            aggregate_type,
            aggregate_id,
            event_type,
            payload,
            created_at
        )
        VALUES ($1, $2, $3, $4, $5, $6)
    `

    _, err := tx.ExecContext(
        ctx,
        query,
        event.ID,
        event.AggregateType,
        event.AggregateID,
        event.Type,
        event.Payload,
        event.CreatedAt,
    )

    if err != nil {
        return fmt.Errorf("insert outbox event: %w", err)
    }

    return nil
}
Enter fullscreen mode Exit fullscreen mode

Now let's use it in an order service.

func (s *OrderService) Create(
    ctx context.Context,
    req CreateOrderRequest,
) error {
    tx, err := s.db.BeginTx(ctx, nil)
    if err != nil {
        return fmt.Errorf("begin transaction: %w", err)
    }

    defer tx.Rollback()

    order := Order{
        ID:        uuid.NewString(),
        Customer:  req.CustomerID,
        Amount:    req.Amount,
        CreatedAt: time.Now().UTC(),
    }

    if err := s.orders.Insert(ctx, tx, order); err != nil {
        return fmt.Errorf("insert order: %w", err)
    }

    payload, err := json.Marshal(OrderCreated{
        OrderID:   order.ID,
        Customer:  order.Customer,
        Amount:    order.Amount,
        CreatedAt: order.CreatedAt,
    })
    if err != nil {
        return fmt.Errorf("marshal event: %w", err)
    }

    event := outbox.Event{
        ID:            uuid.New(),
        AggregateType: "order",
        AggregateID:   order.ID,
        Type:          "order.created",
        Payload:       payload,
        CreatedAt:     time.Now().UTC(),
    }

    if err := outbox.Insert(ctx, tx, event); err != nil {
        return fmt.Errorf("insert event: %w", err)
    }

    if err := tx.Commit(); err != nil {
        return fmt.Errorf("commit order transaction: %w", err)
    }

    return nil
}
Enter fullscreen mode Exit fullscreen mode

There is no Kafka call inside this transaction.

That is intentional.

The transaction only guarantees that the order and its event record are persisted together.

The actual event publication happens asynchronously.

What if the commit response is lost?

Suppose PostgreSQL commits successfully, but the connection drops before the application receives the commit response.

The application might not know whether the transaction succeeded.

This is another reason the HTTP operation should be idempotent.

A retry should not blindly create another order and another event. Use the idempotency mechanism from the previous article, or a durable business uniqueness constraint, to recognize the logical operation.

The outbox guarantees atomic persistence of the event record with the business write. It does not replace request-level idempotency.

5. Claiming Events With Multiple Workers

A single worker can process pending events, but production systems often need multiple workers.

Imagine three worker instances:

Worker A
Worker B
Worker C
Enter fullscreen mode Exit fullscreen mode

If all three execute the same query:

SELECT *
FROM outbox_events
WHERE published_at IS NULL;
Enter fullscreen mode Exit fullscreen mode

they can select the same events.

That creates duplicate work immediately.

PostgreSQL provides a useful mechanism for this problem:

FOR UPDATE SKIP LOCKED.

It allows workers to claim different rows without waiting for rows already locked by other workers.

However, we should not hold a database transaction open while publishing to Kafka.

Instead, we use a short transaction to claim a batch, commit it, and publish outside the transaction.

Claiming a batch

WITH candidates AS (
    SELECT id
    FROM outbox_events
    WHERE
        published_at IS NULL
        AND next_attempt_at <= NOW()
        AND (
            lease_until IS NULL
            OR lease_until < NOW()
        )
    ORDER BY id
    FOR UPDATE SKIP LOCKED
    LIMIT $1
)
UPDATE outbox_events AS events
SET
    claim_token = $2,
    lease_until = NOW() + ($3 * INTERVAL '1 second'),
    attempts = attempts + 1
FROM candidates
WHERE events.id = candidates.id
RETURNING
    events.id,
    events.event_id,
    events.aggregate_type,
    events.aggregate_id,
    events.event_type,
    events.payload,
    events.attempts;
Enter fullscreen mode Exit fullscreen mode

Parameters:

  • $1: batch size.
  • $2: unique claim token for this batch.
  • $3: lease duration in seconds.

The FOR UPDATE SKIP LOCKED clause prevents workers from claiming the same locked rows.

The UPDATE records ownership and establishes a lease.

Once the transaction commits, the worker can publish the returned events.

A simplified Go representation:

type ClaimedEvent struct {
    ID            int64
    EventID       uuid.UUID
    AggregateType string
    AggregateID   string
    Type          string
    Payload       []byte
    Attempts      int
}

type Claim struct {
    Token  uuid.UUID
    Events []ClaimedEvent
}
Enter fullscreen mode Exit fullscreen mode

The repository method should execute the claim query in a short transaction and return the rows after committing.

The key design rule:

Never keep database row locks open while waiting for a message broker.

A slow broker should not hold database transactions and locks indefinitely.

6. Publishing Events Outside the Transaction

Define a small publisher interface:

type Publisher interface {
    Publish(
        ctx context.Context,
        event ClaimedEvent,
    ) error
}
Enter fullscreen mode Exit fullscreen mode

The implementation can use Kafka, RabbitMQ, NATS, or another messaging system.

The outbox worker should not depend on the broker's concrete client.

Now let's define the worker:

type Worker struct {
    repo      Repository
    publisher Publisher

    batchSize int
    lease     time.Duration
}
Enter fullscreen mode Exit fullscreen mode

A useful repository interface:

type Repository interface {
    Claim(
        ctx context.Context,
        limit int,
        lease time.Duration,
    ) (Claim, error)

    MarkPublished(
        ctx context.Context,
        id int64,
        token uuid.UUID,
    ) error

    ScheduleRetry(
        ctx context.Context,
        id int64,
        token uuid.UUID,
        delay time.Duration,
        cause error,
    ) error
}
Enter fullscreen mode Exit fullscreen mode

The worker processes one claimed batch:

func (w *Worker) ProcessBatch(
    ctx context.Context,
) error {
    claim, err := w.repo.Claim(
        ctx,
        w.batchSize,
        w.lease,
    )
    if err != nil {
        return fmt.Errorf("claim outbox events: %w", err)
    }

    for _, event := range claim.Events {
        if err := ctx.Err(); err != nil {
            return err
        }

        publishCtx, cancel := context.WithTimeout(
            ctx,
            10*time.Second,
        )

        err := w.publisher.Publish(publishCtx, event)
        cancel()

        if err != nil {
            delay := retryDelay(event.Attempts)

            scheduleErr := w.repo.ScheduleRetry(
                ctx,
                event.ID,
                claim.Token,
                delay,
                err,
            )

            if scheduleErr != nil {
                return fmt.Errorf(
                    "schedule retry for event %s: %w",
                    event.EventID,
                    scheduleErr,
                )
            }

            continue
        }

        if err := w.repo.MarkPublished(
            ctx,
            event.ID,
            claim.Token,
        ); err != nil {
            return fmt.Errorf(
                "mark event %s as published: %w",
                event.EventID,
                err,
            )
        }
    }

    return nil
}
Enter fullscreen mode Exit fullscreen mode

This is a sequential batch processor. You can add bounded parallelism later, but the ownership and completion rules should remain the same.

Notice that the worker does not delete an event immediately after publishing it.

It marks the event as published.

This makes it possible to inspect publication history, measure delivery delays, and troubleshoot failures.

For large systems, published rows should eventually be archived or removed according to a retention policy.

7. Marking Events as Published Safely

The claim token must be checked when updating an event.

UPDATE outbox_events
SET
    published_at = NOW(),
    claim_token = NULL,
    lease_until = NULL,
    last_error = NULL
WHERE
    id = $1
    AND claim_token = $2
    AND published_at IS NULL;
Enter fullscreen mode Exit fullscreen mode

Why is this important?

Consider this sequence:

  1. Worker A claims an event.
  2. Worker A pauses for a long time.
  3. Its lease expires.
  4. Worker B reclaims the event.
  5. Worker A resumes.

Without an ownership check, Worker A could update the event even though Worker B now owns it.

The claim token prevents that stale update.

Your repository should also check RowsAffected().

If no row was updated, the claim is no longer owned by that worker, or the event has already been completed.

However, there is an important limitation.

The claim token protects database state. It cannot stop Worker A from publishing to Kafka after its lease expires.

A lease is not a distributed lock over the external broker.

That is why consumer-side deduplication remains necessary.

8. Retry Strategy and Backoff

Message brokers can be temporarily unavailable.

A worker should not retry a failed event continuously.

That creates unnecessary load and can make an existing outage worse.

Use exponential backoff with jitter.

func retryDelay(attempt int) time.Duration {
    const (
        base = time.Second
        max  = 5 * time.Minute
    )

    if attempt < 1 {
        attempt = 1
    }

    exponent := min(attempt-1, 8)

    delay := base * time.Duration(1<<exponent)

    if delay > max {
        delay = max
    }

    jitter := time.Duration(
        rand.Int64N(int64(delay/2) + 1),
    )

    return delay/2 + jitter
}
Enter fullscreen mode Exit fullscreen mode

This implementation assumes a Go version with math/rand/v2.

The delay grows with each failed attempt but is capped to prevent excessively long retry intervals.

The retry update can look like this:

UPDATE outbox_events
SET
    next_attempt_at = NOW() + ($1 * INTERVAL '1 second'),
    claim_token = NULL,
    lease_until = NULL,
    last_error = $2
WHERE
    id = $3
    AND claim_token = $4
    AND published_at IS NULL;
Enter fullscreen mode Exit fullscreen mode

The repository should pass a bounded, sanitized error message rather than storing arbitrary error output indefinitely.

What about permanently failing events?

Some events will never succeed without intervention.

Examples:

  • Invalid payload.
  • Unsupported schema version.
  • Message exceeds broker limits.
  • A downstream system permanently rejects the event.

A production design needs a poison-event policy.

Possible approaches:

  • Maximum attempt count.
  • Dead-letter table or queue.
  • Manual replay.
  • Alerting and operational investigation.
  • Event quarantine without losing the original payload.

Do not silently delete failed events.

A dead-letter policy should preserve enough information to investigate and replay the operation safely.

Also remember that a maximum retry count is a business and operational decision. Some important events may require indefinite retry with alerting rather than permanent abandonment.

9. The Most Important Failure Window: Publish Succeeds, Database Update Fails

Let's examine the most important scenario.

1. Worker claims event
2. Worker publishes event to Kafka
3. Kafka acknowledges the message
4. Worker crashes
5. published_at is never updated
6. Lease expires
7. Another worker claims and publishes the event again
Enter fullscreen mode Exit fullscreen mode

The event is delivered twice.

This is expected behavior.

The outbox pattern provides durable, at-least-once publication when configured with retries and recovery. It does not provide exactly-once delivery across PostgreSQL and an external broker.

The consumer must be idempotent.

This is not an optional optimization. It is part of the design.

10. Consumer-Side Deduplication

Every event should have a stable identifier.

For example:

{
  "event_id": "b89b5f43-0d7c-45d7-8a9b-9dbcf9fcb1f8",
  "event_type": "order.created",
  "aggregate_id": "order-123",
  "payload": {
    "order_id": "order-123"
  }
}
Enter fullscreen mode Exit fullscreen mode

The consumer stores processed event IDs.

CREATE TABLE processed_events (
    consumer_name TEXT NOT NULL,
    event_id UUID NOT NULL,
    processed_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),

    PRIMARY KEY (consumer_name, event_id)
);
Enter fullscreen mode Exit fullscreen mode

The consumer should insert the deduplication record and apply its business change in the same transaction.

func (c *Consumer) Handle(
    ctx context.Context,
    event Event,
) error {
    tx, err := c.db.BeginTx(ctx, nil)
    if err != nil {
        return fmt.Errorf("begin transaction: %w", err)
    }

    defer tx.Rollback()

    result, err := tx.ExecContext(
        ctx,
        `
        INSERT INTO processed_events (
            consumer_name,
            event_id
        )
        VALUES ($1, $2)
        ON CONFLICT DO NOTHING
        `,
        "inventory-service",
        event.ID,
    )
    if err != nil {
        return fmt.Errorf("insert processed event: %w", err)
    }

    affected, err := result.RowsAffected()
    if err != nil {
        return fmt.Errorf("read affected rows: %w", err)
    }

    if affected == 0 {
        // Already processed.
        return tx.Commit()
    }

    if err := c.inventory.ApplyEvent(ctx, tx, event); err != nil {
        return fmt.Errorf("apply inventory event: %w", err)
    }

    if err := tx.Commit(); err != nil {
        return fmt.Errorf("commit consumer transaction: %w", err)
    }

    return nil
}
Enter fullscreen mode Exit fullscreen mode

The broker message should only be acknowledged after the database transaction commits.

If the consumer crashes before committing, the transaction rolls back and the message can be delivered again.

If the transaction commits but the broker acknowledgement is lost, the message may be delivered again. The processed_events constraint makes the second delivery harmless.

There is still an important boundary:

If the consumer's business operation calls an external API, that external side effect needs its own idempotency or outbox strategy.

Deduplication only protects effects committed inside the same transaction.

11. Event Ordering Is a Separate Problem

The outbox query uses:

ORDER BY id
Enter fullscreen mode Exit fullscreen mode

Does this guarantee that every event reaches consumers in business order?

Not necessarily.

Multiple workers can claim different events and publish them at different speeds.

Consider:

Order 123:

1. order.created
2. order.paid
3. order.shipped
Enter fullscreen mode Exit fullscreen mode

Worker A claims order.created and becomes slow.

Worker B claims order.paid and publishes it first.

The broker might receive the events in the wrong order.

If ordering matters, define the ordering boundary explicitly.

Possible strategies include:

  • Partition events by aggregate ID.
  • Use a stable aggregate key in Kafka.
  • Maintain an aggregate version or sequence number.
  • Ensure only one active publisher processes a given aggregate.
  • Use a per-aggregate dispatch mechanism.

For example, add:

ALTER TABLE outbox_events
ADD COLUMN aggregate_version BIGINT;
Enter fullscreen mode Exit fullscreen mode

Then enforce uniqueness where appropriate:

CREATE UNIQUE INDEX idx_outbox_aggregate_version
ON outbox_events (
    aggregate_type,
    aggregate_id,
    aggregate_version
);
Enter fullscreen mode Exit fullscreen mode

A stable Kafka partition key helps preserve order within a partition, but it does not fix out-of-order publication before messages reach Kafka.

If strict per-aggregate ordering is required, the dispatcher must also enforce it.

This is an important trade-off: stronger ordering constraints can reduce parallelism.

12. Running the Worker Continuously

The worker needs a lifecycle.

A simple polling loop:

func (w *Worker) Run(ctx context.Context) error {
    ticker := time.NewTicker(500 * time.Millisecond)
    defer ticker.Stop()

    for {
        if err := w.ProcessBatch(ctx); err != nil {
            if ctx.Err() != nil {
                return ctx.Err()
            }

            w.logger.Error(
                "outbox batch failed",
                "error",
                err,
            )
        }

        select {
        case <-ctx.Done():
            return ctx.Err()

        case <-ticker.C:
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

This is intentionally simple.

A production worker should also consider:

  • Immediate processing when work is found.
  • Adaptive polling when the queue is empty.
  • Bounded parallel publishing.
  • Graceful shutdown.
  • Lease renewal for long-running publication.
  • Per-event timeouts.
  • Database and broker connection health.
  • Backpressure when the broker slows down.

One detail is worth emphasizing:

The lease duration must be longer than the expected processing time, or the worker must renew its lease while processing.

Otherwise, another worker may reclaim an event while the original worker is still publishing it.

Even with lease renewal, consumer deduplication remains the final protection against duplicate delivery.

13. Testing the Outbox

A few unit tests are useful, but the most important guarantees involve database transactions and concurrent workers.

Use PostgreSQL integration tests for the following scenarios.

Test 1: Business transaction rollback

  • Insert an order.
  • Insert an outbox event.
  • Force the transaction to fail.
  • Verify that neither record exists.

Test 2: Concurrent workers

  • Insert multiple pending events.
  • Start several workers concurrently.
  • Verify that no two active claims own the same row at the same time.
  • Verify that all events eventually become eligible for publication.

Test 3: Publish succeeds, mark-published fails

  • Make the publisher succeed.
  • Simulate a database failure during MarkPublished.
  • Let the lease expire.
  • Verify that the event can be published again.
  • Verify that the consumer processes the event only once.

Test 4: Worker crash and lease recovery

  • Claim an event.
  • Simulate a worker crash.
  • Wait until the lease expires.
  • Verify that another worker can claim it.

Test 5: Stale worker

  • Worker A claims an event.
  • Its lease expires.
  • Worker B reclaims it.
  • Worker A attempts to update the row.
  • Verify that the old claim token cannot update the event.

Test 6: Poison event

  • Configure the publisher to fail permanently.
  • Verify retry scheduling.
  • Verify that the event reaches the configured dead-letter or alerting policy.

Test 7: Ordering

If your application requires per-aggregate ordering:

  • Insert multiple events for the same aggregate.
  • Process them with multiple workers.
  • Verify that consumers observe the expected sequence.

Avoid testing only the happy path. Most outbox bugs appear between two successful operations.

14. Observability: Know When Events Are Falling Behind

A transactional outbox can be functioning correctly while downstream systems are still experiencing serious delays.

You need visibility into the queue.

Useful metrics include:

outbox_pending_events
outbox_oldest_pending_age_seconds
outbox_publish_attempts_total
outbox_publish_failures_total
outbox_publish_duration_seconds
outbox_events_published_total
outbox_lease_expirations_total
outbox_dead_letter_events_total
Enter fullscreen mode Exit fullscreen mode

Some useful operational signals:

Signal What it might indicate
Pending event count increasing Publisher throughput is below event creation rate
Oldest event age increasing Events are not being delivered quickly enough
Retry count increasing Broker or network instability
Lease expirations increasing Workers are crashing or taking too long
Dead-letter count increasing Events require investigation

Avoid using event IDs, aggregate IDs, or customer IDs as metric labels. They create high-cardinality metrics.

For logs and traces, event IDs and aggregate IDs are useful correlation fields.

A particularly valuable metric is:

NOW() - MIN(created_at)
Enter fullscreen mode Exit fullscreen mode

for unpublished events.

The oldest pending event age often tells you more about the health of the pipeline than the total pending count.

A queue containing 10,000 events might be healthy if they were created seconds ago. A queue containing 20 events might be unhealthy if the oldest has been waiting for an hour.

15. Scaling and Operational Trade-offs

The transactional outbox is straightforward to introduce, but the operational model deserves attention.

Polling vs. CDC

The implementation in this article uses polling.

A worker periodically queries PostgreSQL for pending events.

Advantages:

  • Simple architecture.
  • No additional CDC infrastructure.
  • Easy to inspect and operate.
  • Works with standard PostgreSQL deployments.

Disadvantages:

  • Polling creates database queries even when the queue is empty.
  • High event volume can increase database load.
  • Batch sizing and polling intervals need tuning.

An alternative is Change Data Capture (CDC).

Tools such as Debezium can capture database changes from the PostgreSQL write-ahead log and publish them to a broker.

Advantages:

  • Avoids repeated polling queries.
  • Can scale well for high-volume event streams.
  • Integrates naturally with Kafka-based architectures.

Disadvantages:

  • Additional infrastructure and operational complexity.
  • Schema and connector management.
  • More complex replay and recovery behavior.
  • Still requires idempotent consumers.

The right choice depends on throughput, latency requirements, operational capacity, and existing infrastructure.

Retention and cleanup

Published events should not accumulate forever.

A cleanup job can delete old published events in batches:

WITH old_events AS (
    SELECT id
    FROM outbox_events
    WHERE published_at < NOW() - INTERVAL '7 days'
    ORDER BY id
    LIMIT 1000
)
DELETE FROM outbox_events
WHERE id IN (
    SELECT id FROM old_events
);
Enter fullscreen mode Exit fullscreen mode

The retention period should reflect debugging, replay, compliance, and recovery requirements.

For larger systems, consider partitioning by creation time or moving published events to an archive.

Never remove unpublished events just because they are old.

Backpressure

If event creation is faster than publication, the outbox grows.

A worker cannot solve an indefinitely increasing backlog by polling more frequently.

Monitor the publication rate and pending age.

Consider bounded publishing concurrency, broker capacity, database capacity, and alert thresholds.

The outbox is durable storage, not infinite capacity.


16. Production Checklist

Before deploying a transactional outbox, verify the following:

  • [ ] Business changes and outbox inserts share one database transaction.
  • [ ] Every event has a stable unique ID.
  • [ ] Event payloads have an explicit schema and versioning strategy.
  • [ ] Workers claim rows using FOR UPDATE SKIP LOCKED.
  • [ ] Database transactions are short and never span broker publication.
  • [ ] Claims have bounded leases.
  • [ ] Stale workers cannot update records owned by newer claims.
  • [ ] Retry scheduling uses exponential backoff and jitter.
  • [ ] Poison events have an explicit handling policy.
  • [ ] Consumers deduplicate events using durable event IDs.
  • [ ] Consumer deduplication and business writes share a transaction.
  • [ ] Ordering requirements are explicitly defined.
  • [ ] Graceful shutdown and lease recovery are tested.
  • [ ] Pending event age and retry rates are monitored.
  • [ ] Published event retention and cleanup are configured.
  • [ ] HTTP operations that create events are idempotent.

Conclusion

Reliable event publishing is not just about calling Kafka after a database insert.

It is about handling the failure windows between independent systems.

The transactional outbox pattern gives your application a durable boundary: when a business transaction commits, the event that represents that change is committed with it.

A separate worker handles publication, retries failures, and recovers abandoned work.

But the pattern does not guarantee exactly-once delivery.

A publisher can crash after the broker accepts an event. A lease can expire while a worker is still running. A message can be delivered more than once.

That's why the complete design needs stable event IDs, consumer-side deduplication, explicit ordering rules, and operational visibility.

Don't try to make two independent systems commit together. Make the intent durable in one transaction, then deliver it reliably.

That's the foundation of a dependable event-driven architecture in Go.


Further Reading

Top comments (0)