One PHP process published 10,000 messages to Kafka, waited for the broker to confirm every single one, and stopped the clock at 64 milliseconds. That is 157,266 confirmed messages per second, from PHP, with no delivery guarantee traded away. This post is the receipts: what I measured, how, and what surprised me along the way.
TL;DR: Ecotone's high-throughput publishing sends messages as provider-native batches and collects all broker confirmations before the business operation completes. Measured from one PHP process: Kafka 157,266 msg/s, Redis 115,638, RabbitMQ 67,143 (amqp-ext), Postgres 53,711, SQS 7,549 — every message individually confirmed before the operation completes.
ℹ️ Note: High-throughput publishing is a paid Ecotone Enterprise capability. A trial licence is available at ecotone.tech/pricing#trial, so every number below is reproducible on your machine.
Table of contents
- How I measured
- What each provider can actually do
- The numbers
- Where each number comes from: the two mechanisms
- Draining an outbox: the other half of the path
- What actually happens to a published message
- The code behind the benchmark
- Common questions
How I measured
I did not want to claim a number I could not hand to a skeptic. So the benchmark is a runnable demo — public at github.com/SimplyCodedSoftware/ecotone-publishing-throughput-demo — a Docker Compose setup with Kafka, RabbitMQ (on both Enqueue AMQP transports), LocalStack SQS, Redis and Postgres, plus one PHP process that publishes 10,000 messages and awaits every broker confirmation before stopping the clock, reporting the median of five iterations.
The rules I held myself to:
- The guarantee is constant. Every scenario waits for per-message broker confirmation. Comparing confirmed publishing against unconfirmed publishing would make the numbers look better and mean nothing.
- Warm, isolated runs. Each provider is measured on its own, best warm run reported, with the stable band across repeated runs recorded next to it.
- One PHP process. No worker pools, no parallel producers. The point is what a single process can do.
ℹ️ Note: Prerequisites for running it yourself — Docker Compose and an Ecotone Enterprise trial licence from ecotone.tech/pricing#trial. PHP itself runs inside the demo's containers.
What each provider can actually do
The numbers below only make sense with one thing said up front: high-throughput publishing is two independent mechanisms, not one, and a provider only gets the mechanism its protocol has.
| Provider | Batching | Non-blocking confirmation |
|---|---|---|
| Kafka | native batch produce | delivery reports collected at the end |
| RabbitMQ | one publisher-confirms round trip | confirms collected at the end |
| SQS | native batch send requests, sent concurrently | responses collected at the end |
| Postgres | one multi-row INSERT | not offered — the INSERT confirms itself |
| Redis | one scripted round trip | not offered — the reply is the confirmation |
Redis and Postgres confirm the write in the reply to the write, so there is nothing to defer — batching is their entire feature, and their configuration call takes no parameters. That is why they show one fast scenario in the results and the other three providers show two.
The numbers
Best isolated warm runs, one PHP process, 10,000 messages per run, median of five, brokers running locally:
| Provider | Batched publishing (absolute) | Band across isolated runs |
|---|---|---|
| Kafka | 157,266 msg/s — 10,000 confirmed in 64ms | 98,962–157,266 |
| Redis | 115,638 msg/s — 10,000 confirmed in 86ms | 111,374–115,638 |
| RabbitMQ (amqp-ext) | 67,143 msg/s — 10,000 confirmed in 149ms | 62,854–67,143 |
| Postgres | 53,711 msg/s | 48,887–53,711 |
| RabbitMQ (amqp-lib) | 36,294 msg/s | 31,222–36,294 |
| SQS (LocalStack) | 7,549 msg/s | 7,190–7,549 |
Two things in this table surprised me.
First, Redis lands second overall at 115,638 msg/s, behind only Kafka — despite having no deferred-confirmation mode at all. Redis acknowledges each command in its reply, so there is nothing to defer and batching is the entire feature; the same is true of Postgres at 53,711. The "boring" providers benefit as much as the streaming ones.
Second, RabbitMQ appears twice, because the two Enqueue AMQP transports are genuinely different code paths: amqp-lib hands the whole batch to the socket in a single write, amqp-ext writes per message — and amqp-ext still wins by roughly 1.8×, because php-amqplib encodes the AMQP protocol in pure PHP and that overhead costs more than the single socket write saves. It stays the recommended transport, and it is the one the RabbitMQ numbers here quote; both transports support the full feature set.
For context on why this matters: publishing one message at a time is round-trip-bound. Each publish serializes, sends, waits for the broker's confirmation, and only then starts the next one. Ten thousand of those waits, one after another, is what makes the per-message path take seconds. The sequencing is the cost. That is the cap being removed here — the common expectation for confirmed publishing from PHP is a few hundred messages per second.
Where each number comes from: the two mechanisms
The headline row is both mechanisms at once. Measured separately, they buy different things — worth knowing before you decide which switch you need.
Batching — gather and send together. Everything the handler published during the operation is collected and handed to the channel as one message instead of N sends; the provider maps it onto its own batch primitive. N round-trips become one, and the application never builds a batch. Redis and Postgres isolate this mechanism perfectly, since it is the only one they have:
| Provider | Per-message | Batched |
|---|---|---|
| Redis | 32,184 msg/s | 115,638 msg/s |
| Postgres | 16,421 msg/s | 53,711 msg/s |
Collapsing round-trips is worth the most exactly where the infrastructure is least forgiving: the less headroom the storage has, the more of the per-message path is pure waiting.
Non-blocking confirmation — stop waiting between messages. Every message is still written on its own, but the process no longer waits for each confirmation before writing the next: Kafka produces without flushing and drains delivery reports as it goes, RabbitMQ coalesces its publisher confirms, SQS dispatches requests concurrently. The waiting happens once, at the end of the scope, before the transaction commits:
| Provider | Per-message | Non-blocking confirm |
|---|---|---|
| Kafka | 9,053 msg/s | 23,421 msg/s — 2.6x |
| RabbitMQ (amqp-ext) | 13,754 msg/s | 20,261 msg/s — 1.5x |
| RabbitMQ (amqp-lib) | 9,776 msg/s | 15,252 msg/s — 1.6x |
| SQS (LocalStack) | 660 msg/s | 1,143 msg/s — 1.7x |
Every transport gains here, and the size of the gain tells you where it was spending its time. Kafka's 2.6x is the outlier: producing without flushing lets the client keep working while delivery reports drain behind it. The AMQP transports and SQS land between 1.5x and 1.7x, because each message is still its own write and only the confirmation wait gets coalesced.
Both, which is the default. The batch removes the round-trips, the deferred confirmation removes the waiting between what is left, and every confirmation is still collected before the transaction commits — Kafka 157,266 msg/s, RabbitMQ (amqp-ext) 67,143, RabbitMQ (amqp-lib) 36,294, SQS 7,549. Those are the rows in the table above.
Draining an outbox: the other half of the path
Every number above measures the write into the broker. Systems that use the outbox pattern have a second leg to pay for: the message is committed into the database with the business change, and a separate process relays it onward. That relay normally consumes the outbox like any other channel — one message per poll cycle, deserialized and republished on its own.
The same demo times that leg (./run.sh outbox). Batched forwarding replaces the outbox consumer with a publishing endpoint that claims rows straight from the database, groups them by target and hands over whole batches in wire format; the target has high-throughput publishing on, so a claim becomes one native broker batch. 10,000 messages waiting in the outbox:
| Target | Message by message | Batched, 100 rows per cycle | One claim of 10,000 |
|---|---|---|---|
| Kafka | 34.49s — 290 msg/s | 0.318s — 31,461 msg/s | 0.200s — 50,052 msg/s |
| RabbitMQ | 29.75s — 336 msg/s | 0.393s — 25,418 msg/s | 0.264s — 37,808 msg/s |
| Redis | 29.35s — 341 msg/s | 0.293s — 34,093 msg/s | 0.232s — 43,063 msg/s |
| SQS (LocalStack) | 43.67s — 229 msg/s | 1.889s — 5,293 msg/s | 1.472s — 6,794 msg/s |
Message by message, every row pays a poll cycle, a deserialize, its own transaction and its own publish — around 3ms each, more on SQS where each publish is an HTTP round trip. Batched forwarding claims rows in blocks and hands the target whole batches, which is the entire difference.
Which is the finding I did not expect: the relay was the bottleneck, not the broker. Every target lands within a tenth of a second of the others once the rows are claimed in one batch — except SQS, whose API caps a batch request at 10 entries, so 10,000 messages are still 1,000 HTTP round trips.
Batch size is a smaller knob than the batching itself: a hundred cycles of a hundred rows cost 1.3x to 1.6x what a single claim of 10,000 does, depending on the target, and the single claim buys that by holding the whole batch in memory inside one transaction. Delivery guarantees do not move either way. A failed delivery is released for redelivery rather than duplicating what already went out, and a connection failure rolls the cycle back for a clean retry.
What actually happens to a published message
The mechanism in one sentence: the waiting happens once, at the end. Messages fire to the broker the instant your handler emits them and travel while it keeps working; Ecotone collects every confirmation once, right before the transaction commits.
The business operation cannot complete until every confirmation is in hand. If a delivery fails, the operation fails, or the specific failed message routes to the error channel — individually, not as a whole batch. The consumer side never knows batching happened: it sees individual messages, retries individual messages, dead-letters individual messages.
What you observe when logging is exactly the order the test suite asserts:
transaction started
command handler executed
published batch of 2 messages to broker
delivery confirmations awaited
transaction committed
Confirmations are awaited once, after the handler finishes and before the commit — not per message.
The code behind the benchmark
The scenario behind the headline number is two lines:
// From the runnable throughput demo
$publisher->publishDeferred(buildBatch(), MediaType::TEXT_PLAIN)->resolve();
// Kafka: 10,000 confirmed in 64ms — 157,266 msg/sec
publishDeferred returns a Future; resolve() blocks until every broker confirmation arrives.
That is the explicit form. The form most applications will use requires no publishing code at all — a handler that publishes events stays exactly as it always was:
#[CommandHandler]
public function place(PlaceOrder $command, EventBus $eventBus): void
{
$eventBus->publish(new OrderWasPlaced($command->orderId));
$eventBus->publish(new OrderConfirmationRequested($command->orderId));
}
No batch objects, no futures. The handler publishes events one call at a time, as before.
High-throughput publishing is switched on in configuration, per channel or publisher:
final class MessagingConfiguration
{
// Events published from handlers are gathered, sent as one
// native batch, and confirmed before the operation completes
#[ServiceContext]
public function ordersChannel(): KafkaMessageChannelBuilder
{
return KafkaMessageChannelBuilder::create('orders')
->withHighThroughputPublishing();
}
// Message Publisher — enables publishDeferred() with a Future
#[ServiceContext]
public function orderPublisher(): AmqpMessagePublisherConfiguration
{
return AmqpMessagePublisherConfiguration::create()
->withHighThroughputPublishing();
}
}
One builder call per channel. The same call maps to each broker's native batching underneath.
Each provider gets what it does best under that one API: Kafka produces without flushing per message, RabbitMQ defers confirms, SQS sends native batch requests concurrently, Postgres writes one multi-row INSERT, Redis pipelines commands. The handler code never sees the difference.
Common questions
Were these runs against remote brokers?
No — brokers ran on the same machine, which is the right setup for measuring the publishing path itself. Over a real network the round-trip per message gets more expensive, not less, so per-message publishing degrades further while the batched path pays that latency once per batch.
Do any of these numbers skip broker confirmations?
No. Every scenario awaits per-message broker confirmation before the clock stops. A failed delivery fails the operation or routes that specific message to the error channel. That constraint is held constant across every broker and transport in the table.
Why is SQS so much slower than the others?
SQS itself caps how many entries fit in one native batch request, so more round-trips remain per 10,000 messages. It still lands at 7,549 msg/s confirmed — well above what a per-message loop achieves against the same endpoint.
Do my consumers need to understand batches?
No. Batching never survives the wire. Consumers receive individual messages; retries and dead-lettering operate on individual messages. Nothing downstream changes.
Run it yourself
Numbers you cannot reproduce are just claims. The demo is public at github.com/SimplyCodedSoftware/ecotone-publishing-throughput-demo: Docker Compose, five brokers, six configurations, one PHP process, and the clock only stops when the last confirmation arrives. Request a trial licence, run it on your laptop, and see where your band lands.
Trial licences are available at ecotone.tech/pricing#trial.
What throughput do you actually get out of PHP publishing in your own stack? The demo runs on any machine with Docker — I would genuinely like to see numbers that disagree with mine.
Originally published at blog.ecotone.tech.
Top comments (0)