DEV Community: yuwnloyblog

Benchmarking Real-Time Messaging Without Mixing Up ACKs, Deliveries, and Fan-Out

yuwnloyblog — Mon, 20 Jul 2026 10:17:33 +0000

A messaging benchmark can report an impressive number and still tell you almost nothing.

“Messages per second” might mean WebSocket frames written by a load generator, publishes accepted by a server, messages committed to storage, or callbacks observed by recipients. In group chat, one accepted publish may produce hundreds or thousands of deliveries. Connection storms and steady-state delivery exercise different parts of a system, yet they are often rolled into one average.

We ran into this problem while improving performance transparency for JuggleIM, an open-source messaging server written in Go. Before publishing bigger numbers, we wanted a benchmark whose semantics were explicit and whose results could be reproduced.

The result is a small Go harness for private and group chat that reports connection setup, server acknowledgements, and recipient delivery separately. This article explains the decisions behind it, the mistakes it tries to avoid, and how to run it yourself.

Start by defining what a message means

For a real-time messaging system, at least four events matter:

Connection established: the client completed the production authentication and WebSocket handshake.
Publish attempted: the load generator handed a message to a connected client.
Publish acknowledged: the server accepted the publish and returned its protocol-level ACK.
Delivery observed: a recipient client decoded the message and invoked its application callback.

These events answer different questions.

Connection latency tells you how quickly a deployment can admit clients. ACK latency covers client serialization, transport, server processing, and the ACK return path. Delivery latency covers the full sender-to-recipient path. Delivery count reveals fan-out.

Combining them into one number hides the system behavior we actually want to understand.

Two workloads, two different shapes

The harness implements two workloads using JuggleIM's production Protobuf-over-WebSocket protocol.

Private chat: a ring

Every connected client is a sender. Clients form a ring:

client 1 -> client 2
client 2 -> client 3
...
client N -> client 1

The configured rate is the aggregate publish rate across the ring. This spreads work across clients without generating a quadratic number of conversations or concentrating all sends on one connection.

For every measured publish, the harness records:

publish-to-ACK latency;
whether the ACK succeeded or failed;
publish-to-recipient-callback latency;
the number of recipient callbacks observed.

Group chat: explicit fan-out

All clients join one synthetic group. A configurable subset acts as senders while every connected member can receive messages.

The report does not call recipient callbacks “message throughput.” It reports two rates:

ACK throughput: accepted group publishes per second;
delivery throughput: callbacks observed across group members per second.

If 100 group members receive one accepted publish, that is one successful ACK and roughly 100 delivery events. Both values are useful. Treating them as interchangeable is not.

Measure connection establishment separately

Opening thousands of authenticated WebSocket connections exercises token validation, connection state, file descriptors, goroutine scheduling, and network setup. Steady-state messaging exercises routing, acknowledgements, persistence, conversation state, and recipient fan-out.

The harness therefore completes and records all connection attempts before warm-up begins. It refuses to publish a partial baseline if any configured client fails to connect.

Connection metrics include attempts, successes, failures, error codes, and P50/P95/P99 latency. They are never mixed into steady-state message latency.

Warm up before recording

The first few operations of a process may include lazy initialization, empty caches, new database connections, or one-time allocations. Measuring them together with steady state makes short tests especially noisy.

Each workload has three phases:

setup -> connect -> warm up -> measure -> delivery grace

Warm-up messages use the real protocol and storage behavior but are tagged as warmup and excluded from the report. Measured messages carry a run ID, phase, sequence number, nanosecond send timestamp, and enough padding to reach the requested payload size.

After sending stops, a configurable grace period collects deliveries already in flight.

Use production semantics, not a convenient mock

The benchmark creates synthetic users through JuggleIM's signed server API, obtains real client tokens, and connects through the same WebSocket protocol used by client SDKs. Group workloads create actual group membership before clients connect.

Stored and counted text messages are enabled by default. Turning persistence off would produce a different and usually much easier workload, so the setting is included in every report.

The benchmark application secret is read only from an environment variable. It is not placed in process arguments or written into result files.

Percentiles are more useful than one average

An average can look healthy while a meaningful fraction of users experience long delays. The report includes minimum, mean, P50, P95, P99, and maximum latency for:

connection establishment;
successful publish acknowledgements;
observed deliveries.

Failures are counted by error code instead of being silently discarded or folded into successful latency.

The JSON report also records the workload and environment:

server commit and dirty-tree state;
client count and group sender count;
target rate, warm-up, duration, and delivery grace;
payload size and persistence flags;
OS, architecture, Go version, CPU model, logical CPU count, and memory;
database and deployment labels.

Without that context, two benchmark files are not meaningfully comparable.

A verified smoke run

We checked the full workflow against the repository's local Docker Compose stack. The server and load generator shared one Apple M2 host, the measurement lasted only ten seconds, and the working tree contained the new harness.

That makes this a functional smoke baseline, not a production capacity claim.

The workload used 20 connected clients, stored and counted 256-byte messages, a three-second warm-up, a ten-second measurement, and a target of 50 publishes per second.

Scenario	Connections	Successful ACKs	ACK P95 / P99	Observed deliveries	Delivery P95 / P99
Private ring	20/20	499/499	2.37 / 17.09 ms	499	8.47 / 36.75 ms
20-member group, 2 senders	20/20	499/499	4.54 / 30.19 ms	9,492	35.29 / 115.88 ms

The group workload makes the distinction visible: 499 accepted publishes created 9,492 observed client deliveries. Saying the system handled either “49.9 messages per second” or “949.2 messages per second” without naming the metric would be misleading.

The full private-chat JSON and group-chat JSON are public.

Run both workloads locally

The one-command runner requires Docker Compose, Go, curl, jq, openssl, and shasum:

git clone https://github.com/juggleim/im-server.git
cd im-server
scripts/run-benchmark.sh all

It builds and starts the isolated local stack, creates a benchmark application, registers synthetic users, executes both workloads, and writes JSON reports under benchmark-results/.

The defaults are intentionally modest. Override them with environment variables:

BENCH_CLIENTS=500 \
BENCH_RATE=1000 \
BENCH_WARMUP=30s \
BENCH_DURATION=2m \
BENCH_PAYLOAD_BYTES=1024 \
scripts/run-benchmark.sh all

The CLI rejects non-loopback targets by default. A separate --allow-non-loopback flag exists for dedicated benchmark environments, but the runner should never be pointed at shared staging or production infrastructure.

You can inspect the complete benchmark methodology and CLI reference before running it.

What a serious capacity test still needs

A reproducible harness is necessary, but it is not sufficient for a defensible capacity claim.

For a publishable performance study, we would additionally:

Run the server, database, and load generator on separate named hosts.
Use a clean server commit and preserve every result file.
Record container limits, MySQL configuration, storage type, network topology, and TLS status.
Run multiple trials after a longer warm-up and publish all trials, not only the best one.
Collect server CPU, memory, network, disk, database, and Go profile data.
Increase load until errors or latency show the actual saturation point.
Test private chat and multiple group sizes independently.
Compare systems only when persistence, acknowledgement, payload, fan-out, and delivery semantics are equivalent.

Large numbers without these controls are demonstrations, not benchmarks.

What we learned

The most important part of the harness was not the percentile function or the rate limiter. It was deciding what each counter means.

For messaging systems:

connections are not messages;
attempted publishes are not acknowledged publishes;
acknowledged publishes are not recipient deliveries;
group deliveries are not sender throughput;
local smoke results are not production capacity.

Once those boundaries are explicit, optimization becomes easier too. A high connection P99, high ACK P99, and high group-delivery P99 point to different parts of the system.

The harness is open source in the JuggleIM repository. If you work on real-time systems, feedback on the workload model, timing semantics, and missing scenarios is especially welcome.

JuggleIM: An Open-Source, Self-Hosted Messaging Backend Built in Go

yuwnloyblog — Thu, 16 Jul 2026 06:05:11 +0000

Real-time messaging looks simple until you try to build it.

A prototype can send a message over a WebSocket in an afternoon. A production messaging system must also handle reconnects, acknowledgements, duplicate messages, offline users, conversation state, unread counts, groups, message history, push notifications, multiple devices, moderation, storage, and operational visibility.

That is a large amount of infrastructure to build before chat becomes useful to your actual product.

JuggleIM is an open-source, self-hosted messaging backend designed to provide that foundation. It is written in Go, uses Protobuf over WebSocket for client connections, exposes REST APIs for business services, and includes SDKs for major client platforms.

At the time of writing, the project has earned more than 3,500 GitHub stars and 360 forks. More importantly, it can be started locally with one Docker Compose command.

What can you build with it?

JuggleIM is not a complete social product or customer-support application. It is the messaging infrastructure underneath those products.

Typical use cases include:

Private and group chat inside a mobile or web application
Customer-service messaging
Marketplace conversations between buyers and sellers
Live-stream and community chatrooms
Internal collaboration tools
Device-to-cloud messaging for IoT products
Real-time conversations with AI assistants and bots
Multi-tenant communication features for SaaS products

The server provides the messaging layer while your business backend continues to own product-specific concepts such as accounts, permissions, subscriptions, orders, or CRM records.

Why not build it from scratch?

Sending bytes is only the beginning. A useful messaging system needs answers to questions such as:

What happens when a client reconnects after losing its network?
How do you prevent a retried publication from creating duplicate messages?
How are messages synchronized across a user's phone, browser, and desktop client?
How do you maintain unread counts and conversation ordering?
When should an offline push notification be sent?
How do group membership, mute settings, blocks, and moderation affect delivery?
Where do message history and delivery state live?
How does one deployment isolate multiple applications or tenants?

JuggleIM already models these concerns. The server includes message IDs and sequence numbers, QoS-aware acknowledgements, duplicate-publication filtering, sendbox and history paths, conversation state, group membership, online presence, push routing, and tenant-scoped credentials.

This lets a team spend more time on its product experience and less time rebuilding messaging plumbing.

A modular Go architecture

The open-source server runs as a single Go process with clear internal module boundaries.

There are four primary entry points:

Entry point	Default port	Purpose
Server API Gateway	`9001`	REST APIs used by your trusted business backend
Navigator	`9002`	Validates client tokens and returns the WebSocket endpoint
Connect Manager	`9003`	Maintains Protobuf-over-WebSocket client connections
Admin Gateway	`8090`	Serves the administration console and APIs

Requests from these gateways enter an internal actor and RPC runtime. Domain modules then handle messaging, users, friends, presence, conversations, groups, history, push, file storage, subscriptions, bots, moderation, and RTC room signaling.

Actor methods are routed by a stable target ID such as a user, group, or conversation. The runtime supports synchronous queries, asynchronous commands, grouped routing, and broadcast. This keeps service boundaries explicit without requiring the operational overhead of a distributed microservice deployment for the community edition.

You can read the full English architecture guide or its Chinese version.

Storage without unnecessary complexity

The default deployment requires MySQL 8. It stores application configuration and the core domain data needed by the messaging system.

MongoDB is optional. When configured as the message storage engine, it provides alternative collections for message, history, and push workloads. A local LevelDB-backed KV store is also available for internal timestamp-ordered data.

Attachments can integrate with S3-compatible storage, MinIO, Alibaba Cloud OSS, or Qiniu. Offline notifications can be delivered through APNs, FCM, and supported Android vendor channels.

The result is a small starting topology that can grow with the product instead of requiring a long list of infrastructure dependencies on day one.

Start it locally

The fastest way to try JuggleIM is Docker Compose:

git clone https://github.com/juggleim/im-server.git
cd im-server
docker compose up -d

After the containers become healthy, open the admin console:

http://127.0.0.1:8090

The development credentials are:

username: admin
password: 123456

These credentials are intended only for local development and must be changed before a production deployment.

Create the first tenant from the admin API:

curl --request POST \
  --url http://127.0.0.1:8090/admingateway/apps/create \
  --header 'Content-Type: application/json' \
  --data '{"app_key":"my-app","app_name":"My App"}'

The response contains an app_key and app_secret. The secret belongs only on your trusted business backend; it must never be embedded in a client application.

From there, the business backend can use the server API on port 9001, while client SDKs discover and connect to the WebSocket service on port 9003.

The complete setup instructions are available in the deployment guide.

More than one client platform

A messaging backend is only useful if clients can integrate with it. The JuggleIM organization provides SDKs and demo applications across the ecosystem, including:

Android
iOS
Web
React Native
Flutter
HarmonyOS
Desktop applications
Go and Java server-side SDKs

There are also demo business services and web clients that show how authentication, users, friends, groups, and application-specific workflows can sit above the core IM server.

Browse the complete ecosystem from the JuggleIM GitHub organization.

Multi-tenancy is part of the design

JuggleIM carries an app_key through API requests, internal RPC calls, storage operations, and message delivery. This allows one deployment to host multiple isolated applications.

That is useful for:

SaaS products that provision a separate messaging space per customer
Teams running staging and production applications on shared infrastructure
Platforms operating several brands or regional applications
Developers building a reusable communication service for multiple products

Tenant isolation is not something added as an afterthought; it is part of the request and routing context.

A transparent community-edition boundary

The open-source repository implements a modular single-node server. Its internal package is named gmicro.Cluster, but the community implementation routes work to the current node.

Multi-node discovery, cross-node routing, failover, horizontal scaling, and commercial support belong to the professional offering. We prefer to make this boundary explicit rather than imply that the community repository provides behavior it does not contain.

For many product teams, a self-hosted single-node deployment is an effective way to develop, validate, and operate an initial messaging workload. Teams that later need a distributed topology can evaluate the professional edition against their measured requirements.

Security starts with clear boundaries

JuggleIM uses tenant-scoped credentials and token-based client authentication. In a production environment, HTTP and WebSocket traffic should be protected with HTTPS and WSS through a trusted TLS termination layer.

Operators should also restrict the admin console, diagnostics endpoint, database ports, log uploads, and storage credentials. Default development credentials should never remain enabled in production.

The project does not claim automatic end-to-end encryption between chat participants. If your product requires application-layer content encryption, evaluate it separately as part of the client and key-management design.

Clear security claims are more valuable than broad promises.

Where the project is going

The project is actively improving its developer experience and technical transparency. Current community work includes:

Reproducible performance benchmarks
Broader CI and automated security checks
More complete configuration references
End-to-end API examples
Docker Compose troubleshooting documentation

These tasks are tracked publicly in GitHub Issues, including issues labeled good first issue for new contributors.

Try it, inspect it, and tell us what is missing

If you are adding messaging to a product, JuggleIM gives you a practical starting point that you can run, inspect, and self-host.

If the project solves a problem for you, a GitHub star helps other developers discover it. If it does not yet fit your use case, open a discussion and describe what you are building. Concrete feedback is how open-source infrastructure becomes more useful.