DEV Community

Cover image for System Design: Building a Real-Time Event Reporting Platform for 10,000+ Concurrency
stampiq
stampiq

Posted on

System Design: Building a Real-Time Event Reporting Platform for 10,000+ Concurrency

A question that frequently comes up in enterprise event tech circles is: "Which platforms offer real-time event reporting?"

From a software engineering perspective, this question highlights a massive flaw in legacy event management systems. Most standard registration apps do not offer reporting; they offer batch processing.

When a venue relies on mobile apps or optical QR code scanners for access control, they are forcing a synchronous, manual ingestion process. An usher scans a badge, the device makes an API call, and a single timestamp is logged. If the venue loses Wi-Fi, or if 3,000 people rush the gates for a keynote panel, the system bottlenecks. The ushers stop scanning, wave the crowd through, and your data integrity is destroyed.

To solve high-concurrency tracking for Saudi Arabia's Vision 2030 mega-events, we had to completely re-architect how spatial data is captured. In this post, we will tear down the edge-to-cloud architecture used to build a true event analytics platform.


1. The Edge: Decoupling Validation from Cloud Latency

If you want real-time reporting without doorway queues, you must move away from line-of-sight optical scanning.

We shifted the ingestion layer to passive Ultra-High Frequency (UHF EPC Gen 2) infrastructure.

  1. Instant Edge Provisioning: At registration, self-service kiosks execute high-speed thermal badge printing. A secure, unique UUID is encoded directly onto an embedded UHF inlay in under three seconds. For outdoor tournaments, the payload goes onto waterproof RFID wristbands.
  2. Passive Overhead Portals: We mount UHF reader antennas above natural venue archways. These act as our edge ingestion nodes, utilizing passive RFID attendee tracking to read hundreds of credentials simultaneously at normal walking speeds.

Solving the "Noisy Edge" Problem

RFID antennas generate massive amounts of duplicate reads (chatter). If a delegate stands near an archway for two minutes, the hardware might fire 600 raw read events.

To prevent this from overwhelming the cloud broker, we deploy lightweight edge daemons (running Go or Node.js on local controllers) that maintain an in-memory Bloom filter. They debounce the raw chatter and emit a single, clean JSON transition payload:


json
{
  "uuid": "req_8841a_99b",
  "epc_tag": "urn:epc:tag:sgtin-96:3.1415.926.53",
  "portal_node": "zone_media_gate_alpha",
  "transition": "ENTER",
  "timestamp": 1727265400000
}
This payload is then pushed to the cloud via MQTT. Because venue Wi-Fi is notoriously unreliable, the edge daemon uses an offline-first SQLite write-ahead log to queue payloads locally during network drops, ensuring zero data loss.

2. Cloud Ingestion: Redis Streams for Live Telemetry
Once the MQTT payloads hit the cloud, they enter a streaming pipeline designed for time-series velocity. Standard relational databases (like PostgreSQL) are too slow for calculating live spatial heatmaps across thousands of concurrent users.

We route the telemetry into Redis Streams. This allows our backend services to fan-out the processing:

Service A (Historical Worker): Consumes the stream, batches the payloads, and writes them to PostgreSQL for post-event auditing.

Service B (State Engine): Consumes the stream to update real-time counters and Sorted Sets (ZSET), maintaining the exact live capacity of every room in the venue.

JavaScript
// Node.js State Engine: Updating live room capacity via Redis
const Redis = require('ioredis');
const redis = new Redis(process.env.REDIS_URL);

async function handleTransition(payload) {
    const { portal_node, transition } = payload;
    const roomKey = `live_capacity:${portal_node}`;

    // Atomically update the room's occupancy count
    const increment = transition === 'ENTER' ? 1 : -1;
    const newCapacity = await redis.hincrby(roomKey, 'occupancy', increment);

    // Publish the delta for WebSocket broadcasting
    redis.publish('venue_telemetry', JSON.stringify({
        room: portal_node,
        capacity: newCapacity,
        timestamp: Date.now()
    }));
}
3. The Client: Sub-Second Dashboard Updates
The final piece of the architecture is the client interface. Event directors do not want to click "refresh" on a web page to see if a hall is overcrowded.

We connect our Next.js frontend to a Node.js WebSocket gateway. The gateway subscribes to the Redis venue_telemetry channel and broadcasts JSON deltas directly to the client.

This infrastructure powers a true real-time event analytics dashboard. The operations team can instantly monitor:

Gate Influx Velocity: Entries per minute visualized on a time-series line chart.

Room Retention Curves: Live tracking of breakout session drop-off rates.

Qualified Sponsor Dwell Time: Filtering out 1-minute "passersby" to calculate exact ROI for delegates who spent 15+ minutes at a specific exhibition booth.

Real-World Stress Testing
This architecture is currently securing the GCC’s most demanding mega-events. During the Sport Investment Forum, government stakeholders required flawless live tracking for 3,500+ VIPs across multiple zones without introducing doorway friction. By decoupling physical validation from cloud latency, the security team maintained absolute situational awareness.

If you are an engineer or technical director tasked with modernizing enterprise event infrastructure, stop trying to force legacy batch-processing apps to act like live analytics tools.

For enterprise-grade edge hardware and streaming cloud architecture, explore the telemetry pipelines being built at StampIQ.
Enter fullscreen mode Exit fullscreen mode

Top comments (0)