Last month a video about a Norwegian fjord drone crash went from 4,000 to 1.1 million views in nine hours on our discovery feed. The problem wasn't ingesting that traffic — it was the front end. Every visitor on the trending page had a setInterval firing a fetch('/api/views?id=...') every five seconds, and with a few thousand concurrent viewers that turned into a steady 600+ requests per second of nothing but "give me the current number." Each one opened a SQLite read, serialized a tiny JSON payload, and burned a LiteSpeed worker slot. The counts were stale by up to five seconds anyway, and the polling cost more CPU than the actual video metadata API.
Polling is the wrong tool for a value that changes continuously and that the server already knows about the instant it changes. The right tool — for a one-directional, server-to-client stream of small updates — is Server-Sent Events (SSE). At ViralVidVault we run a European viral-video discovery feed, and live view counts are the single most-watched number on the site. This post is how we replaced polling with SSE on a PHP 8.4 / SQLite (WAL) / LiteSpeed / Cloudflare stack, the buffering traps that cost me an afternoon, and how we keep it GDPR-clean.
Why SSE and not WebSockets
The knee-jerk answer to "live updates" is WebSockets, but WebSockets are a bidirectional, stateful protocol. View counts only ever flow one way: server to browser. The client never has anything to say back except "I'm still here," which TCP already handles. SSE gives us exactly that one-way channel over plain HTTP, and it brings three properties that matter on our stack:
- It's just HTTP. No protocol upgrade, no separate port, no special LiteSpeed module. It rides through Cloudflare and our existing TLS termination unchanged.
-
Automatic reconnect is built in. The browser's
EventSourcereconnects on drop and replays theLast-Event-IDheader, so we get resumable streams for free. -
It degrades to text. An SSE stream is a
text/event-streambody you cancurland read with your eyes. Debugging a WebSocket frame stream is a much worse afternoon.
The one real limitation — the ~6 connections-per-domain cap in HTTP/1.1 — disappears under HTTP/2, which both LiteSpeed and Cloudflare give us by default. Over h2 a browser multiplexes dozens of SSE streams down one connection.
The data side: counting views without melting SQLite
Before streaming anything we need a count that's cheap to read and safe to write under load. We use SQLite in WAL (write-ahead logging) mode, which lets readers and a single writer proceed concurrently — exactly the shape of a view counter: many reads, frequent small writes from one ingest process.
The naive approach — UPDATE videos SET views = views + 1 on every hit — serializes every viewer behind a row lock and thrashes the WAL. Instead we buffer increments in memory and flush them on an interval. Here is the writer side, a long-running PHP CLI process that drains an in-memory delta map into SQLite:
<?php
declare(strict_types=1);
final class ViewCounter
{
private \PDO $db;
/** @var array<int,int> videoId => pending delta */
private array $pending = [];
public function __construct(string $dbPath)
{
$this->db = new \PDO('sqlite:' . $dbPath, null, null, [
\PDO::ATTR_ERRMODE => \PDO::ERRMODE_EXCEPTION,
]);
// WAL: concurrent readers + one writer, durable enough for counters.
$this->db->exec('PRAGMA journal_mode = WAL');
$this->db->exec('PRAGMA synchronous = NORMAL');
$this->db->exec('PRAGMA busy_timeout = 5000');
}
public function record(int $videoId): void
{
$this->pending[$videoId] = ($this->pending[$videoId] ?? 0) + 1;
}
/** Flush buffered deltas in one transaction. Call every ~1s. */
public function flush(): void
{
if ($this->pending === []) {
return;
}
$stmt = $this->db->prepare(
'UPDATE videos SET views = views + :d, updated_at = unixepoch() WHERE id = :id'
);
$this->db->beginTransaction();
try {
foreach ($this->pending as $id => $delta) {
$stmt->execute([':d' => $delta, ':id' => $id]);
}
$this->db->commit();
$this->pending = [];
} catch (\Throwable $e) {
$this->db->rollBack();
error_log('view flush failed: ' . $e->getMessage());
}
}
}
Batching turns thousands of per-request writes into one transaction per second. With synchronous = NORMAL under WAL, that single commit is cheap and still crash-safe up to the last checkpoint — acceptable for a vanity counter where losing a sub-second of increments on a hard crash is a non-event.
A note on GDPR, because this is Europe and it matters: a raw increment carries no personal data. We do not log IP, user-agent, or any identifier against the view. The counter is a pure integer aggregate. Our actual analytics dedupe is done with a rotating, salted, daily-bucketed hash that we never persist past the aggregation window — but the count that feeds the SSE stream is just videos.views, an anonymous total. No consent banner is required to display a public number going up.
The SSE endpoint in PHP 8.4
Now the read side. The endpoint holds the connection open and pushes a new value each time the count changes. The critical detail on LiteSpeed (and behind Cloudflare) is disabling every layer of output buffering, or your events get held in a buffer and the client sees nothing until the buffer fills or the connection closes — which defeats the entire point.
<?php
declare(strict_types=1);
// --- Disable buffering everywhere it can hide ---
@ini_set('zlib.output_compression', '0'); // no gzip on a stream
@ini_set('output_buffering', '0');
while (ob_get_level() > 0) {
ob_end_flush();
}
ignore_user_abort(false); // let us notice when the client leaves
set_time_limit(0);
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache, no-transform');
header('Connection: keep-alive');
// LiteSpeed / nginx: do NOT buffer this response.
header('X-Accel-Buffering: no');
$videoId = (int)($_GET['id'] ?? 0);
if ($videoId <= 0) {
http_response_code(400);
exit;
}
$db = new PDO('sqlite:/var/www/data/vault.db', null, null, [
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
]);
$db->exec('PRAGMA query_only = 1'); // this connection only reads
$stmt = $db->prepare('SELECT views FROM videos WHERE id = :id');
// Resume support: client sends Last-Event-ID on reconnect.
$lastSent = isset($_SERVER['HTTP_LAST_EVENT_ID'])
? (int)$_SERVER['HTTP_LAST_EVENT_ID']
: -1;
$idleTicks = 0;
while (!connection_aborted()) {
$stmt->execute([':id' => $videoId]);
$views = (int)$stmt->fetchColumn();
if ($views !== $lastSent) {
// id: lets the browser resume; event name groups handlers client-side
echo "id: {$views}\n";
echo "event: views\n";
echo 'data: ' . json_encode(['v' => $views], JSON_THROW_ON_ERROR) . "\n\n";
$lastSent = $views;
$idleTicks = 0;
} else {
// Heartbeat comment every ~15s keeps Cloudflare / proxies from
// dropping an idle connection. Comments start with ':'
if (++$idleTicks >= 15) {
echo ": keepalive\n\n";
$idleTicks = 0;
}
}
flush();
sleep(1);
}
A few things are doing real work here:
-
The
id:field carries the last value. On reconnect the browser sends it back asLast-Event-ID, so we can skip re-emitting a number the client already has. We reuse the count itself as the event id since it's monotonic. -
event: viewsnames the event so the client can bind a specific handler and ignore heartbeats. -
The
: keepalivecomment is an SSE comment line (anything after a leading colon is ignored by the parser). It exists purely to push bytes through idle proxies. Cloudflare will eventually close a connection with no traffic; a 15-second heartbeat keeps it warm without sending a fake count. -
connection_aborted()+sleep(1)is a deliberately boring poll-the-DB-once-a-second loop. SQLite reads under WAL are sub-millisecond on an indexed primary key, so one read per connection per second is far cheaper than the old model of every client polling over HTTP. We moved the polling from the network edge to a local file read.
That last point is the whole trick: SSE doesn't make the data magically push itself. The server still has to learn the new value somehow. But doing it as a local SQLite read inside one held-open process is orders of magnitude cheaper than thousands of clients each making a full HTTP round-trip through Cloudflare and LiteSpeed.
The browser client
The front-end code is almost embarrassingly small, which is the point. EventSource handles connection, parsing, and reconnect-with-backoff for us.
const counter = document.querySelector('#view-count');
const videoId = counter.dataset.videoId;
const es = new EventSource(`/sse/views?id=${videoId}`);
es.addEventListener('views', (e) => {
const { v } = JSON.parse(e.data);
counter.textContent = new Intl.NumberFormat('en-GB').format(v);
});
// EventSource auto-reconnects on error; we only log.
es.addEventListener('error', () => {
// readyState 0 = connecting (browser is already retrying)
if (es.readyState === EventSource.CLOSED) {
console.warn('view stream closed by server');
}
});
// Stop streaming when the tab is hidden — saves a connection and respects
// the user's battery/data. Resume when they come back.
document.addEventListener('visibilitychange', () => {
if (document.hidden) {
es.close();
}
});
We close the stream on visibilitychange to hidden. A user who tabbed away doesn't need a live counter, and dropping the connection frees a worker slot. (To resume cleanly you'd re-instantiate EventSource on visible — left out for brevity, but it's a few lines.) This one habit cut our average concurrent stream count by roughly 40% in production, because a large share of "viewers" are background tabs.
Fanning out at scale with a Go broker
The per-connection PHP loop is fine up to a point, but PHP holding thousands of long-lived connections means thousands of LiteSpeed workers parked in sleep(1). Past a few hundred concurrent streams per video you want a single process reading the DB and fanning the result out to every subscriber. Go's goroutines and channels are built for exactly this, so for our hottest videos a small Go broker sits behind the SSE path:
package main
import (
"fmt"
"net/http"
"sync"
"time"
)
type Broker struct {
mu sync.RWMutex
clients map[chan int]struct{}
latest int
}
func NewBroker() *Broker {
return &Broker{clients: make(map[chan int]struct{})}
}
// Publish is called by the single DB-poller goroutine.
func (b *Broker) Publish(v int) {
b.mu.Lock()
b.latest = v
for ch := range b.clients {
select {
case ch <- v:
default: // slow client: drop this tick, never block the broker
}
}
b.mu.Unlock()
}
func (b *Broker) ServeHTTP(w http.ResponseWriter, r *http.Request) {
fl, ok := w.(http.Flusher)
if !ok {
http.Error(w, "streaming unsupported", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("X-Accel-Buffering", "no")
ch := make(chan int, 4)
b.mu.Lock()
last := b.latest
b.clients[ch] = struct{}{}
b.mu.Unlock()
defer func() {
b.mu.Lock()
delete(b.clients, ch)
b.mu.Unlock()
}()
// Send current value immediately so a new tab isn't blank.
fmt.Fprintf(w, "event: views\ndata: {\"v\":%d}\n\n", last)
fl.Flush()
heartbeat := time.NewTicker(15 * time.Second)
defer heartbeat.Stop()
for {
select {
case <-r.Context().Done():
return // client disconnected
case v := <-ch:
fmt.Fprintf(w, "id: %d\nevent: views\ndata: {\"v\":%d}\n\n", v, v)
fl.Flush()
case <-heartbeat.C:
fmt.Fprint(w, ": keepalive\n\n")
fl.Flush()
}
}
}
Now one goroutine reads SQLite once a second and calls Publish; every connected client gets the update without anyone touching the database again. The select { case ch <- v: default: } pattern is the important defensive bit — a slow or stalled client must never block the broadcast, so we drop a tick for that client rather than back-pressuring everyone. A single modest Go process handles tens of thousands of these streams where the PHP-per-connection model would have exhausted worker pools long before.
We route only the genuinely viral videos through the Go broker (a Cloudflare Worker checks a "is hot" flag and rewrites the path); the long tail of cold videos stays on the simple PHP endpoint, which is more than adequate when a video has three concurrent viewers.
Load-testing the stream
Numbers you didn't measure are wishes. SSE load behaves differently from request/response load: the metric isn't requests-per-second, it's concurrent held connections and whether updates actually arrive on time. A quick Python harness with aiohttp opens N streams and records inter-event latency:
import asyncio, time, aiohttp
URL = "https://viralvidvault.com/sse/views?id=12345"
CONCURRENCY = 2000
async def one(session, stats):
try:
async with session.get(URL) as resp:
last = time.monotonic()
async for raw in resp.content:
line = raw.decode("utf-8", "replace").strip()
if line.startswith("data:"):
now = time.monotonic()
stats.append(now - last)
last = now
except Exception as e:
print("stream error:", e)
async def main():
stats: list[float] = []
timeout = aiohttp.ClientTimeout(total=None) # streams never "finish"
conn = aiohttp.TCPConnector(limit=CONCURRENCY)
async with aiohttp.ClientSession(timeout=timeout, connector=conn) as s:
tasks = [asyncio.create_task(one(s, stats)) for _ in range(CONCURRENCY)]
await asyncio.sleep(60)
for t in tasks:
t.cancel()
if stats:
stats.sort()
p50 = stats[len(stats)//2]
p99 = stats[int(len(stats)*0.99)]
print(f"events: {len(stats)} p50={p50:.2f}s p99={p99:.2f}s")
asyncio.run(main())
The key assertions are: do all 2,000 connections stay open for the full minute, and is p99 inter-event latency close to your tick interval (~1s) rather than spiking to your heartbeat interval (15s, which would mean updates aren't actually flowing). The first time I ran this against LiteSpeed I saw a clean 1.0s p50 locally but a 15s p99 through Cloudflare — the smoking gun that something between the origin and the client was buffering. The fix was the Cache-Control: no-transform header plus confirming zlib.output_compression was off; Cloudflare was trying to apply compression to the stream and holding bytes to do it.
Buffering traps, ranked by how long they cost me
If SSE "doesn't work," it's almost always buffering. In rough order of how much time each one ate:
-
PHP/zlib output compression. Gzip needs a buffer to compress; a stream never fills it. Set
zlib.output_compression = 0and sendCache-Control: no-transformso the edge doesn't re-add it. -
LiteSpeed / nginx proxy buffering. Send
X-Accel-Buffering: no. Without it the proxy collects your output and forwards in chunks. -
Leftover output buffers. A framework or
auto_prepend_filemay have startedob_start(). Drain them all with theob_end_flush()loop before the first event. -
Cloudflare idle timeout. No bytes for ~100s and the connection drops. The 15-second heartbeat comment prevents it and is invisible to the
EventSourceparser. - HTTP/1.1 connection cap. Six streams per domain max. Serve over HTTP/2 (default on our stack) and it's a non-issue.
Conclusion
Server-Sent Events let us delete a whole category of wasteful traffic. The trending page went from ~600 polling requests/second to a set of held-open connections that each cost one local SQLite read per second, the counts update in near real time instead of on a 5-second sawtooth, and the client code shrank to a dozen lines because EventSource does the reconnect work. The architecture scales in tiers: a dead-simple PHP loop for the cold long tail, a Go broker for the handful of genuinely viral videos, and Cloudflare deciding which is which at the edge.
The two lessons worth carrying off this stack: SSE doesn't push data for free — you still need a cheap way to learn the new value, and batched SQLite-under-WAL is that cheap way; and when SSE looks broken, it's buffering, every time, so kill it at PHP, the proxy, and the CDN before you suspect anything else. And because a public view count is an anonymous aggregate, none of this added a line to our consent flow — which, when you operate in Europe, is exactly the kind of feature you want.
Top comments (0)