Real-time data is hard.
When building LogWard, one feature was non-negotiable: Live Tail—the ability to watch logs appear in real-time as your application generates them.
Think tail -f for the web, but prettier.
The challenge? Rendering 50-100 log lines per second without:
- Freezing the browser
- Eating all available RAM
- Creating a janky, unusable UI
Here's how I built it using Server-Sent Events (SSE) and Svelte 5 Runes.
Why SSE Instead of WebSockets?
Everyone reaches for WebSockets first for real-time features. I almost did too.
But for this use case, Server-Sent Events are perfect:
| Feature | WebSockets | SSE |
|---|---|---|
| Bidirectional | ✅ Yes | ❌ No (server → client only) |
| Protocol | Custom | Standard HTTP |
| Reconnection | Manual | Automatic |
| Compatibility | Needs library | Native browser API |
| HTTP/2 Multiplexing | ❌ No | ✅ Yes |
For logs, I don't need bidirectional communication. The server streams data, the client displays it. SSE is simpler.
Backend: Fastify SSE Endpoint
The backend sends log chunks as they arrive in real-time:
// packages/backend/src/routes/stream.ts
import { FastifyRequest, FastifyReply } from 'fastify';
fastify.get('/api/v1/stream', async (req: FastifyRequest, reply: FastifyReply) => {
// Validate API key
const apiKey = req.headers['x-api-key'];
if (!apiKey) return reply.code(401).send({ error: 'Missing API key' });
// Set SSE headers
reply.raw.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
});
// Subscribe to Redis Pub/Sub for new logs
const subscriber = redis.duplicate();
await subscriber.connect();
await subscriber.subscribe('logs:stream', (message) => {
const logs = JSON.parse(message);
// Send logs as SSE event
reply.raw.write(`data: ${JSON.stringify(logs)}\n\n`);
});
// Cleanup on disconnect
req.raw.on('close', () => {
subscriber.unsubscribe();
subscriber.quit();
});
});
Key points:
- Logs are published to Redis when ingested
- Each connected client subscribes to the stream
- Connection stays open until the client disconnects
Frontend: Svelte 5 Runes State Management
Here's where Svelte 5 shines. Managing a real-time stream of data with fine-grained reactivity is beautiful:
// lib/stores/live-tail.svelte.ts
export class LiveTailStore {
// Reactive state using Runes
logs = $state([]);
isConnected = $state(false);
private eventSource: EventSource | null = null;
// Derived values
logCount = $derived(this.logs.length);
connect(apiKey: string) {
if (this.eventSource) return; // Already connected
const url = `${API_URL}/api/v1/stream`;
this.eventSource = new EventSource(url, {
headers: { 'X-API-Key': apiKey } // ⚠️ Note: Not all browsers support headers in EventSource
});
this.eventSource.onopen = () => {
this.isConnected = true;
};
this.eventSource.onmessage = (event) => {
const newLogs: LogEntry[] = JSON.parse(event.data);
// Prepend new logs (newest first)
this.logs.unshift(...newLogs);
// Keep only last 1000 logs in memory (prevent RAM explosion)
if (this.logs.length > 1000) {
this.logs.length = 1000;
}
};
this.eventSource.onerror = () => {
this.isConnected = false;
this.disconnect();
// Auto-reconnect after 5 seconds
setTimeout(() => this.connect(apiKey), 5000);
};
}
disconnect() {
this.eventSource?.close();
this.eventSource = null;
this.isConnected = false;
}
clear() {
this.logs = [];
}
}
export const liveTail = new LiveTailStore();
Why This Pattern Works
1. Memory Management
The if (this.logs.length > 1000) check is critical. Without it, after 10 minutes of high-volume logs, you'd eat gigabytes of RAM.
2. Automatic Reactivity
With $state, any component that reads liveTail.logs re-renders automatically when logs arrive. No manual subscriptions.
3. Derived State
$derived lets us compute values that auto-update. For example, showing "Displaying 247 logs" without manual counting.
UI Component: Virtualized List
Rendering 1000 DOM nodes is slow. I use svelte-virtual for virtualization:
<script lang="ts">
import { liveTail } from '$lib/stores/live-tail.svelte';
import VirtualList from 'svelte-virtual';
let isPaused = $state(false);
// When paused, freeze the current logs snapshot
let displayLogs = $derived(isPaused ? [...liveTail.logs] : liveTail.logs);
</script>
<div class="live-tail-container">
<div class="controls">
<button on:click={() => isPaused = !isPaused}>
{isPaused ? '▶ Resume' : '⏸ Pause'}
</button>
<button on:click={() => liveTail.clear()}>🗑 Clear</button>
<span>📊 {liveTail.logCount} logs</span>
</div>
<VirtualList items={displayLogs} let:item>
<LogRow log={item} />
</VirtualList>
</div>
The VirtualList only renders ~20 visible rows at a time, even if there are 1000 logs in memory. Buttery smooth.
Performance Results
Testing with 100 logs/second:
| Metric | Without Virtualization | With Virtualization |
|---|---|---|
| FPS | 15 fps (janky) | 60 fps (smooth) |
| RAM Usage | 800MB+ | ~150MB |
| Time to Freeze | ~2 minutes | Never |
Try It Yourself
The full implementation is in the LogWard repo. If you're building real-time dashboards, feel free to steal this pattern.
- 💻 Source Code: github.com/logward-dev/logward
- 🚀 Live Demo: logward.dev
Have you built real-time features before? What patterns worked (or didn't work) for you? Let's discuss! 👇
Top comments (0)