In this guide, I'll show you how to create a real-time log streaming system using Server-Sent Events (SSE) with a Node.js backend and React frontend. We'll build a system where you can watch logs appear instantly as your backend processes run.
What We'll Cover
- What are Server-Sent Events?
- Why use SSE for log streaming?
- Building the backend log streaming service
- Creating the React frontend to display logs
- Handling errors and connection issues
- Testing the system
What are Server-Sent Events?
Server-Sent Events (SSE) is a technology that lets servers push data to web browsers in real-time. Unlike WebSockets, which provide two-way communication, SSE is one-way (server to client) and is perfect for scenarios where you only need to send data from server to client, like log streaming.
Why SSE for Log Streaming?
- Simple to implement - easier than WebSockets
- Automatic reconnection if connection drops
- Built into browsers - no extra libraries needed
- Perfect for one-way data flow
- Works well with HTTP/HTTPS
Backend Implementation
Let's start with the Node.js backend. Here's how to set up the log streaming service:
// logStream.js
const express = require('express');
const cors = require('cors');
const app = express();
app.use(cors());
app.get('/api/logs/stream', (req, res) => {
// Set headers for SSE
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
// Handle client disconnect
req.on('close', () => {
// Clean up if needed
console.log('Client disconnected');
});
// Your process that generates logs
const streamLogs = async () => {
try {
// Example: simulate a process with multiple steps
const steps = [
'Starting process...',
'Loading data...',
'Processing...',
'Finishing up...'
];
for (const message of steps) {
// Send log as SSE
res.write(`data: ${JSON.stringify({
timestamp: new Date(),
message: message
})}\n\n`);
// Simulate some work
await new Promise(resolve => setTimeout(resolve, 1000));
}
} catch (error) {
res.write(`data: ${JSON.stringify({
error: error.message
})}\n\n`);
}
};
streamLogs();
});
Understanding the SSE Protocol Format
One crucial detail when implementing Server-Sent Events is the message format. Each message sent from the server must start with data: followed by your content and end with two newlines (\n\n). This isn't just a convention β it's a requirement of the SSE protocol. Let's look at why this matters:
// β Wrong - won't work
res.write(JSON.stringify({ message: 'Processing...' }));
// β Wrong - missing double newline
res.write(`data: ${JSON.stringify({ message: 'Processing...' })}\n`);
// β
Correct format
res.write(`data: ${JSON.stringify({ message: 'Processing...' })}\n\n`);
If you forget the data: prefix or the double newline, your client's onmessage event won't fire, and it might seem like your code is working (the connection is established) but no logs appear in your frontend. This is a common gotcha that can be confusing to debug because the server appears to be sending data, but the client never receives it. The SSE specification requires this format to properly parse and trigger events on the client side.
The format breaks down like this:
-
data:- Tells the browser this is a message payload - Your JSON data or message
-
\n\n- Two newlines mark the end of a message
Think of it like addressing a letter - without the proper address format, your message won't reach its destination, even if the content is perfect!
Frontend Implementation
Now let's create a React component to display these streaming logs:
// LogStream.jsx
import React, { useState, useEffect } from 'react';
const LogStream = () => {
const [logs, setLogs] = useState([]);
const [isConnected, setIsConnected] = useState(false);
const startLogStream = () => {
// Create SSE connection
const eventSource = new EventSource('http://localhost:3000/api/logs/stream');
// Handle incoming messages
eventSource.onmessage = (event) => {
const data = JSON.parse(event.data);
setLogs(prevLogs => [...prevLogs, data]);
};
// Handle connection open
eventSource.onopen = () => {
setIsConnected(true);
};
// Handle errors
eventSource.onerror = (error) => {
console.error('SSE error:', error);
setIsConnected(false);
eventSource.close();
};
// Clean up on unmount
return () => {
eventSource.close();
setIsConnected(false);
};
};
return (
<div>
<button onClick={startLogStream}>
Start Streaming Logs
</button>
<div>
<h3>Status: {isConnected ? 'Connected' : 'Disconnected'}</h3>
<div className="logs-container">
{logs.map((log, index) => (
<div key={index} className="log-entry">
<span>{new Date(log.timestamp).toLocaleTimeString()}</span>
<span>{log.message}</span>
</div>
))}
</div>
</div>
</div>
);
};
Real-world Usage Examples
- Processing Large Files
// Backend
const processLargeFile = async (file) => {
const totalLines = await countFileLines(file);
let processedLines = 0;
const readStream = createReadStream(file);
readStream.on('data', chunk => {
processedLines += chunk.toString().split('\n').length;
// Send progress update
res.write(`data: ${JSON.stringify({
type: 'progress',
percentage: (processedLines / totalLines) * 100,
message: `Processing line ${processedLines} of ${totalLines}`
})}\n\n`);
});
};
- Database Operations
// Backend
const backupDatabase = async () => {
const tables = await getTables();
for (const table of tables) {
res.write(`data: ${JSON.stringify({
type: 'backup',
table: table.name,
message: `Backing up table ${table.name}`
})}\n\n`);
await backupTable(table);
}
};
Error Handling Best Practices
- Backend Error Handling
try {
// Your process here
} catch (error) {
res.write(`data: ${JSON.stringify({
type: 'error',
message: error.message,
timestamp: new Date()
})}\n\n`);
} finally {
res.write(`data: ${JSON.stringify({
type: 'end',
message: 'Stream ended'
})}\n\n`);
res.end();
}
- Frontend Error Handling
const [error, setError] = useState(null);
eventSource.onerror = (error) => {
setError('Connection lost. Retrying...');
// Implement exponential backoff
setTimeout(() => {
startLogStream();
}, 2000);
};
Tips and Best Practices
-
Memory Management
- Clean up old logs periodically
- Implement pagination or virtual scrolling for large log volumes
- Close EventSource when component unmounts
-
Performance
- Batch log messages when possible
- Use timestamps for ordering
- Consider debouncing updates for high-frequency logs
-
User Experience
- Show connection status
- Provide auto-scroll option
- Add search/filter capabilities
- Include timestamp and log level
-
Security
- Implement authentication
- Validate client permissions
- Sanitize log content
- Use HTTPS in production
Conclusion
Server-Sent Events provide a simple yet powerful way to stream logs in real-time from your Node.js backend to your React frontend. This approach is perfect for monitoring long-running processes, showing progress updates, or any scenario where you need real-time updates from server to client.
The key benefits are:
- Simple implementation
- Real-time updates
- Automatic reconnection
- Good browser support
- Works over standard HTTP
Remember to handle errors properly, manage memory usage, and consider security implications when implementing this in a production environment.
Top comments (1)
β₯