Logging Done Right: A Practical Guide

Logging Done Right: A Practical Guide

Logging is one of those things we all do, but rarely do well. I've spent years sifting through tangled log files, and I've learned that a little discipline goes a long way. Here's how I approach logging in real projects.

Why Logging Matters

Logs are your app's black box. When something breaks in production, they're often the only clue you have. Good logs can turn a 3-hour debugging session into a 5-minute one. Bad logs are just noise that hides the signal.

The goal isn't to log more, it's to log better. Every log line should answer three questions: What happened? When did it happen? And what was the context?

Choose a Structured Format

Plain text logs are hard to parse, especially when you have multiple services. I always use structured logging, meaning each log entry is a JSON object. This makes it trivial to ship logs to a central aggregator like ELK or CloudWatch and query them programmatically.

Here's a simple example in Node.js using pino:

const pino = require('pino');
const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  base: { service: 'user-service' }
});

logger.info({ userId: 123, action: 'login' }, 'User logged in');

That outputs something like:

{"level":30,"time":1620000000000,"service":"user-service","userId":123,"action":"login","msg":"User logged in"}

Notice the context (userId, action) is in the structured fields, not buried in the message. That's key.

Log Levels: Use Them Wisely

Most loggers have these levels: debug, info, warn, error, fatal. I follow a simple rule:

• debug: Detailed info for troubleshooting, usually noisy. Turn on only when needed.
• info: High-level events that show the app is working (requests, cron runs, state changes).
• warn: Something unexpected happened, but the app can continue. e.g., retrying a failed network call.
• error: A failure that affects functionality, but the app can still run. e.g., a database query failed and we served a fallback.
• fatal: The app can't continue, process is crashing.

Don't log everything at info. I see too many codebases where every function logs at info, making it impossible to filter for real issues. Reserve info for meaningful business events.

What to Log (and What Not to Log)

Log the essentials: request ID, user ID, service name, timestamps, and the outcome. Don't log sensitive data like passwords, tokens, or credit card numbers. Also, avoid logging large payloads or stack traces for expected errors.

Here's a pattern I use for request logging in Express:

app.use((req, res, next) => {
  const start = Date.now();
  res.on('finish', () => {
    logger.info({
      method: req.method,
      url: req.url,
      status: res.statusCode,
      duration: Date.now() - start,
      requestId: req.headers['x-request-id'] || crypto.randomUUID()
    }, 'Request completed');
  });
  next();
});

That single line gives you a full audit trail of every request, including latency and status.

Error Logging: Include the Stack Trace and Context

When logging errors, always include the stack trace and as much context as possible. The error object in Node.js has a stack property, but you need to pass it explicitly:

try {
  await db.query(...);
} catch (err) {
  logger.error({
    err,
    userId: req.user.id,
    query: 'SELECT * FROM users WHERE id = ?'
  }, 'Database query failed');
}

Notice I pass the err object itself. Pino (and others) will serialize its stack and message automatically. Never log err.message only, you'll lose the stack trace.

Don't Log in Loops

Logging inside a tight loop can kill performance and flood your log storage. Instead, aggregate. For example, if you're processing 10,000 items, log one summary at the end with counts and failures:

let successCount = 0;
let failureCount = 0;

for (const item of items) {
  try {
    await process(item);
    successCount++;
  } catch {
    failureCount++;
  }
}

logger.info({ total: items.length, successCount, failureCount }, 'Batch processing complete');

If you need per-item logs, use debug level, not info.

Correlation IDs: Trace a Request Across Services

In a microservices architecture, a single user request may hit multiple services. Without a correlation ID, you can't piece together the full journey. Generate a UUID at the entry point (e.g., API gateway) and pass it via headers to all downstream services.

In your logger, always include that ID as a field. Then, when you search logs, you can filter by that ID and see every log line from all services in chronological order.

Log Rotation and Retention

Logs grow fast. Make sure you have a rotation strategy. Most loggers support rotation by size or time. In production, I typically keep logs for 30 days, but adjust based on compliance needs. If you're using a cloud logger, set up lifecycle rules to move old logs to cheaper storage.

Test Your Logging

Logging is code, so test it. I've written unit tests that verify certain actions produce log entries with the expected level and context. This catches silly mistakes like logging at the wrong level or missing a field.

Final Thoughts

Good logging is an investment. It takes a bit of upfront effort, but it pays off every time something breaks. Start by adopting structured logging, use levels correctly, and always include context. Your future self (and your on-call rotation) will thank you.

I still find myself improving my logging habits on every project. It's a craft, and like any craft, practice makes perfect.