Structured Logging: From console.log to Production-Ready Observability

#devops #backend #webdev #node

Structured Logging: From console.log to Production-Ready Observability

console.log is fine for development. In production, you need logs you can search, filter, and alert on.

Why Structured Logging

// Unstructured — hard to query
console.log('User 123 placed order 456 for $29.99 at 2024-01-15T10:30:00Z');

// Structured — queryable as JSON
logger.info('order.created', {
  userId: '123',
  orderId: '456',
  amount: 2999,
  currency: 'usd',
});

Pino: Fast Structured Logger for Node.js

npm install pino pino-pretty

import pino from 'pino';

export const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  ...(process.env.NODE_ENV === 'development' ? {
    transport: { target: 'pino-pretty', options: { colorize: true } }
  } : {}),
  // Add base fields to every log
  base: {
    service: 'api',
    version: process.env.npm_package_version,
    env: process.env.NODE_ENV,
  },
  // Redact sensitive fields
  redact: ['req.headers.authorization', 'body.password', 'body.cardNumber'],
});

Request Context Logging

import { AsyncLocalStorage } from 'async_hooks';

const requestContext = new AsyncLocalStorage<{ requestId: string; userId?: string }>();

// Middleware: set context for entire request lifecycle
app.use((req, res, next) => {
  const requestId = crypto.randomUUID();
  requestContext.run({ requestId }, next);
});

// Logger with context
function log(level: 'info' | 'warn' | 'error', message: string, data?: object) {
  const ctx = requestContext.getStore();
  logger[level]({ ...ctx, ...data }, message);
}

// Any log from within a request automatically includes requestId
log('info', 'order.created', { orderId: '123' });
// → { requestId: 'uuid', userId: '456', orderId: '123', message: 'order.created' }

Log Levels

logger.trace('Entering function');  // verbose debugging
logger.debug('Query executed', { sql, duration });  // development
logger.info('User logged in', { userId });   // normal operations
logger.warn('Rate limit approaching', { current, limit }); // might be a problem
logger.error('Payment failed', { error, userId, orderId }); // needs attention
logger.fatal('Database unreachable'); // service is down

Sending to a Log Aggregator

// Send to Loki (Grafana's log storage)
const logger = pino(
  { level: 'info' },
  pino.transport({
    target: 'pino-loki',
    options: {
      host: process.env.LOKI_URL,
      labels: { app: 'api', env: process.env.NODE_ENV },
    },
  })
);