DEV Community: Theo Gravity

Why LogLayer is the Logging Abstraction Framework for TypeScript

Theo Gravity — Sun, 16 Nov 2025 18:18:14 +0000

Logging is one of those fundamental concerns that every application must address, yet it's often treated as an afterthought. In the JavaScript and TypeScript ecosystem, developers face a bewildering array of logging libraries: Pino, Winston, Bunyan, and countless others. Each library has its own API, quirks, and conventions. This fragmentation creates real problems: inconsistent codebases, vendor lock-in, and the constant need to rewrite logging code when requirements change.

LogLayer solves these problems by providing a unified abstraction layer that sits on top of any logging library while offering a consistent, fluent API that makes logging both powerful and pleasant to use.

The Problem with Traditional Logging

Before diving into LogLayer's solutions, let's examine the problems it addresses. Most logging libraries share a common set of methods like info, warn, and error, but they diverge significantly in how they handle structured metadata and error objects.

Consider the inconsistency across popular libraries:

// Winston: message first, then metadata
winston.info('User logged in', { userId: '123', ip: '192.168.1.1' })

// Bunyan: metadata first, then message
bunyan.info({ userId: '123', ip: '192.168.1.1' }, 'User logged in')

// Pino: similar to Bunyan but with different error handling
pino.info({ userId: '123', err: error }, 'User logged in')

This inconsistency means that switching between logging libraries requires rewriting every log statement in your codebase. It also means that teams working across multiple projects must remember different APIs, leading to mistakes and reduced productivity.

Error handling is particularly problematic. Some libraries expect error objects as the first parameter, others as a property in a metadata object, and still others require manual serialization. This leads to ad-hoc solutions, inconsistent error logging, and lost stack traces.

Consistent API and Easy Swapping

The most immediate benefit of LogLayer is consistency. Regardless of whether you're using Pino, Winston, Bunyan, or any other supported transport, your logging code remains identical:

import { LogLayer } from 'loglayer'
import { PinoTransport } from '@loglayer/transport-pino'
import pino from 'pino'

const log = new LogLayer({
  transport: new PinoTransport({ logger: pino() })
})

// This code works the same way regardless of the transport
log.withMetadata({ userId: '123', ip: '192.168.1.1' })
   .withError(new Error('Connection failed'))
   .error('User login failed')

If you later decide to switch to Winston or a cloud provider transport, you only change the transport configuration. Your application code remains untouched. This consistency extends across your entire codebase and different projects, reducing cognitive load and eliminating API-related bugs. This flexibility is especially valuable in microservices architectures where different services might benefit from different logging backends, or when migrating legacy applications to modern logging solutions.

Multi-Transport Support

LogLayer doesn't limit you to a single logging destination. You can send logs to multiple transports simultaneously, enabling powerful use cases:

import { LogLayer } from 'loglayer'
import { PinoTransport } from '@loglayer/transport-pino'
import { DatadogTransport } from '@loglayer/transport-datadog'
import pino from 'pino'

const log = new LogLayer({
  transport: [
    new PinoTransport({ logger: pino() }),  // Local logging
    new DatadogTransport({ ... })            // Cloud logging
  ]
})

// Single log call sends to both destinations
log.info('Request processed')

This multi-transport capability enables gradual migrations, A/B testing of logging backends, and redundancy for critical applications. You can ship logs directly to cloud providers like DataDog or New Relic without relying on their APM packages or sidecars, giving you more control over your observability pipeline.

Fluent API for Better Developer Experience

LogLayer's fluent API makes logging code more readable and maintainable. The separation of message, metadata, and errors into distinct method calls creates a clear, chainable interface:

log.withContext({ requestId: 'abc123', userId: '456' })
   .withPrefix('[API]')
   .withMetadata({ endpoint: '/users', method: 'GET' })
   .withError(error)
   .error('Request failed')

This approach has several advantages. First, it's self-documenting: the code clearly shows what context is being added, what metadata is included, and what error is being logged. Second, it's composable: you can build up log statements incrementally, making it easy to add or remove information. Third, it prevents common mistakes like passing errors in the wrong format or forgetting to include metadata.

The fluent API also supports context that persists across multiple log calls, which is invaluable for request-scoped logging. LogLayer provides sophisticated context management through its context manager system, supporting isolated contexts, linked contexts, and custom context managers for advanced use cases like async context propagation:

// Set context once for a request
const requestLogger = log.withContext({ requestId: 'abc123', userId: '456' })

// All subsequent logs include this context
requestLogger.info('Processing started')
requestLogger.withMetadata({ step: 'validation' }).info('Validating input')
requestLogger.withMetadata({ step: 'database' }).info('Querying database')

This context management is particularly valuable in serverless environments, microservices, and any application where you need to correlate logs across asynchronous operations.

Standardized Error Handling

Error handling is one of LogLayer's strongest features. It provides consistent error serialization across all transports, ensuring that error objects are always properly formatted with their stack traces, messages, and types preserved:

log.withError(new Error('Database connection failed')).error('Operation failed')

This works the same way regardless of whether you're using Pino (which has good error support), Winston (which requires configuration), or a cloud provider transport. LogLayer handles the serialization automatically, ensuring that stack traces are never lost and error information is always structured consistently.

Powerful Plugin System

LogLayer's plugin system allows you to extend and modify logging behavior at various points in the log lifecycle. Plugins can transform data, filter logs, enrich metadata, and more:

import { redactionPlugin } from '@loglayer/plugin-redaction'

const log = new LogLayer({
  transport: new PinoTransport({ logger: pino() }),
  plugins: [
    redactionPlugin({
      paths: ['password', 'apiKey', 'token'],
      censor: '[REDACTED]'
    })
  ]
})

// Sensitive data is automatically redacted
log.withMetadata({ password: 'secret123', userId: '456' }).info('User login')

The plugin system is extensible, allowing you to create custom plugins for your specific needs. Official plugins include redaction for sensitive data, filtering for log levels, OpenTelemetry integration, and DataDog APM trace injection.

Comprehensive Transport Ecosystem

LogLayer supports an extensive ecosystem of transports covering virtually every logging need:

Traditional Logging Libraries: Pino, Winston, Bunyan, Log4js, Roarr, Tracer, TSLog, Consola, Signale, Loglevel, Logtape
Cloud Providers: DataDog, New Relic, Google Cloud Logging, AWS CloudWatch Logs, Sumo Logic, Axiom, Better Stack, Dynatrace, Victoria Logs
Specialized Systems: OpenTelemetry, Sentry, Electron Log, AWS Lambda Powertools, Log File Rotation, Pretty Terminal, HTTP
Browser Environments: DataDog Browser Logs, Console Transport

The transport system is extensible, allowing you to create custom transports for proprietary or specialized logging systems.

Testing Support

LogLayer includes built-in testing utilities that make it easy to verify logging behavior in tests:

import { MockLogLayer } from 'loglayer'

const log = new MockLogLayer()

// Your application code uses log normally
myFunction(log)

// Verify logs were called correctly
expect(log.info).toHaveBeenCalledWith('Expected message')

The mock implementation ensures that no actual logging occurs during tests, keeping test output clean and tests fast. This is especially valuable in large codebases where logging is pervasive.

Conclusion

Logging is too important to be fragmented across incompatible APIs and locked into single-vendor solutions. LogLayer provides the abstraction layer that modern applications need: a consistent, powerful API that works with any logging backend, in any environment, with any runtime.

Whether you're starting a new project or migrating an existing one, LogLayer gives you the freedom to choose the best logging solution for each use case while maintaining consistency across your codebase. It's not just another logging library. It's the abstraction layer that makes all logging libraries work together seamlessly.

LogLayer is open source and available under the MIT license. You can find the source code, contribute, and learn more at the LogLayer GitHub repository or visit the official documentation to get started.

Adding Metrics as You Write Your Logs in Node.js

Theo Gravity — Mon, 03 Nov 2025 00:42:04 +0000

When building production applications, observability is crucial for understanding system behavior, diagnosing issues, and optimizing performance. However, metrics collection is often treated as an afterthought, bolted onto applications only when problems arise or when someone remembers to add instrumentation.

This creates a challenge: by the time you realize you need metrics, you may have already missed critical data points during development and testing phases. More importantly, it separates metrics from logs conceptually, even though they often represent the same events from different perspectives.

The Auto-Instrumentation Approach

Many observability solutions attempt to solve this by providing auto-instrumentation. These tools automatically wrap HTTP frameworks, database drivers, and other common libraries to collect metrics without requiring explicit code changes.

Auto-instrumentation is powerful for baseline metrics, but it has limitations:

Generic metrics only: You get request counts, durations, and error rates, but not business-specific metrics
Limited context: Auto-instrumented metrics often lack business context that your logs contain
Performance overhead: Automatic wrapping can introduce overhead, even if minimal
Coverage gaps: Custom logic, background jobs, and business-specific operations aren't covered

For example, auto-instrumentation might tell you that /api/users took 150ms, but it won't tell you that user ID 123 specifically requested their own profile data, or that this particular request involved a cache miss followed by a database query. That context lives in your logs, but connecting it to metrics requires additional effort.

LogLayer's Approach: A Clean, Fluent API

LogLayer, a TypeScript logging library, addresses this by providing a single, consistent API for both logging and metrics.

It offers a mixin that integrates with the StatsD client hot-shots to add a fluent metrics API directly to LogLayer instances. The mixin provides a builder pattern that makes metrics collection as clean and expressive as LogLayer's logging API.

LogLayer also supports a wide range of logging libraries and cloud transports, allowing you to use your preferred logging backend while adding metrics collection through the same interface. Instead of juggling two different APIs, you use one consistent, fluent interface for both logging and metrics.

Here's how it works in practice:

import { LogLayer, useLogLayerMixin, ConsoleTransport } from 'loglayer';
import { StatsD } from 'hot-shots';
import { hotshotsMixin } from '@loglayer/mixin-hot-shots';

// Set up StatsD client
const statsd = new StatsD({
  host: 'localhost',
  port: 8125
});

// Register the metrics mixin (must be called before creating LogLayer instances)
useLogLayerMixin(hotshotsMixin(statsd));

// Create logger instance
const log = new LogLayer({
  transport: new ConsoleTransport({
    logger: console
  })
});

// Use the same log instance for both metrics and logging
log.stats.increment('request.count').send();
log.stats.timing('request.duration', 150).send();
log.withMetadata({ userId: '123', endpoint: '/api/users' })
   .info('Request processed');

Notice how both metrics and logging use the same log instance with a consistent, fluent API. The stats property provides access to a builder pattern that lets you chain configuration methods before sending the metric:

// The fluent builder pattern makes metrics configuration clean and readable
log.stats.increment('api.requests')
  .withValue(1)
  .withTags(['env:production', 'service:api'])
  .withSampleRate(0.1)
  .send();

Why Using the Same Object Matters

The key advantage of LogLayer's approach is that metrics and logging share the same object instance. This might seem like a small detail, but it has significant practical benefits:

No separate client management: You don't need to import, configure, or pass around a separate StatsD client. Your log instance already has everything you need. When you're already logging, adding a metric is just accessing a property on the same object.

Consistent context: Since you're using the same log instance throughout your code, you can easily add metrics wherever you're already logging. If you're logging a request, you can immediately add a metric without needing to import or inject another dependency.

Consider this common scenario:

// Traditional approach - need both logger and statsd
function processRequest(logger: Logger, statsd: StatsD, userId: string) {
  logger.info('Processing request', { userId });
  statsd.increment('requests.total');
  // ... rest of function
}

// With LogLayer - just one object
function processRequest(log: LogLayer, userId: string) {
  log.info('Processing request', { userId });
  log.stats.increment('requests.total').send();
  // ... rest of function
}

The LogLayer version is simpler because you're working with one object instead of two. When you're already logging, adding metrics becomes a natural extension of what you're doing, not a separate concern.

Conclusion

Observability shouldn't be an afterthought. By providing a single, fluent API for both logging and metrics, LogLayer makes it natural to instrument your code as you write it. You don't need to juggle multiple APIs or remember different patterns for logging vs. metrics.

The hot-shots mixin's fluent builder pattern makes metrics configuration clean and expressive, while the unified API means your instrumentation code stays consistent throughout your application. The result is more comprehensive observability with less effort, better code organization, and metrics that naturally live alongside your logs.

If you're building Node.js applications and want to improve your observability while reducing code complexity, consider giving LogLayer's fluent logging and metrics API a try. You might find that the best way to ensure you collect metrics is to make them as easy to use as your logging.

Why LogLayer is the Logging Framework for Deno

Theo Gravity — Mon, 27 Oct 2025 03:10:46 +0000

Deno developers face a logging dilemma. Console logging is simple but limited. Traditional Node.js libraries like Winston or Pino aren't built for Deno's environment. LogLayer solves this by providing a unified logging layer designed for Deno, offering flexibility to adapt your logging infrastructure as your application grows without changing your code.

What Makes LogLayer Different

The framework provides a consistent, fluent API for specifying log messages, metadata, and errors, regardless of which underlying transport you're using. This means you can start with simple console logging during development, then seamlessly transition to more sophisticated logging solutions as your application grows, all without changing your logging code.

Here's a quick example of LogLayer's expressive API in action:

log.withContext({ userId: 123, environment: "production" })
  .withError(new Error("Connection failed"))
  .withMetadata({ retryCount: 3 })
  .error("Failed to connect to database")

All the context, error details, and metadata are captured in one fluent chain, making it easy to attach structured data to your log entries without cluttering your code.

Works With Your Favorite Logging Libraries

You can also use LogLayer with Deno-compatible logging libraries like LogTape and tslog.

Start with console logging first, then switch over to a better logging library later without changing the log entries in your application!

Installation and Setup

Getting started with LogLayer in a Deno project is straightforward. You have two primary options for installation. The first approach uses npm specifiers directly in your import statements:

import { LogLayer, ConsoleTransport } from "npm:loglayer@latest";

For better dependency management in production, LogLayer works with Deno's import map system. Create a deno.json file:

{
  "imports": {
    "loglayer": "npm:loglayer@latest"
  }
}

Then import using clean module paths:

import { LogLayer, ConsoleTransport } from "loglayer";

const log = new LogLayer({
  transport: new ConsoleTransport({
    logger: console
  })
});

log.withMetadata({ userId: "123" }).info("Processing user request");

If you want to use a logging library like tslog:

import { Logger } from "npm:tslog@latest";
import { LogLayer } from "npm:loglayer@latest";
import { TsLogTransport } from "npm:@loglayer/transport-tslog@latest";

const tslog = new Logger();

const log = new LogLayer({
  transport: new TsLogTransport({
    logger: tslog
  })
});

log.withMetadata({ userId: "123" }).info("Processing user request");

All your logging code remains unchanged when swapping transports, making it easy to experiment with different backends without rewriting your application code.

Basic Example Usage

The framework's fluent API makes it easy to attach metadata and errors to log entries.

log.withContext({
  userId: 12345,
  environment: "production"
});

log.withMetadata({
  action: "user_login",
  ipAddress: "192.168.1.1"
}).info("User logged in successfully");

log.withError(new Error("Database connection failed"))
  .withMetadata({ query: "SELECT * FROM users" })
  .error("Query execution failed");

The context data persists across all subsequent log calls, automatically including the user ID and environment in every log entry. Meanwhile, metadata can be added per log call to provide specific details about that particular event. Errors are handled consistently, with LogLayer serializing error objects in a standardized format that works across all transports.

Pretty Terminal Logging

For development, LogLayer supports the Simple Pretty Terminal transport, which provides beautifully formatted, color-coded log output in your Deno terminal:

import { LogLayer } from "npm:loglayer@latest";
import { getSimplePrettyTerminal } from "npm:@loglayer/transport-simple-pretty-terminal@latest";

const log = new LogLayer({
  transport: getSimplePrettyTerminal({
    runtime: "node", // Use "node" for Deno
    viewMode: "inline", // "inline" | "message-only" | "expanded"
  })
});

log.withMetadata({ userId: 123, action: "login" }).info("User logged in");

This transport offers multiple view modes and customizable themes, making it easier to read and debug logs during development. The transport works seamlessly with Deno and doesn't require any Node-specific dependencies.

Conclusion

LogLayer provides Deno developers with a production-ready logging solution that scales from simple console output to sophisticated backends like LogTape and tslog, all without changing your code. The framework's transport-based architecture and fluent API make it an ideal choice for any Deno project, whether you're building a CLI tool or a production application.

Why use LogLayer as the logger for your SDK

Theo Gravity — Sun, 12 Oct 2025 23:04:52 +0000

When building Typescript / JavaScript SDKs and libraries, choosing the right logging solution is crucial for both developer experience and maintainability. LogLayer stands out as the ideal choice for SDK and library developers due to its unique combination of flexibility, consistency, and powerful features designed specifically for complex software ecosystems.

The Logging Challenge in SDK Development

SDK and library developers face unique logging challenges that traditional logging solutions often fail to address adequately:

Dependency conflicts: Different consumers may use different logging libraries
Inconsistent APIs: Each logging library has its own parameter order and method signatures
Context management: Libraries need to maintain context across different execution scopes
Flexibility requirements: Different deployment environments require different logging destinations
Error handling: Inconsistent error serialization across different logging backends

LogLayer solves these problems by providing a unified, consistent interface that works across all major logging libraries and environments.

Key Benefits for SDK and Library Developers

1. Developer-friendly API

LogLayer provides an intuitive, fluent API that makes logging simple and expressive. Here's how easy it is to get started:

import { LogLayer, ConsoleTransport } from 'loglayer'

// Simple setup - works out of the box
const log = new LogLayer({
  transport: new ConsoleTransport({ logger: console })
})

// Clean, readable logging
log.info('User logged in successfully')

// Add structured metadata
log.withMetadata({ userId: '1234', action: 'login' }).info('User action completed')

// Handle errors elegantly
log.withError(new Error('Database connection failed')).error('Operation failed')

// Chain multiple operations
log
  .withMetadata({ requestId: 'req-123' })
  .withError(new Error('Validation failed'))
  .error('Request processing failed')

This fluent API makes your SDK's logging code clean, maintainable, and easy to understand for both you and your users.

2. Universal Compatibility

LogLayer supports over 30 different logging libraries and cloud providers, ensuring your SDK works seamlessly regardless of what your users prefer:

Logging Libraries: Pino, Winston, Bunyan, Log4js, tslog, and many more
Cloud Providers: Datadog, New Relic, Google Cloud Logging, AWS Lambda Powertools, Better Stack, and others
Runtime Support: Node.js, Deno, Bun, and browser environments

This means your SDK can integrate with any logging infrastructure your users have, without forcing them to adopt a specific logging library.

3. Consistent API Across All Environments

Unlike traditional logging libraries that have different parameter orders and method signatures, LogLayer provides a consistent, fluent API:

// LogLayer - consistent everywhere
log.withMetadata({ userId: '1234' }).withError(error).error('Operation failed')

// Traditional libraries - different APIs
winston.info('message', { data })     // winston
bunyan.info({ data }, 'message')      // bunyan
pino.info({ data }, 'message')        // pino

This consistency reduces cognitive load for developers using your SDK and eliminates the need to remember different logging patterns.

4. Flexible Transport System

Your SDK users may want to send logs to different destinations. LogLayer's transport system allows them to choose:

// User can configure multiple transports simultaneously
const log = new LogLayer({
  transport: [
    new PinoTransport({ logger: pino() }),           // Local logging
    new DatadogTransport({ apiKey: 'xxx' }),         // Cloud logging
    new ConsoleTransport({ logger: console })        // Development
  ]
})

5. Built-in Plugin System

LogLayer's plugin system allows you to add powerful features without bloating your SDK:

const log = new LogLayer({
  transport: new ConsoleTransport({ logger: console }),
  plugins: [
    redactionPlugin({ paths: ['password', 'token'] }), // Security
    sprintfPlugin()                                    // String formatting
  ]
})

6. Standardized Error Handling

Error handling is notoriously inconsistent across logging libraries. LogLayer standardizes this:

// Works the same way regardless of underlying logging library
log.withError(new Error('Database connection failed')).error('Operation failed')

// Automatic error serialization with stack traces
log.errorOnly(new Error('Validation error'))

7. Zero Configuration for Users

Your SDK users can start with minimal configuration and scale up as needed:

// Minimal setup - works out of the box
const log = new LogLayer({
  transport: new ConsoleTransport({ logger: console })
})

// Advanced setup - when users need more features
const log = new LogLayer({
  transport: [
    new PinoTransport({ logger: pino() }),
    new DatadogTransport({ apiKey: process.env.DATADOG_API_KEY })
  ],
  plugins: [redactionPlugin({ paths: ['password'] })],
  contextFieldName: 'context',
  metadataFieldName: 'metadata'
})

8. Testing and Mocking Support

LogLayer makes testing your SDK much easier with built-in support for mocking and test-friendly configurations:

// In your SDK tests
import { MockLogLayer } from 'loglayer'
import { vi } from 'vitest'

describe('SDK Functionality', () => {
  let sdk: MySDK

  beforeEach(() => {
    sdk = new MySDK({ 
      logger: new MockLogLayer() // no-op logger so tests don't produce logs
    })
  })
})

Implementation Example

Here's how you might integrate LogLayer into an SDK:

// sdk.ts
import { type ILogLayer, LogLayer, ConsoleTransport } from 'loglayer'

export interface SDKConfig {
  // Use the ILogLayer interface to allow different LogLayer versions to be used by end-users
  logger?: ILogLayer
  sdkVersion?: string
  sdkName?: string
}

export class MySDK {
  private log: ILogLayer

  constructor(config?: SDKConfig) {
    // Allow users to provide their own LogLayer instance
    this.log = config?.logger || this.createDefaultLogger()

    // Set SDK-level context
    this.log.withContext({
      sdk_version: config?.sdkVersion || '1.0.0',
      sdk_name: config?.sdkName || 'my-awesome-sdk'
    })
  }

  private createDefaultLogger(): ILogLayer {
    // Fallback to a simple console logger if none provided
    return new LogLayer({
      transport: new ConsoleTransport({ logger: console }),
      contextFieldName: 'context',
      metadataFieldName: 'metadata'
    })
  }
}

// Usage examples:
// 1. SDK creates its own default logger
const sdk = new MySDK({
  sdkVersion: '2.0.0',
  sdkName: 'my-sdk'
})

// 2. User provides their own LogLayer instance
const userLogger = new LogLayer({
  transport: new PinoTransport({ logger: pino() }),
  plugins: [redactionPlugin({ paths: ['password'] })]
})

const sdk = new MySDK({
  logger: userLogger,
  sdkVersion: '2.0.0',
  sdkName: 'my-sdk'
})

Conclusion

By choosing LogLayer for your SDK or library, you're providing your users with a logging solution that adapts to their needs rather than forcing them to adapt to yours.

Send logs to Sumo Logic in Node.js using LogLayer

Theo Gravity — Mon, 03 Mar 2025 03:31:50 +0000

Sumo Logic support in the Javascript / Node.js ecosystem is lacking. The sumo-logger package has not been updated in years, most likely due to their adoption of Open Telemetry (otel), and wants you to integrate using an otel stack instead.

What if you don't care about otel, and just want to send logs?

LogLayer offers a Sumo Logic Transport that you can use to send log entries to Sumo Logic.

What is LogLayer

LogLayer is the modern Typescript logging library built for structured logging. It wraps around your favorite logging libraries like pino and winston.

The logging APIs were built for the best developer experience possible:

import { LogLayer, ConsoleTransport } from 'loglayer'

const log = new LogLayer({
  // swap with pino, or multiple loggers
  transport: [
    new ConsoleTransport({
      logger: console,
    })
  ],
})

// Basic logging
log.info('Hello world!')

// Logging with metadata
log.withMetadata({ user: 'john' }).info('User logged in')

// Logging with context (persists across log calls)
log.withContext({ requestId: '123' })

log.info('Processing request') // Will include requestId

// Logging errors
log.withError(new Error('Something went wrong')).error('Failed to process request')

Integrating LogLayer with Sumo Logic

Once you create an HTTP source in Sumo Logic, you can integrate by doing the following:

npm install @loglayer/transport-sumo-logic serialize-error loglayer

import { LogLayer } from "loglayer";
import { SumoLogicTransport } from "@loglayer/transport-sumo-logic";
import { serializeError } from "serialize-error";

const transport = new SumoLogicTransport({
  url: "YOUR_SUMO_LOGIC_HTTP_SOURCE_URL",
});

const logger = new LogLayer({
  errorSerializer: serializeError,  // Important for proper error serialization
  transport
});

logger.info("Hello from LogLayer!");

You should now see logs sent to Sumo Logic!

For more information

Writing Logs to Files in Node.js with LogLayer

Theo Gravity — Sun, 02 Mar 2025 20:41:46 +0000

Writing logs to files is a crucial aspect of application monitoring and debugging in production environments. In this article, we'll explore how to effectively manage file-based logging in Node.js applications using LogLayer and the Log File Rotation Transport, a powerful and flexible logging library.

What is LogLayer?

LogLayer is a modern, TypeScript-first logging library that provides a clean and intuitive API for structured logging, and it acts as a layer on top of your favorite logging library like winston or pino.

Why LogLayer for File Logging?

LogLayer provides a robust solution for file-based logging with several advantages:

Automatic log file rotation based on size or time
Support for log compression
Batching capabilities for improved performance
Flexible log formatting
Automatic cleanup of old log files

Getting Started

First, install the necessary packages:

npm install loglayer @loglayer/transport-log-file-rotation serialize-error

Basic File Logging Setup

Here's a basic example of setting up file logging:

import { LogLayer } from "loglayer";
import { LogFileRotationTransport } from "@loglayer/transport-log-file-rotation";
import { serializeError } from "serialize-error";

const logger = new LogLayer({
  errorSerializer: serializeError,
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log"
    }),
  ],
});

// Start logging!
logger.withMetadata({
  port: 3000,
}).info("Application started");

logger
  .withError(new Error("Something went wrong"))
  .error("An error occurred");

Adding Context and Metadata

LogLayer provides several ways to enrich your logs with additional information:

Using Context

Context is persistent data that applies to all subsequent log entries:

import { hostname } from "node:os";

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log"
    }),
  ],
});

// Add context that will be included in all subsequent logs
const contextLogger = logger.withContext({
  hostname: hostname(),
  environment: process.env.NODE_ENV,
  version: process.env.npm_package_version
});

contextLogger.info("Server starting");  // Will include context
contextLogger.error("Connection failed");  // Will include same context

Using Metadata

Metadata is one-time data that only applies to the current log entry:

// Add metadata for a single log entry
logger.withMetadata({
  requestId: "123",
  duration: 45,
  statusCode: 200
}).info("Request processed");

// Different metadata for different log entries
logger.withMetadata({
  userId: "user-456",
  action: "login"
}).info("User logged in");

// Combine context and metadata
const userLogger = logger.withContext({ service: "user-service" });

userLogger.withMetadata({
  duration: 123,
  status: "success"
}).info("Operation completed");

Static Data in Transport

You can also configure static data at the transport level:

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log",
      staticData: {
        hostname: hostname(),
        environment: process.env.NODE_ENV,
        version: process.env.npm_package_version
      }
    }),
  ],
});

This will produce logs like:

{
  "level": "info",
  "message": "Request processed",
  "timestamp": "2024-01-17T12:34:56.789Z",
  "hostname": "prod-server-1",
  "environment": "production",
  "version": "1.0.0",
  "requestId": "123",
  "duration": 45,
  "statusCode": 200
}

Advanced Configuration

Daily Log Rotation

For applications that need to rotate logs daily:

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app-%DATE%.log",
      frequency: "daily",
      dateFormat: "YMD",
      compressOnRotate: true,
      maxLogs: "7d" // Keep logs for 7 days
    }),
  ],
});

Size-based Rotation

For high-volume applications, rotating based on file size:

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log",
      size: "10M", // Rotate when file reaches 10 MB
      maxLogs: 5, // Keep last 5 log files
      compressOnRotate: true
    }),
  ],
});

Performance Optimization with Batching

For high-throughput applications, enable batching to reduce disk I/O:

const logger = new LogLayer({
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log",
      batch: {
        size: 1000,    // Write after 1000 logs are queued
        timeout: 5000  // Or after 5 seconds, whichever comes first
      }
    }),
  ],
});

Best Practices

Use Rotation: Always configure log rotation to prevent disk space issues:

   new LogFileRotationTransport({
     filename: "./logs/app-%DATE%.log",
     frequency: "daily",
     maxLogs: "30d",
     compressOnRotate: true
   })

Enable Batching for High Volume: For applications with high log volume:

   new LogFileRotationTransport({
     filename: "./logs/app.log",
     batch: {
       size: 1000,
       timeout: 5000
     }
   })

Separate Logs by Concern: Use different transports for different types of logs:

   const logger = new LogLayer({
     transport: [
       new LogFileRotationTransport({
         filename: "./logs/app.log"
       }),
       new LogFileRotationTransport({
         filename: "./logs/errors.log",
         levelMap: {
           error: "ERROR",
           fatal: "FATAL"
         }
       })
     ],
   });

Use Context and Metadata Appropriately:

   // Use context for values that remain constant
   const serviceLogger = logger.withContext({
     service: "user-service",
     version: "1.0.0"
   });

   // Use metadata for request-specific information
   serviceLogger.withMetadata({
     requestId: "req-123",
     duration: 45,
     statusCode: 200
   }).info("Request completed");

Error Handling

Proper error serialization is crucial for debugging:

import { serializeError } from "serialize-error";

const logger = new LogLayer({
  errorSerializer: serializeError,
  transport: [
    new LogFileRotationTransport({
      filename: "./logs/app.log"
    }),
  ],
});

try {
  throw new Error("Something went wrong");
} catch (error) {
  logger.withError(error).error("Operation failed");
}

Conclusion

LogLayer provides a powerful and flexible solution for file-based logging in Node.js applications. With features like automatic rotation, compression, and batching, it helps you maintain clean and efficient logs while ensuring you don't miss important information for debugging and monitoring.

For more detailed information about LogLayer's file rotation transport, check out:

Taking pretty printing of logs to the next level with LogLayer

Theo Gravity — Mon, 24 Feb 2025 02:33:53 +0000

LogLayer, the modern logging library for Javascript / Typescript that supports shipping logs to loggers like Pino / Winston and SaaS providers like DataDog, now has a powerful pretty printing feature offered by the Pretty Terminal Transport:

🔍 Interactive Selection Mode - Browse and inspect logs in a full-screen interactive view
📝 Detailed Log Inspection - Examine individual log entries with formatted data and context
🔎 Search/Filter Functionality - Find specific logs with powerful filtering capabilities
💅 JSON Pretty Printing - Beautifully formatted structured data with syntax highlighting
🎭 Configurable Themes - Choose from pre-built themes or customize your own colors

Try it out

npm install loglayer @loglayer/transport-pretty-terminal serialize-error

import { LogLayer } from 'loglayer';
import { getPrettyTerminal } from '@loglayer/transport-pretty-terminal';
import { serializeError } from "serialize-error";

// Create LogLayer instance with the transport
const log = new LogLayer({
  errorSerializer: serializeError,
  transport: [
    getPrettyTerminal()
  ],
});

// Start logging!
log.withMetadata({ foo: 'bar' }).info('Hello from Pretty Terminal!');

Simple View

You start out in the simple view, where your logs will print out. You can pause the log stream with "P".

Selection / search view

Pressing up or down in the simple view will switch to the selection / search view, where you can search through your logs:

Start typing to filter out logs:

Detailed view

This is useful for very nested JSON data or arrays. Selecting a log entry from the selection view will bring up the detailed view:

You can scroll through the data, and use keyboard commands like "C" to expand out arrays to see the underlying data in them.

JSON view

Pressing "J" in the detailed view will print out the raw JSON, giving you an opportunity to properly copy the JSON data for use elsewhere:

For more information

For using LogLayer in general, visit the site to get started.

If you want to learn more about the different themes (including creating your own), and keyboard shortcuts for the pretty print, check out the Pretty Terminal Transport documentation.

Custom / structured logging in Next.js

Theo Gravity — Tue, 28 Jan 2025 02:14:34 +0000

Next.js does not have a way to use a custom logger for server-side uncaught exceptions and rejections.

While you can use a library like next-logger to help, you will be limited to using Pino only. This doesn't help if you want to use another logging library or even ship the logs to a cloud provider like DataDog.

Instead, you can use the LogLayer logging library to capture these exceptions and ship them off to your favorite logging library like Pino and DataDog at the same time.

Check the LogLayer site for supported loggers and cloud providers.

Install

This guide assumes you already have Next.js set up.

First, install the required packages. You can use any transport you prefer - we'll use Pino in this example:

npm i loglayer @loglayer/transport-pino pino serialize-error

Setup

You will need to create an instrumentation file in the root of your project.

// instrumentation.ts
import { LogLayer, type ILogLayer } from 'loglayer';
import { PinoTransport } from "@loglayer/transport-pino";
import pino from "pino";
import { serializeError } from "serialize-error";

/**
 * Strip ANSI codes from a string, which is something Next.js likes to inject.
 */
function stripAnsiCodes(str: string): string {
  return str.replace(
    /[\u001b\u009b][[()#;?]*(?:[0-9]{1,4}(?:;[0-9]{0,4})*)?[0-9A-ORZcf-nqry=><]/g,
    "",
  );
}

/**
 * Create a console method that logs to LogLayer
 */
function createConsoleMethod(log: ILogLayer, method: "error" | "info" | "warn" | "debug" | "log") {
  let mappedMethod = method;

  if (method === "log") {
    mappedMethod = "info";
  }

  return (...args: unknown[]) => {
    const data: Record<string, unknown> = {};
    let hasData = false;
    let error: Error | null = null;
    const messages: string[] = [];

    for (const arg of args) {
      if (arg instanceof Error) {
        error = arg;
        continue;
      }

      if (typeof arg === "object" && arg !== null) {
        Object.assign(data, arg);
        hasData = true;
        continue;
      }

      if (typeof arg === "string") {
        messages.push(arg);
      }
    }

    let finalMessage = stripAnsiCodes(messages.join(" ")).trim();

    // next.js uses an "x" for the error message when it's an error object
    if (finalMessage === "⨯" && error) {
      finalMessage = error?.message || "";
    }

    if (error && hasData && messages.length > 0) {
      log.withError(error).withMetadata(data)[mappedMethod](finalMessage);
    } else if (error && messages.length > 0) {
      log.withError(error)[mappedMethod](finalMessage);
    } else if (hasData && messages.length > 0) {
      log.withMetadata(data)[mappedMethod](finalMessage);
    } else if (error && hasData && messages.length === 0) {
      log.withError(error).withMetadata(data)[mappedMethod]("");
    } else if (error && messages.length === 0) {
      log.errorOnly(error);
    } else if (hasData && messages.length === 0) {
      log.metadataOnly(data);
    } else {
      log[mappedMethod](finalMessage);
    }
  };
}

export async function register() {
  const logger = new LogLayer({
    errorSerializer: serializeError,
    transport: [
      new PinoTransport({
        logger: pino(),
      }),
    ]
  })

  if (process.env.NEXT_RUNTIME === "nodejs") {
    console.error = createConsoleMethod(logger, "error");
    console.log = createConsoleMethod(logger, "log");
    console.info = createConsoleMethod(logger, "info");
    console.warn = createConsoleMethod(logger, "warn");
    console.debug = createConsoleMethod(logger, "debug");
  }
}

Test

If you threw an error from page.tsx that is uncaught, you should see this in the terminal:

{"err":{"type":"Object","message":"test","stack":"Error: test\n    at Page (webpack-internal:///(rsc)/./src/app/page.tsx:12:11)","digest":"699232626","name":"Error"},"msg":"test"}

For more information

The best Javascript library for structured logging

Theo Gravity — Tue, 21 Jan 2025 06:13:46 +0000

Modern JavaScript applications need structured logging. As applications grow in complexity, being able to search, analyze, and monitor logs becomes critical. Yet most logging solutions make this surprisingly difficult.

Traditional logging libraries in JavaScript have a fundamental problem: they were designed for string-based logging first, with structured data added as an afterthought. This leads to several critical issues in modern applications:

Inconsistent Data Structure: Each log entry can have different shapes of data, making it hard to query and analyze logs at scale.
Poor Type Safety: Most loggers accept any object as metadata, leading to runtime errors and inconsistent data types.
Mixed Concerns: Message, data, and errors are often jumbled together, making it difficult to parse and process logs programmatically.
Limited Context Support: Adding application-wide context often requires manual string concatenation or messy object merging.
Inflexible APIs: Simple tasks like adding both error and metadata to a log require awkward workarounds.

Let's look at how popular logging libraries handle these challenges:

Winston

// Mixing message and data with no clear structure
winston.info("Processing order", { orderId, userId, amount });

Bunyan

// Requires creating child loggers just to add context
const orderLogger = logger.child({ orderId, userId });
orderLogger.info({ amount }, "Processing order");

Pino

// Similar issues - metadata and message mixed together
logger.info({ orderId, userId, amount }, "Processing order");

While these libraries have attempted to add structured logging capabilities, their string-first design makes them cumbersome for modern, data-driven logging needs. They each have their own way of handling structured data, but none provide a truly satisfying solution.

A Better Approach to Structured Logging

What if there was a logging library designed from the ground up for structured logging? One that enforced consistent log structure, provided type safety, and cleanly separated different types of data?

This is where LogLayer (MIT Licensed) comes in. Here's how it handles the same logging task:

// Clean separation of concerns with type-safe builder pattern
logger
  .withContext({ userId })  // application-wide context
  .withMetadata({          // request-specific data
    orderId,
    amount
  })
  .info("Processing order");

LogLayer was built from the ground up with structured logging as its core focus. It provides:

Type-safe builder pattern API
Clear separation between context, metadata, and messages
Consistent structure across your entire application
Powerful plugin system for data transformation
Support for multiple transports without changing log structure
First-class error handling and serialization

Let's explore how LogLayer solves each of the traditional logging problems:

Rich Structured Data Support

LogLayer makes it easy to include structured data with your logs:

// Add context that will be included with all logs
logger.withContext({ 
  service: "payment-api",
  version: "1.2.0"
});

// Add metadata for specific log entries
logger.withMetadata({ 
  orderId: "12345",
  amount: 99.99
}).info("Payment processed successfully");

First-Class Error Handling

Error logging is a core feature with dedicated support:

try {
  // ... some code that might throw
} catch (error) {
  logger.withError(error)
        .withMetadata({ userId: "123" })
        .error("Failed to process payment");
}

// Or log just the error
logger.errorOnly(error);

Flexible Configuration

LogLayer offers extensive configuration options:

Custom error serialization
Configurable field names for errors, context, and metadata
Enable/disable logging at runtime
Multiple transport support
Plugin system for extending functionality

Conclusion

LogLayer brings structure and consistency to your application's logging while remaining flexible and extensible. Its rich feature set and clean API make it an excellent choice for applications that need robust, structured logging capabilities.

LogLayer: A Modern Logging Library for TypeScript / JavaScript

Theo Gravity — Thu, 16 Jan 2025 01:09:14 +0000

Ever found yourself wrestling with different logging libraries across projects? Or maybe you've spent hours trying to standardize how errors and metadata are logged across your codebase?

I built LogLayer, an open-source solution to solve these common logging challenges by improving the developer experience around writing logs.

What is LogLayer?

LogLayer is a unified logging layer that sits on top of your favorite logging libraries (like Pino, Winston, Bunyan) and cloud providers (AWS, Google Cloud, DataDog, etc).

It provides a consistent, fluent API for logging messages, metadata, and errors while routing your logs to wherever they need to go.

Here's a quick example of what logging with LogLayer looks like:

import { LogLayer } from 'loglayer'
import { PinoTransport } from '@loglayer/transport-pino'
import pino from 'pino'

// Setup LogLayer with Pino
const log = new LogLayer({
  transport: new PinoTransport({
    logger: pino()
  })
})

// Log with metadata
log.withMetadata({ userId: '123', action: 'login' })
   .info('User logged in successfully')

// Logging with context (persists across log calls)
log.withContext({ requestId: '123' })
log.info('Processing request') // Will include requestId

try {
  // ... some code that might throw
} catch (error) {
  log.withError(error)
     .withMetadata({ userId: '123' })
     .error('Failed to process user request')
}

Benefits of using LogLayer

Swap Logging Libraries

Maybe you've discovered late in the game that Next.js and Pino don't play very well without additional configuration when you're building your Next.js project for the first time. You might want to immediately switch to another library to save yourself the trouble, but doing that might mean you would need to refactor all your logging code.

With LogLayer, you can easily swap logging libraries without changing your code.

Consistent API Across All Logging Libraries

No more remembering different parameter orders or method names:

// With LogLayer - consistent API everywhere
log.withMetadata({ some: 'data' }).info('my message')

// Without LogLayer - different APIs for different libraries
winston.info('my message', { some: 'data' })     // winston
bunyan.info({ some: 'data' }, 'my message')      // bunyan

Multi-Transport Support Out of the Box

Want to send logs to both DataDog and your logger? No problem:

import { LogLayer } from 'loglayer'
import { datadogLogs } from '@datadog/browser-logs'
import { PinoTransport } from "@loglayer/transport-pino"
import { DatadogBrowserLogsTransport } from "@loglayer/transport-datadog-browser-logs"

// Initialize Datadog
datadogLogs.init({
  clientToken: '<CLIENT_TOKEN>',
  site: '<DATADOG_SITE>',
  forwardErrorsToLogs: true,
  sampleRate: 100
})

const log = new LogLayer({
  transport: [
    new PinoTransport({
      logger: pino()
    }),
    new DatadogBrowserLogsTransport({
      logger: datadogLogs
    })
  ]
})

// Logs go to both Pino and DataDog
log.info('User logged in successfully')

Clean Error Handling

Standardized error handling that works the same way across all logging libraries:

// Error handling works consistently
log.withError(new Error('test'))
   .withMetadata({ userId: '123' })
   .error('Operation failed')

Powerful Plugin System

Extend functionality with plugins for things like redacting sensitive information:

const log = new LogLayer({
  plugins: [{
    onBeforeDataOut: (params) => {
      // Redact sensitive information before logging
      if (params.data?.password) {
        params.data.password = '***'
      }
      return params.data
    }
  }]
})

Easy Testing

Built-in mocks make testing a breeze:

import { MockLogLayer } from 'loglayer'

// Use MockLogLayer in your tests - no real logging will occur
const log = new MockLogLayer()

Getting Started

Installation is straightforward:

npm install loglayer

import { LogLayer, ConsoleTransport } from 'loglayer'

const log = new LogLayer({
  transport: new ConsoleTransport({
    logger: console,
  }),
})

log.info('Hello world!')