DEV Community

Cover image for Elixir logging to (multiple) file(s) using metadata_filter
Ahsan Nabi Dar
Ahsan Nabi Dar

Posted on

 

Elixir logging to (multiple) file(s) using metadata_filter

This is a follow up post for JSON Logging in Elixir using a custom formatter where we looked in to how to log request logs to a file using a custom formatter. We require below hex dependencies in our project to log request logs and other app logs.

{:plug_logger_json, "~> 0.7.0"},
{:logger_file_backend, "~> 0.0.12"}
Enter fullscreen mode Exit fullscreen mode

Unlike non functional languages where you have log objects per log file and can use them to write to different log

require "logger"
request_logger = Logger.new('/src/obliviator/log/obliviator_request.log')
request_logger.debug("I'm a request debug log")

stream_logger = Logger.new('/src/obliviator/log/stream.log')
stream_logger.info("I'm a stream info log")

Enter fullscreen mode Exit fullscreen mode

but in Elixir you only have a Logger object and need to differentiate based on metadata assigned to write to the associated log. metadata_filter can be assigned on multiple fields. We can separate logs by each application running on our VM or by key identifying our file.

# log by identifying application 
metadata_filter: [application: :plug_logger_json]
Enter fullscreen mode Exit fullscreen mode
#log by tagging log by key value
metadata_filter: [log: :stream]
Enter fullscreen mode Exit fullscreen mode

The full config would be as below

use Mix.Config

config :plug_logger_json,
  filtered_keys: ["password", "authorization"],
  suppressed_keys: ["api_version", "log_type"]

config :logger,
  backends: [
    {LoggerFileBackend, :request_log},
    {LoggerFileBackend, :message_log},
    {LoggerFileBackend, :debugging_log}
  ],
  utc_log: true,
  handle_otp_reports: true

# configuration for the {LoggerFileBackend, :request_log} backend
config :logger, :request_log,
  format: {Obliviator.Formatter.RequestLog, :format},
  metadata: :all,
  path: "/src/obliviator/log/obliviator_request.log",
  level: :info,
  metadata_filter: [application: :plug_logger_json]

config :logger, :message_log,
  format: {Obliviator.Formatter.MessageLog, :format},
  metadata: :all,
  path: "/src/obliviator/log/stream.log",
  level: :info,
  metadata_filter: [log: :stream]

config :logger, :debugging_log,
  format: {Obliviator.Formatter.MessageLog, :format},
  metadata: :all,
  path: "/src/obliviator/log/obliviator_debugging.log",
  level: :debug,
  metadata_filter: [log: :debugging]
Enter fullscreen mode Exit fullscreen mode

Once the config is in place data can be logged as follows

# log to :stream
Logger.info("Stream msg", log: :stream)

#log to :debugging
Logger.debug("Debugging", log: :debugging)
Enter fullscreen mode Exit fullscreen mode

Elixir logging might look limiting at the start but with a powerful custom formatter and metadata filtering ability its is sufficiently better than many ;)

Top comments (0)

An Animated Guide to Node.js Event Loop

Node.js doesn’t stop from running other operations because of Libuv, a C++ library responsible for the event loop and asynchronously handling tasks such as network requests, DNS resolution, file system operations, data encryption, etc.

What happens under the hood when Node.js works on tasks such as database queries? We will explore it by following this piece of code step by step.