Elixir logging to (multiple) file(s) using metadata_filter

This is a follow up post for JSON Logging in Elixir using a custom formatter where we looked in to how to log request logs to a file using a custom formatter. We require below hex dependencies in our project to log request logs and other app logs.

{:plug_logger_json, "~> 0.7.0"},
{:logger_file_backend, "~> 0.0.12"}

Unlike non functional languages where you have log objects per log file and can use them to write to different log

require "logger"
request_logger = Logger.new('/src/obliviator/log/obliviator_request.log')
request_logger.debug("I'm a request debug log")

stream_logger = Logger.new('/src/obliviator/log/stream.log')
stream_logger.info("I'm a stream info log")

but in Elixir you only have a Logger object and need to differentiate based on metadata assigned to write to the associated log. metadata_filter can be assigned on multiple fields. We can separate logs by each application running on our VM or by key identifying our file.

# log by identifying application 
metadata_filter: [application: :plug_logger_json]

#log by tagging log by key value
metadata_filter: [log: :stream]

The full config would be as below

use Mix.Config

config :plug_logger_json,
  filtered_keys: ["password", "authorization"],
  suppressed_keys: ["api_version", "log_type"]

config :logger,
  backends: [
    {LoggerFileBackend, :request_log},
    {LoggerFileBackend, :message_log},
    {LoggerFileBackend, :debugging_log}
  ],
  utc_log: true,
  handle_otp_reports: true

# configuration for the {LoggerFileBackend, :request_log} backend
config :logger, :request_log,
  format: {Obliviator.Formatter.RequestLog, :format},
  metadata: :all,
  path: "/src/obliviator/log/obliviator_request.log",
  level: :info,
  metadata_filter: [application: :plug_logger_json]

config :logger, :message_log,
  format: {Obliviator.Formatter.MessageLog, :format},
  metadata: :all,
  path: "/src/obliviator/log/stream.log",
  level: :info,
  metadata_filter: [log: :stream]

config :logger, :debugging_log,
  format: {Obliviator.Formatter.MessageLog, :format},
  metadata: :all,
  path: "/src/obliviator/log/obliviator_debugging.log",
  level: :debug,
  metadata_filter: [log: :debugging]

Once the config is in place data can be logged as follows

# log to :stream
Logger.info("Stream msg", log: :stream)

#log to :debugging
Logger.debug("Debugging", log: :debugging)

Elixir logging might look limiting at the start but with a powerful custom formatter and metadata filtering ability its is sufficiently better than many ;)