Any server or application needs some kind of logging system. It is common to skip logging in personal or practice project. But it is a must to have a fairly robust logging system in production ready servers and apps.
Benefits of a logging system:
- Logs are used by product managers and UX designers for planning & design.
- It helps marketers track performance of various features related to advertising campaigns.
- It gives us the idea of how much traffic is the application getting.
- Track errors, attacks and unsuccessful requests in production.
Getting started with Morgan
Morgan is a logging package for node.js. It can generate log files for all the traffic that comes to your server. It has some cool tricks to help us in both development and production. So let's get started.
Install Morgan
$ npm install morgan --save
Basic Morgan Usage
Adding morgan to middlewares in express is enough to get you started.
const express = require('express');
const morgan = require('morgan'); // import morgan
const app = express();
// setup morgan
app.use(morgan("dev"));
app.listen(3000, () => {
console.debug('App listening on :3000');
});
Above setup will start logging requests in console. You can see, I have specified dev in morgan setup. There are many presets available in morgan and one them is dev. Some other presets are:
-
tiny shortdev-
common: Follows Apache like log format
Logging requests to file
Now we will learn how to output logs to a file. Instead of creating simple logging to file, we will create rotating logs. Rotating logs are logs that covers a part of activity on the server in a file and then creates a new log file. This way we can keep track of past activities. We don't have to keep one single log file open for continuous logging. So, let's setup this.
For rotating logs, we will install a package named rotating-file-stream. It will help us automate log rotation.
$ npm install rotating-file-stream --save
Setup Morgan
const express = require('express');
const morgan = require('morgan'); // import morgan
const rfs = require("rotating-file-stream");
const app = express();
// MORGAN SETUP
// create a log stream
const rfsStream = rfs.createStream("log.txt", {
size: '10M', // rotate every 10 MegaBytes written
interval: '1d', // rotate daily
compress: 'gzip' // compress rotated files
})
// add log stream to morgan to save logs in file
app.use(morgan("dev", {
stream: rfsStream
}));
// another logger to show logs in console as well
app.use(morgan("dev"));
app.listen(3000, () => {
console.debug('App listening on :3000');
});
That was enough to setup our logging system but we can add environment variables to make it more intuitive to enable and disable logging to file
Adding environment variables to our setup
You will need dotenv package to load environment variables in nodejs from .env file
$ npm install dotenv --save
Now create a .env file in the root directory of project and add the following variables.
LOG_FILE=log.txt
LOG_FORMAT=common
LOG_SIZE=10M
LOG_INTERVAL=1d
Let's change our configuration to use these variables
require('dotenv').config() // load variables from .env file
const express = require('express');
const morgan = require('morgan'); // import morgan
const rfs = require("rotating-file-stream");
const app = express();
// MORGAN SETUP
// create a log stream
const rfsStream = rfs.createStream(process.env.LOG_FILE || 'log.txt', {
size: process.env.LOG_SIZE || '10M',
interval: process.env.LOG_INTERVAL || '1d',
compress: 'gzip' // compress rotated files
});
// if log file defined then use rfs stream else print to console
app.use(morgan(process.env.LOG_FORMAT || "dev", {
stream: process.env.LOG_FILE ? rfsStream : process.stdout
}));
// if log file is defined then also show logs in console
// else it will use the previous process.stdout to print to console
if(process.env.LOG_FILE) {
app.use(morgan(process.env.LOG_FORMAT || "dev"));
}
app.listen(3000, () => {
console.debug('App listening on :3000');
});
With above configuration we can relax and let morgan handle the job of logging. Whenever an error occurs, visit the log file and you can track where things went wrong.
I hope this article helped you understand why it important to have a good logging system and how to add one in express.
Top comments (7)
Thanks for taking your time to explain it in detail.
About rotating logs, does the API create new log.txt file and wipes the previous when the limit (10M / 1day) is reached or does it create additional files, like log.txt, log1.txt, log2.txt, etc?
I don't know about it just yet. But I will figure that out and let you know if I get the answer.
I went ahead to try it myself using a local server and nodemon and I found it a bit disappointing that your code doesn't even run. Instead it prints an error:
TypeError: Router.use() requires a middleware function but got a ObjectAfter short investigation I figured what the problem is. Rather than passing options Object to a morgan middleware, you passed it as a separate entity to app.use (as if it's a middleware).
app.use(morgan('dev'), { stream: rfsStream });The correct code looks like this (with options passed as a second param to morgan)
app.use(morgan('dev', { stream: rfsStream }))With that out of the way, I set interval to only 2 minutes and ran the app locally to see what happens.
The end result is very interesting.
Every log that occurs will be written in log.txt file. When the timer runs out, the RFS library creates a new log file with timestamp as prefix 20220108-1214-01-log.txt and stores all of the logs from the previous period in the newly created file.
As for the original, RFS wipes the content inside it and populates it with new logs.
The process is then repeated again and again until you shutdown the server or increase the interval.
And that's how it works 😎
Nice one. Congrats that you get it to run.
Thank you for pointing out the error, I will update the code :)
Morgan is a bit antiquated, we've opted for something more modern: dev.to/wparad/hacking-your-product...
I see. Thanks for the great reference :)
Awesome Guide !