Great, I wrote the original reply from a point of view of taking care of centralized logging in a polyglot environment (including node.js) and being quite depressed with how people seem to be stuck reinventing doing logging the wrong way.
The way you need to think is in terms of many independent components that each generate logs that need to be captured somehow. You use stuff like Elastic Beats, logstash, syslog, etc. to collect logs from different sources and send them to remote systems where they get aggregated, analyzed and stored. Each application typically runs in a docker container. Docker comes with its own ecosystem of logging stuff (some of it quite crappy, beware). The bare minimum is simply capturing stdout and pumping it to some remote aggregator annotated with some metadata about where it came from for. For example logstash comes with a lot of functionality to help you make sense of all the crap you get from legacy systems doing whatever on stdout (e.g. nginx access logs, mysql logs, systemd, syslog) and pick that apart. A lot is lost in translation this way. The best way is to not lose stuff and output in a format that your remote aggregator understands, which is typically json or some well defined line based format like syslog.
What you built is basically a custom log formatter. What happened in the java world is that logging apis you use in your code were separated from log output generation pretty early on. That's a good idea, that needs adopting.
So, it is easy to switch any Java application to whatever output format you need and people do that all the time. Most other languages seem to not do that and so you end up with basically some hardcoded syntactic sugar around puts in ruby or console.log in javascript and log output that cannot be customized and lacks usable context. The best you can hope for is doing some application specific parsing of the output to grab things like timestamps and log levels. Garbage in, garbage out, basically.
So, if you want to facilitate your stuff being used in a cloud based environment, you want some option to output json instead of human readable stuff and be able to switch between both. It's OK to output that to stdout or stderr. In a dockerized environment that is both acceptable and easy to deal with. You get the best of both worlds.
I guess I need to be more clear when describing the reasoning behind and the goal of Ratlog.
The setup you are describing is definitely the right choice for cloud-scale infrastructure. There JSON output is a good choice for further processing and aggregation. That's why winston and bunyan do things the way they do. They are definitely a better choice for logging in a big system with centralized logging.
They also provide support for other formatting options than JSON so one could implement a pretty-printed version - maybe even using Ratlog.
Ratlog is custom format (mostly) for being consumed directly by humans. It doesn't need processing to be readable. Not all software is cloud-scale. I saw the need for Ratlog because we used these complex logging tools and in the end we were reading JSON manually. We don't want to have to setup a centralized logging system when running a simple system on a single machine.
The other issue Ratlog solves for us is giving us the right context for logs using tags and fields - kinda like what you described with Mapped Diagnostic Context.
You are definitely right, that this is still missing in the JS ecosystem. It would be nice if also in cloud-based scenarios logging libraries give you more tools to add context to logs in a standard way.
That's also an interesting problem. But not one Ratlog is trying to solve.
The point is that Ratlog is about output, not about the log api itself. These should be independent. So, why not separate those; like other frameworks do. So you have 1 way to log and multiple ways to target either human readable output or machine readable output. That would solve a real problem in the node.js world where most logging frameworks are naive and simplistic. I'd love to get better logging from node.js.
Just stopping by to let you know I appreciate the feedback you gave!
While I like to keep Ratlog.js as simple as possible to use for the simple use case, I realize the value of Ratlog's logging semantics separate from its output format.
I extended the API in a way which allows using those semantics independent from the output format and which should make it easy to use it on top of other logging frameworks, libraries and services:
Great, I wrote the original reply from a point of view of taking care of centralized logging in a polyglot environment (including node.js) and being quite depressed with how people seem to be stuck reinventing doing logging the wrong way.
The way you need to think is in terms of many independent components that each generate logs that need to be captured somehow. You use stuff like Elastic Beats, logstash, syslog, etc. to collect logs from different sources and send them to remote systems where they get aggregated, analyzed and stored. Each application typically runs in a docker container. Docker comes with its own ecosystem of logging stuff (some of it quite crappy, beware). The bare minimum is simply capturing stdout and pumping it to some remote aggregator annotated with some metadata about where it came from for. For example logstash comes with a lot of functionality to help you make sense of all the crap you get from legacy systems doing whatever on stdout (e.g. nginx access logs, mysql logs, systemd, syslog) and pick that apart. A lot is lost in translation this way. The best way is to not lose stuff and output in a format that your remote aggregator understands, which is typically json or some well defined line based format like syslog.
What you built is basically a custom log formatter. What happened in the java world is that logging apis you use in your code were separated from log output generation pretty early on. That's a good idea, that needs adopting.
So, it is easy to switch any Java application to whatever output format you need and people do that all the time. Most other languages seem to not do that and so you end up with basically some hardcoded syntactic sugar around puts in ruby or console.log in javascript and log output that cannot be customized and lacks usable context. The best you can hope for is doing some application specific parsing of the output to grab things like timestamps and log levels. Garbage in, garbage out, basically.
So, if you want to facilitate your stuff being used in a cloud based environment, you want some option to output json instead of human readable stuff and be able to switch between both. It's OK to output that to stdout or stderr. In a dockerized environment that is both acceptable and easy to deal with. You get the best of both worlds.
I guess I need to be more clear when describing the reasoning behind and the goal of Ratlog.
The setup you are describing is definitely the right choice for cloud-scale infrastructure. There JSON output is a good choice for further processing and aggregation. That's why winston and bunyan do things the way they do. They are definitely a better choice for logging in a big system with centralized logging.
They also provide support for other formatting options than JSON so one could implement a pretty-printed version - maybe even using Ratlog.
Ratlog is custom format (mostly) for being consumed directly by humans. It doesn't need processing to be readable. Not all software is cloud-scale. I saw the need for Ratlog because we used these complex logging tools and in the end we were reading JSON manually. We don't want to have to setup a centralized logging system when running a simple system on a single machine.
The other issue Ratlog solves for us is giving us the right context for logs using tags and fields - kinda like what you described with Mapped Diagnostic Context.
You are definitely right, that this is still missing in the JS ecosystem. It would be nice if also in cloud-based scenarios logging libraries give you more tools to add context to logs in a standard way.
That's also an interesting problem. But not one Ratlog is trying to solve.
The point is that Ratlog is about output, not about the log api itself. These should be independent. So, why not separate those; like other frameworks do. So you have 1 way to log and multiple ways to target either human readable output or machine readable output. That would solve a real problem in the node.js world where most logging frameworks are naive and simplistic. I'd love to get better logging from node.js.
Just stopping by to let you know I appreciate the feedback you gave!
While I like to keep Ratlog.js as simple as possible to use for the simple use case, I realize the value of Ratlog's logging semantics separate from its output format.
I extended the API in a way which allows using those semantics independent from the output format and which should make it easy to use it on top of other logging frameworks, libraries and services:
You can find more in the API docs