I wanted to write this post because the most common question I get when I issue a pull request for an optimization is, "How the hell did you find t...
For further actions, you may consider blocking this person and/or reporting abuse
Great article Molly!
I also give a big priority to my logs because it's the only feedback I can have from my app.
We use a JSON logs specification to have a common logs format on all the apps in my project and to allow to build common kibana dashboards for all of them.
But with a high trafic, logs can become a bottleneck and may sometimes be replaced by some "code instrumentalization" (e.g. Prometheus exporter or whatever).
Interesting! I have never heard of Prometheus, that looks neat. For production we have Fluentd setup for exporting logs to our Elasticsearch logging cluster and we also use Kibana for creating dashboards and saving searches. We have found Fluentd works really nice with Ruby which is why we choose it over something like logstash.
We also use Fluentd rather than logstash in our logs stack.
Prometheus is a great tool, but it's not based on the logs. Your app has to provide an API from which Prometheus will grab your metrics to build real-time aggregated dashboards.
Prometheus + Graphana = graphing heaven. :)
I heart common log formatting. Done right it is very helpfuil; done wrong, the problem you stated.
Delayed follow up (it took a while for grep to finish!): we had a conversation about /how/ to do logging back in '17 that fits nicely with Molly's why & when:
dev.to/grhegde09/logging-done-righ...
That's a great post! I love that we both linked back to Star Trek on it, great minds think alike 😉
Great article!
I used to work in a project, in which my team was developing a windows driver. We invested heavily in logging. The goal was to have a logging system that can be enabled/disabled in runtime at any time. Then we did our own tool to improve log parsing.
What we've achieved was truly impressive. There were single cases when I needed to use WinDbg for something else than extracting logs from a memory dump. However, the most important was, that we were able to diagnose in field issue, by asking a customer to run a command and send us a log output (btw. logs were binary and could only be read if we shared symbol files). One cannot praise enough such feature if your product is shipped with millions of computers.
Excellent article! I have a question: what log level do you set for your production environment?
info! Let me know if you have any other questions :)
LOL! :D
Fabulous post Molly
Thank you! This community you have formed seems really awesome, I am excited to contribute more to it :)
This is one of those things that come with experience
What software development skills only come with experience?
Ben Halpern
Tracking bugs is fun (most of the times).
Nice post :)
So true!
Logs can be really intimidating when you are a new developer bc the first time you see them flash by there are so many. I think in the future I will try to encourage new developers to look more at the logs when they are starting out. Even if it's just for small pieces of code, I think it would help them get more familiar with the code and how logging works.