Know your Linux 06: Logs

Hello and welcome to the sixth and final entry in "Know Your Linux" series. This time let’s talk about Linux’s logging.

Before we dive right in 🏄‍♂️, make sure to check out the previous entry.

Article No Longer Available

Why logging?

Well, Logging acts as your friend and advisor when the customer comes yelling WHY IS IT NOT WORKING?, It helps you understand the flow of the application and identify the causes of different problems. Without logging, It could be very hard to answer the big question What went wrong?

Logging in Linux

Linux - like other operating systems - provides logs on everything from kernel events to user actions. We can thank two services for that systemd-journald and rsyslog. All the collected logs by default are stored in the directory /var/log.

Wanna know more about what services and daemons are? checkout the previous entry.

Article No Longer Available

Wanna know more about how Linux structures its directories?

Article No Longer Available

Configuring logs

The rsyslog configuration file is /etc/rsyslog.conf, It has the following syntax :

facility.priority;facility.priority;... path

For example :

*.info;mail.none;authpriv.warning; /var/log/logfile

We have 3 terminologies at play here :

• Facility: The type of application producing the log files, ex: mail, cron or authpriv
• Priority: The severity of the log produced by a certain application
• Path: Location of the log file where logs should be stored

We can override the /etc/rsyslog.conf in any file *.conf file under the /etc/rsyslog.d directory.

To prevent a certain facility from logging, the priority field can be set to none, which means that none of the logs from this facility will be added to the specified log file.

From the example above, We can see that the file /var/log/logfile will contain the following :

1. *.info Logs with priority set to info from all facilities.
2. mail.none None of the mail facility logs.
3. authpriv.warning Only warnings from the authpriv facility.

Log Priorities

The table below shows all priorities and their meaning.

Code	Priority	Severity
0	emerg	System is unstable
1	alert	Action must be taken immediately
2	crit	Critical condition
3	err	non-critical error
4	warning	Warning condition
5	notice	Normal but significant event
6	info	Informational event
7	debug	Debug-level messages

Do logs stay forever?

Typically the log files are saved under /var/log which is a persistent directory but not forever. The service responsible for rotating the logs is called logrotate which is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large. Isn't it awesome? 🕶️

That’s all from my side ✋, That's also the last entry in "Know Your Linux" series, Make sure to check out previous entries. Also, Don't forget to tell me your thoughts in the comments bellow 🙄

As always,
Happy coding 🔥🔥
“كود بسعادة”

Comments

[Thomas H Jones II]

Probably want to qualify this statement:

Typically the log files are saved under /var/log which is a persistent directory but not forever.

Unless you've created /var/log as a pseudo-filesystem, everything under it is persistent. It's simply other processes that may be configured to help ensure you don't blow out whatever filesystem /var/log is sitting on.

While many of the log files that get placed in /var/log are accounted for in the default logrotate configurations, it's not safe to simply assume that things will be defaulted away (or, even if they are defaulted for rotation, your system's logging activity may happen too quickly for the default-settings to account for). A couple of things that can be gotchas:

• The logs produced by auditd typically aren't rotated by the logrotate service
• Depending on how you've set up your syslog – whether you've altered the default logging-paths or added non-default log-outputters – your log files very well may end up being wholly persistent …Even up to the point where your system becomes unusable or even crashses due to having run out of space.
• Similarly, if you've configured any of your systemd units to handle their own logging activities (or installed applications that do similarly), you can likewise overflow your system
• If you've installed software that don't log into /var/log (Splunk, McAfee and a few other "enterprise" agents are great for this)

In short, always understand what and where your software/services are logging, how quickly they're doing and update your log-rotation utilities configuration(s) to account for it. And, if you're paranoid (or operate under configuration-mandates), it can be helpful to make sure your logging directories are on separate filesystems from your / filesystem.

[Bassem]

Yeah! That's for sure. Thanks for clarifying!!

[DUVERGIER Claude]

What about "user actions"? I see no guide/example about this in the article.

I was looking for something like Windows Auditing of Windows Events such as login/logoff, file creation/deletion/modification, etc. but for a Linux OS.

[Robin Palotai]

What tripped me, why call it maillog, if exactly the mail logs are missing :)

[Bassem]

Ohh, Sorry about that! 😂 Will fix it now!

[Eugene Vedensky]

Came for the article, stayed for the beautiful timex banner.

[Bassem]

Sure, Whatever works! 😂 Happy that you liked it. To be honest tho. I don't really like it. But I feel like I gotta stick with the one banner that series started with.