DEV Community

koh-sh
koh-sh

Posted on • Originally published at koh-sh.hatenablog.com

Finally I am using journalctl

I know it is too late but now I found out journalctl is convenient.

What is journalctl

From the output of man journalctl

NAME
       journalctl - Query the systemd journal

SYNOPSIS
       journalctl [OPTIONS...] [MATCHES...]

DESCRIPTION
       journalctl may be used to query the contents of the systemd(1) journal as written by
       systemd-journald.service(8).
Enter fullscreen mode Exit fullscreen mode

OK, then what is journald?
Let's see man systemd-journald

DESCRIPTION
       systemd-journald is a system service that collects and stores logging data. It creates and
       maintains structured, indexed journals based on logging information that is received from a
       variety of sources:
Enter fullscreen mode Exit fullscreen mode

To put it simply, journald corrects and manages many kinds of logs.
It can correct the below logs. Pretty much every log.

  • Kernel log messages, via kmsg
  • Simple system log messages, via the libc syslog(3) call
  • Structured system log messages via the native Journal API, see sd_journal_print(3)
  • Standard output and standard error of service units. For further details see below.
  • Audit records, originating from the kernel audit subsystem

Work with journalctl

So let's play with journalctl.
Environment: CentOS 7.5.1804 on Vagrant.

Without any options, it prints all logs stated above.
Older entries come top and operated with a pager like less command.

% sudo journalctl

-- Logs begin at Sat 2019-04-13 14:05:21 JST, end at Sat 2019-04-13 14:26:56 JST. --
Apr 13 14:05:21 localhost.localdomain systemd-journal[84]: Runtime journal is using 6.1M (max allowed 49.5M, trying to leave 74.3M free of 489.6M available <E2><86><92> current limit 49.5M).
Apr 13 14:05:21 localhost.localdomain kernel: Initializing cgroup subsys cpuset
Apr 13 14:05:21 localhost.localdomain kernel: Initializing cgroup subsys cpu
<ellipsis>
Apr 13 14:20:01 Vag2 systemd[1]: Stopping User Slice of root.
Apr 13 14:20:32 Vag2 systemd[1]: Starting Cleanup of Temporary Directories...
Apr 13 14:20:32 Vag2 systemd[1]: Started Cleanup of Temporary Directories.
Apr 13 14:23:18 Vag2 sudo[2428]:  vagrant : TTY=pts/0 ; PWD=/home/vagrant ; USER=root ; COMMAND=/bin/journalctl

Enter fullscreen mode Exit fullscreen mode

Options can be used to filter logs.
Here are some useful options.

Kernel messages

-k prints kernel messages like dmesg command.

% sudo journalctl -k

-- Logs begin at Sat 2019-04-13 14:05:21 JST, end at Sat 2019-04-13 14:34:48 JST. --
Apr 13 14:05:21 localhost.localdomain kernel: Initializing cgroup subsys cpuset
Apr 13 14:05:21 localhost.localdomain kernel: Initializing cgroup subsys cpu
Apr 13 14:05:21 localhost.localdomain kernel: Initializing cgroup subsys cpuacct
Apr 13 14:05:21 localhost.localdomain kernel: Linux version 3.10.0-862.11.6.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 21:49:04 UTC 2018
Apr 13 14:05:21 localhost.localdomain kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64 root=/dev/mapper/centos-root ro net.ifnames=0 biosdevname=0 crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8
Enter fullscreen mode Exit fullscreen mode

Units

-u prints specific unit logs.
Printing HAproxy's log for example.

% sudo journalctl -u haproxy

-- Logs begin at Sat 2019-04-13 14:05:21 JST, end at Sat 2019-04-13 15:00:48 JST. --
Apr 13 14:05:31 Vag2 systemd[1]: Started HAProxy Load Balancer.
Apr 13 14:05:31 Vag2 systemd[1]: Starting HAProxy Load Balancer...
Apr 13 14:05:31 Vag2 haproxy-systemd-wrapper[1019]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
Apr 13 15:00:45 Vag2 systemd[1]: Reloaded HAProxy Load Balancer.
Apr 13 15:00:45 Vag2 haproxy-systemd-wrapper[1019]: haproxy-systemd-wrapper: re-executing on SIGUSR2.
Apr 13 15:00:45 Vag2 haproxy-systemd-wrapper[1019]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds -sf 1036
Enter fullscreen mode Exit fullscreen mode

Priority

-p prints a specific priority level of logs.
Specified with below numbers or strings.

  • emerg(0)
  • alert(1)
  • crit(2)
  • err(3)
  • warning(4)
  • notice(5)
  • info(6)
  • debug(7)

% sudo journalctl -p err

-- Logs begin at Sat 2019-04-13 14:05:21 JST, end at Sat 2019-04-13 14:48:50 JST. --
Apr 13 14:05:31 Vag2 systemd[1]: Failed to start Crash recovery kernel arming.
Enter fullscreen mode Exit fullscreen mode

Priorities can be multiple.

% sudo journalctl -p err -p warning

-- Logs begin at Sat 2019-04-13 14:05:21 JST, end at Sat 2019-04-13 14:49:58 JST. --
<ellipsis>
Apr 13 05:05:29 Vag2 kernel: 00:00:00.000268 main     Executable: /opt/VBoxGuestAdditions-5.2.18/sbin/VBoxService
                             00:00:00.000268 main     Process ID: 906
                             00:00:00.000269 main     Package type: LINUX_64BITS_GENERIC
Apr 13 05:05:29 Vag2 kernel: 00:00:00.003419 main     5.2.18 r124319 started. Verbose level = 0
Apr 13 14:05:31 Vag2 chronyd[647]: Forward time jump detected!
Apr 13 14:05:31 Vag2 systemd[1]: Failed to start Crash recovery kernel arming.
Apr 13 14:05:31 Vag2 systemd[1]: kdump.service failed.
Apr 13 14:05:41 Vag2 kernel: 00:00:10.017183 timesync vgsvcTimeSyncWorker: Radical guest time change: 32 411 887 294 000ns (GuestNow=1 555 131 941 050 133 000 ns GuestLast=1
Enter fullscreen mode Exit fullscreen mode

Other than filtering

There are some options which change how to print.

Reverse order

-r prints newer logs first.

% sudo journalctl -r

-- Logs begin at Sat 2019-04-13 14:05:21 JST, end at Sat 2019-04-13 14:56:44 JST. --
Apr 13 14:56:44 Vag2 sudo[2609]:  vagrant : TTY=pts/0 ; PWD=/home/vagrant ; USER=root ; COMMAND=/bin/journalctl -r
Apr 13 14:56:24 Vag2 sudo[2605]:  vagrant : TTY=pts/0 ; PWD=/home/vagrant ; USER=root ; COMMAND=/bin/journalctl -r
Apr 13 14:50:01 Vag2 systemd[1]: Stopping User Slice of root.
Apr 13 14:50:01 Vag2 systemd[1]: Removed slice User Slice of root.
Apr 13 14:50:01 Vag2 CROND[2596]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Enter fullscreen mode Exit fullscreen mode

Follow

-f keep printing new entries while using it. Like tail -f command.

% sudo journalctl -f

-- Logs begin at Sat 2019-04-13 14:05:21 JST. --
Apr 13 14:50:01 Vag2 systemd[1]: Created slice User Slice of root.
Apr 13 14:50:01 Vag2 systemd[1]: Starting User Slice of root.
Apr 13 14:50:01 Vag2 systemd[1]: Started Session 8 of user root.
Apr 13 14:50:01 Vag2 systemd[1]: Starting Session 8 of user root.
Apr 13 14:50:01 Vag2 CROND[2596]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 13 14:50:01 Vag2 systemd[1]: Removed slice User Slice of root.
Apr 13 14:50:01 Vag2 systemd[1]: Stopping User Slice of root.
Apr 13 14:56:24 Vag2 sudo[2605]:  vagrant : TTY=pts/0 ; PWD=/home/vagrant ; USER=root ; COMMAND=/bin/journalctl -r
Enter fullscreen mode Exit fullscreen mode

Non pager

-no-pager prints logs without pager.

% sudo journalctl --no-pager

-- Logs begin at Sat 2019-04-13 14:05:21 JST, end at Sat 2019-04-13 15:02:30 JST. --
Apr 13 14:05:21 localhost.localdomain systemd-journal[84]: Runtime journal is using 6.1M (max allowed 49.5M, trying to leave 74.3M free of 489.6M available → current limit 49.5M).
Apr 13 14:05:21 localhost.localdomain kernel: Initializing cgroup subsys cpuset
Apr 13 14:05:21 localhost.localdomain kernel: Initializing cgroup subsys cpu
<ellipsis>
Apr 13 15:01:01 Vag2 anacron[2654]: Will run job `cron.daily' in 30 min.
Apr 13 15:01:01 Vag2 anacron[2654]: Will run job `cron.weekly' in 50 min.
Apr 13 15:01:01 Vag2 anacron[2654]: Jobs will be executed sequentially
Apr 13 15:02:30 Vag2 sudo[2661]:  vagrant : TTY=pts/0 ; PWD=/home/vagrant ; USER=root ; COMMAND=/bin/journalctl --no-pager
Enter fullscreen mode Exit fullscreen mode

Conclusion

I think if you are familiar with these options, journalctl should be very convenient for you.
I know some people don't like it but I personally like it.

Top comments (2)

Collapse
 
ferricoxide profile image
Thomas H Jones II

Oof:

_CentOS 7.5.1804 _

Damn, son: 7.7 is rolling up on three months old. Time to upgrade. Gives you access to some other nifty features and rebases some software to less-buggy versions.

=)

Collapse
 
koh_sh profile image
koh-sh

Thanks for the heads up.
I will try 7.7, or maybe CentOS8.