DEV Community

Kamal Mustafa
Kamal Mustafa

Posted on

Quick sed lesson

I was looking for a way to "grep" logs within certain time range. The first one I found (and use all the time since then) is using awk:-

awk '/run_cmd/ && "00:00:00" < $3 && $3 <= "04:35:00"' /var/log/local1
Enter fullscreen mode Exit fullscreen mode

But what if I want just logs from 5 minutes ago? From certain time to now is easy:-

awk '/ERROR/ && "10:40:00" < $3 && $3 <= strftime("%H:%M:%s")' /var/log/local1
Enter fullscreen mode Exit fullscreen mode

But I can't find a way to get "time ago" with awk. Of course we can use date command to generate the date and pass it to awk:-

time_ago=$(date --date='10 minutes ago' '+%H:%M') && awk -v time_ago="$time_ago" '/ERROR/ && time_ago < $3 && $3 <= strftime("%H:%M:%s")' /var/log/local1
Enter fullscreen mode Exit fullscreen mode

That starting to get out of hand however. As usual, stackoverflow has better answer. Not using awk but sed.

sed -n "/^$(date --date='5 minutes ago' '+%b %_d %H:%M')/,\$p" /var/log/local1
Enter fullscreen mode Exit fullscreen mode

While it work, I don't really understand it. Time for some quick sed lesson.

Basic sed - sed -n 10,15p file.txt print line 10 - 15 of the file.

10,15 is an address range. Address range can be a line number or regex pattern.

sed -n /^813/p file.txt this function similar to grep ^813 file.txt.

Oh, p there is a command, which is print. Another command usually use is d for delete.

Remember that address can be a range?

sed -n /^813/, \$p file.txt
Enter fullscreen mode Exit fullscreen mode

Here we specify a range, which simply the end of the file. So sed will print starting from the match to the end of the file.

sed -n "/^$(date --date='5 minutes ago' '+%b %_d %H:%M')/,\$p" /var/log/local1
Enter fullscreen mode Exit fullscreen mode

Now back to the original command. date --date='5 minutes ago' '+%b %_d %H:%M' will return something like Oct 12 10:14.

So basically we're doing:-

sed -n "/^Oct 12 10:14/,\$p" /var/log/local1
Enter fullscreen mode Exit fullscreen mode

Which mean print from match of Oct 12 10:14 to the end of file.

Oh wait, why the -n flag there? sed by default will print each line. So -n is for quiet so we instruct it to not print anything. Instead we use p to print only the match.

End of lesson ;)

Discussion (5)

Collapse
yonootz321 profile image
yonootz321

What if there's no log with that specific time?

Collapse
ferricoxide profile image
Thomas H Jones II

You can make the time less and less unique by eliminating the further-right time-fields. That said, once you start getting into the range where your granularity is hours, maybe you don't need to be cutting down your logs all that much, any way?

Collapse
k4ml profile image
Kamal Mustafa Author

Good question. Obviously I made assumption on the time format, as that all logs I have. What kind of format do you have?

Collapse
tux0r profile image
tux0r

And after you find your way through sed, nothing is stopping you from adapting ed as your daily text editor.

Collapse
letsfindcourse profile image
letsfindcourse

This tutorial means that this is a brief introductory guide to SED that will help give beginners a solid foundation about concrete tasks.
letsfindcourse.com/tutorials/sed-t...