Quick sed lesson

k4ml profile image Kamal Mustafa ・2 min read

I was looking for a way to "grep" logs within certain time range. The first one I found (and use all the time since then) is using awk:-

awk '/run_cmd/ && "00:00:00" < $3 && $3 <= "04:35:00"' /var/log/local1

But what if I want just logs from 5 minutes ago? From certain time to now is easy:-

awk '/ERROR/ && "10:40:00" < $3 && $3 <= strftime("%H:%M:%s")' /var/log/local1

But I can't find a way to get "time ago" with awk. Of course we can use date command to generate the date and pass it to awk:-

time_ago=$(date --date='10 minutes ago' '+%H:%M') && awk -v time_ago="$time_ago" '/ERROR/ && time_ago < $3 && $3 <= strftime("%H:%M:%s")' /var/log/local1

That starting to get out of hand however. As usual, stackoverflow has better answer. Not using awk but sed.

sed -n "/^$(date --date='5 minutes ago' '+%b %_d %H:%M')/,\$p" /var/log/local1

While it work, I don't really understand it. Time for some quick sed lesson.

Basic sed - sed -n 10,15p file.txt print line 10 - 15 of the file.

10,15 is an address range. Address range can be a line number or regex pattern.

sed -n /^813/p file.txt this function similar to grep ^813 file.txt.

Oh, p there is a command, which is print. Another command usually use is d for delete.

Remember that address can be a range?

sed -n /^813/, \$p file.txt

Here we specify a range, which simply the end of the file. So sed will print starting from the match to the end of the file.

sed -n "/^$(date --date='5 minutes ago' '+%b %_d %H:%M')/,\$p" /var/log/local1

Now back to the original command. date --date='5 minutes ago' '+%b %_d %H:%M' will return something like Oct 12 10:14.

So basically we're doing:-

sed -n "/^Oct 12 10:14/,\$p" /var/log/local1

Which mean print from match of Oct 12 10:14 to the end of file.

Oh wait, why the -n flag there? sed by default will print each line. So -n is for quiet so we instruct it to not print anything. Instead we use p to print only the match.

End of lesson ;)

Posted on by:

k4ml profile

Kamal Mustafa


Python/Django Developer at xoxzo.com.


Editor guide

What if there's no log with that specific time?


You can make the time less and less unique by eliminating the further-right time-fields. That said, once you start getting into the range where your granularity is hours, maybe you don't need to be cutting down your logs all that much, any way?


Good question. Obviously I made assumption on the time format, as that all logs I have. What kind of format do you have?


And after you find your way through sed, nothing is stopping you from adapting ed as your daily text editor.


This tutorial means that this is a brief introductory guide to SED that will help give beginners a solid foundation about concrete tasks.