GREP - A Practical Guide 🚀

Problem:
As a developer, you need to efficiently search through log files to find specific API requests or errors, but manually scanning through the logs is time-consuming and error-prone.

Solution:
Utilize the grep command in the terminal to search for API requests within log files.

Examples are mostly based on API logs but can be used with any file.
If you want to go throw all commands live - Git Clone Terminal/Grep

#Basics of Grep 📝

The basic syntax for grep is:

grep "pattern" file 

where:
pattern is the regular expression you want to search for
file is the name of the file you want to search

Grep works on all Unix-like systems.

Grep will print each line in the file that matches the regular expression.

By default, grep is case-sensitive, so "gnu" is different from "GNU" or "Gnu." You can make it ignore capitalization with the --ignore-case option.

1. Search for a pattern within a file

# grep "search pattern" path/to/file
grep "error" api_server.log

2. Search stdin for lines that do match a pattern

Many times we want to pipe Grep with another command.

# cat path/to/file | grep "search_pattern"
cat api_server.log | grep error

3. Search pattern in multiple files in the current directory with .txt extension

cd logs
# grep "search pattern" *.txt
grep ERROR *.txt

Most Important Flags 🚩

-i

Grep default is case-sensitive. Use this flag to make it search case-insensitive.

grep error api_server.log -i

-v

Invert the match, print all lines where the pattern does not match.

grep INFO api_server.log -v

-w

Search for the whole word. Sometimes there is a relative pattern match but we want an exact word. In that case, this flag is useful.

grep INFO api_server.log -w

-n

Show line numbers along with matching lines.

grep POST api_server.log -n

-l

Find file names that match the pattern.

#grep "pattern" *.ext -l 
grep ERROR *.txt -l

-R

If you only know the folder name and it contains subdirectories, you must retrieve all file names and then search recursively within the directories.

grep ERROR -l -R

-o

Only print the matching part of the line (not the whole line)

grep "Internal Server Error" api_server.log -o

-c

Let's say you have one deprecated API now you want to track how many users still use it throw logs. This flag will return the count.

grep "/api/v1/deprecated" api_server.log -c
# In multiple files
grep "/api/v1/deprecated" ./logs/*.txt -c

-E

Interpret the pattern as an extended regular expression.

grep -E "user_id=[0-9]{4}" api_server.log

Line Context Search 🔍

-A: (Lines Above)

To display the line containing the error and the line directly preceding it, you can use -A 1:
Example:

-B: (Lines Below)

Continuing from the previous example, to display the line containing the error and the line directly following it, you can use -B 1:
Example:

-C: (Lines Containing)

To display the line containing the error and the lines directly above and below it, you can use -C 1:
Example:

# Real Life Examples 💡

If you're not familiar with REGEX, I'll explain it next.

Codebase Exploration:

I know we have a vs-code search. But searching through the terminal creates a great impression 😎

grep -r "getUserById" ./

Parsing and Extracting Information

grep -o -E "User: (\w+) performed action: (\w+)" user_log.log

This command uses a regular expression to capture user names and their corresponding actions.

User: Alice performed action: login
User: Bob performed action: view_profile
User: Alice performed action: post_comment
User: Charlie performed action: login
User: Alice performed action: view_profile
User: Bob performed action: post_comment

Pipe with another command to extract data

docker ps | grep -oE '^[0-9a-f]+'

This will output a list of container IDs for all running Docker containers.

f9e5f041b25a
2ab9d3fc5f8e

# Advance REGEX Search 🧠

Search for any four consecutive digits in api_server.log

grep -E "user_id=[0-9]{4}" api_server.log

Matching Words Starting with 'A' or 'B':

grep -E '\b[A-Ba-b]\w+\b' api_server.log

Match either/or

grep '400\|500' api_server.log|

Bonus Tip ✨

If you have big files:
Ripgrep is much faster than grep.

# Install
sudo apt-get install ripgrep
#or
brew install ripgrep
# Syntax 
rg <search_pattern> <filename>

Blog Link

https://www.priya.today/blogs/grep

Conclusion

In conclusion, grep is a powerful tool that enables users to search, filter, and manipulate text data efficiently from the command line. Mastering grep can significantly enhance productivity and streamline text processing tasks in the terminal environment.

Happy Coding 👩‍💻