Problem:
As a developer, you need to efficiently search through log files to find specific API requests or errors, but manually scanning through the logs is time-consuming and error-prone.
Solution:
Utilize the grep command in the terminal to search for API requests within log files.
Examples are mostly based on API logs but can be used with any file.
If you want to go throw all commands live - Git Clone Terminal/Grep
#Basics of Grep 📝
The basic syntax for grep is:
grep "pattern" file
where:
pattern is the regular expression you want to search for
file is the name of the file you want to search
Grep works on all Unix-like systems.
Grep will print each line in the file that matches the regular expression.
By default, grep is case-sensitive, so "gnu" is different from "GNU" or "Gnu." You can make it ignore capitalization with the --ignore-case option.
1. Search for a pattern within a file
# grep "search pattern" path/to/file
grep "error" api_server.log
2. Search stdin for lines that do match a pattern
Many times we want to pipe Grep with another command.
# cat path/to/file | grep "search_pattern"
cat api_server.log | grep error
3. Search pattern in multiple files in the current directory with .txt extension
cd logs
# grep "search pattern" *.txt
grep ERROR *.txt
Most Important Flags 🚩
-i
Grep default is case-sensitive. Use this flag to make it search case-insensitive.
grep error api_server.log -i
-v
Invert the match, print all lines where the pattern does not match.
grep INFO api_server.log -v
-w
Search for the whole word. Sometimes there is a relative pattern match but we want an exact word. In that case, this flag is useful.
grep INFO api_server.log -w
-n
Show line numbers along with matching lines.
grep POST api_server.log -n
-l
Find file names that match the pattern.
#grep "pattern" *.ext -l
grep ERROR *.txt -l
-R
If you only know the folder name and it contains subdirectories, you must retrieve all file names and then search recursively within the directories.
grep ERROR -l -R
-o
Only print the matching part of the line (not the whole line)
grep "Internal Server Error" api_server.log -o
-c
Let's say you have one deprecated API now you want to track how many users still use it throw logs. This flag will return the count.
grep "/api/v1/deprecated" api_server.log -c
# In multiple files
grep "/api/v1/deprecated" ./logs/*.txt -c
-E
Interpret the pattern as an extended regular expression.
grep -E "user_id=[0-9]{4}" api_server.log
Line Context Search 🔍
-A: (Lines Above)
To display the line containing the error and the line directly preceding it, you can use -A 1:
Example:
-B: (Lines Below)
Continuing from the previous example, to display the line containing the error and the line directly following it, you can use -B 1:
Example:
-C: (Lines Containing)
To display the line containing the error and the lines directly above and below it, you can use -C 1:
Example:
# Real Life Examples 💡
If you're not familiar with REGEX, I'll explain it next.
Codebase Exploration:
I know we have a vs-code search. But searching through the terminal creates a great impression 😎
grep -r "getUserById" ./
Parsing and Extracting Information
grep -o -E "User: (\w+) performed action: (\w+)" user_log.log
This command uses a regular expression to capture user names and their corresponding actions.
User: Alice performed action: login
User: Bob performed action: view_profile
User: Alice performed action: post_comment
User: Charlie performed action: login
User: Alice performed action: view_profile
User: Bob performed action: post_comment
Pipe with another command to extract data
docker ps | grep -oE '^[0-9a-f]+'
This will output a list of container IDs for all running Docker containers.
f9e5f041b25a
2ab9d3fc5f8e
# Advance REGEX Search 🧠
Search for any four consecutive digits in api_server.log
grep -E "user_id=[0-9]{4}" api_server.log
Matching Words Starting with 'A' or 'B':
grep -E '\b[A-Ba-b]\w+\b' api_server.log
Match either/or
grep '400\|500' api_server.log|
Bonus Tip ✨
If you have big files:
Ripgrep is much faster than grep.
# Install
sudo apt-get install ripgrep
#or
brew install ripgrep
# Syntax
rg <search_pattern> <filename>
Blog Link
https://www.priya.today/blogs/grep
Conclusion
In conclusion, grep is a powerful tool that enables users to search, filter, and manipulate text data efficiently from the command line. Mastering grep can significantly enhance productivity and streamline text processing tasks in the terminal environment.
Happy Coding 👩💻
Top comments (0)