My Favorite Linux Commands: My Silent Heroes in the Console

For twenty years, I've been immersed in these systems, at the console, staring at that black screen, solving problems and building things. During this process, I've used hundreds of commands; some just did their job and moved on, while others became true lifesavers for me. When I say "My Favorite Linux Commands," what comes to mind are not just simple tools, but also symbols of past difficult moments, sleepless nights, and ultimately, victories.

In this post, instead of just presenting a list of commands, I'll explain what these console heroes mean to me and why I love them so much. I definitely have a memory, a rescue operation, associated with each one. For me, a favorite command isn't the one I use most often, but the one that was there for me in a crisis and made me say, "that's it!"

My Wizards for Text Processing: awk and sed

When working in a production ERP, sometimes you encounter log files that are like searching for a needle in a haystack of terabytes of data. Even worse, you might have to process data sets that come in a specific format but contain inconsistencies. It's precisely in these moments that awk and sed transform into wizards for me.

Once, during a supply chain integration, a thousands-line CSV file kept arriving with an incorrect format for a specific product code. It wasn't possible to fix it from the source system, and doing it manually would take days. I remember using sed to correct all the wrong formats in that file in seconds, and then awk to extract only the fields I wanted and create a new output. At that moment, the time and effort these two commands saved me were invaluable.

# Change all occurrences of a specific pattern in a file
sed -i 's/WRONG_CODE_FORMAT/CORRECT_CODE_FORMAT/g' input.csv

# Extract only the 1st and 3rd columns from a CSV file
awk -F',' '{print $1 "," $3}' input.csv > output.csv

These commands are more than just text processing tools; they are the cornerstones of data manipulation and automation for me. Transforming complex datasets into meaningful information in seconds incredibly speeds up my problem-solving process.

Uncovering the Unseen: strace and lsof

As a system administrator, my least favorite thing is saying "it's not working." Because everything has a reason, and finding that reason is my job. This is where commands like strace and lsof come into play. The fastest way to understand why an application isn't starting, why it can't open a specific file, or why it can't establish a network connection is to trace system calls.

In a client project, a Java-based application was unexpectedly throwing a "Permission Denied" error. I checked file permissions and user rights multiple times but couldn't find the problem. Finally, I ran the application with strace and saw that it was actually trying to write the log file to a different directory than intended, and indeed, it didn't have write permissions in that directory. This was just one of the times strace solved an hour-long headache for me in 5 minutes. lsof is an indispensable friend for understanding why a port is busy or which application is holding a file open.

💡 Golden Rule of Debugging

If you can't find the answer to "why" in a problem, look at low-level tools like strace or lsof. Sometimes the problem is hidden in a much more fundamental layer than you think.

# Trace system calls of an application
strace -f -o output.log /path/to/my_application

# See processes using a port or file
lsof -i :8080
lsof /var/log/syslog

These commands allow me to take an X-ray of the system's inner workings. They clearly reveal problems in unseen layers and illuminate the path to a solution.

Understanding System Behavior: journalctl and systemd-analyze

In modern Linux systems, systemd is an indispensable part of our lives. I can't take a single step without journalctl and systemd-analyze to understand service status, the boot process, and system events. These commands are critically important, especially when setting journald rate limits or trying to understand why a particular service keeps restarting.

On the backend of my own side product, a service would shut down periodically without any errors. When I meticulously examined the service's logs with journalctl, I actually encountered an OOM-killed message. The service was silently terminated due to insufficient memory. Analyzing boot process delays or checking cgroup limits using systemd-analyze are also among my routine tasks. This makes it much easier to get to the root of performance issues.

# Follow logs of a specific service
journalctl -u my_service.service -f

# List the slowest systemd units to start
systemd-analyze blame

These two commands are powerful diagnostic tools that help me find answers to "why did this happen?" questions in the system. They allow me to monitor the pulse of my system and detect potential problems early.

My Eyes and Ears in the Network Layer: tcpdump and ss

Network problems can sometimes be among the most stubborn issues. In situations where packets are lost, connections are inexplicably dropped, or performance falls below expectations, tcpdump and ss are indispensable for me. I once experienced a scenario at a company exit where, when using 3 different ISPs, voice packets were breaking up because DSCP marking was not done correctly. By monitoring the real-time packet flow with tcpdump, I was able to identify which packets were incorrectly marked and where they were dropped.

ss (socket statistics), as a modern and more powerful alternative to netstat, allows me to quickly see active connections, listening ports, and network statistics. When there's an unexpected connection surge on a server or an application isn't managing its connection pool correctly, ss provides me with an instant visual diagnosis.

# Monitor HTTP traffic passing through a specific interface
sudo tcpdump -i eth0 'port 80 and host 192.168.1.1'

# See all TCP connections and their states
ss -tuna

These commands allow me to delve into the depths of network traffic and transform abstract network problems into concrete data. They help me work like a detective in the unseen network layer.

The Secret to My Productivity: tmux and htop

Finally, perhaps the simplest but most impactful commands for my daily workflow: tmux and htop. tmux allows me to manage multiple terminal sessions in a single window, and even resume my sessions exactly where I left off after closing and reopening them. When I start a long-running process on a server, knowing that the process will continue to run even if my SSH connection drops is a huge relief.

htop, on the other hand, is a much more user-friendly and visual alternative to the classic top command. It's constantly at my fingertips for monitoring server resource usage, processes, and memory status in real-time. Especially when I see an abnormal increase in an application's CPU or memory consumption, htop is the first tool to give me a clue.

# Start a new tmux session
tmux new -s my_session

# Monitor system resources with htop
htop

These commands, rather than directly solving a problem, are tools that make my problem-solving and system administration process more efficient and expand my comfort zone.

Conclusion: Commands Are Just the Beginning

For me, the list of "My Favorite Linux Commands" is not just a technical preference, but also a reflection of 20 years of experience, countless events, and a continuous learning process. Each command has taught me something, helped me solve a problem, or made my job easier. These silent heroes in the console offer me not just tools, but also a philosophy of problem-solving.

So, what are your favorite Linux commands? Which commands rescued you in the toughest moments or completely changed your workflow? I'd love for you to share them with me in the comments.