DEV Community: NJEI

The Bar has Always Been Moving. Are you?

NJEI — Sat, 28 Feb 2026 00:21:03 +0000

Today's Wizardry Is Tomorrow's Bare Minimum

And Africa is running out of time to not care.

There is a story — the kind that gets passed around in African classrooms — about a colonial-era quack doctor who sold bottled cures to rural crowds.

His secret weapon wasn't the concoction. It was the label.

Printed paper. English words. A logo, if you could call it that.

The author who told this story noted, with quiet humor, that in that era, if you could print your claims, you were generally considered truthful. Literacy was new. The printed word carried the full weight of authority. The bottle didn't need to work. The label just needed to exist.

I think about that story every time I watch a new technology go from miracle to minimum.

The Ladder We Keep Forgetting We're Climbing

Every generation has its wizardry. Every generation eventually forgets it was ever wizardry at all.

In the early 2000s across much of Africa, having an email address made you someone. Not just anyone — someone. A Yahoo address was a kind of credential. It said: I am connected to the world beyond this street.

By 2005, having a website meant you were serious. Businesses printed their URLs on flyers like they were printing certificates.

By 2010, a Facebook page could legitimize an entire company in the eyes of its customers.

By 2015, if your e-commerce store accepted card payments — genuinely processed a Visa transaction — you were operating at a level most of your competitors couldn't touch.

Each of these things, in its moment, was the difference between being taken seriously and being invisible.

Each of them is now table stakes.

This is not a uniquely African story, but it hits differently here — because the gap between when the world adopts something and when we adopt it has historically cost us. Every cycle we miss is compounding interest on a debt we didn't choose but keep paying.

The Cycle Is Happening Again. Right Now.

Here is what is currently being normalized in the rooms you are not yet in:

Teams are training lightweight, fine-tuned models on their own proprietary data — not because they have PhD researchers, but because the tooling now makes it accessible to people who are simply paying attention.

Businesses are building internal AI-powered tools that automate entire workflow layers — not SaaS products, not VC-backed startups, just quiet internal infrastructure that makes ten people do the work of fifty.

Developers who understand what a dataset is — not theoretically, but practically, in terms of what it enables — are making decisions that others don't even know are available to them.

The raw ability to open ChatGPT and ask it a question? That's the Yahoo email address of 2025. Useful, yes. Impressive to exactly no one who matters.

What the Moment Actually Demands

Let me be precise, because vagueness is comfortable and comfort is the enemy of timing.

The floor right now — the floor — is:

1. Understanding what LLMs can be trained to do, specifically.
Not generally. Not "AI can do a lot of things." Specifically: what does it mean to fine-tune a model on a dataset? What changes? What doesn't? You don't need to write the code from scratch. You need to understand the concept well enough to direct someone who can, or to use the tools that now make it possible without a machine learning background.

2. Knowing what a dataset is and how data quality shapes model behavior.
Garbage in, garbage out is a cliché because it's accurate. The ability to look at a business problem and ask "what data do we have, and is it enough?" is fast becoming a foundational professional skill — not just for data scientists, but for product managers, founders, consultants, writers building tools.

3. Building something. Anything.
A small internal tool. A prompt pipeline with logic. An AI-assisted workflow that saves your team hours a week. The era of watching and waiting to see if AI is real is over. It was over eighteen months ago. The question now is whether you're building or being built around.

A Word for African Technologists Specifically

The geopolitical battle for AI dominance — data centers, foundational models, compute infrastructure — that's being fought between the US and China, with Europe occasionally raising its hand to ask about privacy.

That battle is not ours to fight right now. And that's okay.

But the application layer? The last mile? The translation of these tools into products and workflows that serve African markets, African languages, African business contexts?

Nobody is coming to do that for us. And nobody else should.

The window in which we can enter this layer as builders rather than consumers is not permanently open. It is open now. It has been open. It will not always be.

The bottled cure in the colonial story worked because people didn't know enough to ask questions. The audit we owe ourselves — as a continent, as a tech community — is whether we're still letting others print the labels on things we should be producing ourselves.

The Uncomfortable Close

The printed label was once enough.

Then the website.

Then the social media presence.

Then the card payment integration.

Now? Now you need to be able to look at a business problem and build an AI-shaped solution for it. Not describe one. Not tweet about one. Build one.

Tomorrow, that will be the bare minimum too.

The question is never whether the bar will rise.

The question is whether you're rising with it — or realizing, five years from now, that you were standing at the bottom, watching, waiting for someone to explain to you that the ladder had already moved.

If this made you feel something uncomfortable, that's the right reaction. Sit with it, then do something about it.

File Management Beyond Copy-Paste: Compressing, Splitting, and Why tar.gz Isn't Scary

NJEI — Thu, 18 Dec 2025 16:53:36 +0000

File Management Beyond Copy-Paste: Compressing, Splitting, and Why tar.gz Isn't Scary

The Problem: Moving and Managing Large Files

You need to transfer a project with 1,000 files to another server. Do you copy them one by one? That's slow and error-prone.

Or you have a 10GB log file. You can't email it. Can't open it easily. Need to break it into smaller chunks.

Or you download project.tar.gz and have no idea how to open it.

Linux has tools for all of this. Once you understand them, managing files becomes simple.

Comparing Files: diff and cmp

Before we dive into compression, let's cover comparing files.

diff: Line-by-Line Comparison

diff file1.txt file2.txt

Shows which lines are different:

2c2
< This is the old line
---
> This is the new line

Useful for:

Comparing config files
Checking what changed between versions
Reviewing code changes

cmp: Byte-by-Byte Comparison

cmp file1.txt file2.txt

Stops at the first difference:

file1.txt file2.txt differ: byte 45, line 3

Useful for:

Binary files
Quick check if files are identical
Finding corruption

If files are identical, no output is shown.

Understanding tar: The Tape Archive

tar (tape archive) bundles multiple files into one container. It doesn't compress—just packages.

Creating a tar Archive

tar cvf filename.tar /path/to/directory/

Breaking it down:

c - Create archive
v - Verbose (show progress)
f - File (specify filename)

Example:

# Archive entire project directory
tar cvf project.tar /home/user/project/

# Archive multiple directories
tar cvf backup.tar /etc/ /var/log/

Extracting a tar Archive

tar xvf filename.tar

x - Extract
v - Verbose
f - File

Example:

tar xvf project.tar
# Extracts all files to current directory

Important: tar Alone Doesn't Compress

# This creates 1000MB tar from 1000MB of files
tar cvf files.tar large_directory/

The archive is the same size as the original files. To actually compress, we need gzip.

gzip: Actual Compression

gzip compresses files to save space.

Compress a File

gzip filename.tar
# Creates: filename.tar.gz
# Original filename.tar is deleted

The One-Step tar.gz

Most commonly, you create and compress in one command:

tar cvzf filename.tar.gz /path/to/directory/

Added flag:

z - Compress with gzip

Example:

# Create compressed archive
tar cvzf project.tar.gz /home/user/project/

# Result: One compressed file containing everything

Decompress gzip Files

Two ways:

# Method 1: gzip -d
gzip -d filename.tar.gz
# Creates: filename.tar (decompressed)

# Method 2: gunzip (same thing)
gunzip filename.tar.gz
# Creates: filename.tar

Extract tar.gz in One Step

tar xvzf filename.tar.gz

x - Extract
v - Verbose
z - Decompress with gzip
f - File

Example:

tar xvzf project.tar.gz
# Decompresses and extracts everything

Understanding tar.gz

When you see filename.tar.gz:

It's a tar archive (many files bundled)
Compressed with gzip (smaller size)

Think of it like a zip file in Windows.

Real-World Example: Backing Up a Website

# Create compressed backup
tar cvzf website-backup-2024-12-09.tar.gz /var/www/html/

# Check size
ls -lh website-backup-2024-12-09.tar.gz
# Output: -rw-r--r-- 1 user user 45M Dec 09 10:00 website-backup-2024-12-09.tar.gz

# Later, restore it
tar xvzf website-backup-2024-12-09.tar.gz

Truncate: Shrink or Extend File Size

truncate changes file size. Warning: This loses data.

Basic Usage

truncate -s 10 filename

Sets file size to 10 bytes. Everything beyond is deleted.

Common Uses

# Shrink file to 1MB
truncate -s 1M largefile.log

# Shrink file to 10 bytes
truncate -s 10 file.txt

# Empty a file (set to 0 bytes)
truncate -s 0 file.txt

Real Example: Clear Log File

# Log file is 5GB, causing disk issues
ls -lh /var/log/app.log
# -rw-r--r-- 1 root root 5.0G Dec 09 10:00 app.log

# Empty it
sudo truncate -s 0 /var/log/app.log

# Verify
ls -lh /var/log/app.log
# -rw-r--r-- 1 root root 0 Dec 09 10:01 app.log

Note: Truncate doesn't delete the file, just its contents.

Combining Files: cat

cat (concatenate) combines multiple files into one.

Basic Syntax

cat file1 file2 file3 > combined.txt

This appends file1, file2, and file3 into combined.txt.

Example: Combine Logs

# Multiple log files from different days
cat app-2024-12-01.log app-2024-12-02.log app-2024-12-03.log > december.log

# Verify
wc -l december.log
# Shows total lines from all three files

Example: Combine Parts

# Downloaded file in parts
cat download.part1 download.part2 download.part3 > complete-file.zip

Splitting Files: split

split breaks large files into smaller chunks.

Split by Lines

split -l 300 file.txt childfile

-l 300 - Split every 300 lines
file.txt - Input file
childfile - Prefix for output files

Output:

childfileaa  (first 300 lines)
childfileab  (next 300 lines)
childfileac  (next 300 lines)
...

Split by Size

# Split into 10MB chunks
split -b 10M largefile.log part_

# Output: part_aa, part_ab, part_ac, ...

Real Example: Split Large Database Dump

# Database dump is 5GB
ls -lh database.sql
# -rw-r--r-- 1 user user 5.0G Dec 09 10:00 database.sql

# Split into 100MB chunks for easier transfer
split -b 100M database.sql db_part_

# Results:
ls -lh db_part_*
# db_part_aa (100MB)
# db_part_ab (100MB)
# db_part_ac (100MB)
# ...

# Later, recombine on destination server
cat db_part_* > database.sql

Real-World Scenarios

Scenario 1: Backup and Transfer

# Create compressed backup
tar cvzf backup.tar.gz /home/user/documents/

# Transfer to remote server
scp backup.tar.gz user@remote-server:/backups/

# On remote server, extract
ssh user@remote-server
cd /backups
tar xvzf backup.tar.gz

Scenario 2: Send Large File via Email

Email has size limits. Split the file:

# Split into 10MB chunks
split -b 10M presentation.pdf part_

# Email part_aa, part_ab, part_ac separately

# Recipient combines them
cat part_* > presentation.pdf

Scenario 3: Archive Old Logs

# Logs taking up space
du -sh /var/log/app/
# 10G    /var/log/app/

# Archive and compress old logs
tar cvzf old-logs-2024-11.tar.gz /var/log/app/2024-11-*.log

# Verify compression
ls -lh old-logs-2024-11.tar.gz
# -rw-r--r-- 1 user user 500M Dec 09 10:00 old-logs-2024-11.tar.gz
# (Compressed from 2GB to 500MB)

# Delete originals after verifying
rm /var/log/app/2024-11-*.log

Scenario 4: Clear Growing Log Files

# Check disk space
df -h
# Filesystem      Size  Used Avail Use% Mounted on
# /dev/sda1        50G   48G     0  98% /

# Find large log files
du -sh /var/log/* | sort -h | tail -5

# Truncate the largest
sudo truncate -s 0 /var/log/massive-app.log

# Check disk space again
df -h
# Filesystem      Size  Used Avail Use% Mounted on
# /dev/sda1        50G   20G   28G  42% /

Scenario 5: Deploy Application

# Developer creates release
tar cvzf myapp-v1.2.3.tar.gz /opt/myapp/

# Upload to server
scp myapp-v1.2.3.tar.gz user@prod-server:/tmp/

# On production server
ssh user@prod-server
cd /tmp
tar xvzf myapp-v1.2.3.tar.gz -C /opt/
# -C flag extracts to specific directory

Common tar Flags

Creating Archives

tar cvf file.tar dir/       # Create, verbose
tar czf file.tar.gz dir/    # Create, gzip
tar cjf file.tar.bz2 dir/   # Create, bzip2 (better compression)

Extracting Archives

tar xvf file.tar            # Extract, verbose
tar xzf file.tar.gz         # Extract, gzip
tar xjf file.tar.bz2        # Extract, bzip2

Viewing Contents

tar tvf file.tar            # List contents without extracting
tar tzf file.tar.gz         # List contents of gzipped archive

Extract to Specific Directory

tar xvzf file.tar.gz -C /destination/path/

Quick Reference

Comparison

diff file1 file2            # Line-by-line
cmp file1 file2             # Byte-by-byte

tar Operations

# Create
tar cvf file.tar dir/
tar cvzf file.tar.gz dir/   # With gzip

# Extract
tar xvf file.tar
tar xvzf file.tar.gz        # With gzip

# View
tar tvf file.tar

Compression

# Compress
gzip file.tar               # Creates file.tar.gz
tar cvzf file.tar.gz dir/   # Create and compress

# Decompress
gzip -d file.tar.gz         # Creates file.tar
gunzip file.tar.gz          # Same thing
tar xvzf file.tar.gz        # Extract compressed

File Operations

# Truncate
truncate -s 10M file.log    # Shrink to 10MB
truncate -s 0 file.log      # Empty file

# Combine
cat file1 file2 > combined

# Split
split -l 300 file.txt prefix    # By lines
split -b 10M file.log prefix    # By size

Common Mistakes

Mistake #1: Forgetting the 'z' flag

# Wrong - creates tar but doesn't compress
tar cvf file.tar.gz dir/

# Right - actually compresses
tar cvzf file.tar.gz dir/

Mistake #2: Extracting in wrong directory

# Extracts to current directory (messy)
tar xvzf file.tar.gz

# Better - extract to specific location
tar xvzf file.tar.gz -C /target/directory/

Mistake #3: Truncating without backup

# Data is lost forever
truncate -s 0 important.log

# Better - backup first
cp important.log important.log.backup
truncate -s 0 important.log

Mistake #4: Wrong split prefix

# Splits to: childfileaa, childfileab
split -l 300 file.txt childfile

# If you wanted: part_aa, part_ab
split -l 300 file.txt part_
# Note the underscore

Tips for Efficiency

Tip 1: View before extracting

# Check what's inside first
tar tvzf file.tar.gz | less

# Then extract
tar xvzf file.tar.gz

Tip 2: Compress multiple directories

# Backup multiple locations at once
tar cvzf backup.tar.gz /etc/ /var/www/ /home/user/

Tip 3: Exclude files

# Archive but skip certain files
tar cvzf backup.tar.gz --exclude='*.log' --exclude='*.tmp' /home/user/

Tip 4: Use pbzip2 for faster compression

# Much faster on multi-core systems
tar cvf - dir/ | pbzip2 > file.tar.bz2

Key Takeaways

tar bundles files - Doesn't compress by itself
gzip compresses - Makes files smaller
tar.gz = bundled + compressed - Like a zip file
Use 'z' flag - tar cvzf and tar xvzf
truncate loses data - Use carefully
cat combines files - Simple concatenation
split breaks files - By lines or size
Always test with 't' flag - View before extracting

Managing files isn't just about copy-paste. Compression, splitting, and archiving are essential skills for working with servers, backups, and large datasets.

What's your most common tar command? Share your file management workflows in the comments.

The Next Decade of AI and Human Interaction: A Developer's Perspective on What's Really Coming

NJEI — Mon, 15 Dec 2025 23:33:22 +0000

I've been coding for some years. I've seen frameworks rise and fall, languages evolve, and entire paradigms shift. But nothing—nothing—has made me question the future of my profession quite like the past three years of AI development.

Last month, I watched Claude Opus 4 autonomously work on a codebase for nearly seven hours—a full corporate workday. No prompts. No hand-holding. Just... working.

That same week, OpenAI CEO Sam Altman admitted he's worried about "the rate of change that's happening in the world right now", calling it the fastest technology adoption in history, with over 800 million people using ChatGPT weekly.

And then Anthropic CEO Dario Amodei predicted AI could eliminate half of entry-level white-collar jobs within one to five years, while maintaining a 25% chance AI development goes "really badly."

So here we are. Standing at the edge of the 2030s, watching the most powerful minds in tech issue both promises and warnings. What does the next decade actually look like for developers, for knowledge workers, for humans?

Let me share what I've learned—not from breathless hype or dystopian fear, but from the data, the leaders, and the patterns emerging right now.

Where We Are Right Now (December 2025)
The Timeline: What Industry Leaders Are Actually Saying
2025-2027: The Arrival of AI Colleagues
- What This Means for Developers
- The First Wave of Job Changes
2028-2030: The Partnership Era
- Human-AI Symbiosis Emerges
- New Skills, New Roles
- The Economic Reality
2031-2035: The Redefinition
- What Work Becomes
- The Skills That Endure
- The New Value Creation
The Two Paths Ahead
- Path 1: The Optimistic Scenario
- Path 2: The Turbulent Scenario
What This Means for You Right Now
- For Junior Developers
- For Mid-Level Developers
- For Senior Developers and Architects
The Uncomfortable Truths We Need to Accept
The Skills That AI Can't Replace (Yet)
Preparing for 2035: A Practical Roadmap
Why I'm Still Optimistic (But Realistic)
The Questions We Should Be Asking
Final Thoughts: Partnership, Not Replacement

Where We Are Right Now (December 2025)

Let's establish the baseline. As of late 2025, we've reached some remarkable milestones:

Model Capabilities:

GPT-5 achieves 94.6% on advanced mathematics problems, 74.9% on real-world software engineering tasks, and has drastically reduced hallucinations
50% of developers now use AI coding tools daily, with that number rising to 65% in top-performing organizations
Anthropic now commands 40% of enterprise LLM spending, up from 12% in 2023, while OpenAI's share dropped from 50% to 27%

Economic Impact:

ChatGPT reached 700 million weekly active users, up from 500 million in March
OpenAI hit its first $1 billion revenue month in July 2025
AI coding tools generated $4 billion in revenue in 2025, up from $550 million

Real-World Deployment:

Companies report 15%+ development velocity gains
AI agents can now work autonomously for hours
Models can maintain context across entire codebases

This isn't hype. This is happening right now.

The Timeline: What Industry Leaders Are Actually Saying

The CEOs building these systems aren't being coy about timelines. Here's what they're publicly predicting:

Dario Amodei (Anthropic CEO):
Expects the singularity by 2026. Stated that "over the next two or three years, I am relatively confident that we are indeed going to see models that show up in the workplace, that consumers use—that are assistants to humans but that gradually get better than us at almost everything".

Sam Altman (OpenAI CEO):
Wrote "We are now confident we know how to build AGI as we have traditionally understood it". Predicts AGI by 2035, mentioning "a few thousand days" in his blog "The Intelligence Age".

Demis Hassabis (Google DeepMind CEO):
Said AGI is "probably a handful of years away", defining it as systems capable of exhibiting all cognitive capabilities humans possess.

Jensen Huang (Nvidia CEO):
Predicted in March 2024 that within five years, AI would match or surpass human performance on any test—by 2029.

Let that sink in. The people building these systems expect human-level AI within 3-10 years. Not in some distant future. Within the next decade.

2025-2027: The Arrival of AI Colleagues

What This Means for Developers

OpenAI believes 2025 may see the first AI agents "join the workforce" and materially change company output. We're not talking about better autocomplete. We're talking about AI that can:

Understand a feature request
Design an architecture
Implement it across multiple files
Write tests
Deploy it
Monitor for issues

All without human intervention for hours at a time.

Anthropic's Claude Code can already autonomously modify a codebase, and by 2027, junior developer roles face automation by coding agents, with code review becoming instant and comprehensive through AI.

In practice, this means:

By late 2025, a senior developer might say: "Build a user authentication system with OAuth, email verification, and rate limiting" and come back two hours later to working, tested, documented code.

By 2027, entire features—from database schema to frontend components—could be generated and refined by AI agents working in teams.

The First Wave of Job Changes

About $2.9 trillion of economic value could be unlocked in the United States by 2030 if organizations prepare their people and redesign workflows around people, agents, and robots working together.

But here's the uncomfortable part: While 92 million jobs may be displaced by automation, 170 million new jobs will be created, with workers expecting 39% of their existing skill sets to be transformed or become outdated by 2030.

What gets automated first:

Boilerplate code generation
Bug fixing in well-understood domains
Documentation writing
Code review for style and common issues
Test generation
Database query optimization

What remains human:

Architecture decisions for novel systems
Stakeholder communication
Product vision and strategy
Complex debugging across systems
Ethical considerations in design

The gap between junior and senior developer value will widen dramatically. Not because juniors are replaceable, but because the tasks they typically do are.

2028-2030: The Partnership Era

Human-AI Symbiosis Emerges

The concept of "Symbiotic AI" represents a deliberate design philosophy where AI solutions are built specifically around humans, focusing on the collective inference capabilities that emerge when humans and algorithms collaborate effectively.

By 2030, AI systems become adept at anticipating human actions, enhancing the synergy between humans and machines, producing results that surpass what either could achieve independently.

This is where it gets interesting. The best outcomes won't come from AI replacing humans or humans working without AI. They'll come from partnership.

What this looks like in practice:

Scenario 1: Architecture Design

Human: "We need to scale this to 10M users with sub-100ms latency"
AI: Analyzes current architecture, proposes three approaches with tradeoffs
Human: Evaluates business context, picks approach
AI: Implements the solution across the codebase
Human: Reviews critical paths and security implications
AI: Refines based on feedback and deploys

Scenario 2: Complex Debugging

AI: Detects anomaly in production, traces it across 47 services
Human: Recognizes similar pattern from different context
AI: Applies pattern to current problem, generates fix
Human: Validates fix doesn't introduce edge cases
AI: Implements, tests, and monitors deployment

The key shift: AI assists with routine tasks such as scheduling or data entry, and augments our abilities to identify insights, enabling faster, more informed decisions, allowing humans to focus on empathy, creativity, critical thinking and innovation.

New Skills, New Roles

76% of employees believe that AI will create entirely new skills that don't yet exist. Here are the roles emerging by 2030:

AI Orchestrators:
Developers who design and manage AI agent workflows. Think Kubernetes for AI agents—someone needs to architect how they communicate, when they hand off tasks, and how they escalate to humans.

Human-AI Interface Designers:
Creating the interaction patterns between humans and AI systems. Good collaboration between humans and AI depends on clear communication—someone needs to design those communication protocols.

AI Behavior Designers:
Defining what "good" looks like for AI agents in different contexts. Teaching them company-specific practices, domain knowledge, and ethical boundaries.

Prompt Engineers (Evolved):
Skills like prompt engineering, which involves crafting precise inputs to optimize AI outputs, are becoming indispensable. But by 2030, it's less about single prompts and more about designing entire prompt chains and agent behaviors.

The Economic Reality

Collaboration between humans and AI could unlock up to $15.7 trillion in economic value by 2030, with global productivity growth potentially reaching 3.4% by 2030 due to AI and automation.

But who captures this value? As workers collaborate with AI, they apply skills like writing and research in new ways rather than being made obsolete. The economic gains go to those who learn to leverage AI, not those who resist it.

2031-2035: The Redefinition

What Work Becomes

By 2035, Sam Altman suggests graduates "could very well be leaving on a mission to explore the solar system on a spaceship in some completely new, exciting, super well-paid, super interesting job".

Hyperbole? Maybe. But the fundamental point stands: the jobs of 2035 don't exist yet.

What we can predict:

Creative Direction Dominates: AI executes, humans envision. Your value comes from knowing what to build and why, not how to build it.
Context Becomes Currency: While machines are set to deliver unprecedented speed and precision through rapidly advancing processing power, assuming they will render human labor obsolete by 2035 would be a misconception. Humans provide context AI can't: business strategy, user empathy, organizational politics, cultural nuance.
Ethical Oversight Intensifies: As AI systems make more decisions, humans become the guardrails. Someone needs to ask "should we?" not just "can we?"
Continuous Learning is the Job: Continuous learning has become a skill in its own right. The half-life of technical skills shrinks to months. Learning itself becomes the meta-skill.

The Skills That Endure

Low-exposure skills—those in the bottom quartile of automation risk—are likely to endure, often grounded in human connection and care, such as leadership and healthcare skills.

For developers specifically:

High-Value Skills:

System thinking (understanding how components interact)
Stakeholder translation (turning business needs into technical solutions)
Risk assessment (security, privacy, scalability)
Team coordination (aligning people toward goals)
Ethical reasoning (making value-based decisions)

Medium-Value Skills:

Domain expertise (healthcare, finance, etc.)
Performance optimization for unique constraints
Integration architecture across legacy and modern systems
Mentoring and knowledge transfer

Low-Value Skills:

CRUD operations
Boilerplate generation
Standard algorithm implementation
Routine debugging
Documentation writing

Notice a pattern? The closer a skill is to human judgment in context, the more valuable it remains.

The New Value Creation

Here's where it gets exciting. While 92 million jobs may be displaced, 170 million new jobs will be created. What are these new jobs?

Possibilities for 2035:

AI Ecosystem Architects: Designing how hundreds of AI agents work together across an organization
Synthetic Data Engineers: Creating training data for specialized AI models
AI Behavior Auditors: Ensuring AI systems remain aligned with human values
Human Experience Designers: Crafting how humans interact with AI-powered systems
AI-Human Team Managers: Leading hybrid teams of people and agents
Consciousness Interface Developers: Building systems that feel natural to human cognition
Emergence Pattern Analysts: Studying unexpected behaviors in multi-agent systems

Sound sci-fi? So did "Cloud DevOps Engineer" in 1995.

The Two Paths Ahead

The future isn't predetermined. We're choosing between paths right now.

Path 1: The Optimistic Scenario

By 2032, AI systems become adept at anticipating human actions, enhancing the synergy between humans and machines, producing results that surpass what either could achieve independently.

In this future:

Education Adapts: Companies increase investments in AI training programs by 2030 to equip workers with skills to work alongside advanced automation
Regulation Provides Structure: Governance frameworks like the EU AI Act provide fairness, transparency, and accountability
Value is Distributed: New roles emerge faster than old ones disappear
Augmentation Wins: AI amplifies human capabilities rather than replacing them
Creativity Flourishes: Freed from tedious work, humans focus on innovation

Real-world example:
A developer in 2035 wakes up to AI agents having implemented overnight the feature they discussed yesterday. They spend their day reviewing the approach, ensuring it aligns with business strategy, considering user experience implications, and working with stakeholders on the next evolution. Their value lies in direction, not execution.

Path 2: The Turbulent Scenario

In the absence of clear regulatory frameworks, ethical and safety considerations are often neglected as developers rush to market, creating systems that prioritize efficiency over human well-being.

In this future:

Displacement Outpaces Creation: Jobs disappear faster than new ones emerge
Skills Gap Widens: Without coherent strategy for integrating human and machine labor, companies swing between underusing or over-relying on automation, resulting in subpar outcomes
Inequality Deepens: Those who adapt thrive; those who don't struggle
Trust Erodes: AI systems frequently malfunction, misinterpreting data or sending false alarms, with little support or training
Human Skills Atrophy: Over-reliance on AI leads to loss of fundamental capabilities

Real-world example:
A developer in 2035 watches AI agents generate code they don't understand, can't debug, and can't improve. When the AI fails (and it will), they're helpless. Their role becomes babysitting systems they can't control, creating value for neither themselves nor their organization.

What This Means for You Right Now

For Junior Developers

The Hard Truth:
Entry-level positions will shrink. AI could eliminate half of entry-level white-collar jobs within one to five years.

The Opportunity:
Learn to work with AI from day one. Teams report 15%+ velocity gains as they've adopted AI tools across the software development lifecycle. Junior developers who can leverage AI to perform at mid-level capacity will be invaluable.

Action Items:

Master AI-assisted development NOW (GitHub Copilot, Claude Code, Cursor)
Focus on skills AI can't easily replicate: system design, architecture, business context
Build projects that showcase human judgment + AI capability
Learn prompt engineering and AI orchestration
Develop strong communication skills—explaining and defending technical decisions

For Mid-Level Developers

The Hard Truth:
Your current skillset has a shelf life. 39% of existing skill sets will be transformed or become outdated by 2030.

The Opportunity:
You're in the sweet spot. You have enough experience to provide context AI needs, and enough adaptability to learn new paradigms. Nearly all occupations have at least one highly disrupted skill, and a third of occupations will see more than 10% of their skills highly changed.

Action Items:

Transition from "doer" to "director"—focus on what to build and why
Develop expertise in one vertical (healthcare, finance, etc.) where domain knowledge matters
Learn to manage AI agents as you would junior developers
Build skills in system architecture and integration
Start mentoring—teaching is a uniquely human skill

For Senior Developers and Architects

The Hard Truth:
Your experience is valuable, but only if applied to new contexts. Yesterday's architecture patterns might not apply in an AI-first world.

The Opportunity:
You've seen paradigm shifts before. While machines deliver unprecedented speed and precision, they won't render human labor obsolete—humans are better positioned to engage in creative and stimulating work.

Action Items:

Learn to architect systems where AI agents are first-class citizens
Focus on organizational and business strategy—AI won't replace that
Develop expertise in AI safety and ethics—someone needs to be responsible
Mentor the next generation in human-AI collaboration
Position yourself as a "translator" between business needs and AI capabilities

The Uncomfortable Truths We Need to Accept

AI Will Get Better: Exponentially. GPT-5 represents a significant leap in intelligence over all previous models, and we're only getting started.
Some Jobs Will Disappear: Junior developer roles face automation by coding agents. This is happening, not hypothetical.
We Don't Know the Timeline: Predictions for AGI range from 2026 to beyond 2050. Uncertainty is the only certainty.
Adaptation is Mandatory: Continuous learning will be non-negotiable, and reskilling programs and lifelong education should empower workers to adapt.
The Gap Will Widen: Those who embrace AI will pull further ahead of those who resist. The productivity difference will be measured in orders of magnitude.
It's Already Too Late to Ignore: ChatGPT has been adopted by the world faster than any other technology in history. The wave is here.

The Skills That AI Can't Replace (Yet)

What is easy for humans is often difficult for machines, and vice versa—high-level reasoning requires relatively little computation compared to low-level sensorimotor skills.

The Human Advantage:

Contextual Understanding: Knowing why something matters to the business, not just what to build
Emotional Intelligence: Reading team dynamics, stakeholder concerns, user frustrations
Ethical Reasoning: Making value-based decisions in ambiguous situations
Creative Synthesis: Combining ideas from disparate domains in novel ways
Strategic Thinking: Planning years ahead while adapting to daily changes
Relationship Building: Trust, rapport, and collaboration with humans
Cultural Navigation: Understanding organizational politics and human dynamics

Notice these are all soft skills. The irony? As AI handles more technical work, human skills become more valuable.

Preparing for 2035: A Practical Roadmap

2025 (Now):

Start using AI tools daily in your workflow
Experiment with prompt engineering and AI pair programming
Read about AI capabilities and limitations
Stay informed on AI developments

2026-2027:

Take online courses on AI/ML fundamentals (you don't need to be an expert, but understand the basics)
Build side projects using AI agents
Develop expertise in one domain (healthcare, finance, etc.)
Practice "directing" AI rather than just using it

2028-2030:

Position yourself as an AI-human collaboration expert
Learn to manage hybrid teams (humans + AI agents)
Develop strong communication and leadership skills
Focus on architecture and system design
Consider roles in AI safety, ethics, or governance

2031-2035:

Pivot to strategic and creative work
Mentor the next generation
Help shape how your organization uses AI responsibly
Build expertise in emerging fields (AI orchestration, behavior design, etc.)
Stay curious—the roles of 2035 don't exist yet

Why I'm Still Optimistic (But Realistic)

Despite the warnings and uncertainties, here's why I believe in a positive future:

1. We've Been Here Before

The internet didn't destroy jobs—it transformed them. Cloud computing didn't eliminate IT—it elevated it. AI will do the same. While 92 million jobs may be displaced, 170 million new jobs will be created.

2. Human Creativity is Boundless

Humans focus on strengths of empathy, creativity, critical thinking and innovation. These aren't disappearing—they're becoming more valuable.

3. Partnership > Replacement

The real breakthrough lies in understanding where AI's capabilities end and human strengths take over. The future isn't AI or humans—it's AI and humans.

4. We Control the Narrative

Skill measurement is more than a technical requirement—it's a moral imperative that empowers organizations to define roles clearly, address skill gaps, and prepare workforces for the future. We choose how to deploy AI. We choose how to distribute the benefits. We choose what kind of future we want.

5. The Best Work is Ahead

While machines deliver unprecedented speed and precision, humans are better positioned to engage in creative and stimulating work. Imagine a world where tedious work is automated, freeing humans for the challenging, meaningful problems we actually want to solve.

The Questions We Should Be Asking

Instead of "Will AI take my job?", ask:

How can I use AI to become 10x more effective?
What uniquely human value can I provide?
What problems become solvable when I have AI as a partner?
How do I ensure AI development benefits everyone, not just a few?
What skills should I develop that will remain valuable for decades?
How do we build AI systems that augment rather than replace?
What safeguards do we need to prevent the turbulent scenario?
How do we distribute the economic gains from AI fairly?

The quality of our future depends on the quality of our questions today.

Final Thoughts: Partnership, Not Replacement

Sam Altman's concern about "the rate of change that's happening in the world right now" is valid. Making sure we introduce this to the world in a responsible way, where people have time to adapt, to give input, to figure out how to do this is critical.

But here's what I've learned studying these trends, talking to people building these systems, and experimenting with these tools:

The future isn't AI replacing humans. It's AI and humans discovering what they can accomplish together.

Yes, jobs will change. Skills will evolve. Roles will transform. But humans will need to hone the skills that machines can't replicate—and those skills are more important than ever.

The developer of 2035 won't be obsolete. They'll be more creative, more strategic, more impactful than we can imagine today. They'll direct AI agents like conductors leading an orchestra, creating symphonies of software we can barely conceive of now.

But only if we prepare. Only if we adapt. Only if we embrace the partnership.

Even if job disruption is swift, it will be offset by entirely new types of work. The question isn't whether you'll have a role in 2035. It's what role you'll choose to play.

So start now. Learn to work with AI. Develop your uniquely human skills. Stay curious. Stay adaptable. Stay optimistic.

The next decade will be the most transformative in human history. Let's make sure it's also the most empowering.

What's your take? Are you optimistic or concerned about AI and human collaboration? What skills are you developing for the next decade? Drop a comment—I'd love to hear your perspective.

And if this resonated with you, share it with your team. We're all navigating this together.

Resources for Deeper Learning

Remember: The future isn't happening to us. It's happening with us. Let's build it together. 🚀

The Magic of Linux Text Editors

NJEI — Mon, 15 Dec 2025 22:46:23 +0000

The Magic of Linux Text Editors: vi/vim and sed for Editing Like a Pro

The Problem: Editing Files on Remote Servers

You SSH into a server. Need to edit a config file. No GUI. No VS Code. Just the terminal.

You open the file with vi and... nothing works. You're stuck. Can't exit. Can't save. Panic sets in.

Or you need to replace a word in 100 files. Opening each one manually? That'll take hours.

This is why vi/vim and sed exist. One for interactive editing, one for automated text transformation.

Understanding vi/vim

vi (visual editor) is on every Linux system. vim (vi improved) is the enhanced version with more features.

Two Modes: The Key Concept

vi/vim has two modes:

Command Mode (default) - Navigate, delete, copy, search
Insert Mode - Type text normally

This confuses beginners, but it's powerful once you understand it.

Starting vi/vim

vi filename      # Opens file in vi
vim filename     # Opens file in vim

You start in Command Mode.

Insert Mode: Adding Text

From Command Mode, press:

Insert at cursor position

i     # Insert mode at cursor

Insert with space

a     # Insert mode after cursor (adds space)

New line and insert

o     # New line below, enter insert mode
O     # New line above, enter insert mode (capital O)

Press ESC to return to Command Mode.

Command Mode: Navigation and Editing

Basic Movement

h     # Left
j     # Down
k     # Up
l     # Right

# Or use arrow keys

Deleting

x     # Delete character under cursor
dd    # Delete entire line
dw    # Delete word
D     # Delete from cursor to end of line

Undo

u     # Undo last change
Ctrl+r     # Redo

Replace

r     # Replace single character (type new char)
R     # Replace mode (keep typing to replace multiple chars)

Note: r can be unreliable for multiple characters. Use R for replacing continuously or use search/replace.

Search

/keyword     # Search forward for "keyword"
?keyword     # Search backward for "keyword"
n            # Next occurrence
N            # Previous occurrence

Copy and Paste

yy    # Copy (yank) current line
p     # Paste below cursor
P     # Paste above cursor (capital P)

# Copy multiple lines
3yy   # Copy 3 lines

Save and Exit

:w           # Save (write)
:q           # Quit (only if no changes)
:wq          # Save and quit
:wq!         # Save and quit (force)
:q!          # Quit without saving (discard changes)

# Shortcut for save and quit
Shift+ZZ     # Same as :wq

Important: Type these commands only in Command Mode (press ESC first).

Search and Replace in vim

While in Command Mode:

:%s/old/new/g     # Replace all occurrences in file
:%s/old/new/gc    # Replace all, with confirmation
:s/old/new/g      # Replace in current line only

Breaking down :%s/old/new/g:

: - Enter command
% - Apply to entire file
s - Substitute
/old/new/ - Replace "old" with "new"
/g - Global (all occurrences on each line)

Replace on Specific Lines

:1,10s/old/new/g      # Lines 1-10
:5s/old/new/g         # Line 5 only

Example: Change Variable Name

# Replace "userName" with "userId" everywhere
:%s/userName/userId/g

vi vs vim

vi - Original editor, basic features, always available

vim - Enhanced version with:

Syntax highlighting
Multiple undo levels
Split windows
Plugins
Better search/replace

Check out openvim.com for interactive vim tutorial.

Most systems have vim. If not:

sudo apt install vim     # Ubuntu/Debian
sudo yum install vim     # CentOS/RHEL

The sed Command: Stream Editor

sed (stream editor) transforms text non-interactively. Perfect for scripts and bulk operations.

Basic Syntax

sed 'command' filename

Find and Replace

# Replace first occurrence per line
sed 's/Name/NewName/' filename

# Replace all occurrences (global)
sed 's/Name/NewName/g' filename

# Replace and save to file (in-place)
sed -i 's/Name/NewName/g' filename

Important:

Without -i, output goes to screen (original file unchanged)
With -i, modifies file directly
/g flag makes it global (all occurrences on each line)

Delete Text

# Delete a specific word
sed 's/Name//g' filename

# Delete lines containing keyword
sed '/keyword/d' filename

# Delete empty lines
sed '/^$/d' filename

# Delete first two lines
sed '1,2d' filename

# Delete lines 5-10
sed '5,10d' filename

Replace Special Characters

# Replace tabs with spaces
sed -i 's/\t/ /g' filename

# Replace spaces with tabs
sed -i 's/ /\t/g' filename

Display Specific Lines

# Show lines 12-18
sed -n '12,18p' filename

# Show all except lines 12-18
sed '12,18d' filename

# Show first 10 lines (like head)
sed -n '1,10p' filename

Add Blank Lines

# Add blank line after each line
sed G filename

# Double-space the file
sed G filename > newfile.txt

Advanced Replace: Skip Lines

# Replace in all lines EXCEPT line 1
sed '1!s/word/newword/g' filename

# Replace in lines 10-20 only
sed '10,20s/word/newword/g' filename

Real-World Scenarios

Scenario 1: Update Configuration

You need to change a port number in a config file:

# Check current value
grep "port" /etc/app/config.yml

# Replace with sed
sed -i 's/port: 8080/port: 3000/g' /etc/app/config.yml

# Verify change
grep "port" /etc/app/config.yml

Scenario 2: Clean Log Files

Remove empty lines and lines with "DEBUG":

# Remove empty lines
sed -i '/^$/d' app.log

# Remove DEBUG lines
sed -i '/DEBUG/d' app.log

# Or chain them
sed -i '/^$/d; /DEBUG/d' app.log

Scenario 3: Bulk File Updates

Replace API endpoint in all JavaScript files:

# Find all .js files and replace
find . -name "*.js" -exec sed -i 's/api.old.com/api.new.com/g' {} \;

Scenario 4: Edit Config on Server

SSH into server and edit nginx config:

ssh user@server
sudo vim /etc/nginx/nginx.conf

# Press i to enter insert mode
# Make changes
# Press ESC
# Type :wq to save and exit

Scenario 5: Remove Comments

Remove lines starting with # (comments):

sed -i '/^#/d' script.sh

Combining vi/vim and sed

Use vi/vim for interactive editing. Use sed for automated changes.

When to Use vim

Editing single files
Need to see context
Making multiple different changes
Interactive work

When to Use sed

Batch processing multiple files
Scripted changes
Simple find/replace operations
Automated deployments

Example Workflow

# 1. Check what needs changing
grep "old_value" *.conf

# 2. Test sed command (without -i)
sed 's/old_value/new_value/g' config.conf

# 3. Apply to all files
sed -i 's/old_value/new_value/g' *.conf

# 4. Verify one file in vim if needed
vim config.conf

Quick Reference

vim Commands

Command	Action
`i`	Insert mode at cursor
`a`	Insert after cursor
`o`	New line below
`ESC`	Back to command mode
`x`	Delete character
`dd`	Delete line
`u`	Undo
`r`	Replace character
`/keyword`	Search
`yy`	Copy line
`p`	Paste
`:w`	Save
`:q`	Quit
`:wq`	Save and quit
`:q!`	Quit without saving
`Shift+ZZ`	Save and quit
`:%s/old/new/g`	Replace all

sed Commands

Command	Action
`sed 's/old/new/g' file`	Replace all
`sed -i 's/old/new/g' file`	Replace in-place
`sed '/keyword/d' file`	Delete lines with keyword
`sed '/^$/d' file`	Delete empty lines
`sed '1,5d' file`	Delete lines 1-5
`sed -n '10,20p' file`	Show lines 10-20
`sed 's/\t/ /g' file`	Tabs to spaces
`sed G file`	Double-space

Common Mistakes

Mistake #1: Forgetting to exit insert mode

You're typing commands but they appear as text. Press ESC first.

Mistake #2: Using sed without testing

# Wrong - modifies file immediately
sed -i 's/old/new/g' important_file.txt

# Right - test first
sed 's/old/new/g' important_file.txt
# Check output, then add -i

Mistake #3: Not escaping special characters

# Wrong
sed 's/http://old.com/http://new.com/g' file

# Right - escape forward slashes or use different delimiter
sed 's|http://old.com|http://new.com|g' file

Mistake #4: Forgetting the /g flag

# Only replaces first occurrence per line
sed 's/old/new/' file

# Replaces all occurrences
sed 's/old/new/g' file

Tips for Efficiency

Tip 1: Create backups before sed -i

# Create backup with .bak extension
sed -i.bak 's/old/new/g' file.txt

Tip 2: Use vim for learning, sed for automation

Learn editing in vim interactively. Once you know what to change, automate with sed.

Tip 3: Chain sed commands

# Multiple operations at once
sed -i '/^$/d; /DEBUG/d; s/old/new/g' file.txt

Tip 4: Vim line numbers

# In vim command mode
:set number        # Show line numbers
:set nonumber      # Hide line numbers

Practical Examples

Example 1: Update Database Connection

# Old: localhost
# New: db.server.com

sed -i 's/localhost/db.server.com/g' config.php

Example 2: Change All Ports

# Update port 8080 to 3000 in all configs
find /etc/app -name "*.conf" -exec sed -i 's/:8080/:3000/g' {} \;

Example 3: Remove Trailing Whitespace

sed -i 's/[[:space:]]*$//' file.txt

Example 4: Add Text to Beginning of Lines

# Add "# " to beginning of each line (comment out)
sed 's/^/# /' file.txt

Example 5: Replace in Specific Section

# Replace only in lines 10-50
sed '10,50s/old/new/g' file.txt

Key Takeaways

vim has two modes: Command (default) and Insert (press i)
ESC returns to command mode - Always remember this
:wq saves and quits - Or use Shift+ZZ
sed is for automation - Use -i to modify files
Test sed without -i first - Preview changes before applying
Use /g for global replace - Without it, only first occurrence changes
vim for interactive, sed for batch - Choose the right tool
Practice on openvim.com - Interactive vim tutorial

vim and sed aren't relics. They're essential tools for server management, automation, and efficient text editing. Master them and you'll edit files faster than anyone clicking through a GUI.

What's your most useful vim or sed command? Share your go-to editing tricks in the comments.

Text Processing in Linux: grep, awk, and Pipes That Actually Get Work Done

NJEI — Mon, 15 Dec 2025 22:31:43 +0000

Text Processing in Linux: grep, awk, and Pipes That Actually Get Work Done

The Problem: Manually Searching Through Files

You need to find all error messages in a 10,000-line log file. Or extract usernames from system files. Or count how many times a specific IP address appears in access logs.

Opening the file in an editor and searching manually? That's slow and error-prone.

Linux text processing tools turn these tasks into one-line commands.

The cut Command: Extract Columns

cut extracts specific characters or fields from each line.

By Character Position

# Get first character from each line
cut -c1 file.txt

# Get characters 1-3
cut -c1-3 file.txt

# Get characters 1, 2, and 4
cut -c1,2,4 file.txt

Real Example: Extract File Permissions

ls -l | cut -c1-10
# Output: drwxr-xr-x, -rw-r--r--, etc.

The awk Command: Pattern Scanning and Processing

awk is powerful for extracting and manipulating fields (columns).

Basic Field Extraction

# Print first column
awk '{print $1}' file.txt

# Print first and third columns
awk '{print $1, $3}' file.txt

# Print last column (NF = number of fields)
ls -l | awk '{print $NF}'
# Shows filenames from ls -l output

Search and Print

# Find lines containing "Jerry" and print them
awk '/Jerry/ {print}' file.txt

# Or shorter:
awk '/Jerry/' file.txt

Change Field Delimiter

By default, awk uses spaces. Use -F to change it:

# Use colon as delimiter (common in /etc/passwd)
awk -F: '{print $1}' /etc/passwd
# Output: List of all usernames

Modify Fields

# Replace second field with "JJ"
echo "Hello Tom" | awk '{$2="JJ"; print $0}'
# Output: Hello JJ

Filter by Length

# Get lines longer than 15 characters
awk 'length($0) > 15' file.txt

Real-World Example: Extract IP Addresses

# Get IP addresses from access log
awk '{print $1}' /var/log/nginx/access.log

# Count unique IPs
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c

The grep Command: Search Text

grep (global regular expression print) searches for keywords in files or output.

Basic Search

# Find keyword in file
grep keyword filename

# Search in output
ls -l | grep Desktop

Useful Flags

# Count occurrences
grep -c keyword file.txt

# Ignore case
grep -i keyword file.txt
# Finds: keyword, Keyword, KEYWORD

# Show line numbers
grep -n keyword file.txt
# Output: 5:line with keyword

# Exclude lines with keyword (invert match)
grep -v keyword file.txt

Real-World Example: Find Errors in Logs

# Find all error lines
grep -i error /var/log/syslog

# Count errors
grep -i error /var/log/syslog | wc -l

# Find errors but exclude specific ones
grep -i error /var/log/syslog | grep -v "ignore_this_error"

The egrep Command: Multiple Keywords

egrep (or grep -E) searches for multiple patterns at once.

# Search for keyword1 OR keyword2
egrep -i "keyword1|keyword2" file.txt

# Find lines with error or warning
egrep -i "error|warning" /var/log/syslog

The sort Command: Alphabetical Ordering

# Sort alphabetically
sort file.txt

# Reverse sort
sort -r file.txt

# Sort by second field
sort -k2 file.txt

Real Example: Sort By File Size

# Sort files by size (5th column in ls -l)
ls -l | sort -k5 -n
# -n flag for numerical sort

The uniq Command: Remove Duplicates

uniq filters out repeated lines. Important: File must be sorted first.

# Remove duplicates
sort file.txt | uniq

# Count duplicates
sort file.txt | uniq -c
# Output: 3 line_content (appears 3 times)

# Show only duplicates
sort file.txt | uniq -d

Real Example: Most Common Log Entries

# Find most common errors
grep error /var/log/syslog | sort | uniq -c | sort -rn | head -10

Breaking it down:

grep error - Find error lines
sort - Sort so duplicates are together
uniq -c - Count duplicates
sort -rn - Sort by count (reverse numerical)
head -10 - Show top 10

The wc Command: Count Lines, Words, Bytes

wc (word count) reads files and counts.

# Count lines, words, bytes
wc file.txt
# Output: 45 300 2000 file.txt

# Only lines
wc -l file.txt

# Only words
wc -w file.txt

# Only bytes
wc -c file.txt

Real Examples

# Count files in directory
ls -l | wc -l

# Count how many times keyword appears
grep keyword file.txt | wc -l

# Count total lines of code in Python files
find . -name "*.py" -exec wc -l {} \; | awk '{sum+=$1} END {print sum}'

Comparing Files

diff: Line-by-Line Comparison

# Compare files
diff file1.txt file2.txt

# Output shows differences:
# < line in file1
# > line in file2

cmp: Byte-by-Byte Comparison

# Compare files
cmp file1.txt file2.txt

# Output: first byte that differs
# No output if files are identical

Combining Commands with Pipes

The real power comes from chaining commands together.

Example 1: Find and Count

# How many users have /bin/bash as their shell?
grep "/bin/bash" /etc/passwd | wc -l

Example 2: Top 5 Largest Files

ls -lh | sort -k5 -h -r | head -5

Example 3: Extract and Sort

# Get all usernames and sort them
awk -F: '{print $1}' /etc/passwd | sort

Example 4: Search, Extract, Count

# Find IP addresses that accessed /admin
grep "/admin" /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn

This shows which IPs hit /admin most frequently.

Example 5: Log Analysis

# Find most common error types
grep -i error /var/log/app.log | awk '{print $5}' | sort | uniq -c | sort -rn | head -10

Real-World Scenarios

Scenario 1: Find Large Files

# Files larger than 100MB
find / -type f -size +100M 2>/dev/null | xargs ls -lh | awk '{print $5, $NF}'

Scenario 2: Monitor Active Connections

# Count connections per IP
netstat -an | grep ESTABLISHED | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn

Scenario 3: Check Failed Login Attempts

# Count failed SSH attempts by IP
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn

Scenario 4: Disk Usage by Directory

# Top 10 directories by size
du -h /var | sort -h -r | head -10

Scenario 5: Extract Email Addresses

# Find all email addresses in file
grep -Eo '[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}' file.txt | sort | uniq

Common Patterns

Pattern 1: Search, Extract, Sort, Count

grep pattern file | awk '{print $2}' | sort | uniq -c | sort -rn

Pattern 2: Filter and Process

cat file | grep -v exclude_pattern | awk '{print $1}'

Pattern 3: Multiple Conditions

egrep "error|warning" file | grep -v "ignore" | wc -l

Quick Reference

cut

cut -c1-3 file        # Characters 1-3
cut -d: -f1 file      # First field (delimiter :)

awk

awk '{print $1}' file              # First column
awk -F: '{print $1}' file          # Custom delimiter
awk '/pattern/ {print}' file       # Pattern matching
awk '{print $NF}' file             # Last column
awk 'length($0) > 15' file         # Lines > 15 chars

grep

grep pattern file                   # Search
grep -i pattern file               # Ignore case
grep -c pattern file               # Count
grep -n pattern file               # Line numbers
grep -v pattern file               # Invert (exclude)
egrep "pat1|pat2" file             # Multiple patterns

sort

sort file                          # Alphabetical
sort -r file                       # Reverse
sort -k2 file                      # By second field
sort -n file                       # Numerical

uniq

sort file | uniq                   # Remove duplicates
sort file | uniq -c                # Count occurrences
sort file | uniq -d                # Show only duplicates

wc

wc file                            # Lines, words, bytes
wc -l file                         # Lines only
wc -w file                         # Words only
wc -c file                         # Bytes only

diff/cmp

diff file1 file2                   # Line comparison
cmp file1 file2                    # Byte comparison

Tips for Efficiency

Tip 1: Use pipes instead of temporary files

# Instead of:
grep pattern file > temp.txt
sort temp.txt > sorted.txt

# Do:
grep pattern file | sort

Tip 2: Combine grep with awk

# Filter then extract
grep error log.txt | awk '{print $1, $5}'

Tip 3: Use awk instead of multiple cuts

# Instead of:
cut -d: -f1 file | cut -d- -f1

# Do:
awk -F: '{print $1}' file | awk -F- '{print $1}'

Tip 4: Test patterns on small samples first

# Test on first 10 lines
head -10 large_file.txt | grep pattern

Key Takeaways

cut - Extract characters or fields
awk - Process fields, pattern matching, calculations
grep - Search for patterns
egrep - Search multiple patterns
sort - Order lines
uniq - Remove duplicates (must sort first)
wc - Count lines, words, bytes
Pipes (|) - Chain commands together
diff/cmp - Compare files

These commands aren't just for showing off. They solve real problems:

Analyzing logs
Extracting data
Monitoring systems
Processing reports
Debugging issues

Master these tools and manual file searching becomes a thing of the past.

What text processing task do you do most often? Share your go-to command combinations in the comments.

I Automated My Boring Dev Tasks With Simple Bash Scripts (You Don't Need DevOps Experience)

NJEI — Wed, 10 Dec 2025 13:03:36 +0000

Three months ago, I was wasting 2-3 hours every week on repetitive tasks. Setting up environments. Backing up files. Renaming batches of images. Running the same 6 commands every time I started a new feature.

I kept telling myself: "I should automate this."

But automation felt like this mysterious DevOps thing that required learning Docker, Kubernetes, CI/CD pipelines, and sacrificing a weekend to figure it all out.

Then I discovered something: Most of my boring tasks could be automated with simple bash scripts—15-20 lines of code that take 10 minutes to write.

No DevOps experience needed. No complex tools. Just bash, which you already have installed.

Let me show you the scripts that gave me back those hours.

Why Bash Scripts (And Not Fancy Tools)
Your First Script: The 5-Minute Setup
- Making Scripts Executable
- Running Your Script
Script 1: Project Setup Automation
- The Manual Way (Before)
- The Automated Way (After)
- How It Works
Script 2: Smart Backup System
- Why This Matters
- The Script
- Breaking It Down
- Scheduling Automatic Backups
Script 3: Batch File Renaming
- The Problem
- The Solution
- Advanced: Pattern-Based Renaming
Script 4: Git Workflow Automation
- The Tedious Workflow
- The One-Command Solution
- Safety Features
Script 5: Environment Cleanup
- The Bloat Problem
- The Cleanup Script
- What Gets Cleaned
Script 6: Port Killer (Because Something is Always Using Port 3000)
- The Frustration
- The Fix
- Cross-Platform Version
Making Scripts Globally Available
- Method 1: Add to PATH
- Method 2: Create Aliases
- Method 3: Symlinks
Bash Scripting Essentials You Need to Know
- Variables
- User Input
- Conditionals
- Loops
- Functions
- Exit Codes
Debugging Your Scripts
- Common Errors and Fixes
Real-World Impact: Time Saved
Your Script Library: Getting Started
Next Steps: Progressive Automation
Key Takeaways

Why Bash Scripts (And Not Fancy Tools)

Before we dive in, you might be thinking: "Why not use Python? Or Node.js? Or some automation platform?"

Here's why bash:

Already installed - Every Mac and Linux has bash. Windows has WSL or Git Bash.
No dependencies - No npm install, no pip, no virtual environments.
Perfect for system tasks - File operations, process management, running commands.
Instant execution - No compilation, no startup time.
Universal - Works on servers, containers, local machines.

Python is great. Node.js is powerful. But for quick automation of repetitive tasks? Bash is unbeatable.

Your First Script: The 5-Minute Setup

Let's start with the absolute basics. Create a file called hello.sh:

#!/bin/bash

echo "Hello from your first script!"
echo "Current directory: $(pwd)"
echo "Current user: $(whoami)"
echo "Today's date: $(date)"

Making Scripts Executable

chmod +x hello.sh

That's it. The +x flag makes the file executable.

Running Your Script

./hello.sh

Output:

Hello from your first script!
Current directory: /Users/john/projects
Current user: john
Today's date: Wed Dec 11 14:23:45 PST 2024

What just happened:

#!/bin/bash tells the system to use bash to run this script
echo prints to the terminal
$(command) runs a command and inserts its output
chmod +x made it executable
./ runs the script from the current directory

You just automated your first task. Let's get practical.

Script 1: Project Setup Automation

The Manual Way (Before)

Every time you start a new project, you do this:

mkdir my-new-project
cd my-new-project
git init
echo "# My New Project" > README.md
echo "node_modules/" > .gitignore
echo ".env" >> .gitignore
echo ".DS_Store" >> .gitignore
mkdir src
touch src/index.js
npm init -y
npm install express dotenv
git add .
git commit -m "Initial commit"
code .

That's 13 commands. Every. Single. Time.

The Automated Way (After)

Create newproject.sh:

#!/bin/bash

# Check if project name was provided
if [ -z "$1" ]; then
    echo "Usage: ./newproject.sh <project-name>"
    exit 1
fi

PROJECT_NAME=$1

# Create project structure
echo "📁 Creating project: $PROJECT_NAME"
mkdir "$PROJECT_NAME"
cd "$PROJECT_NAME"

# Initialize git
echo "🔧 Setting up Git..."
git init

# Create README
echo "📝 Creating README..."
echo "# $PROJECT_NAME" > README.md
echo "" >> README.md
echo "## Getting Started" >> README.md
echo "" >> README.md
echo "\`\`\`bash" >> README.md
echo "npm install" >> README.md
echo "npm start" >> README.md
echo "\`\`\`" >> README.md

# Create .gitignore
echo "🚫 Creating .gitignore..."
cat > .gitignore << EOF
node_modules/
.env
.DS_Store
dist/
build/
*.log
.vscode/
coverage/
EOF

# Create project structure
echo "📂 Creating directories..."
mkdir -p src/{components,utils,config}
mkdir -p tests

# Create initial files
echo "📄 Creating initial files..."
cat > src/index.js << EOF
console.log('Hello from $PROJECT_NAME!');
EOF

cat > .env.example << EOF
PORT=3000
NODE_ENV=development
EOF

# Initialize npm
echo "📦 Initializing npm..."
npm init -y

# Install common dependencies
echo "⬇️  Installing dependencies..."
npm install express dotenv

echo "⬇️  Installing dev dependencies..."
npm install -D nodemon

# Update package.json scripts
echo "🔧 Configuring npm scripts..."
npm pkg set scripts.start="node src/index.js"
npm pkg set scripts.dev="nodemon src/index.js"
npm pkg set scripts.test="echo \"Error: no test specified\" && exit 1"

# Initial commit
echo "💾 Creating initial commit..."
git add .
git commit -m "Initial commit: project setup"

# Open in VS Code
echo "🚀 Opening in VS Code..."
code .

echo ""
echo "✅ Project $PROJECT_NAME created successfully!"
echo ""
echo "To get started:"
echo "  cd $PROJECT_NAME"
echo "  npm run dev"

How It Works

Usage:

./newproject.sh my-awesome-app

What it does:

✅ Creates project directory
✅ Initializes git repository
✅ Creates structured README with getting started instructions
✅ Sets up comprehensive .gitignore
✅ Creates organized folder structure (src, tests, components)
✅ Generates starter files with boilerplate code
✅ Initializes npm with package.json
✅ Installs express, dotenv, and nodemon
✅ Configures npm scripts for start/dev
✅ Makes initial git commit
✅ Opens project in VS Code

Time saved: 5 minutes per project × 20 projects/year = 100 minutes saved

New concepts introduced:

$1 - First command-line argument
[ -z "$1" ] - Check if variable is empty
cat > file << EOF - Create multi-line files (here document)
mkdir -p - Create nested directories
npm pkg set - Modify package.json programmatically

Script 2: Smart Backup System

Why This Matters

You're working on a feature. You want to try something risky. You need a quick backup—not a full git commit, just a safety net.

Manually copying files is tedious. This script does it intelligently.

The Script

Create backup.sh:

#!/bin/bash

# Configuration
BACKUP_DIR="$HOME/backups"
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
PROJECT_NAME=$(basename "$(pwd)")
BACKUP_NAME="${PROJECT_NAME}_${TIMESTAMP}"
BACKUP_PATH="$BACKUP_DIR/$BACKUP_NAME"

# Colors for output
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

echo -e "${BLUE}🗄️  Backup Script${NC}"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

# Create backup directory if it doesn't exist
mkdir -p "$BACKUP_DIR"

# Check if we're in a git repo
if [ -d ".git" ]; then
    echo -e "${YELLOW}📍 Git repository detected${NC}"

    # Check for uncommitted changes
    if [[ -n $(git status -s) ]]; then
        echo -e "${YELLOW}⚠️  You have uncommitted changes${NC}"
        git status -s
        echo ""
    fi
fi

# Create the backup
echo -e "${BLUE}📦 Creating backup...${NC}"
echo "Source: $(pwd)"
echo "Destination: $BACKUP_PATH"

# Use rsync for smart copying (only changed files)
rsync -av \
    --exclude='node_modules' \
    --exclude='.git' \
    --exclude='dist' \
    --exclude='build' \
    --exclude='*.log' \
    --exclude='.DS_Store' \
    --exclude='coverage' \
    . "$BACKUP_PATH"

if [ $? -eq 0 ]; then
    echo -e "${GREEN}✅ Backup created successfully!${NC}"
    echo ""
    echo "Backup location: $BACKUP_PATH"

    # Calculate backup size
    SIZE=$(du -sh "$BACKUP_PATH" | cut -f1)
    echo "Backup size: $SIZE"

    # List recent backups
    echo ""
    echo -e "${BLUE}Recent backups:${NC}"
    ls -lth "$BACKUP_DIR" | grep "$PROJECT_NAME" | head -5

    # Cleanup old backups (keep last 10)
    echo ""
    echo -e "${YELLOW}🧹 Cleaning up old backups...${NC}"
    cd "$BACKUP_DIR"
    ls -t | grep "$PROJECT_NAME" | tail -n +11 | xargs -I {} rm -rf {}
    echo "Kept last 10 backups"

else
    echo -e "${RED}❌ Backup failed!${NC}"
    exit 1
fi

echo ""
echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
echo -e "${GREEN}Done!${NC}"

Breaking It Down

Key features:

Smart naming - Uses timestamp: myproject_20241211_142335
Excludes bloat - Skips node_modules, .git, build folders
Uses rsync - Only copies changed files (fast!)
Colored output - Green/blue/yellow for better UX
Git awareness - Warns about uncommitted changes
Auto cleanup - Keeps only last 10 backups
Shows size - You know how much space you're using

Usage:

./backup.sh

Output:

🗄️  Backup Script
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📍 Git repository detected
⚠️  You have uncommitted changes
 M src/index.js
?? temp.txt

📦 Creating backup...
Source: /Users/john/projects/myapp
Destination: /Users/john/backups/myapp_20241211_142335
✅ Backup created successfully!

Backup location: /Users/john/backups/myapp_20241211_142335
Backup size: 145M

Recent backups:
drwxr-xr-x  myapp_20241211_142335
drwxr-xr-x  myapp_20241211_093022
drwxr-xr-x  myapp_20241210_165544

🧹 Cleaning up old backups...
Kept last 10 backups

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Done!

Scheduling Automatic Backups

Want backups every day at 6 PM? Use cron:

crontab -e

Add this line:

0 18 * * * cd /path/to/your/project && /path/to/backup.sh

Now your project backs itself up daily.

Script 3: Batch File Renaming

The Problem

You have 47 screenshots named:

Screenshot 2024-12-11 at 10.23.45 AM.png
Screenshot 2024-12-11 at 10.24.12 AM.png
Screenshot 2024-12-11 at 10.25.33 AM.png
...

You need them named:

feature-demo-01.png
feature-demo-02.png
feature-demo-03.png
...

Doing this manually? 20 minutes of mind-numbing work.

The Solution

Create rename-files.sh:

#!/bin/bash

# Check arguments
if [ $# -lt 2 ]; then
    echo "Usage: ./rename-files.sh <pattern> <new-prefix>"
    echo "Example: ./rename-files.sh '*.png' 'screenshot'"
    exit 1
fi

PATTERN=$1
PREFIX=$2
COUNTER=1

echo "🔄 Renaming files matching: $PATTERN"
echo "New prefix: $PREFIX"
echo ""

# Preview mode first
echo "Preview (no changes will be made):"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

for file in $PATTERN; do
    if [ -f "$file" ]; then
        extension="${file##*.}"
        new_name=$(printf "%s-%02d.%s" "$PREFIX" "$COUNTER" "$extension")
        echo "$file  →  $new_name"
        COUNTER=$((COUNTER + 1))
    fi
done

echo ""
echo "Total files to rename: $((COUNTER - 1))"
echo ""

# Ask for confirmation
read -p "Proceed with renaming? (y/n): " -n 1 -r
echo ""

if [[ $REPLY =~ ^[Yy]$ ]]; then
    COUNTER=1
    for file in $PATTERN; do
        if [ -f "$file" ]; then
            extension="${file##*.}"
            new_name=$(printf "%s-%02d.%s" "$PREFIX" "$COUNTER" "$extension")
            mv "$file" "$new_name"
            echo "✅ Renamed: $new_name"
            COUNTER=$((COUNTER + 1))
        fi
    done
    echo ""
    echo "🎉 Done! Renamed $((COUNTER - 1)) files"
else
    echo "❌ Cancelled"
fi

Usage:

./rename-files.sh "Screenshot*.png" "feature-demo"

Output:

🔄 Renaming files matching: Screenshot*.png
New prefix: feature-demo

Preview (no changes will be made):
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Screenshot 2024-12-11 at 10.23.45 AM.png  →  feature-demo-01.png
Screenshot 2024-12-11 at 10.24.12 AM.png  →  feature-demo-02.png
Screenshot 2024-12-11 at 10.25.33 AM.png  →  feature-demo-03.png

Total files to rename: 3

Proceed with renaming? (y/n): y

✅ Renamed: feature-demo-01.png
✅ Renamed: feature-demo-02.png
✅ Renamed: feature-demo-03.png

🎉 Done! Renamed 3 files

Advanced: Pattern-Based Renaming

Want to replace spaces with dashes?

#!/bin/bash

for file in *\ *; do
    if [ -f "$file" ]; then
        new_name="${file// /-}"
        mv "$file" "$new_name"
        echo "Renamed: $file → $new_name"
    fi
done

Before:

my vacation photo.jpg
final draft version 2.docx

After:

my-vacation-photo.jpg
final-draft-version-2.docx

Script 4: Git Workflow Automation

The Tedious Workflow

Your typical feature branch workflow:

git checkout main
git pull origin main
git checkout -b feature/user-auth
# ... make changes ...
git add .
git commit -m "Add user authentication"
git push origin feature/user-auth
# ... create PR manually on GitHub ...

Every. Single. Time.

The One-Command Solution

Create gitflow.sh:

#!/bin/bash

# Colors
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m'

# Function to print colored output
print_step() {
    echo -e "${BLUE}▶ $1${NC}"
}

print_success() {
    echo -e "${GREEN}✓ $1${NC}"
}

print_error() {
    echo -e "${RED}✗ $1${NC}"
}

# Check if we're in a git repo
if [ ! -d ".git" ]; then
    print_error "Not a git repository!"
    exit 1
fi

# Get current branch
CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)

echo -e "${BLUE}╔════════════════════════════════════╗${NC}"
echo -e "${BLUE}║   Git Workflow Automation Tool     ║${NC}"
echo -e "${BLUE}╚════════════════════════════════════╝${NC}"
echo ""

# Check for uncommitted changes
if [[ -n $(git status -s) ]]; then
    print_error "You have uncommitted changes!"
    git status -s
    echo ""
    read -p "Commit these changes? (y/n): " -n 1 -r
    echo ""

    if [[ $REPLY =~ ^[Yy]$ ]]; then
        read -p "Commit message: " COMMIT_MSG
        git add .
        git commit -m "$COMMIT_MSG"
        print_success "Changes committed"
    else
        print_error "Please commit or stash your changes first"
        exit 1
    fi
fi

# Sync with main
print_step "Syncing with main branch..."
git checkout main
git pull origin main

if [ $? -ne 0 ]; then
    print_error "Failed to sync with main"
    exit 1
fi
print_success "Synced with main"

# Get feature name
echo ""
read -p "Feature branch name (e.g., user-auth): " FEATURE_NAME

if [ -z "$FEATURE_NAME" ]; then
    print_error "Feature name cannot be empty"
    exit 1
fi

BRANCH_NAME="feature/$FEATURE_NAME"

# Create and checkout feature branch
print_step "Creating branch: $BRANCH_NAME"
git checkout -b "$BRANCH_NAME"

if [ $? -ne 0 ]; then
    # Branch might already exist
    print_step "Branch exists, checking out..."
    git checkout "$BRANCH_NAME"
fi

print_success "On branch: $BRANCH_NAME"

echo ""
echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
echo -e "${GREEN}Ready to work on $BRANCH_NAME${NC}"
echo -e "${GREEN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
echo ""
echo "When done, run:"
echo "  git add ."
echo "  git commit -m 'your message'"
echo "  git push origin $BRANCH_NAME"

Usage:

./gitflow.sh

Interactive output:

╔════════════════════════════════════╗
║   Git Workflow Automation Tool     ║
╚════════════════════════════════════╝

▶ Syncing with main branch...
✓ Synced with main

Feature branch name (e.g., user-auth): user-authentication

▶ Creating branch: feature/user-authentication
✓ On branch: feature/user-authentication

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ready to work on feature/user-authentication
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

When done, run:
  git add .
  git commit -m 'your message'
  git push origin feature/user-authentication

Safety Features

The script includes:

✅ Checks if you're in a git repo
✅ Warns about uncommitted changes
✅ Offers to commit changes for you
✅ Syncs with main before creating branch
✅ Handles existing branch names gracefully
✅ Provides clear next steps

Time saved: 2 minutes per feature × 30 features/month = 60 minutes saved

Script 5: Environment Cleanup

The Bloat Problem

Your node_modules folders are everywhere. Your Docker images are piling up. Your disk space is disappearing.

Time for a cleanup.

The Cleanup Script

Create cleanup.sh:

#!/bin/bash

# Colors
BLUE='\033[0;34m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m'

# Function to get directory size
get_size() {
    du -sh "$1" 2>/dev/null | cut -f1
}

# Function to format numbers
format_number() {
    echo "$1" | sed ':a;s/\B[0-9]\{3\}\>/,&/;ta'
}

echo -e "${BLUE}╔═══════════════════════════════════════╗${NC}"
echo -e "${BLUE}║   Development Environment Cleanup     ║${NC}"
echo -e "${BLUE}╚═══════════════════════════════════════╝${NC}"
echo ""

# 1. Find and clean node_modules
echo -e "${YELLOW}📦 Scanning for node_modules directories...${NC}"
NODE_MODULES_DIRS=$(find . -name "node_modules" -type d -prune 2>/dev/null)
NODE_MODULES_COUNT=$(echo "$NODE_MODULES_DIRS" | grep -c "node_modules")

if [ "$NODE_MODULES_COUNT" -gt 0 ]; then
    echo "Found $NODE_MODULES_COUNT node_modules directories:"
    echo ""

    TOTAL_SIZE=0
    while IFS= read -r dir; do
        if [ -n "$dir" ]; then
            SIZE=$(get_size "$dir")
            echo "  📁 $dir ($SIZE)"
        fi
    done <<< "$NODE_MODULES_DIRS"

    echo ""
    read -p "Delete all node_modules folders? (y/n): " -n 1 -r
    echo ""

    if [[ $REPLY =~ ^[Yy]$ ]]; then
        while IFS= read -r dir; do
            if [ -n "$dir" ]; then
                rm -rf "$dir"
                echo -e "${GREEN}✓ Deleted: $dir${NC}"
            fi
        done <<< "$NODE_MODULES_DIRS"
        echo ""
        echo -e "${GREEN}✓ Cleaned up node_modules${NC}"
    fi
else
    echo "No node_modules directories found"
fi

echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""

# 2. Clean npm cache
echo -e "${YELLOW}🗑️  Checking npm cache...${NC}"
NPM_CACHE_SIZE=$(get_size ~/.npm)
echo "Current npm cache size: $NPM_CACHE_SIZE"
echo ""
read -p "Clear npm cache? (y/n): " -n 1 -r
echo ""

if [[ $REPLY =~ ^[Yy]$ ]]; then
    npm cache clean --force
    echo -e "${GREEN}✓ npm cache cleared${NC}"
fi

echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""

# 3. Clean Docker (if installed)
if command -v docker &> /dev/null; then
    echo -e "${YELLOW}🐳 Checking Docker...${NC}"

    # Count images and containers
    IMAGE_COUNT=$(docker images -q | wc -l | tr -d ' ')
    CONTAINER_COUNT=$(docker ps -a -q | wc -l | tr -d ' ')
    VOLUME_COUNT=$(docker volume ls -q | wc -l | tr -d ' ')

    echo "Docker images: $(format_number $IMAGE_COUNT)"
    echo "Docker containers: $(format_number $CONTAINER_COUNT)"
    echo "Docker volumes: $(format_number $VOLUME_COUNT)"
    echo ""

    # Show disk usage
    docker system df
    echo ""

    read -p "Run Docker system prune? (removes unused data) (y/n): " -n 1 -r
    echo ""

    if [[ $REPLY =~ ^[Yy]$ ]]; then
        docker system prune -af --volumes
        echo -e "${GREEN}✓ Docker cleaned${NC}"
    fi
else
    echo -e "${YELLOW}Docker not installed, skipping...${NC}"
fi

echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""

# 4. Clean build artifacts
echo -e "${YELLOW}🏗️  Scanning for build artifacts...${NC}"

BUILD_DIRS=$(find . -type d \( -name "dist" -o -name "build" -o -name ".next" -o -name "out" \) -prune 2>/dev/null)
BUILD_COUNT=$(echo "$BUILD_DIRS" | grep -c -E "dist|build|\.next|out")

if [ "$BUILD_COUNT" -gt 0 ]; then
    echo "Found $BUILD_COUNT build directories:"
    echo ""

    while IFS= read -r dir; do
        if [ -n "$dir" ]; then
            SIZE=$(get_size "$dir")
            echo "  📁 $dir ($SIZE)"
        fi
    done <<< "$BUILD_DIRS"

    echo ""
    read -p "Delete build artifacts? (y/n): " -n 1 -r
    echo ""

    if [[ $REPLY =~ ^[Yy]$ ]]; then
        while IFS= read -r dir; do
            if [ -n "$dir" ]; then
                rm -rf "$dir"
                echo -e "${GREEN}✓ Deleted: $dir${NC}"
            fi
        done <<< "$BUILD_DIRS"
        echo -e "${GREEN}✓ Build artifacts cleaned${NC}"
    fi
else
    echo "No build artifacts found"
fi

echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""

# 5. Clean log files
echo -e "${YELLOW}📝 Scanning for log files...${NC}"

LOG_FILES=$(find . -type f -name "*.log" 2>/dev/null)
LOG_COUNT=$(echo "$LOG_FILES" | grep -c ".log")

if [ "$LOG_COUNT" -gt 0 ]; then
    echo "Found $LOG_COUNT log files"
    echo ""

    read -p "Delete all .log files? (y/n): " -n 1 -r
    echo ""

    if [[ $REPLY =~ ^[Yy]$ ]]; then
        while IFS= read -r file; do
            if [ -n "$file" ]; then
                rm "$file"
                echo -e "${GREEN}✓ Deleted: $file${NC}"
            fi
        done <<< "$LOG_FILES"
        echo -e "${GREEN}✓ Log files cleaned${NC}"
    fi
else
    echo "No log files found"
fi

echo ""
echo -e "${GREEN}╔═══════════════════════════════════════╗${NC}"
echo -e "${GREEN}║         Cleanup Complete!              ║${NC}"
echo -e "${GREEN}╚═══════════════════════════════════════╝${NC}"
echo ""

# Show disk space freed
echo "Check your available disk space:"
df -h .

What Gets Cleaned

This script intelligently finds and removes:

node_modules/ - Often 200MB+ per project
npm cache - Can grow to several GB
Docker images/containers - Unused ones pile up fast
Build artifacts - dist/, build/, .next/, out/
Log files - *.log files everywhere

Safety features:

✅ Shows what it found before deleting
✅ Asks for confirmation at each step
✅ Shows sizes so you can decide
✅ Never deletes without permission

Usage:

./cleanup.sh

Real impact: Recovered 47GB on my machine last week.

Script 6: Port Killer (Because Something is Always Using Port 3000)

The Frustration

You try to start your dev server:

npm start

Error:

Error: listen EADDRINUSE: address already in use :::3000

Then you go hunting through Activity Monitor or lsof commands you half-remember.

The Fix

Create killport.sh:

#!/bin/bash

# Check if port number provided
if [ -z "$1" ]; then
    echo "Usage: ./killport.sh <port-number>"
    echo "Example: ./killport.sh 3000"
    exit 1
fi

PORT=$1

echo "🔍 Checking port $PORT..."
echo ""

# Find process using the port
if [[ "$OSTYPE" == "darwin"* ]]; then
    # macOS
    PID=$(lsof -ti:$PORT)
else
    # Linux
    PID=$(fuser $PORT/tcp 2>/dev/null | awk '{print $1}')
fi

if [ -z "$PID" ]; then
    echo "✓ Port $PORT is free"
    exit 0
fi

# Get process details
if [[ "$OSTYPE" == "darwin"* ]]; then
    PROCESS_NAME=$(ps -p $PID -o comm=)
else
    PROCESS_NAME=$(ps -p $PID -o comm=)
fi

echo "Found process using port $PORT:"
echo "  PID: $PID"
echo "  Process: $PROCESS_NAME"
echo ""

# Show full process info
ps -p $PID -o pid,ppid,%cpu,%mem,command

echo ""
read -p "Kill this process? (y/n): " -n 1 -r
echo ""

if [[ $REPLY =~ ^[Yy]$ ]]; then
    kill -9 $PID

    if [ $? -eq 0 ]; then
        echo "✓ Process killed"
        echo "✓ Port $PORT is now free"
    else
        echo "✗ Failed to kill process (try with sudo)"
    fi
else
    echo "Cancelled"
fi

Usage:

./killport.sh 3000

Output:

🔍 Checking port 3000...

Found process using port 3000:
  PID: 12345
  Process: node

  PID  PPID  %CPU %MEM COMMAND
12345 98765  0.0  0.5 node /Users/john/app/server.js

Kill this process? (y/n): y

✓ Process killed
✓ Port 3000 is now free

Cross-Platform Version

Want it to work everywhere? Enhanced version:

#!/bin/bash

PORT=$1

# Detect OS
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
    # Linux
    fuser -k $PORT/tcp
elif [[ "$OSTYPE" == "darwin"* ]]; then
    # macOS
    lsof -ti:$PORT | xargs kill -9
elif [[ "$OSTYPE" == "msys" || "$OSTYPE" == "cygwin" ]]; then
    # Windows (Git Bash/WSL)
    netstat -ano | findstr :$PORT | awk '{print $5}' | xargs taskkill //PID //F
fi

echo "✓ Port $PORT cleared"

Time saved: 30 seconds × 100 times/year = 50 minutes

Making Scripts Globally Available

Right now, you need to type ./script.sh from the script's directory. Let's make them available everywhere.

Method 1: Add to PATH

Create a bin directory:

mkdir -p ~/bin

Move your scripts there:

mv newproject.sh ~/bin/newproject
mv backup.sh ~/bin/backup
mv killport.sh ~/bin/killport
# etc...

Add to your PATH in ~/.bashrc or ~/.zshrc:

export PATH="$HOME/bin:$PATH"

Reload your shell:

source ~/.bashrc  # or source ~/.zshrc

Now you can run from anywhere:

newproject my-app
backup
killport 3000

Method 2: Create Aliases

Add to ~/.bashrc or ~/.zshrc:

alias newproject='~/scripts/newproject.sh'
alias backup='~/scripts/backup.sh'
alias killport='~/scripts/killport.sh'
alias cleanup='~/scripts/cleanup.sh'
alias gitflow='~/scripts/gitflow.sh'

Method 3: Symlinks

Create symbolic links in /usr/local/bin:

sudo ln -s ~/scripts/newproject.sh /usr/local/bin/newproject
sudo ln -s ~/scripts/backup.sh /usr/local/bin/backup
sudo ln -s ~/scripts/killport.sh /usr/local/bin/killport

Now these commands work system-wide.

Bash Scripting Essentials You Need to Know

Let's cover the fundamentals you'll use in 99% of scripts.

Variables

# Simple assignment
NAME="John"
AGE=30

# Use variables
echo "Hello, $NAME"
echo "You are $AGE years old"

# Command output as variable
CURRENT_DIR=$(pwd)
TODAY=$(date +%Y-%m-%d)

# Environment variables
echo $HOME
echo $USER
echo $PATH

User Input

# Read input
read -p "Enter your name: " USERNAME
echo "Hello, $USERNAME"

# Read with default
read -p "Port [3000]: " PORT
PORT=${PORT:-3000}  # Use 3000 if empty

# Yes/No prompt
read -p "Continue? (y/n): " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then
    echo "Continuing..."
fi

Conditionals

# Check if file exists
if [ -f "package.json" ]; then
    echo "Node.js project detected"
fi

# Check if directory exists
if [ -d "node_modules" ]; then
    echo "Dependencies installed"
fi

# Check if variable is empty
if [ -z "$VAR" ]; then
    echo "Variable is empty"
fi

# Check if variable is not empty
if [ -n "$VAR" ]; then
    echo "Variable has a value"
fi

# Multiple conditions
if [ -f "package.json" ] && [ -d "src" ]; then
    echo "Valid project structure"
fi

# Comparison
if [ $AGE -gt 18 ]; then
    echo "Adult"
fi

# String comparison
if [ "$NAME" = "John" ]; then
    echo "Hello John"
fi

Loops

# Loop through files
for file in *.txt; do
    echo "Processing: $file"
done

# Loop through numbers
for i in {1..10}; do
    echo "Number: $i"
done

# While loop
COUNTER=0
while [ $COUNTER -lt 5 ]; do
    echo "Count: $COUNTER"
    COUNTER=$((COUNTER + 1))
done

# Read file line by line
while IFS= read -r line; do
    echo "Line: $line"
done < file.txt

Functions

# Define function
greet() {
    local name=$1
    echo "Hello, $name!"
}

# Call function
greet "John"

# Function with return value
add() {
    local result=$(($1 + $2))
    echo $result
}

sum=$(add 5 10)
echo "Sum: $sum"

# Function with multiple parameters
create_file() {
    local filename=$1
    local content=$2
    echo "$content" > "$filename"
    echo "Created: $filename"
}

create_file "test.txt" "Hello World"

Exit Codes

# Exit with success
exit 0

# Exit with error
exit 1

# Check exit code of last command
if [ $? -eq 0 ]; then
    echo "Command succeeded"
else
    echo "Command failed"
fi

# Exit on any error
set -e

# Exit on undefined variable
set -u

Debugging Your Scripts

Enable debug mode:

#!/bin/bash
set -x  # Print each command before executing

Or run with debug flag:

bash -x script.sh

Output with debug:

+ PORT=3000
+ echo 'Starting on port 3000'
Starting on port 3000
+ npm start

Check syntax without running:

bash -n script.sh

Common Errors and Fixes

Error: Permission denied

# Fix
chmod +x script.sh

Error: Command not found

# Check if command exists first
if command -v docker &> /dev/null; then
    docker --version
else
    echo "Docker not installed"
fi

Error: Variable not found

# Use parameter expansion with default
PORT=${PORT:-3000}  # Use 3000 if PORT not set

Error: File not found

# Check before operating
if [ -f "$FILE" ]; then
    cat "$FILE"
else
    echo "File not found: $FILE"
fi

Real-World Impact: Time Saved

Let me show you the actual time I've saved with these scripts:

Script	Task	Manual Time	Automated Time	Uses/Month	Monthly Savings
newproject.sh	Project setup	5 min	30 sec	8	36 min
backup.sh	Backup project	3 min	10 sec	20	50 min
gitflow.sh	Branch workflow	2 min	20 sec	25	42 min
killport.sh	Kill port process	1 min	5 sec	30	28 min
cleanup.sh	Environment cleanup	15 min	2 min	2	26 min
rename-files.sh	Batch rename	10 min	1 min	4	36 min

Total monthly savings: 218 minutes (3.6 hours)
Annual savings: 2,616 minutes (43.6 hours)

That's more than a full work week every year.

Your Script Library: Getting Started

Here's how to build your own automation library:

1. Create a scripts directory:

mkdir -p ~/scripts
cd ~/scripts

2. Start with one painful task:

What do you do repeatedly?
What takes 5+ minutes each time?
What makes you think "there has to be a better way"?

3. Write a simple script:

#!/bin/bash
# Start simple - automate just one thing
echo "Starting task..."
# Your commands here
echo "Done!"

4. Test it thoroughly:

chmod +x script.sh
./script.sh

5. Refine over time:

Add error checking
Add user prompts
Add color output
Add help text

6. Make it global:

# Add to PATH
echo 'export PATH="$HOME/scripts:$PATH"' >> ~/.bashrc
source ~/.bashrc

7. Document it:
Create a README in your scripts folder:

# My Automation Scripts

## newproject
Creates a new project with git, npm, and standard structure
Usage: `newproject <project-name>`

## backup
Backs up current directory with timestamp
Usage: `backup`

## killport
Kills process using specified port
Usage: `killport <port>`

Next Steps: Progressive Automation

Start small. Build momentum. Here's your roadmap:

Week 1: The Basics

Install your scripts in ~/scripts
Make them executable
Run them manually when needed

Week 2: Make Them Convenient

Add to PATH or create aliases
Use them daily until they become habit

Week 3: Add Intelligence

Add error checking
Add confirmation prompts
Add colored output

Week 4: Share and Expand

Push to GitHub
Get feedback from team
Automate one more task

Month 2: Advanced Automation

Learn cron for scheduling
Create git hooks
Chain scripts together

Month 3: Team Automation

Share scripts with team
Create onboarding automation
Build deployment scripts

Key Takeaways

Start simple - A 10-line script is still automation
Solve real pain - Automate what frustrates you
Test thoroughly - Use confirmation prompts at first
Build incrementally - Don't try to create the perfect script
Document usage - Add help text and examples
Share with team - Your automation can help everyone
Iterate constantly - Scripts evolve with your needs

Remember: Every minute you spend writing automation saves you hours in the future.

The best time to start automating was yesterday. The second best time is right now.

Your Turn

Pick one task you did manually this week that you'll do again. Write a script for it.

Start with 10 lines. No fancy features. Just automate the basic flow.

Then run it tomorrow instead of doing it manually.

That's how you start.

Have you automated something recently? Share your scripts in the comments—I'd love to see what you're building!

And if this helped you save time, share it with a developer who's still doing everything manually. Let's spread the automation mindset.

Resources

Bash Guide for Beginners
ShellCheck - Lint your bash scripts
Bash Scripting Cheatsheet
My Scripts Repository - Fork and customize

Pro tip: Install shellcheck to catch errors:

brew install shellcheck  # macOS
sudo apt install shellcheck  # Linux

# Check your script
shellcheck script.sh

Happy automating! 🚀

File Ownership in Linux: Mastering chown, chgrp, and Recursive Changes

NJEI — Tue, 09 Dec 2025 11:52:33 +0000

File Ownership in Linux: Mastering chown, chgrp, and Recursive Changes

The Problem: Who Actually Owns This File?

You're setting up a web application. You deploy files as your user account, but the web server runs as www-data. Suddenly:

ls -l /var/www/html/index.html
-rw-r--r-- 1 ubuntu ubuntu 2048 Dec 09 10:00 index.html

The web server can't write to upload directories. Log files can't be created. The application breaks.

Or you're collaborating on a project, but files you create are locked to your user account. Your teammate can't edit them even though they should have access.

This is about ownership. Permissions (rwx) control what you can do. Ownership controls who can do it.

Understanding File Ownership

Every file in Linux has two owners:

User owner - A specific user account
Group owner - A specific group

When you run ls -l:

-rw-r--r-- 1 ubuntu developers 2048 Dec 09 10:00 file.txt
              │      │
              │      └─ Group owner
              └──────── User owner

This file is owned by:

User: ubuntu
Group: developers

Why does this matter?

Permissions (rwx) apply based on who you are:

If you're the user owner, you get the first set of permissions
If you're in the group owner, you get the second set
If you're neither, you get the third set (others)

The chown Command: Change User Ownership

chown (change owner) changes who owns a file.

Basic Syntax

chown new_user filename

Simple Example

# Check current ownership
ls -l script.sh
-rwxr-xr-x 1 ubuntu ubuntu 512 Dec 09 10:00 script.sh

# Change owner to 'john'
sudo chown john script.sh

# Verify change
ls -l script.sh
-rwxr-xr-x 1 john ubuntu 512 Dec 09 10:00 script.sh

Note: You need sudo to change ownership (only root can do this).

Real-World Scenario: Web Server Files

You deployed a website, but the web server (running as www-data) can't write to the uploads directory:

ls -ld /var/www/html/uploads
drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 09 10:00 uploads/

Fix it:

sudo chown www-data /var/www/html/uploads
ls -ld /var/www/html/uploads
drwxr-xr-x 2 www-data ubuntu 4096 Dec 09 10:00 uploads/

Now the web server can write to this directory.

The chgrp Command: Change Group Ownership

chgrp (change group) changes the group owner of a file.

Basic Syntax

chgrp new_group filename

Simple Example

# Check current ownership
ls -l project.txt
-rw-r--r-- 1 ubuntu ubuntu 1024 Dec 09 10:00 project.txt

# Change group to 'developers'
sudo chgrp developers project.txt

# Verify change
ls -l project.txt
-rw-r--r-- 1 ubuntu developers 1024 Dec 09 10:00 project.txt

Real-World Scenario: Team Collaboration

You're working on a shared project. Files should be editable by anyone in the developers group:

# Create shared directory
mkdir /shared/project

# Change group ownership
sudo chgrp developers /shared/project

# Give group write permissions
chmod 775 /shared/project

ls -ld /shared/project
drwxrwxr-x 2 ubuntu developers 4096 Dec 09 10:00 /shared/project/

Now anyone in the developers group can create and edit files here.

Changing Both User and Group with chown

You can change both owners at once using chown:

Syntax

chown user:group filename

Example

# Change both user and group
sudo chown www-data:www-data /var/www/html/config.php

ls -l /var/www/html/config.php
-rw-r--r-- 1 www-data www-data 512 Dec 09 10:00 config.php

This is faster than running chown and chgrp separately.

Change Only Group with chown

You can also change just the group using chown:

# Colon with only group name
sudo chown :developers file.txt

ls -l file.txt
-rw-r--r-- 1 ubuntu developers 1024 Dec 09 10:00 file.txt

This does the same thing as chgrp developers file.txt.

Recursive Ownership Changes: The -R Flag

Often you need to change ownership for entire directories and everything inside them.

The -R (Recursive) Flag

chown -R user:group directory/

This changes ownership for:

The directory itself
All files inside
All subdirectories
All files in subdirectories
Everything nested below

Example: Website Directory

# Check current ownership
ls -l /var/www/html/
drwxr-xr-x 5 ubuntu ubuntu 4096 Dec 09 10:00 assets/
-rw-r--r-- 1 ubuntu ubuntu 2048 Dec 09 10:00 index.html
drwxr-xr-x 3 ubuntu ubuntu 4096 Dec 09 10:00 uploads/

# Change everything to www-data
sudo chown -R www-data:www-data /var/www/html/

# Verify
ls -l /var/www/html/
drwxr-xr-x 5 www-data www-data 4096 Dec 09 10:00 assets/
-rw-r--r-- 1 www-data www-data 2048 Dec 09 10:00 index.html
drwxr-xr-x 3 www-data www-data 4096 Dec 09 10:00 uploads/

All files and directories now owned by www-data:www-data.

Real-World Scenario: Application Deployment

You cloned a repository as your user, but the application needs to run as appuser:

# Clone repo
git clone https://github.com/company/app.git /opt/app

# Everything is owned by you
ls -l /opt/app/
-rw-r--r-- 1 ubuntu ubuntu  156 Dec 09 10:00 config.yml
drwxr-xr-x 3 ubuntu ubuntu 4096 Dec 09 10:00 logs/
-rwxr-xr-x 1 ubuntu ubuntu 8192 Dec 09 10:00 start.sh

# Change ownership recursively
sudo chown -R appuser:appuser /opt/app/

# Verify
ls -l /opt/app/
-rw-r--r-- 1 appuser appuser  156 Dec 09 10:00 config.yml
drwxr-xr-x 3 appuser appuser 4096 Dec 09 10:00 logs/
-rwxr-xr-x 1 appuser appuser 8192 Dec 09 10:00 start.sh

Now the application can access all its files.

Combining Ownership and Permissions

Ownership and permissions work together. You often need to set both:

Example: Secure Application Directory

# Set ownership
sudo chown -R appuser:appuser /opt/app/

# Set permissions: owner full access, group read/execute
sudo chmod -R 750 /opt/app/

ls -ld /opt/app/
drwxr-x--- 5 appuser appuser 4096 Dec 09 10:00 /opt/app/

Result:

appuser has full control
Users in appuser group can read and execute
Others have no access

Example: Shared Team Directory

# Create directory
sudo mkdir /shared/team-project

# Set ownership
sudo chown ubuntu:developers /shared/team-project

# Set permissions: owner and group can write
sudo chmod 770 /shared/team-project

# New files inherit group (optional, advanced)
sudo chmod g+s /shared/team-project

ls -ld /shared/team-project
drwxrws--- 2 ubuntu developers 4096 Dec 09 10:00 /shared/team-project/

Now anyone in developers group can collaborate.

Common Patterns and Use Cases

Pattern 1: Web Server Files

# HTML/CSS/JS files
sudo chown -R www-data:www-data /var/www/html/
sudo find /var/www/html -type f -exec chmod 644 {} \;
sudo find /var/www/html -type d -exec chmod 755 {} \;

Pattern 2: Application with Log Directory

# Application files
sudo chown -R appuser:appuser /opt/app/

# Logs directory needs write access
sudo chown -R appuser:appuser /opt/app/logs/
sudo chmod -R 755 /opt/app/logs/

Pattern 3: Database Files

# PostgreSQL data directory
sudo chown -R postgres:postgres /var/lib/postgresql/
sudo chmod -R 700 /var/lib/postgresql/

Pattern 4: SSH Configuration

# User's SSH directory
sudo chown -R ubuntu:ubuntu /home/ubuntu/.ssh/
sudo chmod 700 /home/ubuntu/.ssh/
sudo chmod 600 /home/ubuntu/.ssh/id_rsa

Common Mistakes

Mistake #1: Forgetting sudo

# This fails unless you're root
chown newuser file.txt
# chown: changing ownership of 'file.txt': Operation not permitted

# Need sudo
sudo chown newuser file.txt

Mistake #2: Using -R on the wrong directory

# DANGEROUS - changes system files
sudo chown -R ubuntu:ubuntu /

# Be specific
sudo chown -R ubuntu:ubuntu /home/ubuntu/project/

Mistake #3: Wrong syntax for user:group

# Wrong - space between user and group
sudo chown user group file.txt

# Right - colon, no spaces
sudo chown user:group file.txt

Mistake #4: Changing ownership without checking permissions

# Change owner
sudo chown www-data file.txt

# But file isn't readable by owner!
ls -l file.txt
--wx-wx-wx 1 www-data www-data 512 Dec 09 10:00 file.txt

# Fix permissions too
sudo chmod 644 file.txt

Checking Current Ownership

View detailed ownership

ls -l filename
stat filename  # Even more details

Find files by owner

# Find all files owned by ubuntu
find /home -user ubuntu

# Find all files owned by www-data group
find /var/www -group www-data

Find files with specific ownership

# Find files where user and group don't match
find /var/www -not -user www-data -o -not -group www-data

Special Case: Changing Ownership of Symbolic Links

By default, chown changes the target of a symbolic link, not the link itself.

# Create a link
ln -s /path/to/file.txt link.txt

# This changes the target file's ownership
sudo chown newuser link.txt

To change the link itself (rarely needed):

sudo chown -h newuser link.txt

Reference vs Non-Reference

You can reference another file's ownership:

# Copy ownership from reference.txt to target.txt
sudo chown --reference=reference.txt target.txt

This is useful in scripts where you want to match existing ownership.

Quick Reference

Basic Commands

# Change user owner
sudo chown newuser file.txt

# Change group owner
sudo chgrp newgroup file.txt

# Change both
sudo chown user:group file.txt

# Change only group (with chown)
sudo chown :group file.txt

# Recursive change
sudo chown -R user:group directory/

Common Patterns

Use Case	Command
Web server files	`sudo chown -R www-data:www-data /var/www/`
Application deploy	`sudo chown -R appuser:appuser /opt/app/`
User's home directory	`sudo chown -R ubuntu:ubuntu /home/ubuntu/`
Shared team directory	`sudo chown user:developers /shared/project/`
Database files	`sudo chown -R postgres:postgres /var/lib/postgresql/`

Verification

# Check ownership
ls -l file.txt

# Detailed info
stat file.txt

# Find by owner
find /path -user username
find /path -group groupname

Advanced: Who Can Change Ownership?

Only root (or users with sudo privileges) can change file ownership. This is a security feature.

Why?

If regular users could change ownership, they could:

Give files to other users without permission
Bypass disk quotas
Frame other users for their actions

Working with Groups

View your groups

groups
# ubuntu adm cdrom sudo dip plugdev developers

View a user's groups

groups username

Add user to group

sudo usermod -aG groupname username
# User must log out and back in for this to take effect

Key Takeaways

Every file has two owners: User and group
chown changes user ownership: sudo chown user file
chgrp changes group ownership: sudo chgrp group file
Change both at once: sudo chown user:group file
Use -R for directories: sudo chown -R user:group dir/
Always need sudo: Only root can change ownership
Combine with permissions: Ownership + chmod = complete access control
Be careful with -R: Double-check your path

Ownership is the foundation of Linux security. Master it alongside permissions, and you'll have complete control over file access on your systems.

What ownership pattern do you use most in your workflow? Share your use cases in the comments.

I Looked Inside .git and You Won't Believe What I Found (Actually, You Will,cos It's Fascinating)

NJEI — Mon, 08 Dec 2025 11:42:12 +0000

We've all done it a thousand times:

git init

And just like that, a .git directory appears. We add files, commit changes, push to remote—all without ever peeking inside that folder. It's the black box we trust but never examine.

Until today.

Last Tuesday, I was debugging a corrupted repository (don't ask), and I had no choice but to venture into the .git directory. What I found wasn't chaos or magic—it was a beautifully organized system that suddenly made everything about Git make sense.

Let me show you what's really in there.

The Moment of Truth: Opening .git
The Directory Structure
HEAD: Your Location Marker
refs/: Where Branches Live
objects/: Git's Time Machine
- What's a Git Object?
- The Three Types of Objects
- Following the Chain
index: The Staging Area Revealed
config: Your Repository Settings
hooks/: Automation Paradise
logs/: Git's Diary
The Files Git Doesn't Want You to See
A Real-World Scenario: Following a Commit
Why This Matters
Commands to Explore Safely
Key Takeaways

The Moment of Truth: Opening .git

Here's what most developers think .git contains: "Uh... git stuff?"

Let me show you what's actually in there. Open any git repository and run:

ls -la .git/

You'll see something like this:

.git/
├── HEAD
├── config
├── description
├── hooks/
├── index
├── info/
├── objects/
├── refs/
└── logs/

Each piece has a purpose. Each file tells a story. Let's decode them one by one.

The Directory Structure

Before we dive deep, here's the map:

Path	What It Does
`HEAD`	Points to your current branch
`config`	Repository-specific settings
`description`	Used by GitWeb (rarely used)
`hooks/`	Scripts that run on git events
`index`	Your staging area (binary file)
`info/`	Global exclude patterns
`objects/`	All your data lives here
`refs/`	Pointers to commits (branches, tags)
`logs/`	History of ref changes (reflog)

The two most important directories? objects/ and refs/. Everything else is supporting infrastructure.

HEAD: Your Location Marker

Let's start simple. Check what's in HEAD:

cat .git/HEAD

Output:

ref: refs/heads/main

That's it. That's the file. HEAD is just a pointer telling Git which branch you're on. When you run git checkout feature-branch, Git rewrites this file:

ref: refs/heads/feature-branch

When you checkout a specific commit (detached HEAD state), it changes to:

a3f2d8b9c1e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8

Mind-blowing realization #1: Your "current branch" is just a text file containing a reference.

refs/: Where Branches Live

Look inside the refs directory:

tree .git/refs/

.git/refs/
├── heads/
│   ├── main
│   ├── feature-auth
│   └── bugfix-login
├── remotes/
│   └── origin/
│       ├── main
│       └── develop
└── tags/
    └── v1.0.0

Each branch is just a file containing a commit hash. Let's prove it:

cat .git/refs/heads/main

Output:

f3e4d5c6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2

That's your latest commit on main. That's literally what a branch is—a 40-character hash pointing to a commit.

When you create a new branch:

git branch feature-new

Git creates .git/refs/heads/feature-new and copies the current commit hash into it. That's it. No magic.

Mind-blowing realization #2: Branches are just files with commit hashes. Creating a branch is almost free because it's just writing 40 bytes to a file.

objects/: Git's Time Machine

This is where the real magic happens. The objects/ directory is Git's database—every commit, every file, every version you've ever created lives here.

ls .git/objects/

00/  0f/  1a/  2b/  3c/  4d/  5e/  6f/  7a/  8b/  9c/  ...
info/  pack/

Those two-character directories? They're organizing objects by their hash prefix (for performance). Let's look deeper:

ls .git/objects/a3/

f2d8b9c1e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8

Combine the directory name with the filename: a3f2d8b9c1e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8

That's a complete SHA-1 hash. But what is it?

What's a Git Object?

Git stores everything as objects. There are only four types:

Blob - File contents
Tree - Directory structure
Commit - Snapshot in time
Tag - Named reference (annotated tags)

You can inspect any object:

git cat-file -t a3f2d8b9  # Type
git cat-file -p a3f2d8b9  # Content

The Three Types of Objects

Let's see them in action. I'll create a simple file and commit it:

echo "Hello Git" > test.txt
git add test.txt
git commit -m "Add test file"

Now let's follow the trail.

1. The Commit Object

git cat-file -p HEAD

Output:

tree 5e8b9c0d1a2b3c4d5e6f7a8b9c0d1e2f3a4b5c6
parent f3e4d5c6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2
author John Doe <john@example.com> 1733654400 +0000
committer John Doe <john@example.com> 1733654400 +0000

Add test file

The commit points to a tree (the directory structure) and has metadata (author, timestamp, message).

2. The Tree Object

git cat-file -p 5e8b9c0d1a2b

Output:

100644 blob a3f2d8b9c1e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8    test.txt
100644 blob 7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6    README.md
040000 tree b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b7    src/

The tree lists files (blobs) and subdirectories (trees). It's like a snapshot of ls -l.

3. The Blob Object

git cat-file -p a3f2d8b9c1e4

Output:

Hello Git

The blob is just the file contents. No filename, no metadata—pure content.

Mind-blowing realization #3: Git doesn't store diffs. It stores complete snapshots. Every commit points to a full tree of all files.

Following the Chain

Here's how it connects:

HEAD
  ↓
refs/heads/main (contains: f3e4d5c6...)
  ↓
Commit f3e4d5c6...
  ├── tree 5e8b9c0d...
  │     ├── blob a3f2d8b9... (test.txt)
  │     ├── blob 7a8b9c0d... (README.md)
  │     └── tree b9c0d1e2... (src/)
  └── parent e2f3a4b5... (previous commit)

When you run git log, Git walks this chain backwards through parent commits.

index: The Staging Area Revealed

The staging area isn't virtual—it's a binary file:

file .git/index

Output:

.git/index: Git index, version 2, 3 entries

When you run git add, Git:

Creates a blob object for the file
Updates the index to point to that blob
Doesn't create a commit yet

You can peek inside (it's binary, but git can read it):

git ls-files --stage

Output:

100644 a3f2d8b9c1e4f5a6b7c8d9e0f1a2b3c4d5e6f7a8 0   test.txt
100644 7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6 0    README.md

Each entry shows: permissions, blob hash, stage number, filename.

Mind-blowing realization #4: git add creates the object immediately. The staging area is just a list of "these blobs should be in the next commit."

config: Your Repository Settings

Open .git/config:

cat .git/config

[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true

[remote "origin"]
    url = git@github.com:username/repo.git
    fetch = +refs/heads/*:refs/remotes/origin/*

[branch "main"]
    remote = origin
    merge = refs/heads/main

[user]
    name = John Doe
    email = john@example.com

This is where git config --local writes settings. Global settings live in ~/.gitconfig, but repository-specific overrides live here.

Want to change your email for just this project?

git config user.email "work@company.com"

Check the file—it's been updated.

hooks/: Automation Paradise

The hooks/ directory contains scripts that run automatically on git events:

ls .git/hooks/

applypatch-msg.sample
pre-commit.sample
pre-push.sample
prepare-commit-msg.sample
post-commit.sample
...

Remove the .sample extension to activate a hook. For example, create .git/hooks/pre-commit:

#!/bin/bash
# Run tests before every commit

npm test
if [ $? -ne 0 ]; then
    echo "Tests failed! Commit aborted."
    exit 1
fi

Make it executable:

chmod +x .git/hooks/pre-commit

Now every git commit runs your tests first. If they fail, the commit is blocked.

Popular hooks:

pre-commit - Lint code, run tests
commit-msg - Enforce commit message format
pre-push - Run full test suite before pushing
post-merge - Install dependencies after pulling

Mind-blowing realization #5: Git is programmable. You can automate almost anything.

logs/: Git's Diary

The logs/ directory tracks every change to refs:

cat .git/logs/HEAD

0000000 a3f2d8b John Doe <john@example.com> 1733654400 +0000    commit (initial): Initial commit
a3f2d8b f3e4d5c John Doe <john@example.com> 1733654500 +0000    commit: Add test file
f3e4d5c e2f3a4b John Doe <john@example.com> 1733654600 +0000    commit: Update README

This is what powers git reflog—a safety net that remembers where you've been, even if you've deleted branches or reset commits.

Accidentally deleted a branch?

git reflog
# Find the commit hash before you deleted it
git checkout -b recovered-branch a3f2d8b

The reflog saved you.

The Files Git Doesn't Want You to See

Some files are auto-generated and usually ignored:

COMMIT_EDITMSG - Last commit message (for re-editing)
FETCH_HEAD - Last fetched branch reference
ORIG_HEAD - Backup of HEAD before risky operations
MERGE_HEAD - Commit being merged (during conflicts)
packed-refs - Compressed version of refs (for performance)

These files help Git track state during operations. They're temporary and rebuilt as needed.

A Real-World Scenario: Following a Commit

Let's put it all together. You run:

git commit -m "Fix login bug"

Here's what Git does behind the scenes:

Creates blob objects for each file in the index
Creates a tree object representing the directory structure
Creates a commit object pointing to:
- The tree
- The parent commit
- Author metadata
- Commit message
Updates the branch ref (.git/refs/heads/main) to point to the new commit
Updates HEAD (if needed)
Logs the change in .git/logs/refs/heads/main

All of this happens in milliseconds. And it's just file operations—no database, no network calls.

Why This Matters

Understanding .git changes how you use Git:

Before: "I'll just run these commands and hope it works."

After:

"I need to recover a deleted branch—I'll check reflog."
"I want to automate linting—I'll use pre-commit hooks."
"This merge conflict makes sense—Git is comparing tree objects."
"Branches are cheap—they're literally just pointers."

You stop fearing Git and start leveraging it.

Commands to Explore Safely

Want to dig deeper? These commands are read-only and safe:

# List all objects
find .git/objects -type f

# Show object type
git cat-file -t <hash>

# Show object content
git cat-file -p <hash>

# Show staging area
git ls-files --stage

# Show commit graph
git log --graph --oneline --all

# Show reflog
git reflog

# Show config
git config --list --show-origin

# Verify repository integrity
git fsck

Explore freely. You can't break anything by reading.

Key Takeaways

Git is just files and directories - No magic, no mystery
Branches are pointers - Creating them is nearly free
Objects are immutable - Once created, they never change
The staging area is real - It's .git/index
Everything is recoverable - Thanks to reflog and objects
Git is hackable - Hooks let you automate workflows
Commits store snapshots - Not diffs

The next time you run git init, remember: you're not just initializing a repository. You're creating a time machine, a database, and an automation system—all stored in plain files.

Have you ever peeked inside .git? What did you find? Drop a comment below—I'd love to hear your discoveries.

And if this helped demystify Git for you, share it with a fellow developer who's still treating .git like a black box. Let's spread the knowledge.

Want to go deeper? Check out Git Internals documentation or run git help hooks to see all available hooks.

Linux File Permissions

NJEI — Mon, 08 Dec 2025 11:02:35 +0000

Linux File Permissions Demystified: Understanding rwx and chmod

The Problem: Access Denied

You created a script. You try to run it: ./deploy.sh

bash: ./deploy.sh: Permission denied

Or you try to edit a config file:

vim /etc/nginx/nginx.conf
# "E212: Can't open file for writing"

This is Linux permissions in action. Understanding how they work isn't optional—it's essential for managing servers, deploying code, and troubleshooting access issues.

The Three Permissions: Read, Write, Execute

Every file and directory has three types of permissions:

Read (r) - View file contents or list directory contents
Write (w) - Modify file or create/delete files in directory
Execute (x) - Run file as program or enter directory

The Three Actors: User, Group, Others

Permissions apply to three categories of people:

User (u) - The file owner (whoever created it)
Group (g) - Users in the file's group
Others (o) - Everyone else on the system

When you run ls -l, you see these permissions:

-rwxr-xr--  1 ubuntu developers  156 Dec 08 10:00 script.sh
 ││││││││
 │││││││└─ Others: r-- (read only)
 ││││││└── Group: r-x (read and execute)
 │││││└─── User: rwx (read, write, execute)
 ││││└──── File type: - (regular file)

Breaking it down:

User (owner): rwx - Can read, write, and execute
Group: r-x - Can read and execute (not write)
Others: r-- - Can only read

Method 1: Using Letters with chmod

The chmod command changes permissions. The letter method is intuitive and readable.

Basic Syntax

chmod [who][operation][permission] filename

Who: u (user), g (group), o (others), a (all)
Operation: + (add), - (remove), = (set exactly)
Permission: r (read), w (write), x (execute)

Adding Permissions

Give read permission to everyone:

chmod ugo+r file.txt
# Or shorter:
chmod a+r file.txt

Make a script executable for the owner:

chmod u+x script.sh

Give write permission to group:

chmod g+w config.yml

Removing Permissions

Remove write permission from owner:

chmod u-w filename

Remove read permission from group:

chmod g-r filename

Remove execute from others:

chmod o-x script.sh

Setting Exact Permissions

Set permissions explicitly (replaces existing):

chmod u=rwx,g=rx,o=r file.txt
# User: read, write, execute
# Group: read, execute
# Others: read only

Real-World Example: Making a Script Executable

You download a deployment script:

ls -l deploy.sh
-rw-r--r--  1 ubuntu ubuntu  256 Dec 08 10:00 deploy.sh

Try to run it:

./deploy.sh
# Permission denied

The file isn't executable. Fix it:

chmod u+x deploy.sh

ls -l deploy.sh
-rwxr--r--  1 ubuntu ubuntu  256 Dec 08 10:00 deploy.sh
# Now it has x for user

Run it:

./deploy.sh
# Works!

Method 2: Using Numbers with chmod

Numbers are faster to type and commonly used in documentation. They might look cryptic, but they follow a logical pattern.

The Three-Digit System

chmod 755 file.txt
       │││
       ││└─ Others permissions
       │└── Group permissions
       └─── User permissions

Each digit represents permissions for one actor:

First digit: User (owner)
Second digit: Group
Third digit: Others

How to Calculate the Numbers

Each permission has a value:

Read (r) = 4
Write (w) = 2
Execute (x) = 1

Add them together to get the digit for each actor:

Number	Calculation	Permissions	Meaning
0	0	`---`	No permissions
1	1	`--x`	Execute only
2	2	`-w-`	Write only
3	2+1	`-wx`	Write and execute
4	4	`r--`	Read only
5	4+1	`r-x`	Read and execute
6	4+2	`rw-`	Read and write
7	4+2+1	`rwx`	Read, write, and execute

Common Permission Patterns

755 - Standard for executable files:

chmod 755 script.sh
# User: 7 (rwx) - read, write, execute
# Group: 5 (r-x) - read, execute
# Others: 5 (r-x) - read, execute

Result: -rwxr-xr-x

644 - Standard for regular files:

chmod 644 document.txt
# User: 6 (rw-) - read, write
# Group: 4 (r--) - read only
# Others: 4 (r--) - read only

Result: -rw-r--r--

700 - Private file, owner only:

chmod 700 private-key.pem
# User: 7 (rwx) - read, write, execute
# Group: 0 (---) - no access
# Others: 0 (---) - no access

Result: -rwx------

600 - Private file, not executable:

chmod 600 secrets.txt
# User: 6 (rw-) - read, write
# Group: 0 (---) - no access
# Others: 0 (---) - no access

Result: -rw-------

444 - Read-only for everyone:

chmod 444 readonly.txt
# User: 4 (r--) - read only
# Group: 4 (r--) - read only
# Others: 4 (r--) - read only

Result: -r--r--r--

Converting Between Letters and Numbers

Let's say you see: -rwxr-xr--

Convert to numbers:

User: rwx = 4+2+1 = 7
Group: r-x = 4+0+1 = 5
Others: r-- = 4+0+0 = 4

Result: 754

Or going the other way, chmod 640 file.txt:

User: 6 = 4+2 = rw-
Group: 4 = 4 = r--
Others: 0 = 0 = ---

Result: -rw-r-----

Real-World Scenarios

Scenario 1: Securing SSH Keys

SSH keys must have strict permissions or SSH refuses to use them:

ls -l ~/.ssh/id_rsa
-rw-rw-r-- 1 ubuntu ubuntu 1679 Dec 08 10:00 /home/ubuntu/.ssh/id_rsa

SSH will reject this (too open). Fix it:

chmod 600 ~/.ssh/id_rsa

ls -l ~/.ssh/id_rsa
-rw------- 1 ubuntu ubuntu 1679 Dec 08 10:00 /home/ubuntu/.ssh/id_rsa
# Now SSH accepts it

Scenario 2: Web Server Files

You're setting up a website. Files need to be readable by the web server but not writable:

# HTML/CSS/JS files
chmod 644 *.html *.css *.js
# Result: -rw-r--r-- (owner can edit, everyone can read)

# Upload directory needs write access
chmod 755 uploads/
# Result: drwxr-xr-x (owner can write, everyone can read/enter)

Scenario 3: Shared Team Directory

Your team needs to collaborate on files:

# Create shared directory
mkdir /shared/project

# Set permissions: owner full access, group can read/write
chmod 770 /shared/project
# Result: drwxrwx--- (owner and group full access, others none)

# Files created inside inherit group access
chmod g+w /shared/project/*

Scenario 4: Protecting Configuration Files

Config files often contain sensitive data:

# Database config
chmod 640 database.conf
# Result: -rw-r----- (owner can edit, group can read, others nothing)

# Even more restrictive
chmod 600 secrets.env
# Result: -rw------- (only owner can access)

Combining Both Methods

You can use letters and numbers together in your workflow:

# Set base permissions with numbers (fast)
chmod 644 file.txt

# Add execute for user with letters (clear intent)
chmod u+x file.txt

# Result: -rwxr--r--

Common Mistakes

Mistake #1: Making everything 777

# NEVER do this
chmod 777 file.txt
# Everyone can read, write, execute - major security risk

Only use 777 in very specific debugging scenarios, and remove it immediately after.

Mistake #2: Forgetting the leading 0 for octal

In scripts, you might see:

chmod 0755 script.sh

The leading 0 indicates octal notation. Both work, but 0755 is more explicit.

Mistake #3: Wrong permissions on SSH directories

SSH is picky:

# SSH directory structure
chmod 700 ~/.ssh/              # Directory
chmod 600 ~/.ssh/id_rsa        # Private key
chmod 644 ~/.ssh/id_rsa.pub    # Public key
chmod 644 ~/.ssh/authorized_keys  # Authorized keys

Mistake #4: Not checking current permissions first

Always check before changing:

ls -l filename  # Check current permissions
chmod 644 filename  # Then change
ls -l filename  # Verify the change

Quick Reference

Letter Method Cheat Sheet

chmod u+x file     # Add execute for user
chmod g-w file     # Remove write from group
chmod o+r file     # Add read for others
chmod a+x file     # Add execute for all
chmod u=rwx file   # Set user to rwx exactly
chmod go-rwx file  # Remove all permissions from group and others

Number Method Cheat Sheet

Permission	Number	Letter	Use Case
`rwxrwxrwx`	777	-	Avoid (security risk)
`rwxr-xr-x`	755	Scripts	Executable files
`rwxr-x---`	750	-	Group executable
`rwx------`	700	-	Private executable
`rw-rw-rw-`	666	-	Avoid (security risk)
`rw-rw-r--`	664	-	Shared editable file
`rw-r--r--`	644	Documents	Standard files
`rw-r-----`	640	Configs	Readable by group
`rw-------`	600	Keys	Private files
`r--r--r--`	444	-	Read-only for all

Advanced: Recursive Permissions

Change permissions for all files in a directory:

# Change all files and subdirectories
chmod -R 755 /var/www/html/

# Be careful with -R - it affects everything below

Better approach - set different permissions for files vs directories:

# Directories: 755 (rwxr-xr-x)
find /var/www/html -type d -exec chmod 755 {} \;

# Files: 644 (rw-r--r--)
find /var/www/html -type f -exec chmod 644 {} \;

Checking Permissions

View permissions:

ls -l filename
stat filename  # More detailed info

Test if you can read a file:

test -r filename && echo "Readable"

Test if you can write:

test -w filename && echo "Writable"

Test if you can execute:

test -x filename && echo "Executable"

Key Takeaways

Three permissions: Read (4), Write (2), Execute (1)
Three actors: User, Group, Others
Letter method is readable: chmod u+x file
Number method is faster: chmod 755 file
Common patterns: 644 (files), 755 (scripts), 600 (private)
Add permissions to calculate: 4+2+1=7 (rwx)
Always verify: ls -l before and after changes
Never use 777: Major security risk

Understanding permissions is fundamental to Linux. Whether you prefer letters (readable) or numbers (fast), both methods give you precise control over who can access your files.

What permission pattern do you use most? Share your use cases in the comments.

Mastering Wildcards in Linux: *, ?, and [] for Efficient File Management

NJEI — Mon, 08 Dec 2025 08:22:36 +0000

The Problem

You need to delete 50 log files. Do you run rm 50 times? Move 100 images—one by one?

Wildcards solve this. They let you match multiple files with a single pattern.

What Are Wildcards?

Wildcards are special characters that represent patterns in filenames.

The three main wildcards:

* - Matches zero or more characters
? - Matches exactly one character
[] - Matches a range of characters

The Asterisk (*): Match Everything

The * matches zero or more characters.

Basic Usage

# List all files starting with "abc"
ls abc*
# Matches: abc, abc123, abc-file.txt, abcdefg

# Delete all .tmp files
rm *.tmp
# Matches: data.tmp, backup.tmp, cache.tmp

# Find files containing "xy"
rm *xy*
# Matches: xyfile, file-xy-data, xy, data-xy.txt

Real-World: Cleaning Log Files

You have hundreds of logs in /var/log/app/:

app-2024-11-01.log
app-2024-11-02.log
...
app-2024-11-30.log

Delete all November logs:

rm /var/log/app/app-2024-11-*.log

Compress before archiving:

gzip /var/log/app/app-2024-11-*.log

The Question Mark (?): Match One Character

The ? matches exactly one character—no more, no less.

Basic Usage

# Match files with one char before "abc-xyz"
ls ?abc-xyz*
# Matches: 1abc-xyz, 2abc-xyz, xabc-xyz
# NOT: abc-xyz, 10abc-xyz

# Match single-character files
ls ?
# Matches: a, 1, x
# NOT: ab, 123

# Match specific patterns
ls file-?.txt
# Matches: file-1.txt, file-a.txt
# NOT: file-10.txt, file.txt

Real-World: Finding Backup Versions

Your backups:

backup-1.tar.gz
backup-2.tar.gz
...
backup-9.tar.gz
backup-10.tar.gz

List only single-digit backups:

ls backup-?.tar.gz
# Shows: backup-1.tar.gz through backup-9.tar.gz
# Excludes: backup-10.tar.gz

Square Brackets ([]): Match Ranges

The [] matches any single character within the brackets.

Basic Usage

# Match files starting with "c" or "d"
ls -l [cd]*
# Matches: cat.txt, dog.log, config.yml
# NOT: apple.txt, bat.sh

# Match files starting with any digit
ls [0-9]*
# Matches: 1file.txt, 2data.log, 9report.pdf
# NOT: file1.txt (doesn't start with digit)

# Match specific characters
ls file[123].txt
# Matches: file1.txt, file2.txt, file3.txt
# NOT: file4.txt

# Match letter ranges
ls [a-f]*
# Matches files starting with: a, b, c, d, e, f

Real-World: Organizing Project Files

Your project structure:

config-dev.yml
config-prod.yml
config-test.yml
data-dev.csv
data-prod.csv
data-test.csv

List only dev and test files (not production):

ls *-[dt]*
# Matches: config-dev.yml, config-test.yml, data-dev.csv, data-test.csv
# Excludes: config-prod.yml, data-prod.csv

Brace Expansion ({}): Create Multiple Files

Not a wildcard, but incredibly useful.

Basic Usage

# Create 9 files
touch abc{1..9}-xyz
# Creates: abc1-xyz, abc2-xyz, ..., abc9-xyz

# Create files with different extensions
touch file.{txt,log,bak}
# Creates: file.txt, file.log, file.bak

# Create numbered directories
mkdir project-{001..010}
# Creates: project-001, project-002, ..., project-010

Real-World: Test Environment Setup

# Create 100 test CSV files
touch test-data-{001..100}.csv

# Create directory structure
mkdir -p {dev,test,prod}/{logs,config,data}
# Creates: dev/logs, dev/config, dev/data, test/logs, etc.

Special Characters

Backslash (): Escape Characters

Match files with wildcard characters in their names:

# Match file literally named "file*.txt"
ls file\*.txt
# Matches: file*.txt (actual filename with asterisk)

# Match files with spaces
ls my\ file.txt
# Or use quotes
ls "my file.txt"

Dollar Sign ($): Variables

Reference variables:

# Use variable in command
USER_DIR=/home/ubuntu
ls $USER_DIR/*.txt
# Expands to: ls /home/ubuntu/*.txt

# Current user's home
ls $HOME/documents
# Same as: ls ~/documents

Using `man` for Documentation

Every command has a manual:

man ls
# Shows complete documentation

Navigate man pages:

Space - Scroll down
b - Scroll up
/ - Search
q - Quit

Quick help:

ls --help
# Brief usage summary

Combining Wildcards

Wildcards are powerful when combined:

# Match all .log files from November or December
ls app-2024-1[12]-*.log
# Matches: app-2024-11-01.log, app-2024-12-15.log

# Delete temp files with single-char extensions
rm temp-*.?
# Matches: temp-data.1, temp-file.x
# NOT: temp-data.txt (3 chars)

# Copy specific backup files
cp backup-[1-5]?.tar.gz /archive/
# Matches: backup-10.tar.gz through backup-59.tar.gz

Common Mistakes

❌ Mistake #1: Not quoting wildcards with find

# Wrong - shell expands * before find sees it
find . -name *.txt

# Right - quote to pass literal * to find
find . -name "*.txt"

❌ Mistake #2: Using * when you need ?

# Too broad
ls file-*.txt
# Matches: file-1.txt, file-10.txt, file-abc.txt

# More specific
ls file-?.txt
# Matches only: file-1.txt, file-a.txt

❌ Mistake #3: Not testing before deleting

# DANGEROUS - always check first
ls *.tmp  # Verify what will be deleted
rm *.tmp  # Then delete

# Or use -i flag for confirmation
rm -i *.tmp

Practical Examples

Clean build artifacts

# Remove compiled files
rm *.o *.class *.pyc

# Remove all files in current directory
rm ./*

Find and process files

# Count lines in all Python files
wc -l *.py

# Search in all config files
grep "database" *.conf

# Copy all images to backup
cp *.{jpg,png,gif} /backup/images/

Batch rename

# Rename .txt to .bak
for file in *.txt; do
    mv "$file" "${file%.txt}.bak"
done

Quick Reference

Wildcard	Matches	Example	Result
`*`	Zero or more chars	`abc*`	abc, abc1, abc-file
`?`	Exactly one char	`?abc`	1abc, xabc
`[]`	Char in brackets	`[abc]*`	apple, bat, cat
`[a-z]`	Range	`[a-z]*`	lowercase start
`[0-9]`	Digits	`file[0-9].txt`	file0.txt-file9.txt
`{}`	Brace expansion	`{1..5}`	1 2 3 4 5

Advanced Patterns

Exclude patterns

# List all files except .txt
ls !(*.txt)
# Requires: shopt -s extglob

Multiple wildcards

# Match complex patterns
ls [a-z]*[0-9].{log,txt}
# Matches: app1.log, data5.txt, test9.log

Case-insensitive matching

# Enable case-insensitive globbing
shopt -s nocaseglob
ls *.TXT  # Also matches .txt, .Txt, .TxT

Performance Tips

Be specific

# Slower - searches everywhere
ls *file*

# Faster - more specific pattern
ls file-*.log

Use quotes when needed

# Shell expands first (can be slow with many files)
echo *

# Faster for large directories
echo "*"

Key Takeaways

* matches zero or more - Broadest pattern
? matches exactly one - Precise matching
[] matches ranges - Specific character sets
{} creates multiple items - File creation tool
Test with ls before rm - Always verify
Quote when needed - Prevent unwanted expansion
Read man pages - Detailed documentation

Wildcards turn repetitive operations into single commands. Master these and you'll work exponentially faster.

What's your most useful wildcard pattern? Share in the comments!

Follow for more practical Linux tips as I document my journey into cloud engineering and DevOps.

Understanding Linux File System: ls -ltr Decoded and Directory Structure Explained

NJEI — Sat, 06 Dec 2025 20:03:56 +0000

The Problem

You run ls and see filenames. But is it a file or folder? Who owns it? Can you even edit it?

On day two with Linux, I learned ls can tell you way more than just names.

The Command: `ls -ltr`

ls -ltr

What the flags mean:

-l = Long format (detailed info)
-t = Sort by time
-r = Reverse (newest at bottom)

Terminal Output Example

drwxr-xr-x  2 ubuntu ubuntu    4096 Dec 01 10:23 documents
-rw-r--r--  1 ubuntu ubuntu     156 Dec 02 14:45 config.txt
lrwxrwxrwx  1 ubuntu ubuntu      15 Dec 03 09:12 link_to_file -> /home/ubuntu/file
-rwxr-xr-x  1 ubuntu ubuntu    8192 Dec 04 16:30 script.sh
drwxr-xr-x  3 root   root      4096 Dec 05 11:20 logs
crw-rw----  1 root   tty     136, 0 Dec 06 08:15 tty0
brw-rw----  1 root   disk      8, 0 Dec 06 10:00 sda
srwxrwxrwx  1 root   root         0 Dec 04 12:30 mysql.sock

Let's decode this.

Reading the Output

Column 1: File Type + Permissions

drwxr-xr-x
│└────┬────└─── Others (r-x)
│     └──────── Group (r-x)
│           └── Owner (rwx)
└────────────── File Type (d)

First Character = File Type:

Symbol	Type	Description
`d`	Directory	It's a folder
`-`	Regular file	Normal file
`l`	Symbolic link	Shortcut to another file
`c`	Character device	Keyboard, terminal
`b`	Block device	Hard drive, USB
`s`	Socket	Inter-process communication
`p`	Named pipe	Process communication

Next 9 Characters = Permissions:

r = Read
w = Write
x = Execute
- = No permission

Example: rwxr-xr-x

Owner: read, write, execute
Group: read, execute (no write)
Others: read, execute (no write)

Columns 2-9: The Details

-rw-r--r--  1 ubuntu ubuntu  156 Dec 02 14:45 config.txt
│           │ │      │       │   │            │
│           │ │      │       │   │            └─ Filename
│           │ │      │       │   └────────────── Last modified
│           │ │      │       └────────────────── Size (bytes)
│           │ │      └────────────────────────── Group owner
│           │ └───────────────────────────────── User owner
│           └─────────────────────────────────── Number of links
└─────────────────────────────────────────────── Type + Permissions

File Types in Detail

Regular Files (-)

-rw-r--r--  1 ubuntu ubuntu  156 Dec 02 14:45 config.txt

Your typical files: text, images, binaries, scripts.

Directories (d)

drwxr-xr-x  2 ubuntu ubuntu 4096 Dec 01 10:23 documents

Folders. Note: size shows metadata, not actual contents.

Symbolic Links (l)

lrwxrwxrwx  1 ubuntu ubuntu   15 Dec 03 09:12 link -> /home/file

Like shortcuts. If original file is deleted, link breaks.

Character Devices (c)

crw-rw----  1 root tty  136, 0 Dec 06 08:15 tty0

Transfer data character by character. Terminals, keyboards. Found in /dev/.

Block Devices (b)

brw-rw----  1 root disk  8, 0 Dec 06 10:00 sda

Transfer data in blocks. Hard drives, USB drives. In /dev/.

Sockets (s)

srwxrwxrwx  1 root root     0 Dec 04 12:30 mysql.sock

For inter-process communication. Usually in /var/run/ or /tmp/.

Named Pipes (p)

prw-r--r--  1 ubuntu ubuntu  0 Dec 03 15:20 mypipe

Allow processes to communicate. Less common.

Linux Directory Structure

Everything starts from / (root):

/                           Root directory
├── bin/                    Essential commands (ls, cp, cat)
├── boot/                   Bootloader, kernel
├── dev/                    Device files
├── etc/                    Configuration files
├── home/                   User directories
│   └── ubuntu/             Your home directory
├── lib/                    Shared libraries
├── media/                  Removable media mount points
├── mnt/                    Temporary mounts
├── opt/                    Optional software
├── proc/                   Process information (virtual)
├── root/                   Root user's home
├── run/                    Runtime process data
├── sbin/                   System admin commands
├── srv/                    Service data (web, FTP)
├── sys/                    Kernel/hardware info (virtual)
├── tmp/                    Temp files (cleared on reboot)
├── usr/                    User programs
│   ├── bin/                User commands
│   ├── lib/                Program libraries
│   └── local/              Locally installed software
└── var/                    Variable data
    ├── log/                Log files
    ├── www/                Web server files
    └── tmp/                Temp files (preserved on reboot)

Key Directories

Directory	Purpose	Example Use
`/home/`	Your personal files	`/home/ubuntu/documents/`
`/etc/`	System configuration	`/etc/nginx/nginx.conf`
`/var/log/`	Log files	`/var/log/syslog`
`/tmp/`	Temporary files	Cleared on reboot
`/dev/`	Device files	`/dev/sda` (hard drive)
`/usr/bin/`	Programs	`/usr/bin/python3`

Practical Commands

Basic listing

ls -ltr

Include hidden files

ls -ltra
# Shows files starting with . like .bashrc

Human-readable sizes

ls -ltrh
# Shows 1.5M instead of 1572864

Check specific directory

ls -ltr /var/log

Only directories

ls -ltrd */

Sort by size

ls -lSrh
# -S = sort by size

Common Mistakes

❌ Mistake #1: Not using `-h`

ls -ltr
# Output: -rw-r--r-- 1 ubuntu ubuntu 1572864 Dec 06 10:00 file.txt
# Who knows what 1572864 bytes is?

✅ Better:

ls -ltrh
# Output: -rw-r--r-- 1 ubuntu ubuntu 1.5M Dec 06 10:00 file.txt
# Instantly readable

❌ Mistake #2: Missing hidden files

ls -ltr
# Doesn't show .bashrc, .ssh/, .gitconfig

✅ Better:

ls -ltra
# Shows all files including hidden ones

❌ Mistake #3: Looking for logs in wrong place

ls /tmp/application.log  # Wrong

✅ Better:

ls /var/log/application.log  # Correct

Real-World Examples

Check who owns a config file

ls -l /etc/nginx/nginx.conf
-rw-r--r-- 1 root root 1234 Dec 01 10:00 /etc/nginx/nginx.conf
# Owned by root, group root

Find recently modified logs

ls -ltr /var/log/*.log | tail -5
# Shows 5 most recently modified logs

Check device files

ls -l /dev/sda*
brw-rw---- 1 root disk 8, 0 Dec 06 10:00 /dev/sda
brw-rw---- 1 root disk 8, 1 Dec 06 10:00 /dev/sda1
# b = block device (hard drive)

Verify symbolic link

ls -l /usr/bin/python
lrwxrwxrwx 1 root root 9 Dec 01 08:00 /usr/bin/python -> python3.8
# Link points to python3.8

Quick Reference Table

Flag	What It Does
`-l`	Long format (detailed)
`-t`	Sort by time
`-r`	Reverse order
`-a`	Show hidden files
`-h`	Human-readable sizes
`-d`	List directories themselves
`-S`	Sort by size
`-R`	Recursive (subdirectories too)

Combining Flags

# Most useful combination
ls -ltrah
# Long format, time-sorted, reversed, all files, human sizes

# Find largest files
ls -lSrh | tail -10

# Recently modified, human-readable
ls -ltrhA

Key Takeaways

ls -ltr is your diagnostic tool - shows type, permissions, owner, size, date
First character = file type - Learn these 7: d, -, l, c, b, s, p
Linux is one tree from / - Everything branches from root
Know your key paths:
- Your files: /home/username/
- Configs: /etc/
- Logs: /var/log/
- Devices: /dev/
Always use -h flag - Human-readable sizes are essential

Once you can read ls -ltr output fluently, navigating Linux feels natural. You're seeing the full story, not just filenames.

What ls flag combo do you use most? Drop it in the comments—I'm collecting useful patterns.

Follow for more Linux learning notes as I document my journey into DevOps and cloud engineering.

Finding Files in Linux: Why locate is 10x Faster Than find published

NJEI — Sat, 06 Dec 2025 13:45:53 +0000

The Problem
You're SSH'd into a server. You created a config file two hours ago. You remember it's called nginx.conf, but you have no idea where it is.
This was me on day one with Linux. Here's what I learned about finding files fast.

The Obvious Choice: find
First attempt:
find / -name 'nginx.conf'
What this does:

find - The search command
/ - Star from root directory (searches everything)
-name 'nginx.conf - Find this exact filename

Result: It worked, but took 30+ seconds and threw "Permission denied" errors everywhere.

Why So Slow?
find searches your file system in real-time. It literally opens every directory and checks every file when you run it.

When find Shines
Use find when you need:

# Files modified in last 24 hours
find /home -mtime -1

# Files larger than 100MB
find / -size +100M

# Specific file type
find . -type f -name "*.log"

The Game Changer: locate
A senior dev saw me waiting and said, "Just use locate
locate nginx.conf
Results: Instant. Less than 1 second. No errors.

How locate Works
Here's the key: locate searches a database, not your filesystem
Linux maintains a database of all files (stored at /var/lib/mlocate/mlocate.db). This database updates automatically once a day via cron

When you run locate, you're querying a database → extremely fast.
The tradeoff? Just-created files won't appear until the database updates.

Updating the Database
Created a file and need to find it now?
sudo updatedb
Takes 1-2 minutes to refresh, then locate knows about all your new files.

Real Example: Finding Logs
Find all error logs:

# Using find (slow but real-time)
find /var/log -name '*error*'

# Using locate (fast but uses database)
locate error | grep /var/log

Common Mistakes

1. Permission Denied Spam

# ❌ This gives tons of permission errors
find / -name 'config.txt'

# ✅ Better options
sudo find / -name 'config.txt'
# OR
find ~/ -name 'config.txt'

2. Forgetting About the Database

# Just created a file
touch /tmp/newfile.txt

# This won't find it yet
locate newfile.txt

# Update database first
sudo updatedb

# Now it works
locate newfile.txt

3. Using find When locate Would Work
If you just need to find a file by name, start with locate. It's faster 99% of the time.

The Decision Framework
Use locate when:

✅ You know the filename
✅ Speed matters
✅ File isn't brand new

Use locate when:

✅ File was just created
✅ Searching by
✅ Need to perform actions on results

Useful Patterns
Case-insensitive search

locate -i nginx.conf  # Finds NGINX.conf, nginx.CONF, etc.

Limit results

locate -n 10 *.log  # Show only first 10 results

Search only existing files

locate -e filename  # Skip files that have been deleted

Find and execute

# Delete all .tmp files older than 7 days
find /tmp -name "*.tmp" -mtime +7 -delete

Performance Comparison
On my test system(16GB data):

# find: ~28 seconds
time find / -name 'nginx.conf' 2>/dev/null

# locate: 0.3 seconds
time locate nginx.conf

That's 93x faster for simple name searches.

Key Takeaways

locate uses a database → Fast but potentially outdated
find searches filesystem → Slower but real-time accurate
Update database with sudo updatedb when needed 4.** Choose based on need**: Speed and Accuracy

The moment I understood this difference, file searching in Linux clicked. Now I always ask: "Do I need real-time results?" If not, locate it is.

What's your go-to file finding trick? Drop in the comments, I'm still learning and would love to hear what works for you!

Follow me for more Linux learning notes as I document my journey from beginner to cloud engineer.

DEV Community: NJEI

The Bar has Always Been Moving. Are you?

Today's Wizardry Is Tomorrow's Bare Minimum

The Ladder We Keep Forgetting We're Climbing

The Cycle Is Happening Again. Right Now.

What the Moment Actually Demands

A Word for African Technologists Specifically

The Uncomfortable Close

File Management Beyond Copy-Paste: Compressing, Splitting, and Why tar.gz Isn't Scary

File Management Beyond Copy-Paste: Compressing, Splitting, and Why tar.gz Isn't Scary

The Problem: Moving and Managing Large Files

Comparing Files: diff and cmp

diff: Line-by-Line Comparison

cmp: Byte-by-Byte Comparison

Understanding tar: The Tape Archive

Creating a tar Archive

Extracting a tar Archive

Important: tar Alone Doesn't Compress

gzip: Actual Compression

Compress a File

The One-Step tar.gz

Decompress gzip Files

Extract tar.gz in One Step

Understanding tar.gz

Real-World Example: Backing Up a Website

Truncate: Shrink or Extend File Size

Basic Usage

Common Uses

Real Example: Clear Log File

Combining Files: cat

Basic Syntax

Example: Combine Logs

Example: Combine Parts

Splitting Files: split

Split by Lines

Split by Size

Real Example: Split Large Database Dump

Real-World Scenarios

Scenario 1: Backup and Transfer

Scenario 2: Send Large File via Email

Scenario 3: Archive Old Logs

Scenario 4: Clear Growing Log Files

Scenario 5: Deploy Application

Common tar Flags

Creating Archives

Extracting Archives

Viewing Contents

Extract to Specific Directory

Quick Reference

Comparison

tar Operations

Compression

File Operations

Common Mistakes

Tips for Efficiency

Key Takeaways

The Next Decade of AI and Human Interaction: A Developer's Perspective on What's Really Coming

Table of Contents

Where We Are Right Now (December 2025)

The Timeline: What Industry Leaders Are Actually Saying

2025-2027: The Arrival of AI Colleagues

What This Means for Developers

The First Wave of Job Changes

2028-2030: The Partnership Era

Human-AI Symbiosis Emerges

New Skills, New Roles

The Economic Reality

2031-2035: The Redefinition

What Work Becomes

The Skills That Endure

The New Value Creation

The Two Paths Ahead

Path 1: The Optimistic Scenario

Path 2: The Turbulent Scenario

What This Means for You Right Now

For Junior Developers

For Mid-Level Developers

For Senior Developers and Architects

The Uncomfortable Truths We Need to Accept

The Skills That AI Can't Replace (Yet)