DEV Community: Sheersh Sinha

My DevOps Journey: Day 14 - AWS Storage and Global Distribution: The Missing Pieces Every Cloud Engineer Must Master

Sheersh Sinha — Fri, 14 Nov 2025 21:42:00 +0000

Introduction

In Day 13, I explored how Amazon S3 changed the way I think about storage - versioning, lifecycle policies, encryption, and automation taught me that data durability is the backbone of cloud systems.

But as my sandbox grew into a multi-instance setup, I faced a new reality:

S3 alone cannot solve all storage needs.

Some data must live with compute.

Some must be shared across servers.

Some must be queried.

Some must be protected.

And some must be delivered globally with low latency.

This realization pushed me into a deeper AWS chapter: EBS, EFS, FSx, RDS, DynamoDB, Route 53, and CloudFront.

Each one filled a gap I didn't even know existed - until the day I faced a real problem. Let me take you through that journey.

1. When My EC2 Lost Its Data - The EBS Lesson

One day, I rebooted my EC2 instance… and my application files were gone.

I had forgotten the most basic AWS truth:

"EC2 storage disappears unless you attach persistent volumes."

That's when I discovered EBS (Elastic Block Store) - block storage volumes that live independently of the EC2 instance.

What I learned about EBS:

Persistent even if the EC2 is stopped
Can be detached & attached to any EC2
Great for OS boot volumes, DBs, or app data

Hands-on Commands:

Key Lesson: Compute and storage must be decoupled for reliability.

2. When Multiple EC2s Needed Shared Storage - Enter EFS

Soon, I deployed two EC2 instances behind a load balancer.

But then something broke:

One instance had updated config files
The other still used old ones
My app behaved inconsistently

That's when I needed shared file storage → Amazon EFS

Why EFS was a game changer:

Multiple EC2s can read/write the same files
Perfect for web servers, content directories, shared uploads
Auto-scaling storage
Pay only for what you use

Mount EFS:

EFS solved consistency issues instantly. No more "it works on server 1, but not server 2."

3. Snapshots & Disaster Recovery - EBS Snapshots

Later, I needed safe backups before making risky changes.

That's when I learned the magic of:

Snapshots are incremental & stored in S3 internally
If anything breaks: restore → create new volume → attach

"Always protect your state before making changes."

4. High-Performance File Systems - FSx

AWS FSx came next. At first, I ignored it - until I learned it powers:

High-performance workloads
Machine learning pipelines
Windows application file shares
Data-intensive HPC workloads

Use EFS for Linux. Use FSx for anything heavy-duty.

5. When My App Needed a Database That Wouldn't Crash - RDS

I deployed MySQL on EC2 once. Bad idea.

Maintenance, backups, failover… all on me.

Then I discovered RDS - AWS-managed SQL databases.

RDS gives you:

Automated backups
Multi-AZ replication
Auto patching
Read replicas
Monitoring
Snapshots
Encryption

Launching RDS:

Suddenly, my backend was stable, consistent, and safe.

6. When Performance Outgrew SQL - DynamoDB Saved Me

A scenario happened:

My app needed:

Millisecond reads
Millions of requests
No downtime
No maintenance

Enter DynamoDB - serverless NoSQL at massive scale.

DynamoDB strengths:

Auto-scaling read/write
Global tables
TTL for auto-expiry
On-demand pricing
Zero maintenance

DynamoDB made my backend "infinitely scalable" without touching servers.

7. When My App Needed a Domain - Route 53

Next problem: I needed a custom domain for my app.

Route 53 gave me:

DNS records
Traffic routing
Health checks
Failover routing
Latency-based routing

Now my app wasn't just functional - it had an online identity.

8. When I Needed Global Speed - CloudFront Saved My Bandwidth

Last issue: My images, CSS files, and videos loaded slowly across regions.

CloudFront fixed everything.

CloudFront advantages:

Distributes content globally
Reduces latency
Caches static files
Secures content with HTTPS
Integrates with S3, EC2, and Load Balancers
DDoS protection via AWS Shield

Now my system felt "global." Users anywhere experienced fast performance.

Final Architecture (Day 14 Vision)

Layer	Service	Purpose
CDN	CloudFront	Global CDN Delivery
DNS/Domain	Route 53	Domain & Routing
Load Balancer	ELB	Distributes Traffic
Web Servers	EC2 (EFS)	Compute/Shared Filesystem
Database	RDS / DynamoDB	Managed DB / NoSQL at Scale
Storage	S3, EBS, FSx/EFS	Object/Block/File Storage

What I Learned (Deep DevOps Mindset)

Don't trust EC2 storage - use EBS
Don't duplicate files across instances - use EFS
Don't run SQL manually - use RDS
Don't force SQL for all workloads - use DynamoDB
Don't expose raw S3 - use CloudFront
Don't buy domains elsewhere - Route 53 is built for AWS
Don't skip backups - use Snapshots

This blog wasn't just content - it was an evolution in the way I think about designing cloud systems.

"DevOps isn't about tools - it's about building systems that don't break when life gets busy."

What's Next (Day 15 - Serverless + Observability)

Coming up next:

Theme	Topics
Serverless	AWS Lambda, Triggers, Events, Destinations
Monitoring	CloudWatch Metrics & Logs, Dashboards, Alarms
Alerts	Amazon SNS, SQS message queues

This is where automation becomes truly event-driven and intelligent.

My DevOps Journey: Part 13 - AWS S3 Essentials: The Backbone of Cloud Storage

Sheersh Sinha — Wed, 05 Nov 2025 02:53:00 +0000

In the last blog (Day 12), I explored how networking in AWS forms the foundation for secure and scalable compute - using VPCs, Subnets, Gateways, and Security Layers.

But then I faced a new challenge:

"Where do I keep the data that shouldn't vanish after my EC2 shuts down?"

That's when I discovered Amazon S3 (Simple Storage Service) - the unsung hero of the cloud.

In this chapter, I'll cover:

S3 Buckets and Objects
Regions
S3 Storage Classes
Versioning
Access Control
Buckets and Keys
S3 Data Consistency Models
Object Lifecycle Management
Encryption

And I'll share how I applied these in my AWS Sandbox project

1. What Is S3?

Amazon S3 (Simple Storage Service) is AWS's object storage platform - used to store any amount of data, anywhere, at any time.

Unlike local drives or block storage, S3 isn't tied to an instance - it's global, durable, and accessible via APIs, SDKs, or the console.

Think of it like this:

"EC2 runs your application, S3 remembers everything it does."

It's where you store logs, backups, images, configurations, or even entire static websites.

2. Buckets and Objects - The Core of S3

Everything in S3 lives inside Buckets (like folders), and the data inside them are called Objects.

Each object can be up to 5TB in size and includes:

Data (file itself)
Metadata (info like type, owner, last-modified)
Key (unique identifier within the bucket)

In my setup:

I created a bucket named my-s3-bucket-mumbai-13 to store log files and system backups from my EC2 sandbox.

Command:

Practical takeaway:

Once uploaded, the object gets a unique key path, like:

3. Regions - Where Your Data Lives

Each S3 bucket exists in a specific AWS region (e.g., ap-south-1 for Mumbai).

This helps reduce latency and ensures compliance with local data laws.

For instance, my project bucket was created in Mumbai region, keeping latency low for my EC2 instance running in the same zone.

Tip: Always keep S3 and EC2 in the same region to avoid unnecessary data transfer costs.

4. S3 Storage Classes - Cost Meets Performance

AWS offers multiple storage classes based on how frequently you access data:

Storage Class	Description	Use Case
Standard	High availability, low latency	Frequently accessed data
Intelligent-Tiering	Auto-moves data between classes	Mixed access patterns
Standard-IA	Lower cost, higher retrieval time	Infrequent access
One Zone-IA	Stored in one AZ	Archival data
Glacier / Deep Archive	Very low cost, long retrieval	Long-term backups

In my setup, I used Standard for live log files and Glacier for archived reports.

Command Example:

5. S3 Versioning - Protecting from Mistakes

Ever deleted or overwritten something important?

That's where Versioning saves lives.

Enabling it keeps every version of every object - even if you delete or overwrite it.

Enable via CLI:

Lesson: In DevOps, versioning isn't just for code - it's for everything.

6. S3 Access Control - The Gatekeepers

Security is crucial when multiple users or applications access your S3.

You can manage access using:

IAM Policies: Control access via user roles (recommended).
Bucket Policies: JSON rules directly applied to a bucket.
ACLs (Access Control Lists): Legacy but still useful for object-level control.

Example Bucket Policy (read-only public access):

Note: Always use IAM roles for internal apps - never expose access keys in code.

7. Buckets and Keys - The Naming Convention

Each object inside S3 is identified by a key - similar to a file path.

Example:

s3://my-s3-bucket-mumbai-13/reports/2025/summary.csv

Here:

my-s3-bucket-mumbai-13 - Bucket
reports/2025/summary.csv - Key

Understanding this helps when automating tasks like uploading logs or parsing S3 URLs in scripts.

8. S3 Data Consistency Models

S3 ensures:

Read-after-write consistency for new objects.
Eventual consistency for overwrites and deletions.

In simpler terms:

New uploads appear instantly, but updates may take a few seconds to propagate globally.

Why it matters:

When building pipelines or backup scripts, always add a short delay after overwrites to ensure consistency.

9. Object Lifecycle Management - Automate Archival

Storage is cheap - but at scale, even "cheap" becomes expensive.

Lifecycle policies help automatically transition or delete old objects.

Result: Fully automated cost control without manual cleanup.

10. S3 Encryption - Security Beyond Access Control

AWS offers three levels of encryption:

Encryption Type	Description
SSE-S3	AWS manages encryption keys
SSE-KMS	Customer-managed keys (via AWS KMS)
SSE-C	You manage your own keys

Enable Server-Side Encryption (KMS):

Tip: Always use SSE-KMS for enterprise-grade compliance and auditing.

My Takeaways

S3 is not just storage - it's the foundation for data resilience in AWS.
Buckets and objects are simple, but their policies and lifecycle define your security posture.
Versioning + Encryption + Lifecycle rules = A production-grade storage strategy.

"If compute is the brain of cloud, storage is its memory - precise, persistent, and priceless."

What's Next (Day 14 - AWS Storage & Distribution Deep Dive)

After mastering object storage with S3, I'll now explore persistent, file, and database-level storage, and how AWS distributes content globally.

In Day 14, I'll cover:

Elastic Block Store (EBS)
Elastic File System (EFS)
EBS Snapshots & FSx
Relational Database Service (RDS)
DynamoDB
Route 53
CloudFront

"S3 taught me how to store data. The next step is learning how to serve it - faster, smarter, and everywhere."

My DevOps Journey: Part 12 - Networking Like a Pro: VPC, Subnets & Secure AWS Connectivity

Sheersh Sinha — Fri, 31 Oct 2025 20:35:31 +0000

In my previous blog Day-11 ,I explored how to build scalable and cost-effective AWS infrastructure using Load Balancers, Auto Scaling Groups, and Launch Templates - the foundation of resilience in cloud compute.

But soon, I faced a realization:

Scaling applications means nothing if your network isn't designed to support it.

So in this chapter, I decided to go deeper - to understand how AWS networking actually works behind the scenes, how instances talk to each other securely, and how data travels safely in and out of the cloud.

What I'll Cover in Day 12

- Virtual Private Cloud (VPC)
- Subnets (Public and Private)
- Internet Gateway (IGW)
- NAT Gateway
- Route Tables
- Elastic IPs
- Security Groups vs NACLs

The Real Problem

It started with a small frustration.

I launched an EC2 instance in my new custom environment... and it refused to connect to the Internet.

No updates. No pings. No outgoing traffic.

That's when I discovered my first real DevOps truth:

"In AWS, nothing connects by default - you build the roads yourself."

Step 1: Designing My Own Virtual Private Cloud (VPC)

A VPC (Virtual Private Cloud) is your own private slice of AWS networking.

Think of it as your company's private data center inside the cloud - completely isolated until you decide what connects to what.

I created a custom VPC:

- Name: MY-VPC
- CIDR Block: 10.0.0.0/22
- Region: ap-south-1 (Mumbai)

This became my cloud playground - the foundation for all my future deployments.

Step 2: Public and Private Subnets

The first rule of networking - not everything should be exposed to the Internet.

So I divided my VPC into two subnets:

Subnet Type	CIDR Block	Purpose
Public Subnet	10.0.0.0/24	Hosts Internet-facing resources (Load Balancer, Bastion)
Private Subnet	10.0.1.0/24	Hosts application and database servers

Each subnet was mapped to a different Availability Zone - ensuring fault tolerance.

"Subnets are the boundaries that define your exposure."

Step 3: Route Tables and Internet Gateway

Once I had my subnets, it was time to connect them properly.

Public Route Table:

- Destination: 0.0.0.0/0
- Target: Internet Gateway (IGW)

Private Route Table:
- Destination: Local traffic only (10.0.0.0/22)
- Target: None (isolated network)

Then, I created an Internet Gateway and attached it to my VPC - the bridge between my cloud and the outside world.

Lesson Learned:

Without a route or gateway, your instance is alive but stranded.

Step 4: Launching EC2 in the Custom VPC

This time, I launched my EC2 instance in the public subnet of my custom VPC and allowed HTTP/HTTPS traffic through the security group.

A quick ping 8.8.8.8 returned success.

That one reply packet was more than a test - it was my first working AWS network.

Step 5: Adding NAT Gateway for Private Subnets

Next challenge:

My private EC2 instances needed to download system updates but still remain unreachable from outside.

Solution: NAT Gateway

Here's what I did:

Created a NAT Gateway in the public subnet.
Allocated an Elastic IP to give it a static public address.
Updated the Private Route Table:

- Destination: 0.0.0.0/0
- Target: NAT Gateway

Now, my private servers could reach the Internet outbound (for updates, API calls) while staying invisible to inbound traffic.

"A NAT Gateway is like a one-way mirror - you can see out, but no one can see in."

Step 6: Elastic IP - Stable Connectivity

Public IPs in AWS are dynamic - they change when you stop/start instances.

But real-world servers need consistency (for whitelisting, SSH, or API calls).

That's where Elastic IPs come in.

They're static, reassignable IPs you can attach to NAT Gateways or EC2s.

I attached one EIP to my NAT and another to my Bastion EC2.

Now, even after reboots, I had predictable, reliable connectivity.

Step 7: Security Groups vs Network ACLs

To secure my environment, I used AWS's two-tier firewall model:

Feature	Security Groups	Network ACLs
Scope	Instance-level	Subnet-level
Stateful	Yes (remembers connections)	No (must allow return traffic manually)
Default Behavior	Deny inbound, allow outbound	Allow all
Use Case	Fine-tuned port access	Broader subnet security

For my setup:
- Security Group: Allowed inbound HTTP/HTTPS + SSH.
- NACL: Blocked everything except ports 22 and 80.

Together, they formed a layered defense system - something every DevOps engineer should think about from Day 1.

Step 8: My Final Architecture

Key Components:

- Public Subnet - Internet-facing
- Private Subnet - Internal workloads
- NAT Gateway + EIP - Outbound access only
- Security Groups + NACLs - Layered security

Step 9: My Practical Takeaway

This project changed how I think about the cloud.

I stopped focusing on individual instances - and started seeing the architecture as a living system.

Key insights:

Isolation is intentional, not accidental.
Every subnet and route is a design decision.
Networking is where security and scalability truly begin.

"In DevOps, understanding networks isn't optional - it's the difference between deploying and designing."

What's Next (Day 13 - AWS S3 Essentials)

Now that I've built a secure and scalable network for compute, it's time to move to the next core AWS service - storage.

In Day 13, I'll dive deep into Amazon S3 (Simple Storage Service) - learning how to store, manage, and secure data efficiently in the cloud.

Here's what I'll cover next:

- S3 Buckets and Objects - the basic building blocks of cloud storage
- Regions and Data Distribution - where and how data physically lives
- S3 Storage Classes - balancing performance and cost
- S3 Versioning - tracking and recovering previous data states
- S3 Access Control - IAM policies and bucket permissions
- Buckets and Keys - organizing and addressing data
- S3 Consistency Models - how AWS ensures data reliability
- Object Lifecycle Management - automating archival and deletion
- S3 Encryption - securing data at rest and in transit

"Compute gives life to your application, Networking connects it, and Storage ensures it remembers."

Stay tuned for Day 13, where I explore S3 - the backbone of cloud storage and how DevOps engineers use it for reliability, versioning, and automation.

My DevOps Journey: Part 11 - Building Scalable and Cost-Effective AWS Infrastructure (Real-World Problem DevOps Solution)

Sheersh Sinha — Sun, 26 Oct 2025 21:09:11 +0000

Every DevOps journey reaches that point where the system starts talking back to you - not in words, but in CPU spikes, slow response times, and growing user traffic.

For me, that moment came one Friday afternoon.

My personal project, hosted on a single EC2 instance, suddenly started slowing down during peak hours.

Logs looked fine. Network was stable. But users were waiting.

That's when I realized - it's not about fixing bugs anymore; it's about designing for growth.

The Real Problem - When One Server Isn't Enough

My single EC2 instance was doing everything:

Serving frontend + backend
Handling API requests
Logging activities
Running cron jobs

It was the perfect "one-man army" - until it wasn't.

Then came the symptoms:

Latency spiked above 700ms
CPU utilization touched 90%
One deployment crash took down the entire site

That's when I understood - in DevOps, resilience > perfection.

I didn't need a stronger machine.

I needed multiple lightweight servers working together, scaling up and down as traffic demanded.

Step 1 - Architecting a Scalable System

I started small - with what a real company would do in this scenario:

"Scale horizontally, not vertically."

Here's the system design I implemented:

This architecture allowed:

Traffic distribution using an Application Load Balancer (ALB).
Automated scaling based on CPU or memory usage via Auto Scaling Groups (ASG).
Health monitoring through CloudWatch Metrics.

Step 2 - Implementation Journey & Challenges

Creating a Launch Template

A launch template is your blueprint for EC2 - defining AMI, instance type, and startup configuration.

It's what ensures every new instance behaves exactly as you intend.

Lesson learned: Templates remove the human error of "I forgot to install X on that instance."

Creating a Load Balancer

Once the ALB was configured (internet-facing on port 80), I connected it to my Target Group where all EC2 instances register automatically.

AWS started routing requests evenly.

No more single-server bottlenecks.

No more downtime during deployment.

"If one instance fails, the others don't even notice - that's the DevOps way."

Setting Up Auto Scaling

Then came automation.

Instead of guessing how many servers I'd need, I let AWS decide.

Scaling Policy:

Minimum Instances → 1
Desired → 2
Maximum → 4
Scale out when CPU > 70%

aws autoscaling put-scaling-policy \
  --auto-scaling-group-name webapp-asg \
  --policy-name scale-out \
  --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{"PredefinedMetricSpecification":{"PredefinedMetricType":"ASGAverageCPUUtilization"},"TargetValue":70.0}'

Now, new instances appeared automatically during load peaks and shut down when traffic cooled off.

That's the moment I realized - true DevOps is not manual control, it's intelligent automation.

Step 3 - The Cost-Effectiveness Perspective

I learned early that "scale" doesn't mean "spend more."

To stay cost-efficient:

Used t2.micro (free-tier) for tests.
Configured scaling cooldowns to prevent unnecessary spin-ups.
Added CloudWatch alarms to shut down idle EC2s.
Reserved spot instances for low-priority workloads.

Result:

System scaled automatically but stayed 40% cheaper than running fixed instances.

That's how you build a production-like system on a student budget.

Step 4 - Remedy & Prevention

When things go wrong, DevOps thinking kicks in.

Here's what I learned to always keep ready:

Problem	Remedy	Preventive Measure
High CPU utilization	Scale horizontally using ASG	Add CloudWatch alarms for CPU > 70%
Instance failure	Health checks via ALB	Enable ELB-based instance replacement
Configuration drift	Use Launch Templates	Version control templates via Git
Cost surge	Review billing dashboard weekly	Set AWS Budgets & alerts

Step 5 - Real-World Architecture (VPC Layer Included)

Here's what my updated system design looked like:

Each subnet, route, and NAT was designed to keep public entry restricted and internal services isolated - something real organizations practice every day.

Step 6 - My Vision as a DevOps Engineer

This experience shifted my perspective forever.

I stopped seeing AWS as a set of services - and started seeing it as an ecosystem.

Every design decision - from choosing an instance type to writing a cron job - impacts cost, scalability, and security.

"DevOps isn't about tools; it's about foresight - knowing what breaks tomorrow, and fixing it today."

This is what I strive for in my journey - not just deploying applications, but designing systems that can handle the unknown.

What's Next (Day 12 - AWS Networking & VPC Deep Dive)

Now that I've learned how to distribute and scale workloads with load balancers and auto scaling groups, it's time to look under the hood - the network layer that powers it all.

In Day 12, I'll explore:

Virtual Private Cloud (VPC) - the foundation of your private AWS network
Route Tables - directing traffic inside your cloud
Security Groups and Network Access Control Lists (NACLs) - understanding inbound/outbound control
Subnets (Public and Private) - isolating workloads securely
Internet Gateway (IGW) - bridging private cloud to the public internet
NAT Gateway - controlled outbound access for private instances
Elastic IP (EIP) - fixed IPs for stable external communication

"If Load Balancing is about distributing requests, Networking is about defining where those requests can even go."

My DevOps Journey: Part 10 - Deep Dive into AWS EC2 (Elastic Compute Cloud)

Sheersh Sinha — Mon, 20 Oct 2025 21:48:06 +0000

After understanding how cloud computing works in my previous post, I was eager to move from concepts to creation. I wanted to actually build something in the cloud - and that's when I discovered Amazon EC2 (Elastic Compute Cloud) - the beating heart of AWS infrastructure.

This was the first time I felt like I wasn't just using the cloud… I was running it.

What is EC2?

EC2 is Amazon's virtual server - it's like renting a computer in the cloud that you can access anytime, configure however you want, and scale instantly.

Think of EC2 as your personal Linux or Windows machine - but hosted globally and available on demand. Each instance (server) you launch can host an application, a script, a database, or even an entire architecture.

When I launched my first EC2 instance, it wasn't just a VM - it was a sandbox for experimentation.

Understanding AMI - The Blueprint of Your Cloud Machines

Every EC2 instance starts from an AMI (Amazon Machine Image).

An AMI is like a template that defines:

The operating system (Ubuntu, Amazon Linux, Windows)
Pre-installed packages
System configuration

When I was testing my Log Analyzer project, I picked an Ubuntu 22.04 AMI, installed git, cron, gzip, and cloned my repository. I later created my own custom AMI - so I could launch pre-configured instances instantly, saving time in future experiments.

Key Lesson: Building custom AMIs means infrastructure can be versioned just like code.

Instance Metadata and User Data - Automating Configuration

One of the most powerful features of EC2 is instance metadata and user data. They allow your instance to know about itself and even auto-configure during boot.

Instance Metadata

This is the instance's "self-awareness." It stores dynamic information like:

Instance ID
Public/Private IP
Security group
Region

Command to view metadata (from inside EC2):

User Data

This is a script that runs automatically when your EC2 instance starts for the first time. I used it to automate dependency installation and log setup:

#!/bin/bash
sudo apt update -y
sudo apt install -y git cron gzip
git clone https://github.com/sheersh123/bash-log-analyzer.git
cd bash-log-analyzer
chmod +x log_analyzer.sh
(crontab -l 2>/dev/null; echo "0 0 * * * /home/ubuntu/bash-log-analyzer/log_analyzer.sh /var/log/syslog") | crontab -

When I launched this instance - it auto-setup my environment. No manual SSH, no copy-paste - everything was ready within minutes.

Key Lesson: Automation doesn't start with tools like Ansible - it starts with User Data scripts.

EC2 Instance Types and Pricing Models - The Cost Optimization Game

AWS gives you flexibility not just in size, but in how you pay.

Instance Types

Each type is designed for a specific workload:

Family	Example	Use Case
General Purpose	t3.micro	Lightweight apps, testing
Compute Optimized	c5.large	High CPU workloads
Memory Optimized	r5.large	Databases, analytics
Storage Optimized	i3.large	High disk I/O operations
GPU Instances	p3.2xlarge	ML, deep learning

Pricing Models

Model	Description	Use Case
On-Demand	Pay hourly - no commitment	Testing, short workloads
Reserved	Commit for 1-3 years, lower cost	Long-term stable apps
Spot	Use spare capacity at huge discounts	Flexible, interruptible tasks
Savings Plan	Flexible compute commitment	Mixed workloads

When I started, I used t2.micro (Free Tier) to experiment. It was enough for scripts, GitHub syncs, and learning automation.

Key Lesson: Cloud computing rewards those who understand efficiency - not just scalability.

AWS CLI - Controlling AWS from the Command Line

The AWS CLI became my favorite DevOps weapon. Instead of clicking through the console, I started managing everything through the terminal.

Installing AWS CLI

sudo apt install awscli -y
aws configure

You'll be prompted for:

AWS Access Key
Secret Key
Default region
Output format (json, text, or table)

Common AWS CLI Commands

Command	Purpose
`aws ec2 describe-instances`	List all EC2 instances
`aws s3 ls`	List S3 buckets
`aws ec2 stop-instances --instance-ids <id>`	Stop a specific instance
`aws s3 cp ./reports s3://my-devops-logs/`	Upload files to S3
`aws iam list-users`	View IAM users

Key Lesson: The CLI is where DevOps engineers truly control the cloud - it's scriptable, repeatable, and fast.

My AWS Task - Launching My First Windows EC2 Instance

After experimenting with Linux instances for scripting and automation, I wanted to test how DevOps workflows translate into Windows environments.

As part of my AWS Task, I decided to:

Create a Windows VM on AWS EC2
Connect via RDP (Remote Desktop Protocol)
Open CMD inside the instance
Verify system details such as hostname, architecture, and OS build

This task helped me understand how cross-platform management works in the cloud.

Tech Stack Used:

AWS EC2 (Windows)
RDP (Client pre-installed in Windows)

Commands Executed in CMD:

Once connected, I could view the instance's IP, processor, and memory details - confirming that my Windows EC2 instance was successfully deployed and live.

Key Lesson: Managing Windows VMs on AWS gives a new perspective - it's not just about Linux automation; DevOps engineers often maintain hybrid environments where both OS types coexist.

This small exercise boosted my confidence in handling multi-OS infrastructure - a key skill when working in enterprise-scale DevOps setups.

My Turning Point - The "Vanishing Instance" Moment

During one of my test runs, I accidentally terminated an EC2 instance without creating an AMI backup. All my logs and configurations vanished.

It was frustrating - but it also taught me one of the most powerful lessons in cloud computing:

"In the cloud, if you didn't back it up, it never existed."

Since then, I've built the habit of creating snapshots and AMIs before every experiment.

Key Takeaways

EC2 is the core of AWS computing - your virtual data center
AMIs are blueprints for consistent deployments
Metadata & User Data enable automation from the first boot
Understanding pricing saves money and mistakes
The AWS CLI turns DevOps engineers into automation pros
Hybrid environments (Linux + Windows) reflect real-world DevOps challenges

What's Next - Load Balancing & Auto Scaling in AWS

Now that I've learned to launch and automate EC2 instances, the next step is understanding how to distribute traffic and maintain high availability.

In my next post, I'll explore how AWS helps scale applications seamlessly through Load Balancers and Auto Scaling Groups.

Here's what's coming next:

Load Balancer - The foundation of traffic management in AWS
Application Load Balancer (ALB) - Handling HTTP/HTTPS traffic intelligently
Network Load Balancer (NLB) - High-performance traffic routing at Layer 4
Launch Templates - Predefined instance configurations for auto-scaling
Types of Load Balancers - Understanding Classic vs Application vs Network
Target Groups and Listeners - The logic behind routing and instance health checks
Auto Scaling Group (ASG) - Automatically adjusting instance count based on demand

"Scaling isn't about adding servers - it's about maintaining stability while the world grows around your system."

Stay tuned - the next post will be all about keeping your cloud architecture resilient, dynamic, and efficient.

My DevOps Journey: Part 9 - Into the Cloud (Cloud Computing, IAM, and AWS Foundations)

Sheersh Sinha — Wed, 15 Oct 2025 21:33:29 +0000

After exploring how to secure systems and networks in Day 8 (Network Security and DevSecOps), I wanted to understand where all these systems actually live.

I had spent enough time protecting the pipelines - now it was time to understand the platform that powers them.

That's when I entered the world of Cloud Computing - the heart of modern DevOps.

What is Cloud Computing?

When I first launched my sandbox project on an AWS EC2 instance, it hit me - the cloud isn't some distant, complex system.

It's computing on demand - available anytime, anywhere, in seconds.

No cables, no datacenters - just configuration, connectivity, and scalability.

Definition: Cloud computing delivers servers, databases, storage, networking, and software services over the internet ("the cloud") - on a pay-as-you-go model.

Cloud Service Models

Model	Meaning	What It Offers	Example
IaaS	Infrastructure as a Service	Compute, storage, networking	AWS EC2, Azure VM
PaaS	Platform as a Service	Managed environment for development	AWS Elastic Beanstalk
SaaS	Software as a Service	Ready-to-use applications	Gmail, Salesforce

Each model defines how much control you have:

IaaS: You manage everything above the OS.
PaaS: Focus only on code - the platform handles scaling.
SaaS: You consume; the vendor manages all.

Lesson: Cloud computing isn't about moving data online - it's about shifting responsibility strategically.

Deployment Models

Type	Description	Example
Public Cloud	Shared by multiple organizations	AWS, Azure, GCP
Private Cloud	Dedicated to one organization	VMware Cloud
Hybrid Cloud	Mix of both	AWS Outposts, Azure Stack

My "Aha!" Moment - Elasticity in Action

During one of my log analyzer tests, I scaled my application on AWS from 1 to 3 instances in under a minute.

Then, I stopped them - and billing stopped too.

That's when I realized what "elasticity" truly meant.

You only pay for what you use, and you scale when you need it.

Lesson: Cloud computing isn't just about running servers - it's about running them intelligently.

Identity and Access Management (IAM) - The Security Backbone of AWS

In traditional setups, access control meant login credentials.

In the cloud, it's Identity and Access Management (IAM) - a system that defines who can do what on which resources.

At first, I underestimated IAM - until I accidentally gave an admin-level policy to a test user.

It taught me a critical lesson about cloud security hygiene.

Key IAM Components

Component	Role
Users	Individual accounts (engineers, admins)
Groups	Collection of users with common permissions
Roles	Temporary access for AWS services (EC2, Lambda)
Policies	JSON-based documents defining permissions

Example IAM Policy (Least Privilege)

This gives read-only access to the S3 bucket - nothing more.

Key takeaway: Always apply the Principle of Least Privilege - give only what's needed.

My First IAM Mistake

While experimenting with AWS, I once created a new IAM user to test access permissions for S3.
I wanted the user to upload log reports from the Bash Log Analyzer project — but every time I tried running:

aws s3 cp ./reports s3://my-devops-logs/ --recursive

I kept getting this frustrating error:

An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

After checking the policy, I realized the user only had s3:GetObject (read-only) permissions - not s3:PutObject (write).

It was such a small oversight, yet it stopped my automation completely.

I updated the policy to:

And instantly, everything worked.

Lesson: In the cloud, permissions can make or break productivity. Always double-check what’s allowed — both read and write.

Introduction to AWS - My First Steps into the Cloud

AWS (Amazon Web Services) felt like a huge universe of services at first.

So I simplified it - I started with the essentials every DevOps engineer must know.

Step 1: Launching an EC2 Instance

My first step was creating a virtual machine (EC2) on AWS.

Commands that became second nature:

ssh -i mykey.pem ubuntu@<public-ip>
sudo apt update && sudo apt install git

That one EC2 instance became my testing ground for every experiment - from bash scripts to log analyzers.

Step 2: Storing Logs in S3

Then I moved my project's log reports to Amazon S3 (Simple Storage Service):

aws s3 cp reports/ s3://my-devops-logs/ --recursive

Suddenly, my reports were globally accessible - securely and durably.

Step 3: Networking with VPC

I created a Virtual Private Cloud (VPC) - my own isolated network inside AWS.

I configured:

Subnets (Public & Private)
Internet Gateway
Routing Tables

That experience made me appreciate how my networking knowledge from Day 7 now directly applied in the cloud.

The DevOps Connection - Cloud as the Engine

In DevOps, cloud computing isn't just about hosting - it's about automation and scalability.

Benefit	Impact
Scalability	Auto-scaling instances during traffic spikes
Reliability	Multi-region deployments
Speed	CI/CD pipelines running globally
Security	IAM, encryption, monitoring
Observability	CloudWatch + custom dashboards

Lesson: DevOps without cloud is like scripting without a terminal - limited and local.

Real Scenario - The Forgotten S3 Policy

Once, while uploading files from EC2 to S3, I kept getting "Access Denied."

Everything seemed fine - until I checked the bucket policy.

It was denying uploads from non-HTTPS connections.

A small oversight, but a great reminder:

"Security isn't a barrier - it's a guide to better configurations."

Key Takeaways

Cloud Computing = Renting intelligence, not just hardware.
IAM = The guardian of every resource.
AWS = The backbone of modern DevOps workflows.
Always prefer roles over credentials.
Test your permissions before you automate them.

What's Next (Day 10 - Deep Dive into AWS EC2)

Now that we've explored the foundations of Cloud Computing and IAM, it's time to roll up our sleeves and build inside the cloud.

In Day 10, I'll dive deep into Amazon EC2 (Elastic Compute Cloud) - where theory meets execution.

Here's what's coming next:

Elastic Compute Cloud (EC2) - Understanding the core of AWS computing.
Amazon Machine Images (AMI) - The blueprints of every instance.
EC2 Instance Metadata & User Data - Automating configuration at launch.
EC2 Instance Types & Pricing Models - Choosing the right compute for the right workload.
AWS CLI - Managing AWS resources from the terminal like a true DevOps engineer.

"This is where cloud theory turns into infrastructure reality."

Stay tuned - Day 10 will be all about launching, configuring, and optimizing instances at scale.

My DevOps Journey: Part 8 - Network Security and DevSecOps: Protecting What Connects

Sheersh Sinha — Mon, 06 Oct 2025 16:56:16 +0000

After mastering how systems communicate in Networking (Day 7) ,I learned that communication without protection is like sending postcards without envelopes - anyone can read them.

So, my next mission was clear: learn how to protect the network.

This part of my journey was where DevOps met Security - the phase where "moving fast" had to evolve into "moving fast securely."

Network Security Attacks - The Reality Every Engineer Faces

My first encounter with security wasn't in a textbook - it was on an EC2 instance.
One morning, I opened /var/log/auth.log and saw dozens of failed SSH login attempts from random IPs.

That's when I realized: even your test servers are visible to the entire internet.

Let's break down a few common network security attacks I studied (and later simulated safely in my sandbox):

Attack	What happens	How to mitigate
DDoS (Distributed Denial of Service)	Servers flooded with fake requests	Use CDN, WAF, rate limiting
Man-in-the-Middle (MITM)	Attacker intercepts communication	Enforce HTTPS/TLS, VPNs
DNS Spoofing	Fake DNS responses redirect users	Use DNSSEC & trusted resolvers
Port Scanning	Attackers find open services	Restrict ports via firewalls
Brute Force	Repeated login attempts	Enable fail2ban, SSH keys only

Real-World Learning

When I faced that SSH attack, I realized my biggest mistake - I had allowed inbound SSH from 0.0.0.0/0.

Within hours, my logs were filled with brute-force attempts.

I fixed it by:

Allowing SSH only from my IP range
Switching to key-based authentication
Installing fail2ban

Lesson: Security breaches don't wait for production - they start with small misconfigurations.

Firewall and ACL Configurations - The Digital Security Guard

A firewall isn't just software; it's the first layer of trust your system builds.

It decides who can knock on your door and who can't.

Example: Linux Firewall with iptables

sudo iptables -A INPUT -p tcp --dport 22 -s 192.168.1.10 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 22 -j DROP

This means:

Only 192.168.1.10 can access SSH.
Everyone else gets blocked.

AWS Security Group Example

Rule type	Port	Source	Purpose
Inbound	22	My IP only	SSH access
Inbound	80	0.0.0.0/0	HTTP traffic
Outbound	All	0.0.0.0/0	Internet access

In one of my sandbox deployments, my web app failed to load because I forgot to allow HTTP in inbound rules - a simple oversight that taught me how much firewalls influence availability.

Lesson: Firewalls don't just block; they define your system's exposure.

TLS, IPsec and Encryption Protocols - The Science of Safe Communication

Encryption is what turns a public network into a private channel.

As I explored it, I realized - security doesn't always mean adding tools; sometimes, it's about using the right protocol.

Protocol	Purpose	Example use
TLS (Transport Layer Security)	Encrypts data in transit	HTTPS, email
IPsec (Internet Protocol Security)	Encrypts IP packets	VPNs, VPC tunnels
SSH (Secure Shell)	Secure remote login	Admin access
DNSSEC	Validates DNS responses	Prevents DNS spoofing

To check if a site supports TLS:

Lesson: Encryption doesn't make systems slower - it makes trust faster.

DevSecOps - Automating Security, Not Just Deployments

Once I got comfortable securing systems manually, I realized - manual security doesn't scale.

That's where DevSecOps comes in: security integrated into DevOps pipelines.

Instead of "deploy first, scan later," DevSecOps shifts everything left - security testing starts during development.

Here are some tools I began experimenting with:

Tool	Purpose
Trivy	Scans Docker images for vulnerabilities
Falco	Monitors runtime threats
HashiCorp Vault	Manages and encrypts secrets
OWASP ZAP	Scans web apps for security issues
SonarQube	Detects code vulnerabilities and bugs

Example: Trivy in Action

trivy image myapp:latest

It listed all the CVEs present in my Docker image - outdated packages, unpatched libraries.

I fixed those and pushed a cleaner, more secure image.

Lesson: Security isn't about fear - it's about visibility before failure.

My Turning Point - The "Allow All" Mistake

In one of my early deployments, I opened up inbound rules for all ports - just to test multiple services.

Within a day, my system logs showed multiple scans and failed access attempts.

That one careless configuration taught me what no textbook could:

"Convenience is the enemy of security."

Now, every environment I build - from testing to production - follows zero-trust principles and least privilege access.

Key Takeaways

Every open port is an invitation - protect it.
Security starts with small steps: IP restriction, SSH keys, encryption.
Firewalls and ACLs define who your system talks to.
DevSecOps = embedding security into every build, commit, and deployment.
Prevention always costs less than recovery.

What's Next (Day 9 - Cloud Foundations and IAM Essentials)

Now that I've learned how to protect systems and secure communication, the next logical step is to explore where those systems live - the cloud.

In Day 9, I'll be exploring:

What is Cloud Computing?
Identity & Access Management (IAM)
Introduction to AWS

That's where this journey moves from protecting connections to building secure infrastructure.

My DevOps Journey: Part 7 - Mastering Networking for DevOps Engineers

Sheersh Sinha — Sat, 04 Oct 2025 21:06:08 +0000

When I started with DevOps, I focused on automation and scripting - but soon realized none of it mattered if systems couldn’t talk to each other. One broken route, a blocked port, or a misconfigured subnet could turn a perfect deployment into a dead service.

That’s when I decided to truly understand Networking, not as theory, but as the heartbeat of DevOps.

Computer Networking Overview

Networking is simply how systems communicate. But in DevOps, it’s the foundation of everything — from connecting CI/CD runners to Kubernetes pods.

Think of it like this:

Your code travels through the network pipeline to reach users.

If that pipeline breaks, your service disappears.

Lesson: In DevOps, networking isn’t optional knowledge - it’s your system’s lifeline.

OSI Model - My Troubleshooting Map

The OSI Model isn’t just textbook theory - it’s a structured way to debug real problems.

Layer --- Name ------ What I Check As DevOps
7 --- Application ------ Web servers, APIs, DNS, SSH
6 --- Presentation ------ SSL/TLS encryption
5 --- Session ------ Long-lived connections
4 --- Transport ------ TCP/UDP, ports
3 --- Network ------ IP routing, subnets
2 --- Data Link ------ MAC addressing, ARP
1 --- Physical ------ NICs, cables, virtual interfaces

Scenario:
Once, my SSH connection to an EC2 instance failed. Using OSI logic:

L3 (ping worked)

L4 (port 22 blocked by security group)

Lesson: OSI = The ultimate DevOps debugging checklist.

Cloud Networking - The Invisible Backbone

When I deployed my first multi-tier app on AWS, the frontend couldn’t talk to the backend.

Cause: The frontend was in a public subnet, backend in a private one, but I forgot to configure NAT Gateway for outbound access.

Fixed it by:

Updating route tables
Attaching a NAT gateway

Lesson: Cloud networking uses old principles with new names - VPC = LAN, NAT = router, SG = firewall.

LAN, Switch, Router, Subnet, Firewall, Gateway

Concept Purpose Real-World DevOps Example

LAN ----- Connects devices locally ----- Kubernetes cluster network
Switch ----- Connects nodes in LAN ----- Docker bridge network
Router ----- Routes traffic between networks ----- VPC internet gateway
Subnet ----- Segments network ----- Private and public subnets in AWS
Firewall ----- Controls access ------ Security groups, NACLs
Gateway ----- Network exit point ------- NAT Gateway for private instances

Real Scenario:

Two Docker containers couldn’t reach each other - turned out they were on different bridge networks.
Fixed it by connecting them to the same subnet.

Lesson: Understanding topology saves hours of debugging.

Microservices Networking - When Services Talk to Each Other

In a microservices world, everything talks - but not everything listens properly.

Incident:

payment-service couldn’t reach user-service.

Used:

kubectl exec -it payment-pod -- nslookup user-service

CoreDNS was down. Restarted it, and communication was restored.

Lesson: In Kubernetes, DNS = glue for microservices. If DNS breaks, your entire mesh collapses.

IP Addresses and Ports - The System’s Identity

An IP is your system’s address; a port is the specific door you knock on.

Service ---- Port ---- Protocol
SSH ---- 22 ---- TCP
HTTP ---- 80 ---- TCP
HTTPS ---- 443 ---- TCP
Jenkins ---- 8080 ---- TCP
MySQL ---- 3306 ---- TCP

Check open connections:

Lesson: Every port tells a story - know which ones your app uses, and why.

DNS - The Internet’s Phonebook

When you type google.com, your system asks a DNS server for the IP.

If DNS fails, even healthy servers look “down.”

Commands:

Scenario:

My app worked with IP but failed with the domain - misconfigured DNS record.

Lesson: DNS is invisible until it breaks - then it’s everything.

Useful Networking Commands Every DevOps Engineer Must Know

Command ------ Purpose

ping ------ Check connectivity
ifconfig / ip a ------ View interfaces
traceroute ------ Show packet path
arp -a ------ View MAC-to-IP mapping
iptables -L -n ------ Firewall rules
netstat -tuln ------ Active ports
route -n ------ Routing table
nslookup / dig ------ DNS resolution
curl -v ------ Test HTTP connections

These commands have saved me more times than I can count.

Real-World Turning Point

During one deployment, everything looked green - yet users couldn’t reach the app.
After hours of panic, I realized the service was configured as ClusterIP (internal-only) instead of LoadBalancer.
One YAML change later, traffic started flowing.

Lesson: In DevOps, most “application outages” are actually network misconfigurations.

Key Takeaways

Networking is the foundation of DevOps.
OSI model = debugging GPS.
Cloud networking = traditional concepts in fancy wrappers.
DNS, IP, and ports define your app’s reachability.
Always test with real tools (ping, curl, traceroute, netstat).

What’s Next (Day 8: Network Security & DevSecOps)

Now that I know how systems talk, the next step is learning how to protect that communication.
In Day 8, I’ll explore:

- Network Security Attacks (DDoS, MITM)
- Firewall & ACL configurations
- TLS/IPsec protocols
- DevSecOps tools for automated security

My DevOps Journey: Part 6 - Building a Bash Log Analyzer for Real DevOps Scenarios

Sheersh Sinha — Mon, 29 Sep 2025 15:18:09 +0000

Why I Chose This Project

If you’ve worked in DevOps, you know the 2 AM drill:
A service goes down.
The first question: “What do the logs say?”
Ten minutes later, you’re still scrolling through thousands of lines of /var/log/syslog

During my internship, I saw this problem first-hand. The logs had the answer — but the signal was buried in noise. By the time the team found the root cause, customers had already noticed the downtime.

That’s when I realized: slow log analysis is not just a technical issue, it’s a business risk.

So I built the Bash Log Analyzer & Error Report Generator
.

What Problem Does It Solve?

In real DevOps environments:

Incident response time matters → every minute of downtime = lost money.
Logs are the first diagnostic tool but are messy to read manually.
Teams need structure fast → not raw lines, but actionable insights.

This project automates that workflow by:

Parsing logs for ERROR, WARNING, and CRITICAL entries.
Generating reports in .txt and .csv for teams and management.
Automating analysis via cron so reports are delivered daily.
Business Value: Faster incident response → reduced MTTR (Mean Time to Recovery) → better uptime and reliability.

Repo: bash-log-analyzer

Sandbox Deployment — My DevOps Action Story

I tested the project on an AWS EC2 Ubuntu sandbox before touching production logs. Here’s what happened:

Dependency Pitfalls

First attempt:

sudo apt install -y grep awk sed cut sort uniq gzip cron git unzip

Result:

awk → “virtual package.”
cut, uniq, sort → “not found.”

Fix: Installed gawk. Learned that others come with coreutils.

DevOps Lesson: Don’t assume base images have the same packages — always verify.

GitHub Battles

Branch divergence blocked pushes → fixed with git pull --rebase.
Password auth failed → switched to Personal Access Token (PAT).

DevOps Lesson: Auth evolves (PAT/SSH > passwords). CI/CD pipelines must adapt too.

Permission Errors

Ran analyzer → Permission denied.
Fixed with:

DevOps Lesson: Permissions are small details that break big things.

Testing Logs

Forgot argument → script scolded me with usage instructions.
Ran with sample logs → reports generated fine.
For real /var/log/syslog, I backed it up first before running the script.

DevOps Lesson: Never test directly on production logs without backups.

Cron Automation

Goal: automate daily reports at midnight.
Cron failed silently. Why? I used relative paths . Fix: switched to absolute paths.

DevOps Lesson: In automation, paths must always be explicit.

Outcome

Analyzer worked in sandbox.
.txt and .csv reports generated successfully.
Cron job automated the process.
GitHub repo synced with latest updates.

DevOps Takeaways

Faster MTTR: Logs parsed into clean reports, no endless scrolling.
Safe Practices: Sandbox testing + backups before touching live logs.
Automation First: Cron removes human dependency.
Scalable Vision: Ready for future integrations (Grafana/ELK).

What’s Next

In Day 7, I’ll explore Networking for DevOps Engineers — because once you can analyze logs and automate insights, the next step is ensuring systems communicate reliably.

Topics coming up:

Computer Networking Basics (in DevOps context)
OSI Model as a troubleshooting map
LAN, Switches, Routers, Subnets, Firewalls, Gateways
Cloud Networking
Microservices Networking

Networking is the glue of distributed systems — without it, even the best automation fails.

This project showed me how even a simple Bash script can shrink troubleshooting time, improve observability, and support real business outcomes.

It gave me hands-on experience in log analysis, automation, and deployment troubleshooting — critical skills for a DevOps engineer.

My DevOps Journey: Part 5 - Shell Scripting Lessons from Real Troubleshooting

Sheersh Sinha — Thu, 18 Sep 2025 19:53:38 +0000

After setting up cronjobs in Part 4 , I realized typing commands manually wasn’t scaling. In the real world, systems don’t wait for you to type - they break when you’re asleep. That’s when I leaned on Shell Scripting.

But instead of learning it the “classroom way,” I picked it up through real troubleshooting scenarios. Each concept - variables, arguments, loops - came alive only when I faced a problem I had to solve.

Here’s how scripting became my real-world rescue tool.

Writing My First Script - The “On-Call Repetition Nightmare”

Scenario: During my internship, disk alerts kept firing at midnight. Every time, I’d log in and type df -h manually. It was exhausting.

Solution:

I wrote a script called diskcheck.sh:

#!/bin/bash
df -h

Now, instead of typing commands half-asleep, I just ran:

./diskcheck.sh

Lesson: Scripts are your second brain during on-call duty.

Variables- The “Multiple Servers, One Script” Challenge

Scenario: I had to check connectivity for 3 different servers. My first script hardcoded the IP. If the server changed, the script broke.

Solution:

Use variables:

Later, I swapped the value, and the script worked everywhere.

Lesson: Variables save scripts from becoming useless with changing infrastructure.

Operators-The “Disk Space Alert” Problem

Scenario: A staging server ran out of space, crashing the app. I needed a script to check disk usage and alert only when it crossed 80%.

Solution:

USED=$(df / | grep / | awk '{ print $5 }' | sed 's/%//g')
THRESHOLD=80

if [ $USED -gt $THRESHOLD ]; then
  echo " Disk usage critical: $USED%"
fi

Lesson: Operators + conditions = automated monitoring without external tools.

Read User Input-The “Backup Flexibility” Problem

Scenario: My backup script only archived /home/, but once I needed /etc/ configs too. Editing the script every time was painful.

Solution: Make the script ask:

echo "Enter path to backup:"
read PATH
tar -cvf backup.tar $PATH

Now I could run it for any directory.

Lesson: Interactivity makes scripts versatile across environments.

Functions-he “Scattered Logging Mess”

Scenario: I had multiple scripts spitting logs in random formats. Debugging failures became a nightmare.

Solution: I created a reusable logging function:

log() {
  echo "[INFO] $(date): $1"
}
log "Backup started"
log "Backup completed"

Now all my scripts spoke the same language.

Lesson: Functions bring consistency to messy automation.

Shell vs sh vs Bash - The “Works on Ubuntu, Breaks on Alpine” Problem

Scenario: A deployment script ran fine on Ubuntu but failed inside a lightweight Alpine Docker container.

The issue: Alpine used /bin/sh, which didn’t support certain Bash features.

Solution: Always specify the interpreter:

#!/bin/bash

Lesson: Explicit is better than implicit. Always define your shell.

Conditionals - The “Don’t Delete While Running” Disaster

Scenario: A cleanup script started wiping logs while nginx was still running. The service crashed.

Solution: Add a safety check:

if pgrep nginx; then
  echo "Nginx running. Skipping cleanup."
else
  rm -rf /var/log/nginx/*
fi

Lesson: Conditionals are the guardrails that keep automation safe.

Arguments - The “One Script Per Environment” Problem

Scenario: I had separate scripts for dev, test, and prod backups. Unmanageable.

Solution:

ENV=$1
echo "Backing up $ENV environment..."
tar -cvf backup_$ENV.tar /var/$ENV/

Run with:

./backup.sh dev
./backup.sh prod

Lesson: Arguments make scripts scale across environments.

Loops - The “50 Log Files” Problem

Scenario: Compressing logs one by one? Impossible during an outage.

Solution:

for file in /var/log/*.log; do
  gzip $file
done

Lesson: Loops are the ultimate time-savers for bulk operations.

The Advanced Turning Point: When My Script Nearly Wiped Production

Late one night, I was asked to “clean up temp files.” I rushed and wrote:

But I forgot my script was running as root inside a container mount. It began deleting critical shared files.
I pulled the brakes with Ctrl+C, but the damage was real — some mounted configs were gone.

How I Recovered:

Restored configs from backup.
Added dry-run safety in my scripts:

echo "Would delete: $file"

before actually running rm.
Enforced set -e and logging for all critical scripts.

Lesson: Scripts are scalpels - powerful, precise, and dangerous. Treat them with respect.

Key Takeaways from Part 5

Shell scripting concepts come alive only through real problems.
Variables, arguments, loops → not “syntax,” but solutions to DevOps pain.
Conditionals and functions prevent disasters.
Always test in a sandbox before production.
Scripts aren’t toys — they’re system lifelines.

What’s Next (Part 6: Networking Basics)

Now that I can script my way out of problems, the next step is to make sure systems can talk to each other.

In Part 6, I’ll explore:

Computer Networking Overview
OSI Model
LAN, Switch, Router, Subnet, Firewall
Cloud Networking
Microservices Networking

Over to You

What was your biggest shell scripting disaster? Did you ever run a command that made you sweat? Share it - maybe we can all save someone else from repeating it.

My DevOps Journey: Part 4 - Archiving, Scheduling, Remote Access & System Administration Essentials

Sheersh Sinha — Mon, 15 Sep 2025 20:59:10 +0000

After learning about permissions, ownership, and package managers in Part 3, I realized that being comfortable with Linux also means being able to manage files, automate tasks, secure access, and work across systems.

This part of my journey dives into some of the most practical everyday tools of a DevOps engineer: archiving, cronjobs, SSH, SCP, virtual machines, and key pairs.

Archiving & Compression

One of my first real struggles was handling log backups. I had thousands of log files in /var/log/, and transferring them individually was a nightmare.

So I used tar:

tar -cvf logs.tar /var/log/

Output:

/var/log/syslog
/var/log/auth.log
/var/log/kern.log
...

Later, to extract:

tar -xvf logs.tar

Then I tried compressing a single file:

gzip demo.txt

Output:

demo.txt → demo.txt.gz

Lesson: Instead of moving hundreds of files, I just tar them into one archive, transfer, and extract. Much cleaner.

Automating Tasks with Cronjobs

I hated repeating backups manually, so I created a cronjob.

crontab -e

I added:

0 0 * * * /home/user/backup.sh

To check:

crontab -l

Output:

0 0 * * * /home/user/backup.sh

Troubleshooting: At first, my cronjob didn’t run. The issue? My script used python3, but cron didn’t recognize it. I fixed it by giving the full path:

/usr/bin/python3 /home/user/backup.py

Security & System Administration Basics

Next, I explored system security.

Creating a new user:

sudo adduser dev

Output:

Adding user `dev' ...
Adding new group `dev' (1002) ...
Creating home directory `/home/dev' ...

Giving sudo rights:

sudo usermod -aG sudo dev

Lesson: I once mistakenly gave a developer sudo rights when he only needed file access. It taught me that least privilege is key in production.

Remote Connections with SSH

When I first tried logging into a server:

ssh user@192.168.1.10

It asked for a password - fine. But when I disabled password login, I hit this error:

Permission denied (publickey).

The fix was to use SSH key pairs.

Generating SSH Key Pairs

I generated a key:

ssh-keygen -t rsa -b 4096 -C "sheershsinha30@gmail.com"

Output:

Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa):

Then copied it to the server:

ssh-copy-id user@192.168.1.10

Now login works without typing a password:

ssh user@192.168.1.10

Creating Virtual Machines

Before trying things on production servers, I practiced on VirtualBox:
Created a VM with Ubuntu.
Configured NAT networking.
Installed nginx inside the VM:

sudo apt install nginx -y

Verified:

systemctl status nginx

Output:

nginx.service - A high performance web server and a reverse proxy server
   Active: active (running)

Lesson: Breaking things in a VM is safe - I just reverted to a snapshot.

Key Takeaways from Part 4

Archiving makes backups portable.
Cronjobs automate repetitive tasks.
System security is about principle of least privilege.
SSH & SCP are lifelines for remote management.
SSH key pairs enable passwordless secure automation.
Virtual machines provide a safe DevOps sandbox.

What’s Next (Part 5: Shell Scripting)

After learning permissions, automation, and SSH, I realized that typing commands again and again isn’t efficient. That’s where Shell Scripting comes in - the real glue of DevOps.

In the next part of my journey, I’ll cover:

Introduction to Shell Scripting - Why scripts matter in DevOps.
Write & execute a simple script - Going from one-liners to automation.
Variables & Operators - Storing values and doing calculations.
Read user input - Making scripts interactive.
Functions & Loops - Reusing and repeating code smartly.
Shell vs sh vs Bash - What’s the difference?
How to write Bash scripts - Best practices for real-world use.
Conditional Statements - Adding logic to decisions.
Passing Arguments to a Script - Making scripts flexible and reusable.

This will be the point where I move from “just running commands” to building automation workflows - a must-have step for any DevOps engineer.

Over to You

What was your first cronjob or SSH key setup experience like? Did it work on the first try, or did you spend hours debugging like me? Share your story

My DevOps Journey: Part 3 - Linux Users, Groups, Permissions & Package Managers

Sheersh Sinha — Fri, 12 Sep 2025 20:40:06 +0000

In Part 2 I explored the Linux file system and system logs. But very soon, I hit another roadblock:

Permission denied.

If you’ve ever worked on Linux, you’ve probably seen those dreaded words. At first, I thought it was just me typing the wrong command. But as I dug deeper, I realized it’s actually one of the most important safeguards in Linux - permissions and ownership.

And soon after that, I discovered another layer of control that keeps systems safe: package managers.

Users, Groups & Permissions - The Basics

User (Owner) → The account that creates or owns the file. By default, the creator becomes the owner.
Group → A collection of users with shared permissions (like a project team).
Permissions → Define who can read (r), write (w), and execute (x) a file or directory.

Example from ls -l:

-rwxrw-r--  1 sheersh devops 0 Sep  9 12:00 demo.txt

Breakdown:

sheersh → owner (user)
devops → group
rwx → owner permissions (full access)
rw- → group permissions (read/write)
r-- → others permissions (only read)
This matches 764 in octal form.

My Hands-On Assignment

I was asked to create a file /home/demo.txt and assign permissions so that:
Owner → read, write, execute
Group → read, write
Others → read

Here’s what I did:

Step 1: Create the file

Step 2: Change permissions

Step 3: Verify

Output:

Perfect! The file now matches the required permissions.

** Ownership in Action**

Permissions don’t mean much if the wrong user owns the file. Ownership decides who can exercise those permissions.

Key Commands:

ls -l → shows file ownership (user & group).

chown newuser:newgroup file.txt → changes ownership.

chgrp groupname file.txt → changes group only.

This is crucial in DevOps because files often get created by root, but need to be used by app users or CI/CD pipelines.

Real-World Troubleshooting Story

The Scenario:

We deployed a simple shell script for rotating logs in /var/log/app/. The script worked perfectly in development. But in staging, developers kept reporting:

bash: ./logrotate.sh: Permission denied

At first, I thought it was missing execute permission (x). But after checking, I found something else.

The Investigation:

Check file permissions & ownership:

ls -l logrotate.sh

Output:

-rwxrwxr-- 1 root root 120 Sep  9 10:00 logrotate.sh

File was owned by root.

Developers were in the dev group, but since the file belonged to root:root, they had no control.

Even though permissions looked fine, the ownership was wrong.

The Solution:

Change ownership to the correct user and group:

sudo chown dev:dev logrotate.sh

Now:

-rwxrwxr-- 1 dev dev 120 Sep  9 10:00 logrotate.sh

Verify developer access:

su - dev
./logrotate.sh

Script runs fine.

Lesson Learned

Permissions control what actions are allowed.
Ownership controls who applies those permissions.
A file owned by the wrong user/group can be just as problematic as missing x.

Package Managers - The Next Layer of Control

Once I got comfortable with users, groups, and permissions, I realized something:

“Permissions decide who can use files, but package managers decide who can install and update software on the system.”
On Linux, package managers are the tools that handle software installation, updates, and dependency management.

** Common Package Managers**

APT (Advanced Package Tool) → Used in Ubuntu/Debian.
Example:

sudo apt update
sudo apt install nginx

YUM/DNF → Used in CentOS, Fedora, RHEL.

Example:

sudo dnf install nginx

Zypper → Used in openSUSE.

Example:

sudo zypper install nginx

Pacman → Used in Arch Linux.

Example:

sudo pacman -S nginx

** Package Managers + Permissions**

Only users with sudo/root privileges can install or update packages.
This prevents ordinary users from accidentally (or intentionally) breaking the system.
In DevOps, package managers are often automated via Ansible, Puppet, Dockerfiles, or CI/CD pipelines — but under the hood, it’s the same mechanism.

** Troubleshooting Example**

I once tried installing Nginx as a normal user:

apt install nginx

And got:

E: Could not open lock file /var/lib/dpkg/lock-frontend - Permission denied

The fix? Run with sudo:

sudo apt install nginx

Because installing software changes system-wide directories (like /usr/bin, /etc/nginx), Linux protects it by requiring elevated privileges.

** Lesson: Just like ownership protects files, package managers + permissions protect the whole OS.**

Key Takeaways from Part 3

Users, groups, and permissions protect files.
Ownership decides who controls those files.
Package managers control software installation - only privileged users (or automated DevOps pipelines) can use them.
Real-world troubleshooting often comes down to a mix of permissions, ownership, and package manager access.

What’s Next (Part 4)

Now that I’ve explored permissions, ownership, and package managers, the next step is to dive into the day-to-day toolkit of a Linux system administrator.

In Part 4, I’ll cover:

Archiving and compressing files (tar, gzip, zip)
Automating tasks with cronjobs
Security basics and system administration
Remote access with SSH & SCP
Creating VMs on VirtualBox or equivalent
Generating SSH Key Pairs for passwordless logins

This is where Linux starts feeling like a real DevOps playground.

** Over to You**

Have you ever faced a “Permission denied” while installing software or running a script? Was it due to permissions, ownership, or missing sudo? Share your experience below