DEV Community: Tejas Shinkar

So… If AI Can Write Code, What Are Software Engineers Getting Paid For? 👀

Tejas Shinkar — Sun, 30 Aug 2026 16:17:23 +0000

A few years ago, being the person who could turn a requirement into working code was a pretty valuable skill.

Now you can give an AI:

“Build me a REST API with authentication, tests and documentation.”

…and get something surprisingly usable in minutes.

So naturally, the uncomfortable question is:

If AI can write the code, what exactly are software engineers getting paid for?

Here's where it gets interesting.

Imagine an AI-generated feature gets merged.

The tests pass.

The deployment succeeds.

And two hours later, production starts throwing errors.

Now someone has to figure out:

Why only certain users are affected?

Why did latency suddenly spike?

Is the application broken, or is the database struggling?

Did the new code introduce a race condition?

Is the fix actually safe to deploy?

AI can help investigate all of this.

But somebody still has to understand the system well enough to know whether the answer makes sense.

And developers themselves seem to know this.

Stack Overflow's 2025 survey found that 84% of developers use or plan to use AI tools, yet 46% said they don't trust the accuracy of AI output. The biggest frustration? AI solutions that are almost right — with 45% saying debugging AI-generated code can take more time.

There's another interesting signal: developers are much less willing to hand AI high-responsibility work like deployment and monitoring.

Maybe that's the shift.

The valuable engineer isn't necessarily the person who can type the most code anymore.

It's the person who can:

understand the problem → design the system → use AI effectively → question its output → debug what breaks → make the right trade-off.

AI might be making code cheaper.

But good engineering judgment?

That's still expensive.

And maybe that's what we're actually getting paid for. 👀

I Thought Knowing AWS Was The Goal. I Was Wrong.

Tejas Shinkar — Sat, 29 Aug 2026 16:27:31 +0000

When I started learning AWS, I had this very simple idea in my head: learn the services, finish the modules, build a few projects, and eventually I’d be “good at cloud.” So I started collecting names — EC2, S3, IAM, VPC, EFS, CloudFront, Route 53, and a growing graveyard of AWS console tabs. 😂 At first, it felt productive because every completed service felt like another box checked. But somewhere during the networking labs, that started falling apart. I realized I could tell you what a NAT Gateway does, but still stop and ask myself, “Wait… why are we creating another subnet for this?”

That question actually changed how I started learning. Instead of asking, “What does this AWS service do?”, I started asking, “What problem are we solving that requires this thing?” Suddenly VPC started making more sense. A route table wasn't just another AWS resource; it was an answer to “Where should this traffic go?” A NAT Gateway wasn't something you create because a tutorial told you to; it existed because a private machine needed controlled outbound internet access without becoming publicly reachable. Even NACLs became interesting when I deliberately broke one and watched an otherwise healthy request just… hang. 💀

And I started noticing the same thing outside networking. IAM isn't really about memorizing policies — it's about deciding who should be allowed to do what, and from where. CloudFront isn't just “AWS CDN” — it's figuring out which responses are safe to reuse, which ones are personalized, and what happens when your cache thinks two requests are the same when they aren't. Docker isn't impressive because you know docker build; it's useful when you understand what should actually live inside a container, what should stay outside it, and how that behaves when the container gets replaced.

I'm still nowhere near the point where I'd call myself a production DevOps engineer. Most of what I'm learning right now comes from courses, labs, breaking things, fixing them, and trying to reason about what these systems would look like in a real environment. And honestly, that's probably been more valuable than simply finishing lectures. Because the more I learn, the more I realize that companies probably don't need someone who can recite 100 AWS services from memory. They need someone who can look at a broken system and ask the right next question.

Maybe that's the difference between learning tools and learning engineering. One teaches you what exists. The other teaches you why it exists, when to use it, and what happens when it breaks.

I'm curious about people who already work in Cloud/DevOps/SRE:

What was the first thing you learned in production that made you realize, “Yeah… courses really didn't prepare me for this”?

I Thought Caching Was Just About Speed. Then I Found the Security Trap. 💀

Tejas Shinkar — Fri, 28 Aug 2026 05:20:01 +0000

I used to think caching was a pretty simple performance trick: store something closer to the user, serve it faster, reduce the load on the backend, and everyone wins. Then I started digging into CloudFront cache keys and realized caching can become a data-isolation problem if one small decision is wrong. Imagine an application with a GET /profile endpoint where User A sends Cookie: session=userA and gets a response like “Hello Alice, your orders: 12, your balance: ₹42,000.” CloudFront can cache that response so future requests don't always have to reach the origin. Sounds great, right? Until another user requests the same URL and the cache doesn't distinguish the thing that actually makes the response personal.

Now User B sends GET /profile with Cookie: session=userB, and the origin would normally return completely different information such as “Hello Bob, your orders: 4, your balance: ₹8,000.” But if the cache key only considers /profile and ignores the user-specific part of the request, CloudFront can see the request as equivalent to the object it already has. In that situation, User B could receive the response cached for User A. Nothing has crashed, the EC2 instance can be healthy, the database can be healthy, and CloudFront can even be behaving exactly according to its configuration. The actual problem is the caching strategy. That's when caching stops being only a performance optimization and becomes a correctness and security concern.

Now compare that with a public product catalogue. A request like GET /products?id=101 might return the same product information for every user: iPhone 17, ₹79,999, In Stock. That's a fantastic candidate for caching because thousands of users can safely reuse the same response. We want something like one cache object serving many requests, not a separate cache entry for every visitor. But this is where another subtle problem appears: if every request contains an analytics cookie such as analytics_id=83A91, analytics_id=72B42, or analytics_id=91K17, and those values don't change the product response, putting them into the cache key can create thousands or even millions of unnecessary cache variants. The application may still be correct, but cache reuse gets worse, cache efficiency drops, and more requests may end up going back to the origin.

That leads to one of the most useful CloudFront distinctions I've learned: Cache Policy and Origin Request Policy are not answering the same question. Cache Policy is essentially asking, “What makes this request a different cached object?” while Origin Request Policy is asking, “What information does the origin need to receive?” For example, an analytics cookie might need to reach the origin for logging or processing, while still not needing to create a unique cache entry for every user. The important part is to understand which request values actually change the response and which ones are simply metadata. A good cache design therefore doesn't blindly include every cookie, header, or query string just because it is available; it deliberately chooses only the dimensions that matter.

The production lesson for me is that a high cache-hit ratio is not automatically a good outcome. A cache that is extremely fast but returns the wrong user's data is a disaster, not an optimization. The real question isn't “How much can I cache?” but “What can I safely reuse, what makes the response different, and how do I make CloudFront understand that difference?” This also changes how I think about CDN architecture: cache design affects performance, origin load, freshness, correctness, and potentially security at the same time. Static assets such as versioned JavaScript, CSS, images, and public product data are usually much easier to cache aggressively, while personalized responses need a much more careful strategy. Caching is not just about speed — it's about knowing exactly when two requests are allowed to share the same answer.

I broke my own AWS infrastructure… without touching the infrastructure.

Tejas Shinkar — Wed, 26 Aug 2026 18:02:23 +0000

I had an EC2 instance running an application.

Everything looked fine.

Then I changed one Security Group rule.

Suddenly…

the application couldn't reach the database.

The instances were healthy.

The database was healthy.

CPU was normal.

No deployment happened.

Just… timeout.

And that's when I learned something production-grade:

A Security Group is stateful, but that doesn't mean your architecture is automatically stateful.

The database allowed inbound traffic from the application's Security Group.

So when the app initiated:

EC2 → DB : 3306

the return traffic didn't need a separate inbound rule on the application side.

But when we accidentally restricted the egress from the application SG…

💀 the connection died.

Nothing was "down."

The network policy was.

That's the scary part about cloud infrastructure:

Your application can be perfectly healthy and still be completely unreachable because the control plane says "no."

And that's why production debugging isn't just:

“Is the server running?”

It's:

“Who is allowed to talk to whom, on which port, in which direction, and through which path?”

AWS CloudFront — Complete Understanding, Production Patterns & Practice

Tejas Shinkar — Wed, 26 Aug 2026 17:53:14 +0000

Part of my AWS learning journey — exploring AWS hands-on and building a deeper understanding of Cloud & DevOps. This session dives into how Amazon CloudFront works as a CDN, from edge locations and cache hits/misses to cache keys, cache policies, TTLs, and invalidation.

📋 Topics Covered

#	Topic	Type
1	What CloudFront Actually Is	Concept
2	Why Do We Need a CDN?	Concept
3	CloudFront Terminology — Distribution, Origin, Edge Location	Concept
4	Complete Request Flow	Concept
5	Cache HIT / Cache MISS	Concept
6	Per-Edge Caching	Concept
7	Cache Key	Concept + Interview
8	Cache-Key Design & The Golden Rule	Concept + Interview
9	Personalization and Caching	Concept + DevOps
10	Cache Policy vs Origin Request Policy	Concept + Interview
11	Query Strings, Cookies & Headers in the Cache Key	Concept
12	TTL — Minimum, Default, Maximum	Concept + Cert
13	Cache-Control & Invalidation	Concept
14	Versioned Assets	Concept + DevOps
15	Cache Behaviors	Concept
16	Real-World Caching Scenarios	Concept + DevOps
17	Request Collapsing & Cache Hit Ratio	Concept + Cert
18	Production Cache Hierarchy	Concept
19	CloudFront + S3 (OAC)	Concept + Lab
20	CloudFront + ALB + EC2	Concept + Lab
21	CloudFront + WAF	Concept + DevOps
22	Production Techniques & Common Mistakes	DevOps
23	Production Mini Architecture	DevOps
24	Practice Labs 1–10	Practice
25	Interview Revision	Interview
26	Final Mental Model	Concept

What CloudFront Actually Is

Amazon CloudFront is AWS's Content Delivery Network (CDN).

The simplest mental model is:

CloudFront keeps reusable copies of content closer to users so that the origin does not have to generate or deliver the same response repeatedly.

Without CloudFront:

User
  |
  v
Origin
(EC2 / ALB / S3 / API Gateway)
  |
  v
Response

With CloudFront:

User
  |
  v
CloudFront Edge Location
  |
  +---- Cache HIT ----> Response to User
  |
  +---- Cache MISS ---> Origin
                           |
                           v
                        Response
                           |
                           v
                    CloudFront caches it
                           |
                           v
                        User

The origin remains the source of truth. CloudFront is a delivery and caching layer in front of it.

CloudFront can use origins such as Amazon S3, an Application Load Balancer, EC2/custom HTTP servers, API Gateway, and other HTTP origins.

Why Do We Need a CDN?

Imagine an application hosted in Mumbai. A nearby user may reach it with relatively low network latency, but a user in Europe has a much longer network path:

Europe User
     |
     | long network distance
     v
Mumbai Origin
     |
     v
Response

With CloudFront:

Europe User
     |
     v
Nearby CloudFront edge
     |
     v
Cached response

If the response is already cached at that edge, the origin does not need to be contacted for that request.

CloudFront can therefore reduce viewer latency, reduce origin load, improve scalability, reduce repeated origin data transfer, provide globally distributed delivery, and integrate with security services such as AWS WAF and Origin Access Control.

Important CloudFront Terminology

Distribution

A CloudFront distribution is the overall configuration that tells CloudFront how to deliver an application. It contains configuration for origins, cache behaviors, cache policies, origin request policies, allowed HTTP methods, viewer protocol settings, TLS/custom domains, security, and logging.

Distribution = the overall CloudFront delivery configuration.

Origin

The origin is where CloudFront obtains the original content — for example CloudFront → S3, CloudFront → ALB → EC2, or CloudFront → API Gateway.

CloudFront
    |
    v
ALB
    |
    +--> EC2
    +--> EC2
    +--> EC2

Here, the ALB is the CloudFront origin.

Edge Location / Point of Presence (PoP)

An edge location is a CloudFront location close to viewers where CloudFront can serve cached content.

                  CloudFront
                     |
       +-------------+-------------+
       |             |             |
     India         Europe         USA
       |             |             |
     Users         Users         Users

Different edge locations can have their own cached copies.

The origin stores the original content; edge locations store temporary cached copies.

CloudFront also has regional edge caches between POPs and origins. These provide another caching layer and can retain less-popular objects longer than individual POP caches.

Complete Request Flow

Suppose the origin contains logo.png and a user requests GET /logo.png. The conceptual flow is:

1. User requests /logo.png
              |
              v
2. DNS routes the viewer to an appropriate CloudFront edge
              |
              v
3. CloudFront determines the cache key
              |
              v
4. Cache lookup
          /             \
        HIT              MISS
         |                |
         v                v
 Return object        Request origin
                           |
                           v
                     Origin response
                           |
                           v
                    CloudFront caches
                           |
                           v
                         User

This is the foundation of CloudFront.

Cache HIT / Cache MISS

A cache hit means CloudFront found a valid cached object matching the viewer's request, so the origin does not need to generate the object for that request. Benefits: lower latency, lower origin load, better scalability, better cache hit ratio.

A cache miss means CloudFront cannot find a valid cached object matching the request at that edge, so it must fetch the response from the origin and cache it. A later matching request can then become a cache hit while the object remains valid.

User                          User
 |                              |
 v                              v
CloudFront                CloudFront
 |                              |
 v                              | MISS
Cache HIT                       v
 |                          Origin
 v                              |
Cached object              response, then cached, then returned to User

Per-Edge Caching

Suppose a user in Delhi requests /logo.png. One edge location may cache it, while a user elsewhere may reach another edge location where that object is not yet cached:

                Origin
                  |
        +---------+---------+
        |         |         |
      Edge A    Edge B    Edge C
      logo      logo       logo

The same object can exist in multiple edge caches. A cache miss at one edge does not mean the entire CloudFront distribution has no copy anywhere.

Cache Key

The cache key determines which viewer requests are considered equivalent for caching.

CloudFront's identity/checklist for deciding whether this request can reuse an existing cached object.

For GET /products?id=10, the cache identity could contain the path (/products) and query string (id=10). If another user sends the same relevant request, it can map to the same cached object.

Why Cache-Key Design Matters

If users A, B, and C all request /products?id=10 and receive the same product information, we want a single shared cache object serving all three — good cache reuse.

But if the cache key unnecessarily includes a unique cookie (e.g. user=A vs user=B), CloudFront now treats /products?id=10 + user=A and /products?id=10 + user=B as separate cache variants. The content may be identical, but cache reuse drops and origin traffic can increase.

The Golden Cache-Key Rule

Can this request value change the response?

If yes, it may need to participate in the cache key. If no, including it may unnecessarily fragment the cache.

For example, an analytics_id cookie that's only used for analytics and doesn't change the response usually shouldn't become a cache-key dimension. But a session_id cookie that determines whether the response is "Hello Alice" or "Hello Bob" is a personalized response and must not be accidentally shared.

Personalization and Caching

This is a major production concern. If GET /profile with Cookie: session=A returns "Hello Alice" and the same path with Cookie: session=B returns "Hello Bob", but CloudFront creates one shared cache object using only the path /profile, the response could be incorrectly reused. That is not merely a performance problem — it can become a data-isolation/security problem.

Common strategies:

Strategy 1 — Don't shared-cache private responses. For highly dynamic/private endpoints, route straight through: User → CloudFront → Origin.

Strategy 2 — Vary the cache identity when appropriate. For example, /profile + user=A → Cache A and /profile + user=B → Cache B. This can become extremely expensive and fragmented when there are millions of users.

Strategy 3 — Separate static and dynamic content. A common production architecture: /static/* is highly cacheable, /api/* is dynamic/private.

Cache Policy vs Origin Request Policy

A Cache Policy controls which request values participate in the cache key, TTL settings, and compression-related cache behavior. The cache key can be influenced by path, query strings, headers, and cookies.

Cache Policy answers: "What makes this request a different cache object, and how long should the object remain cached?"

An Origin Request Policy answers a different question:

What information should CloudFront send to the origin?

They are related, but not the same — this is one of the most important CloudFront interview concepts.

Example — origin needs information but cache doesn't

For a request like GET /products?id=10 with Cookie: analytics_id=ABC123, where id=10 determines the product but analytics_id is only needed by the origin for analytics and doesn't change the response, we want the cache key to stay just /products?id=10 while the origin request still forwards analytics_id=ABC123.

So: the Cache Policy does not include analytics_id in the cache key, while the Origin Request Policy forwards it to the origin. This allows cache reuse while still giving the origin the information it needs.

AWS documents this separation explicitly: values included in the cache key are also sent to the origin, while an Origin Request Policy can add additional headers, cookies, and query strings to origin requests without putting them into the cache key.

Warning with Authorization headers

If a request's Authorization header changes what response the user receives, you cannot simply forward the token and assume shared caching is safe. Always ask: does this request information change the response? If yes, either make the relevant dimension part of the cache design or avoid shared caching for that response.

Query Strings, Cookies & Headers in the Cache Key

Query strings: if id in /products?id=10 vs /products?id=20 changes the response, it must be represented in the cache identity — otherwise different products could map to the wrong cached response.

Cookies: if GET /products?id=10 returns the same response regardless of a user=A vs user=B cookie, don't automatically include the cookie in the cache key. If the response does differ because of the cookie, you need a design that prevents incorrect sharing. The answer is not always "include every cookie" — that can create millions of cache variants.

Headers: an Accept-Language: en request returning "Welcome" vs Accept-Language: fr returning "Bienvenue" means language may need to influence the cache identity if it genuinely changes the response.

Vary the cache only on dimensions that genuinely affect the response.

TTL — Minimum, Default, Maximum

TTL means:

How long a cached object remains fresh according to the CloudFront caching configuration.

With a TTL of 3600 seconds (one hour), an object cached at 12:00 stays a hit through 12:59, expires at 13:00, and needs fresh/validated origin content on the next request after that.

Long TTL (e.g. for app.js) gives high cache reuse, low origin traffic, low latency, and better scalability — at the cost of changes remaining stale longer unless you use versioned assets or invalidation.

Short TTL (e.g. for price.json) makes changes visible sooner, but increases origin requests, lowers the cache hit ratio, raises origin load, and can increase latency.

CloudFront cache policies have a Minimum TTL, Default TTL, and Maximum TTL, which work together with origin caching headers such as Cache-Control: max-age=3600.

Default TTL is used when the origin doesn't provide appropriate caching information.
Minimum TTL sets a lower bound on how long CloudFront caches. Important production warning: if Minimum TTL is greater than zero, CloudFront can cache for at least that duration even if origin headers contain directives such as no-cache, no-store, or private — so be careful using positive Minimum TTLs with private/dynamic content.
Maximum TTL places an upper bound on how long an object can remain fresh based on the relevant origin caching headers.

Cache-Control & Invalidation

The origin can send headers like Cache-Control: max-age=3600, Cache-Control: no-store, or Cache-Control: private to communicate caching requirements. CloudFront's cache policy and TTL configuration determine how those instructions interact with CloudFront caching.

If the origin's logo.png changes but CloudFront still has the old version cached with a 24-hour TTL, the old object can persist until it naturally expires. To remove it sooner, use CloudFront invalidation:

Origin = NEW

CloudFront Cache = OLD
        |
        v
   Invalidation
        |
        v
Cached object invalidated
        |
        v
Next request -> MISS
        |
        v
Origin -> NEW
        |
        v
CloudFront caches NEW

Use TTL when "this content can safely remain cached for this long." Use invalidation when "I changed the content and need the cached version invalidated before its normal lifetime." Production systems commonly use both.

Versioned Assets

Instead of constantly replacing /app.js in place, use a content-hashed filename like /app.a82f91.js. When the application changes, /app.b73c21.js is generated — CloudFront sees a different cache key entirely, which makes long TTLs practical for build artifacts.

A common pattern: HTML gets a shorter TTL or controlled invalidation, while JS/CSS/images get a long TTL plus content-hashed filenames. This reduces the need for broad invalidations and is one of the most common production CDN techniques — using app.v1.js/app.v2.js or hash-based names avoids users being stuck on stale cached versions after a deployment.

Cache Behaviors

A Cache Behavior is a set of CloudFront rules applied to requests matching a URL path pattern (e.g. /static/*, /images/*, /api/*). Different behaviors can use different origins, cache policies, origin request policies, allowed methods, TTL/caching behavior, and viewer/security settings:

                  CloudFront
                      |
          +-----------+-----------+
          |                       |
       /static/*                /api/*
          |                       |
          v                       v
         S3                      ALB
                                  |
                               EC2/ECS

A distribution has a default behavior (e.g. /*), and you can add more specific behaviors (e.g. /api/*, /images/*). The more specific matching behavior is used according to CloudFront's path matching rules — /images/logo.png uses the /images/* behavior, /api/products uses the /api/* behavior.

Real-World Caching Scenarios

Public product catalogue: GET /products?id=10 returning identical data to every user (User A → MISS → Origin → Cache, User B/C/D → HIT) is excellent cache reuse.

Personalized profile: GET /profile with different cookies returning "Hello Alice" vs "Hello Bob" must not become a shared cache object — a common fix is /static/* cached by CloudFront, /profile routed to a dynamic/private origin.

Analytics cookie: if GET /products?id=10 returns the same response regardless of an analytics_id cookie, that cookie shouldn't unnecessarily create separate cache objects — if the origin needs it, forward it via the origin request configuration without adding it to the cache key.

Language-specific content: Accept-Language: en vs fr returning "Welcome" vs "Bienvenue" means the caching design must distinguish the language variants.

Rapidly changing stock price: GET /stock/AAPL changing every second makes a long TTL inappropriate — options include a very short TTL, no caching, or a streaming/WebSocket mechanism for genuinely real-time requirements. CloudFront is not a "cache everything" service; caching is a business and correctness decision.

News website: /articles/* can use a longer TTL, /home a shorter TTL, and /api/personalized/* stays dynamic/private — different content has different freshness requirements.

Deployment: if app.js keeps the same filename across versions and the cached object is still valid, some users may keep receiving the old version. Using app.v1.js/app.v2.js or content hashes (app.abc123.js) avoids this — one of the most common production CDN techniques.

Request Collapsing & Cache Hit Ratio

Request collapsing: CloudFront can reduce duplicate origin requests when simultaneous requests for the same object and same cache key arrive at an edge while the object isn't available — 1,000 simultaneous requests can become a single origin fetch, with the response shared across all waiting requests. This only helps when requests share the same cache key; unnecessarily fragmenting the cache key (e.g. by user) makes the requests no longer equivalent for this purpose.

Cache hit ratio = cache hits ÷ total cacheable requests. For example, 800 hits out of 1,000 requests gives an 80% hit ratio. A higher hit ratio generally means more requests are served from edge cache and fewer reach the origin.

Do not optimize hit ratio at the expense of correctness or security.

A 99.9% hit ratio with users receiving incorrect/private data is a terrible architecture.

Production Cache Hierarchy

Viewer
   |
   v
CloudFront POP / Edge
   |
   v
Regional Edge Cache
   |
   v
Origin

The exact internal routing is managed by AWS, but conceptually CloudFront provides geographically distributed caching layers between viewers and the origin. Your application does not need to manually manage these edge caches.

CloudFront + S3 (OAC)

A classic architecture serves index.html, app.js, styles.css, and images/ out of S3 through CloudFront. For production, keep the S3 bucket private and let CloudFront access it using Origin Access Control (OAC):

Internet
   |
   v
CloudFront
   |
   | OAC
   v
Private S3

This avoids making the S3 bucket itself the public application entry point.

CloudFront + ALB + EC2

Internet
   |
   v
CloudFront
   |
   v
ALB
   |
   +--> EC2-1
   +--> EC2-2
   +--> EC2-3

Responsibilities are kept separate: CloudFront handles CDN/edge caching, global delivery, viewer-facing TLS, WAF integration, and routing by cache behavior. ALB handles load balancing, health checks, and distribution across instances. EC2 handles application execution and business logic. Keeping these responsibilities separate is important when designing AWS architectures.

CloudFront + WAF

AWS WAF can inspect requests before they reach the application:

User
 |
 v
CloudFront
 |
 v
WAF rules
 |
 +---- malicious -> BLOCK
 |
 +---- valid ----> cache/origin

Possible protections include SQL injection rules, XSS-related rules, rate limiting, IP restrictions, AWS managed rule groups, and custom rules. WAF does not replace application authentication and authorization.

Production Techniques & Common Mistakes

Production techniques:

Version static assets (app.abc123.js, styles.72fd12.css) and use long TTLs where appropriate.
Separate static and dynamic paths (/static/*, /images/*, /api/*) with a suitable behavior for each.
Minimize cache-key dimensions — avoid unnecessary cookies, headers, query strings, and tracking IDs.
Use Origin Request Policy appropriately — forward what the origin needs without automatically turning every value into a cache-key dimension.
Choose TTL by freshness requirements: stable static → long, frequently changing → short, private → usually not shared cached.
Use OAC for private S3 origins (CloudFront → OAC → private S3).
Use WAF to protect public CloudFront endpoints.
Monitor cache hit ratio, request count, cache misses, origin request volume, origin latency, 4xx/5xx errors, and WAF blocked requests.

Common mistakes:

Including every cookie in the cache key, creating huge cache fragmentation.
Ignoring personalization, causing incorrect or unsafe response reuse.
Using huge TTLs everywhere, creating stale-content problems.
Using tiny TTLs everywhere, destroying much of the CDN benefit and increasing origin load.
Confusing Cache Policy (what affects cache identity) with Origin Request Policy (what additional information goes to the origin).
Relying only on invalidation — versioned assets are often cleaner for deployments.
Making S3 public unnecessarily instead of using OAC for a private CloudFront-backed origin.

Production Mini Architecture

                         Internet
                            |
                            v
                       CloudFront
                            |
              +-------------+-------------+
              |                           |
          /static/*                    /api/*
              |                           |
              v                           v
             S3                         ALB
                                          |
                                     EC2/ECS
                                          |
                                      Database

Security: CloudFront routes through AWS WAF, uses OAC for S3, and enforces HTTPS/TLS.

Caching: /static/* gets a long TTL with versioned assets; /api/public/* gets carefully configured caching; /api/cart/* and /api/profile/* get no/shared cache.

Practice Labs

Practice Lab 1 — Basic CloudFront + S3

Set up a CloudFront distribution in front of an S3 bucket, update the underlying S3 object, and observe how caching and invalidation affect whether the updated content is served immediately.

Practice Lab 2 — Observe HIT/MISS

Request a static object through CloudFront twice and identify the cache HIT/MISS behavior between the first and subsequent requests.

Practice Lab 3 — Query-String Cache Key

Configure a cache policy to include a specific query string, then verify that requests with different query-string values produce independent cache entries while repeated identical requests result in a cache hit.

Practice Lab 4 — Personalized Response Safety

Build an endpoint that returns different content based on a cookie, evaluate the risk of the cache key not distinguishing between users, and redesign the caching approach safely.

Practice Lab 5 — Cache Policy vs Origin Request Policy

Configure a request so an analytics cookie is forwarded to the origin without being included in the cache key, and confirm that cache reuse is preserved while the origin still receives the value it needs.

Practice Lab 6 — TTL Experiment

Set a short TTL on an object, change the origin content, and compare the object's behavior before and after TTL expiry against manually invalidating it.

Practice Lab 7 — Cache Behaviors

Configure separate cache behaviors for /static/* and /api/* with different origins and caching rules, connecting CloudFront to existing EC2/ALB infrastructure.

Practice Lab 8 — Private S3 + OAC

Configure CloudFront to access a private S3 bucket using Origin Access Control, and confirm that direct public access to the bucket remains blocked.

Practice Lab 9 — CloudFront + ALB + EC2

Route CloudFront traffic through an ALB to multiple EC2 instances, separate /static/* and /api/* paths, and determine which paths should be cached versus served dynamically.

Practice Lab 10 — Production-Style Mini Project

Build a production-style CloudFront architecture combining an S3 origin for static assets and an ALB/EC2 origin for API traffic, incorporating OAC, cache/origin request policies, TTL, invalidation, WAF, and monitoring — then validate the design against a range of caching, security, and failure scenarios.

Interview Revision

What is CloudFront?

CloudFront is AWS's CDN. It delivers content from geographically distributed edge locations and caches reusable origin responses closer to users, reducing latency and origin load.

What is a cache hit?

A cache hit occurs when the viewer request maps to a valid cached object, so CloudFront can return it without fetching that object from the origin.

What is a cache miss?

A cache miss occurs when CloudFront cannot find a valid matching cached object, so it obtains the object from the origin and can cache the response.

What is a cache key?

The cache key identifies a cached object. It can include the path and, when configured, selected query strings, headers, and cookies.

Cache Policy vs Origin Request Policy?

Cache Policy controls what contributes to the cache key and TTL behavior. Origin Request Policy controls additional headers, cookies, and query strings that CloudFront sends to the origin without necessarily making them part of the cache key.

Why can including too many cookies be bad?

It creates many cache variants, reduces cache reuse and can lower the cache hit ratio.

Why can excluding a response-changing cookie be dangerous?

Different users can map to the same cache object and potentially receive the wrong personalized response.

Final Mental Model

                VIEWER REQUEST
                       |
                       v
                  CloudFront
                       |
                Build cache key
                       |
                       v
                 Cache lookup
                  /         \
               HIT           MISS
                |              |
                v              v
             Cached         Origin
             object            |
                |              v
                |          Response
                |              |
                |              v
                |          Cache it
                |              |
                +------<--------+
                       |
                       v
                    Viewer

And this distinction:

Cache Policy → what makes requests different? how long can objects be cached?

Origin Request Policy → what additional information does the origin need?

The single most useful rule:

Cache only what can safely be reused, make the cache key vary only on information that changes the response, and choose TTL according to the required freshness.

Another step in the AWS learning journey — understanding not just what CloudFront does, but why caching, routing, and security decisions matter in real-world systems.

AWS EFS Hands-On Lab

Tejas Shinkar — Wed, 26 Aug 2026 04:49:37 +0000

A practical EFS exercise covering shared file access, NFS security, lifecycle management, persistent mounts, and an EBS-vs-EFS copy experiment.

Problem Statement

Build and validate a shared Amazon EFS filesystem that can be mounted by multiple EC2 instances in the same VPC.

The lab was designed to verify the setup through actual hands-on tests:

Mount the same EFS filesystem on two EC2 instances.
Prove that a file written by one instance is visible from the other.
Deliberately remove NFS access and observe the failure.
Configure EFS Lifecycle Management for a 7-day transition to Infrequent Access.
Make the EFS mount persistent across an EC2 reboot using /etc/fstab.
Compare a copy to local EBS-backed storage with a copy to EFS.

Lab Architecture

                         Default VPC
        ┌─────────────────────────────────────────────┐
        │                                             │
        │   EC2-1                 EC2-2              │
        │   /mnt/efs              /mnt/efs            │
        │      │                     │                │
        │      └────── NFS :2049 ────┘                │
        │                     │                       │
        │              EFS Mount Targets              │
        │                     │                       │
        │                Amazon EFS                   │
        │                                             │
        └─────────────────────────────────────────────┘

Security Groups:

ec2-sg — SSH (22) from My IP
efs-sg — NFS (2049) from ec2-sg

EFS used (EFS_LAB, File System ID fs-083c15edb9f9076c7) was created in the same VPC as the EC2 instances, with mount targets available across the VPC's Availability Zones.

Lab Setup — Two EC2 Instances + One Shared EFS

Two Amazon Linux 2023 EC2 instances (EC2-1, EC2-2) were used. Both were associated with ec2-sg. The EFS filesystem used efs-sg.

Security rules:

ec2-sg inbound: SSH/TCP 22 from My IP
efs-sg inbound: NFS/TCP 2049 from ec2-sg

The EFS NFS rule was intentionally restricted to the EC2 security group instead of opening port 2049 broadly.

On both instances:

sudo yum update -y
sudo yum install -y amazon-efs-utils
sudo mkdir -p /mnt/efs

EFS was mounted with:

sudo mount -t efs fs-083c15edb9f9076c7:/ /mnt/efs

The mount was verified with:

df -h | grep efs

Shared-File Test — EC2-1 Writes, EC2-2 Reads

On EC2-1:

echo "Hello from EC2-1" | sudo tee /mnt/efs/shared-file.txt

On EC2-2:

cat /mnt/efs/shared-file.txt

Output:

Hello from EC2-1

This proved that both instances were using the same EFS filesystem:

EC2-1 writes → EFS (same filesystem) → EC2-2 reads

No manual file transfer between the EC2 instances was performed.

Failure Test — Remove NFS Access and Break the EFS Write

The NFS 2049 inbound rule was deliberately removed from efs-sg.

A new file write was attempted:

echo "Dummy text check from EC2-1" | sudo tee /mnt/efs/shared-file2.txt

The operation could not complete while NFS access was blocked.

The NFS rule was restored (NFS TCP 2049 from ec2-sg). The same write was repeated successfully:

Dummy text check from EC2-1

This demonstrated that the mount directory can remain present while network access to the EFS service is blocked.

Lifecycle Management — Configure 7-Day IA Transition

EFS Lifecycle Management was configured as:

Transition into Infrequent Access (IA): 7 days since last access
Transition into Archive: 90 days since last access (existing setting, unchanged)
Transition into Standard: None (existing setting, unchanged)

The 7-day IA policy was successfully configured.

The actual movement of a file into IA was not observed, because the lab did not wait seven days. The documentation therefore records the configuration rather than claiming an observed storage-class transition.

Persistent Mount Test — `/etc/fstab`

Two different behaviors were demonstrated.

EC2-1 — Manual Mount Only

EC2-1 was manually unmounted:

sudo umount /mnt/efs

The mount was checked:

mount | grep efs

No EFS mount remained.

After:

sudo reboot

the EFS filesystem did not automatically return because no EFS entry existed in /etc/fstab.

EC2-2 — Persistent Mount

The existing /etc/fstab was inspected:

cat /etc/fstab

There was no EFS entry initially. The following entry was added:

sudo sh -c 'echo "fs-083c15edb9f9076c7:/ /mnt/efs efs _netdev,tls 0 0" >> /etc/fstab'

Before rebooting, the configuration was tested safely. The existing mount was removed:

sudo umount /mnt/efs

Then /etc/fstab was applied:

sudo mount -a

The mount was verified:

mount | grep efs

After the test succeeded, EC2-2 was rebooted:

sudo reboot

After reconnecting:

mount | grep efs

showed that EFS was automatically mounted.

The shared file was also still accessible:

cat /mnt/efs/shared-file.txt

Output:

Hello from EC2-1

The final proof was therefore:

/etc/fstab → EC2 reboot → EFS automatically mounted → existing shared data available

EBS vs EFS Copy Experiment

Before the benchmark:

df -h /
df -h /mnt/efs

The root EBS filesystem had approximately 8.0 GB total, 1.8 GB used, and 6.3 GB available.

Initial `/tmp` Problem

The first attempt created the test file under /tmp:

dd if=/dev/zero of=/tmp/efs-ebs-test-1GB.img bs=1M count=1024 status=progress

It failed with:

No space left on device

Only about 479 MB was written.

The cause was identified with:

df -h /tmp

which showed:

tmpfs   457M   457M   0   100%   /tmp

So /tmp was a 457 MB tmpfs, not the EBS-backed root filesystem.

The incomplete file was removed and the test moved to /home/ec2-user.

Create the 1 GiB Test File on EBS

dd if=/dev/zero \
  of=/home/ec2-user/efs-ebs-test-1GB.img \
  bs=1M count=1024 \
  status=progress

The resulting file was approximately 1.0 GiB.

EBS → EBS

time cp \
  /home/ec2-user/efs-ebs-test-1GB.img \
  /home/ec2-user/efs-ebs-test-1GB-copy.img

Observed:

real    0m0.005s
user    0m0.000s
sys     0m0.002s

The copy succeeded.

This was not treated as a physical 1 GiB EBS throughput benchmark, because filesystem/page caching can make a local repeated copy appear extremely fast.

EBS → EFS

The EBS copy was removed.

The first EFS copy attempt without elevated privileges failed:

cp: cannot create regular file ... Permission denied

The EFS mount itself was working; the issue was filesystem write permission.

The copy was repeated with sudo:

time sudo cp \
  /home/ec2-user/efs-ebs-test-1GB.img \
  /mnt/efs/efs-ebs-test-1GB-copy.img

Observed:

real    0m7.506s
user    0m0.037s
sys     0m0.010s

The destination file was verified successfully on EFS.

Benchmark Interpretation

EBS → EBS: ~0.005 seconds

EBS → EFS: ~7.506 seconds

These numbers should not be presented as a universal EBS-vs-EFS performance ratio.

The local EBS result was affected by filesystem/page caching and is not equivalent to a cold physical-storage benchmark.

The useful architectural observation is that EFS introduces network filesystem behavior, while EBS provides block storage directly associated with the EC2 workload.

Problems Encountered and Fixes

Problem	What Happened	Resolution
EFS write stopped after NFS rule removal	Network access to EFS was blocked	Restored TCP 2049 from `ec2-sg` to `efs-sg`
EFS did not persist after manual unmount/reboot	No EFS entry existed in `/etc/fstab`	Added the EFS mount configuration to `/etc/fstab`
`grep` failed after reboot	Verification commands were entered in Windows PowerShell rather than Linux	Reconnected to EC2 through SSH
1 GiB test file failed under `/tmp`	`/tmp` was a 457 MB `tmpfs`	Created the test file under `/home/ec2-user`
EFS copy returned `Permission denied`	Current user lacked write permission at the EFS root	Repeated the copy using `sudo`
EBS copy timing looked unrealistically fast	Local filesystem/page caching affected `cp`	Treated it as an observed lab result, not a production benchmark

Evidence Collected

Two EC2 instances successfully mounted the same EFS filesystem.
EC2-1 wrote shared-file.txt.
EC2-2 read the same file and returned Hello from EC2-1.
Removing NFS 2049 access prevented the EFS write from completing.
Restoring the NFS rule allowed the write again.
EFS Lifecycle Management was configured for a 7-day IA transition.
EC2-1 demonstrated that a manual mount does not persist after reboot.
EC2-2 automatically remounted EFS after reboot through /etc/fstab.
The shared file remained accessible after reboot.
A 1 GiB file was successfully copied to EFS.
The EBS-to-EFS copy completed in approximately 7.5 seconds in this lab environment.

Production-Relevant Takeaways

Restrict EFS NFS access by security group instead of exposing TCP 2049 broadly.
Use mount -a to validate an /etc/fstab change before rebooting an instance.
Use _netdev for persistent network filesystem mounts.
Separate network connectivity from filesystem permissions when troubleshooting EFS.
Use Lifecycle Management according to actual access patterns rather than assuming every file belongs in Standard storage.
EFS is appropriate when multiple compute resources need concurrent access to the same file system; EBS is appropriate for block-storage workloads attached to EC2.
Do not use a simple cp test as a production storage benchmark. Caching, EC2 instance type, EBS configuration, EFS throughput mode, network conditions, file size, and workload characteristics all affect the result.
Check what filesystem actually backs a directory before using it for storage tests; /tmp in this lab was a tmpfs.
Clean up EFS, EC2, security groups, and test files after the lab to avoid unnecessary charges.

Lab Completion

Full EC2 + EFS shared filesystem lab ✅
Break and restore NFS access ✅
Configure 7-day EFS IA lifecycle policy ✅
Persist EFS mount through EC2 reboot ✅
EBS vs EFS copy experiment ✅

The lab went beyond a successful service setup: it included deliberate failure, troubleshooting, permissions, persistence testing, filesystem identification, lifecycle configuration, and a controlled storage experiment.

Your Torrent Isn't Downloading From "The Internet"

Tejas Shinkar — Tue, 25 Aug 2026 13:40:47 +0000

We've All Used Torrents. But Who's Actually Sending You the File? 👀

You've downloaded a 5GB movie/game/software through a torrent.

But there's no giant server sitting somewhere going:

"Yep, I'll upload 5GB to this guy." 😂

Instead, that file is split into tiny pieces.

You download different pieces from different people at the same time — and while doing that, you upload the pieces you already have to others.

So instead of:

Server → You

it's more like:

You ↔ You ↔ You ↔ You ↔ You

Everyone is both a downloader and an uploader.

Basically:

"I'll give you my piece if you give me yours."

And suddenly that torrent client you opened 10 years ago is looking suspiciously like a distributed systems lecture. 💀

AWS VPC Hands-On Lab

Tejas Shinkar — Tue, 25 Aug 2026 11:52:10 +0000

Focus: A practical record of the VPC networking work completed in AWS.
This document records what was built, tested, observed, troubleshot, and learned during the hands-on sessions.

📚 VPC Theory & Concepts

The concepts behind these hands-on practicals are covered in the following VPC articles:

VPC Part 1 — Networking Fundamentals: VPC, Subnets, Internet Gateway, NAT Gateway, Security Groups vs NACLs
https://dev.to/tejas_shinkar/aws-networking-fundamentals-vpc-subnets-igw-nat-the-sg-vs-nacl-battle-eel
VPC Part 2 — Extended Networking Concepts: Route Tables, Blackhole Routes, VPC Flow Logs, VPC Endpoints, Transit Gateway
https://dev.to/tejas_shinkar/aws-vpc-extended-route-tables-blackhole-routes-flow-logs-vpc-endpoints-transit-gateway-og4

These articles cover the underlying concepts in detail, while this document focuses specifically on the hands-on implementation, testing, troubleshooting, and observations from the VPC practicals.

1.Building a VPC from Scratch with Public and Private Subnets

Objective

Build the core VPC networking architecture manually instead of using the AWS VPC and more wizard. The goal was to understand how VPC, subnets, route tables, an Internet Gateway, a NAT Gateway, and EC2 instances fit together to provide public and private connectivity.

Architecture

VPC: 10.200.0.0/16

                         Internet
                            |
                           IGW
                            |
              +-------------+-------------+
              |                           |
       Public Subnet                Private Subnet
       10.200.0.0/24                10.200.1.0/24
              |                           |
         Public EC2                   Private EC2
              |                           |
              +---- NAT Gateway <---------+
                         |
                  Public Subnet

What We Built

VPC: lab-vpc
CIDR: 10.200.0.0/16
Public subnet: 10.200.0.0/24
Private subnet: 10.200.1.0/24
Internet Gateway: lab-igw
Public route table
Private route table
NAT Gateway: lab-nat-gw
Elastic IP for NAT Gateway
Public EC2
Private EC2

Implementation

VPC

Created VPC lab-vpc with CIDR 10.200.0.0/16, using VPC only so the networking components could be built manually.

Subnets

Created:

public-subnet-1 — 10.200.0.0/24 — ap-south-1a
private-subnet-1 — 10.200.1.0/24 — ap-south-1b

The subnets were placed in different Availability Zones to introduce a basic multi-AZ layout.

Internet Gateway

Created and attached lab-igw. The IGW alone was not enough to make the subnet public; the public subnet also needed a route to the IGW.

Public Route Table

Configured 0.0.0.0/0 → Internet Gateway and associated it with public-subnet-1.

NAT Gateway

Created lab-nat-gw. The NAT Gateway was placed in the public subnet and given an Elastic IP.

The private route table was then configured with 0.0.0.0/0 → NAT Gateway. This produced the outbound path:

Private EC2 → Private Route Table → NAT Gateway → Internet Gateway → Internet

EC2 Testing

Created public-ec2 and private-ec2.

The public EC2 had a public IP. The private EC2 had no public IP.

From the public EC2, we connected to the private EC2 using its private IP.

From the private EC2, we tested outbound Internet access:

curl https://checkip.amazonaws.com

The returned public IP corresponded to the NAT Gateway's Elastic IP rather than the private EC2's private address.

Evidence / Result

The lab successfully demonstrated:

Public EC2 → Internet

Private EC2 → NAT Gateway → Internet

The private EC2 could reach the Internet without having a public IP.

Problems / Troubleshooting

EC2 Instance Connect

Browser-based EC2 Instance Connect caused connectivity issues during the lab. The SSH Security Group rule was temporarily broadened to 0.0.0.0/0 to avoid making SSH access itself the blocker.

This was a temporary lab workaround, not a production configuration.

NAT Gateway placement

The placement initially raised the question of why the NAT Gateway belongs in a public subnet. The final working architecture made the reason clear:

NAT Gateway → Public Subnet → Public Route Table → IGW → Internet

Hands-On Takeaways

A subnet uses the route table associated with it.
The public route table pointed to the IGW.
The private route table pointed to the NAT Gateway.
The NAT Gateway provided outbound Internet access for the private EC2.
A private EC2 does not need a public IP to initiate outbound Internet traffic.
NAT Gateway placement and routing must be considered together.

2.Blocking HTTP While Allowing HTTPS with a Custom NACL

Objective

Create a custom Network ACL for a public subnet and use an Apache web server to prove that NACL rules can selectively allow or reject traffic.

The practical goal was simple: HTTP should fail, while HTTPS should work, allowing the NACL decision to be observed through an actual application.

Architecture

Internet
   |
   v
Internet Gateway
   |
   v
Custom NACL
   |
   v
Public Subnet
   |
   v
EC2
 |
Apache

What We Built

Custom VPC for the lab
Public subnet
Custom Network ACL
Security Group
Public EC2
Apache HTTP server
HTTPS configuration using a temporary self-signed certificate

NACL Configuration

The important rules were:

Rule	Port	Action
HTTP	80	DENY
HTTPS	443	ALLOW
SSH	22	ALLOW
Return traffic	1024–65535 (TCP)	ALLOW

The SSH rule was temporarily broadened during connectivity troubleshooting.

Apache Setup

Installed Apache:

sudo dnf install httpd -y

Created a test page:

echo "<h1>NACL Lab - HTTP/HTTPS Test</h1>" | sudo tee /var/www/html/index.html

Enabled HTTPS using mod_ssl and a temporary self-signed certificate. Apache was verified to be listening on port 443.

HTTP Test

Opened http://<EC2-public-IP>.

Result: ERR_CONNECTION_TIMED_OUT — the request was blocked by the NACL.

HTTPS Test

Opened https://<EC2-public-IP>.

The browser displayed the expected self-signed certificate warning. After proceeding through the warning, the Apache page loaded successfully.

Evidence / Result

HTTP :80 → BLOCKED

HTTPS :443 → WORKED

This gave us a direct application-level demonstration of the NACL rules.

Problems / Troubleshooting

During Apache installation, the package manager initially timed out.

The problem turned out to be the NACL's stateless return traffic. The EC2 could send EC2:ephemeral-port → Internet:443, but the response returned to the EC2's ephemeral port and was not initially allowed.

Adding an inbound rule (1024-65535 → ALLOW) resolved the package installation.

This was an important real failure rather than a theoretical example.

Hands-On Takeaways

NACL rules apply at the subnet level.
Security Group and NACL rules both affect whether traffic reaches the instance.
A restrictive NACL must account for the return path.
Ephemeral ports become important when working with stateless NACLs.

3.Observing Network Traffic with VPC Flow Logs and CloudWatch

Objective

Enable VPC Flow Logs and use the resulting CloudWatch records to inspect real network traffic.

The goal was to move from simply configuring networking to observing what actually happened on the network.

What We Configured

Flow Logs were configured for all traffic, sent to CloudWatch Logs, with 1-minute aggregation, in log group /vpc/flow-logs/nacl-lab.

AWS created/used the required IAM service role for publishing Flow Logs to CloudWatch.

Traffic Generated

From the EC2:

curl http://checkip.amazonaws.com
curl https://checkip.amazonaws.com

SSH traffic had also been generated by connecting to the instance.

CloudWatch Investigation

CloudWatch → Logs → Log groups → /vpc/flow-logs/nacl-lab → Log stream

The records exposed fields such as Source IP, Destination IP, Source Port, Destination Port, Protocol, Action, Bytes, and Packets.

One observed HTTPS-related flow contained the important values:

Source IP:         13.213.253.85
Destination IP:    10.50.1.165
Source Port:       443
Destination Port:  32798
Protocol:          6 (TCP)
Action:            ACCEPT

The reverse direction showed the corresponding flow back to the EC2's ephemeral port. We also observed REJECT records.

Evidence / Result

The Flow Logs successfully showed both ACCEPT and REJECT traffic, and allowed us to identify:

who was communicating
where traffic was going
which ports were involved
which protocol was used
whether the network flow was accepted or rejected

Hands-On Takeaways

Flow Logs are useful for troubleshooting network behavior.
They can expose rejected traffic that is otherwise difficult to understand.
They provide network-level evidence but do not replace application or authentication logs.
Flow Log records may take some time to appear because of aggregation.

4.Proving the Stateless NACL Gotcha with a Private EC2

Objective

Build a fresh private-network scenario specifically to demonstrate why a restrictive NACL can allow an outbound HTTPS request but still break the connection when the response returns.

The experiment was deliberately designed to fail first and then be fixed.

Architecture

                         Internet
                            |
                           IGW
                            |
                    Public Route Table
                            |
                      NAT Gateway
                            |
                    Private Route Table
                            |
                     Private Subnet
                            |
                       Private EC2

Lab CIDRs:

VPC: 10.60.0.0/16
Public subnet: 10.60.0.0/24
Private subnet: 10.60.1.0/24

Routing Configuration

Public route table: 0.0.0.0/0 → Internet Gateway
Private route table: 0.0.0.0/0 → NAT Gateway

The NAT Gateway was placed in the public subnet. The EC2 was placed in the private subnet with Auto-assign Public IP: Disabled.

Initial NACL

Outbound: 100 → HTTPS 443 → ALLOW, * → ALL → DENY
Inbound: * → ALL → DENY

The deliberate design was:

Allow the HTTPS request out, but do not allow the return traffic back in.

First Test — Failure

From the private EC2:

curl https://checkip.amazonaws.com

The command hung. The request path was:

EC2 → NACL outbound → HTTPS 443 ALLOW → NAT Gateway → Internet

The response path returned to an ephemeral port:

Internet:443 → EC2:ephemeral-port → NACL inbound → DENY

Fix

Added the missing inbound rule: 100 → TCP 1024-65535 → ALLOW. The catch-all deny remained below it.

Second Test — Success

Ran again:

curl https://checkip.amazonaws.com

Result: 13.202.19.75 — the HTTPS request now completed successfully.

Evidence / Result

Before: Outbound 443 → ALLOW. Return traffic → BLOCKED. curl → timeout.

After: Outbound 443 → ALLOW. Inbound ephemeral ports → ALLOW. curl → public IP returned.

Problems / Troubleshooting

The main issue was initially easy to misinterpret as a NAT or Internet problem.

Following the traffic path isolated the failure to the inbound NACL rule. The lab therefore became a useful troubleshooting exercise:

Request works? → check return path → NACL is stateless → check ephemeral ports

Hands-On Takeaways

A private EC2 can reach the Internet through a NAT Gateway without a public IP.
The private route table must point to the NAT Gateway.
The public route table used by the NAT Gateway must point to the IGW.
Restrictive NACLs must explicitly permit both sides of a connection.
Ephemeral ports are not an abstract networking detail; they directly affected the lab's outcome.
Following the packet path was more useful than randomly changing AWS resources.

Practical Reasoning Drills

P1.Longest Prefix Match — Choosing the Most Specific Route

Objective

Use a route table with overlapping routes and determine which route AWS would select for different destination IPs.

Given:

Destination	Target
`10.0.0.0/16`	Local
`10.0.5.0/24`	`pcx-abc`
`0.0.0.0/0`	`igw-xyz`

Results

10.0.5.10 → 10.0.5.0/24 → pcx-abc (reason: /24 is more specific than /16 and /0)
10.0.9.10 → 10.0.0.0/16 → Local (reason: it does not belong to 10.0.5.0/24, but it does belong to 10.0.0.0/16)
52.1.1.1 → 0.0.0.0/0 → igw-xyz (reason: it does not match either 10.x.x.x route, so the default route is used)

Takeaway

More specific prefix wins: /24 > /16 > /0

p2.NAT Gateway Deletion — Diagnosing a Blackhole Route

Objective

Work through a realistic failure where a NAT Gateway is deleted but private EC2 instances still depend on it for outbound Internet access.

Scenario

Private route table: 0.0.0.0/0 → NAT Gateway. NAT Gateway is deleted. The route can become 0.0.0.0/0 → NAT Gateway (BLACKHOLE).

Troubleshooting Path

Follow the traffic path:

EC2 → Security Group → NACL → Private Route Table → NAT Gateway → Public Route Table → IGW → Internet

In this exact scenario, the private route table is the high-value first check.

Takeaway

Don't immediately recreate the NAT Gateway. First identify where the traffic path breaks and check whether the route points to a valid target.

P3.Choosing Gateway vs Interface VPC Endpoints

Objective

Decide which endpoint type should be used when private workloads need AWS service access without relying on a public Internet path.

Service	Endpoint Type
S3	Gateway Endpoint
DynamoDB	Gateway Endpoint
Secrets Manager	Interface Endpoint

Takeaway

Gateway endpoints integrate with route tables. Interface endpoints use private network interfaces/PrivateLink.

P4.Transit Gateway vs VPC Peering — Scaling the Connectivity Model

Objective

Choose an architecture for an environment that has 3 VPCs today, expects 12 additional VPCs, and also needs VPN connectivity to the head office.

Peering Math

Full-mesh peering connections: N × (N - 1) / 2. For 3 VPCs → 3 connections. For 15 VPCs → 105 connections.

Decision

For this growth scenario, Transit Gateway is the more scalable direction. The model becomes:

VPC-A ─┐
VPC-B ─┤
VPC-C ─┼── Transit Gateway ── VPN ── Head Office
VPC-D ─┤
...    ┘

Takeaway

The decision was based on connectivity scale and centralized routing, not simply on the number of VPCs that exist today.

VPC Flow Logs — Investigating Suspicious SSH Traffic

Objective

Use Flow Logs as network-level evidence when investigating a suspected unauthorized SSH attempt against an EC2 instance.

Investigation

Enable Flow Logs for the relevant VPC/subnet and send records to CloudWatch. Look for TCP traffic on destination port 22, matching the target EC2's destination IP, from a suspicious source IP, and check whether the action was ACCEPT or REJECT.

The investigation can establish that a source IP attempted network-level SSH traffic.

Important distinction

An ACCEPT Flow Log does not prove that SSH authentication succeeded. If network traffic was accepted, OS-level SSH/authentication logs must be checked to determine whether a login actually occurred.

Problems and Lessons Across the Labs

EC2 Instance Connect vs Network Configuration

We encountered multiple browser-based EC2 Instance Connect issues.

The important lesson was to distinguish "can I manage/connect to the instance?" from "is the network experiment itself working?"

Temporary broad SSH rules were used only to remove management connectivity as a variable during the labs.

NAT Gateway and Private Subnet Design

The NAT Gateway placement initially required clarification. The working architecture established the relationship:

Private EC2 → Private Route Table → NAT Gateway → Public Route Table → IGW → Internet

The NAT Gateway therefore belongs in a subnet with a working path to the IGW.

Route Table Associations

A subnet association means:

The subnet uses this route table for its routing decisions.

This distinction became important when building separate public and private routing paths.

Duplicate Default Route

A private route table already containing 0.0.0.0/0 → IGW could not also contain another 0.0.0.0/0 → NAT Gateway. The existing default route had to be replaced with the correct target.

NACL Return Traffic

The most valuable troubleshooting lesson was that a restrictive NACL can allow the outbound request but still break the connection because the response arrives on an ephemeral port.

This was observed directly through failed commands and then fixed by adding TCP 1024-65535 → ALLOW.

Flow Logs as Evidence

Flow Logs provided actual network records instead of relying only on assumptions about what the network was doing. They were useful for identifying source, destination, ports, protocol, and ACCEPT/REJECT status.

Production-Relevant Takeaways from the Hands-On Work

These are intentionally limited to lessons that came directly from the practical work.

Separate public and private traffic paths

A production design should not expose application/database resources unnecessarily. A common model is:

Internet → ALB / Public tier → Private application tier → Private database tier

Plan NAT for both availability and cost

For multi-AZ workloads, a single NAT Gateway can create a cross-AZ dependency. NAT Gateway hourly and data-processing costs also make unnecessary outbound Internet traffic worth considering.

Use least-privilege Security Groups

The labs temporarily used broad SSH access to troubleshoot EC2 Instance Connect. Production should instead restrict management access to known sources or use an appropriate private management path.

Treat NACLs as an explicit network boundary

If restrictive NACLs are used, both directions of a connection need to be considered because NACLs are stateless.

Use VPC Flow Logs for visibility

Flow Logs can be useful for troubleshooting, rejected-traffic analysis, security investigations, and validating routing/security changes. But they provide network-level evidence, not application-level authentication evidence.

Design from the traffic requirement

A useful sequence is:

What traffic is required? → Where does it originate? → Where does it need to go? → What route provides that path? → What security controls permit it? → How will we observe/troubleshoot it?

Remaining VPC Practicals

The following two practicals remain to be completed:

Connecting VPCs with Peering, Then Scaling the Design with Transit Gateway

Build two VPCs, establish private connectivity using VPC Peering, then introduce a third VPC and move toward a Transit Gateway-based connectivity model.

The focus will be on the route changes and traffic paths required for VPC-to-VPC communication.

Private S3 Access with a Gateway Endpoint and Flow Log Verification

Add an S3 Gateway Endpoint to a VPC and verify the resulting traffic path using VPC Flow Logs.

The goal is to confirm how the endpoint changes the path used for S3 access and to distinguish that traffic from an Internet/NAT-based path.

AWS S3 Hands-On Lab

Tejas Shinkar — Fri, 21 Aug 2026 12:36:02 +0000

I recently worked through a hands-on Amazon S3 practical series covering the features I would actually expect to use while working with AWS storage.

Instead of only documenting definitions, this post focuses on what I configured, the commands I used, how I verified the behavior, and what I observed when something went wrong.

For the concepts behind these practicals, I have already covered S3 in two detailed sessions:

Session 1: AWS S3 Deep Dive — Objects, Encryption, Bucket Policies & Everything In Between
Session 2: AWS S3 — Versioning, Static Hosting, CORS, Object Lock & More

This post is the practical companion to those two sessions. The concepts are covered there; here I focus on actually building, testing, verifying, and troubleshooting the S3 features.

The labs covered:

S3 bucket configuration and lifecycle management
Bucket policies with IAM, EC2 and HTTPS-only access
SSE-KMS encryption with CloudTrail verification
Pre-signed URLs
AWS CLI s3 sync
S3 Versioning and version recovery
Static website hosting
S3 CORS
S3 Object Lock

Note: This is a practical write-up, so I have intentionally kept the focus on implementation and verification rather than turning it into a generic S3 theory article.

1. S3 Bucket Configuration and Lifecycle Management

Objective

Create an S3 bucket with a secure baseline and configure a lifecycle rule that automatically transitions objects to lower-cost storage classes over time.

Configuration

For the lab:

Block Public Access remained enabled.
Bucket Versioning was enabled.
Lifecycle rule: s3-lab-lifecycle
The rule applied to all objects.
Current objects transition to:
- Standard-IA after 30 days
- Glacier Flexible Retrieval after 90 days

The lifecycle flow was:

Day 0 → object uploaded → Day 30 → Standard-IA → Day 90 → Glacier Flexible Retrieval

Result

The lifecycle rule was successfully created and enabled, and the S3 console confirmed the configured transition periods.

What This Demonstrates

Instead of manually moving old objects, S3 Lifecycle can automatically transition objects between storage classes based on age and the rules we define.

2. S3 Bucket Policy — IAM Read, EC2 Upload and HTTPS-only Access

Objective

Configure a bucket policy that:

Allows a specific IAM user to read objects.
Allows an EC2 role to upload objects.
Denies S3 access when the request is not using HTTPS.

Resources

Bucket: policytesting-bucket-ts
IAM User: Tejas-IAM
EC2 Role: EC2-S3-ReadOnly-Role
Region: ap-south-1

Policy Design

The policy contained three logical statements:

IAM User → s3:GetObject → S3 bucket

EC2 IAM Role → s3:PutObject → S3 bucket

HTTP request → Explicit Deny → S3 bucket

The important security condition was:

"Bool": {
    "aws:SecureTransport": "false"
}

This denies requests that are not using HTTPS.

Verification

The IAM user successfully downloaded an object from S3 using AWS CLI.

An EC2 instance was launched with EC2-S3-ReadOnly-Role attached, and the role was verified from the instance. The EC2 instance was then able to upload an object to S3.

Result

Both permission paths worked as intended:

IAM user → object read
EC2 role → object upload
Non-HTTPS access → explicitly denied

Practical Takeaway

A useful S3 security pattern is to combine:

identity-based permissions,
resource-based bucket policies,
and explicit security-condition denies.

3. SSE-KMS Encryption with CloudTrail Verification

Objective

Configure S3 default encryption using a customer-managed KMS key and verify the resulting cryptographic operations through CloudTrail.

KMS Configuration

The lab used:

Key alias: s3-lab-key
Key type: Symmetric
Key usage: Encrypt and decrypt
Region: ap-south-1
Key administrator: Tejas-IAM
Key user: Tejas-IAM

Configure S3 Encryption

The bucket sse-kms-bucket-tejas was configured for server-side encryption using the customer-managed KMS key.

Verification

After uploading and downloading an object, I checked CloudTrail Event History. The relevant KMS operations were GenerateDataKey and Decrypt — GenerateDataKey appeared during the encryption flow, while Decrypt events were associated with accessing encrypted data.

Troubleshooting

Initially, the expected cryptographic events were not obvious in CloudTrail. Instead of relying only on the initial event list, I changed the lookup to search by Event name. That exposed the expected GenerateDataKey and Decrypt events.

Result

SSE-KMS was successfully configured and the related KMS operations were visible through CloudTrail.

Practical Takeaway

Encryption configuration alone is not always enough when troubleshooting. CloudTrail can provide the audit trail needed to understand which AWS services and cryptographic operations were involved.

4. S3 Pre-signed URL

Objective

Keep an S3 object private while providing temporary access through a pre-signed URL.

Object

Bucket: sse-kms-bucket-tejas
Object: presigned-test.txt

The object itself remained private. No public permissions were added.

Generate the URL

aws s3 presign s3://sse-kms-bucket-tejas/presigned-test.txt --expires-in 300

300 seconds = 5 minutes.

Test

I opened the generated URL in an Incognito browser window. The private object was accessible successfully.

After the five-minute validity period expired, I refreshed the same URL. S3 returned:

AccessDenied
Request has expired

Result

The test demonstrated the main behavior of a pre-signed URL:

Private S3 object + temporary signed URL → temporary access → URL expires → access denied

Practical Takeaway

A pre-signed URL is useful when an application needs to provide temporary access to a private S3 object without making the object itself public.

5. AWS CLI `s3 sync`

Objective

Use aws s3 sync to synchronize a local directory with S3 and verify that only changed files are uploaded during a subsequent sync.

Create the Test Directory

mkdir s3-sync-lab
cd s3-sync-lab

echo "File 1 - original content" > file1.txt
echo "File 2 - original content" > file2.txt

Initial Sync

aws s3 sync . s3://sse-kms-bucket-tejas/s3-sync-lab/
aws s3 ls s3://sse-kms-bucket-tejas/s3-sync-lab/

The first synchronization uploaded both files.

Modify One File

echo "File 1 - UPDATED content" > file1.txt
aws s3 sync . s3://sse-kms-bucket-tejas/s3-sync-lab/

This time, only file1.txt was uploaded.

Result

The practical demonstrated the differential behavior of aws s3 sync — unchanged content did not need to be uploaded again.

Practical Takeaway

This is useful for scripts, backups and deployment workflows where repeatedly uploading an entire directory would be unnecessary.

6. S3 Versioning and Version Recovery

Objective

Enable Versioning, upload the same object key multiple times, identify the generated Version IDs, retrieve an older version, and remove the current version to demonstrate recovery.

Enable Versioning

Versioning was enabled on sse-kms-bucket-tejas.

Create Multiple Versions

The same object key (version-test-tejas.txt) was reused across three uploads:

echo "Version 1 - Created by Tejas" > version-test-tejas.txt
aws s3 cp version-test-tejas.txt s3://sse-kms-bucket-tejas/version-test-tejas.txt

echo "Version 2 - Updated by Tejas" > version-test-tejas.txt
aws s3 cp version-test-tejas.txt s3://sse-kms-bucket-tejas/version-test-tejas.txt

echo "Version 3 - Final update by Tejas" > version-test-tejas.txt
aws s3 cp version-test-tejas.txt s3://sse-kms-bucket-tejas/version-test-tejas.txt

List Versions

aws s3api list-object-versions \
  --bucket sse-kms-bucket-tejas \
  --prefix version-test-tejas.txt

The practical produced multiple retained versions for the same object key. The latest version was marked as current, while previous versions remained available.

Retrieve an Older Version

aws s3api get-object \
  --bucket sse-kms-bucket-tejas \
  --key version-test-tejas.txt \
  --version-id <VERSION_ID> \
  version-1-retrieved.txt

cat version-1-retrieved.txt

The historical content was:

Version 1 - Created by Tejas

Delete the Current Version

aws s3api delete-object \
  --bucket sse-kms-bucket-tejas \
  --key version-test-tejas.txt \
  --version-id <CURRENT_VERSION_ID>

aws s3api list-object-versions \
  --bucket sse-kms-bucket-tejas \
  --prefix version-test-tejas.txt

The previous version became the latest available version.

Troubleshooting

The version listing output opened in the terminal's less viewer — pressing q returns to the shell.

There was also an initial retrieval mistake where cat version-1-retrieved.txt failed because the file had not yet been created. After running get-object successfully, the historical content could be read.

Result

This lab demonstrated that Versioning can preserve historical object states and allow an older version to be retrieved even after a newer version becomes current.

7. S3 Static Website Hosting

Objective

Host a simple static website directly from an S3 bucket and configure a custom error page.

Bucket

Bucket: s3-static-website-tejas
Region: ap-south-1

For this lab, Block Public Access was disabled because the website content needed to be publicly readable through the S3 website endpoint.

Website Configuration

Static website hosting was enabled with:

Index document: cors-test.html
Error document: error.html

Verification

The S3 region-specific website endpoint was opened in a browser. The website loaded successfully.

A non-existent page was then requested to trigger the configured error document. The custom 404 page was returned successfully.

Result

Both paths were verified:

Valid request → index/website content

Invalid request → custom error.html

Practical Takeaway

S3 website hosting is useful for simple static content, although real production architectures may place additional services such as CloudFront in front of the content.

8. S3 CORS

Objective

Demonstrate a browser cross-origin request failure and then allow the request using an S3 CORS configuration.

This was one of the more useful troubleshooting labs because the browser behavior made the difference between S3 access and browser CORS permission very visible.

Resources

Origin website bucket: s3-cors-origin-tejas
Object bucket: s3-cors-lab-tejas
Object: cat.jpg

The webpage was hosted from one S3 website origin and attempted to fetch the image from another S3 origin.

Test Before CORS

The webpage attempted a JavaScript fetch() request for cat.jpg. The browser blocked the response because the required Access-Control-Allow-Origin header was not present. The browser console showed the CORS failure.

Configure CORS

For the practical, the object bucket was configured with:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET"],
    "AllowedOrigins": ["*"],
    "ExposeHeaders": []
  }
]

Test Again

A fresh pre-signed URL was generated and the browser test was repeated. The final response showed:

Response type: cors
Status: 200
Access-Control-Allow-Origin: *
Blob: image/jpeg

The image was successfully returned as a Blob.

Important Troubleshooting Observation

During the lab, an expired pre-signed URL initially produced HTTP 403 with an XML error response — separate from the CORS configuration itself. After generating a fresh pre-signed URL, the request returned HTTP 200 and included the CORS response header.

Another useful observation was that fetch() does not automatically reject its promise just because the HTTP status is 403. The response status should be checked explicitly, for example:

fetch(url)
  .then(response => {
    console.log("Status:", response.status);
    console.log("CORS header:",
      response.headers.get("Access-Control-Allow-Origin"));
    return response.blob();
  })
  .then(data => console.log("Blob:", data));

Result

The practical demonstrated both sides of CORS:

No CORS header → browser blocks cross-origin access

CORS configured → HTTP 200 + CORS header → JavaScript receives the object

9. S3 Object Lock

Objective

Protect an S3 object version using Governance Mode and verify that a normal delete request is blocked while retention is active.

Bucket Configuration

A new bucket was created: s3-object-lock-tejas. Versioning and Object Lock were enabled.

Default retention was left disabled because retention would be applied to the specific test object version.

Create and Upload the Test Object

echo "Object Lock test created by Tejas" > tejas-object-lock.txt

aws s3 cp tejas-object-lock.txt \
s3://s3-object-lock-tejas/

Find the Version ID

aws s3api list-object-versions \
  --bucket s3-object-lock-tejas \
  --prefix tejas-object-lock.txt

The response provided the Version ID for the uploaded object.

Apply Governance Retention

The object version was configured with Governance Mode and a one-day retention period:

aws s3api put-object-retention \
  --bucket s3-object-lock-tejas \
  --key tejas-object-lock.txt \
  --version-id "<VERSION_ID>" \
  --retention "Mode=GOVERNANCE,RetainUntilDate=<RETENTION_DATE>"

Verify Retention

aws s3api get-object-retention \
  --bucket s3-object-lock-tejas \
  --key tejas-object-lock.txt \
  --version-id "<VERSION_ID>"

The response showed:

{
  "Retention": {
    "Mode": "GOVERNANCE",
    "RetainUntilDate": "<RETENTION_DATE>"
  }
}

Attempt Deletion

A normal delete was attempted:

aws s3api delete-object \
  --bucket s3-object-lock-tejas \
  --key tejas-object-lock.txt \
  --version-id "<VERSION_ID>"

S3 rejected the operation with:

AccessDenied
Access Denied because object protected by object lock.

Result

The retention worked as expected. The object version could not be normally deleted while the Object Lock retention period was active.

Governance vs Compliance

The practical used Governance Mode. The important distinction is:

Governance Mode provides retention protection while allowing appropriately authorized bypass operations.
Compliance Mode is stricter and is designed for scenarios where even privileged administrators should not be able to bypass the retention.

What These Practicals Demonstrated

After completing the labs, the S3 concepts were no longer just console options or documentation terms.

The practical sequence covered:

Feature	Practical verification
Lifecycle	Automatic storage-class transition rules
Bucket Policy	IAM read + EC2 upload + HTTPS-only deny
SSE-KMS	S3 encryption using customer-managed KMS
CloudTrail	`GenerateDataKey` and `Decrypt` events
Pre-signed URL	Temporary private-object access
CLI Sync	Differential synchronization
Versioning	Historical versions and recovery
Static Website	Website endpoint + custom 404
CORS	Browser block → CORS-enabled success
Object Lock	Governance retention → delete blocked

Troubleshooting Lessons

A few issues during the labs were particularly useful because they showed how AWS behaves in real situations.

1. CloudTrail Events Were Not Immediately Obvious

Searching by event name exposed the expected KMS operations:

GenerateDataKey
Decrypt

2. Long AWS CLI Output Can Open in `less`

When a command such as list-object-versions produces a long response, pressing q returns to the shell.

3. Generate the Output File Before Reading It

Running cat version-1-retrieved.txt before running get-object results in No such file or directory. The correct sequence is:

S3 version → get-object → local file created → cat local file

4. CORS and HTTP Errors Are Different Problems

A request can fail because:

the object URL expired,
S3 returned 403,
the object does not exist,
or the browser blocks a successful cross-origin response because the CORS headers are missing.

Testing with a fresh pre-signed URL helped separate the authentication/expiry issue from the CORS issue.

5. Object Lock Actually Blocks Deletion

The Object Lock test was especially clear because S3 returned an explicit AccessDenied when a protected object version was deleted.

Final Takeaway

The biggest value of these S3 labs was not memorizing the S3 console. It was seeing how the different controls interact:

                 Amazon S3
                     |
       +-------------+-------------+
       |             |             |
   Security       Storage        Access
       |             |             |
  IAM / Policy   Lifecycle     Pre-signed URL
  HTTPS-only     Versioning    CORS
  SSE-KMS        Object Lock
       |
   CloudTrail

S3 becomes much easier to reason about once these features are connected to actual operations:

Who can access the object?
How is the object encrypted?
How long should it remain in a storage class?
What happens when the object is overwritten?
Can an old version be recovered?
Can the object be accessed temporarily without making it public?
Can a browser access it cross-origin?
Can the object be deleted during a retention period?

These practicals gave me hands-on verification of each of those questions rather than only theoretical answers.

Commands Used in the Labs

A compact list of the most important commands from the practicals:

# Pre-signed URL
aws s3 presign s3://sse-kms-bucket-tejas/presigned-test.txt --expires-in 300

# Sync local directory to S3
aws s3 sync . s3://sse-kms-bucket-tejas/s3-sync-lab/

# List object versions
aws s3api list-object-versions \
  --bucket sse-kms-bucket-tejas \
  --prefix version-test-tejas.txt

# Retrieve a specific version
aws s3api get-object \
  --bucket sse-kms-bucket-tejas \
  --key version-test-tejas.txt \
  --version-id <VERSION_ID> \
  version-1-retrieved.txt

# Delete a specific version
aws s3api delete-object \
  --bucket sse-kms-bucket-tejas \
  --key version-test-tejas.txt \
  --version-id <VERSION_ID>

# List Object Lock versions
aws s3api list-object-versions \
  --bucket s3-object-lock-tejas \
  --prefix tejas-object-lock.txt

# Apply Object Lock retention
aws s3api put-object-retention \
  --bucket s3-object-lock-tejas \
  --key tejas-object-lock.txt \
  --version-id "<VERSION_ID>" \
  --retention "Mode=GOVERNANCE,RetainUntilDate=<RETENTION_DATE>"

# Verify Object Lock retention
aws s3api get-object-retention \
  --bucket s3-object-lock-tejas \
  --key tejas-object-lock.txt \
  --version-id "<VERSION_ID>"

# Attempt deletion of a protected version
aws s3api delete-object \
  --bucket s3-object-lock-tejas \
  --key tejas-object-lock.txt \
  --version-id "<VERSION_ID>"

Conclusion

This S3 practical series covered the service from basic bucket configuration through security, encryption, access control, synchronization, version recovery, static hosting, browser CORS behavior and immutable-object protection.

The most useful part was not simply getting a "Success" message from AWS. It was deliberately testing the expected behavior — including expired URLs, historical versions, browser CORS failures and an Object Lock deletion failure.

That is the kind of hands-on understanding I want to build as I continue learning AWS, Cloud and DevOps.

AWS EC2 Hands-On Lab

Tejas Shinkar — Thu, 20 Aug 2026 17:08:22 +0000

EC2, IAM Roles, EBS, AMIs, Snapshots, CLI and Boto3

Part of my AWS hands-on practical journey. This post documents the EC2 labs and practical tasks I completed, including what I built, how I verified it, and the problems I encountered along the way. For the concepts behind these EC2 practicals, see EC2 Session.

Practicals Covered

#	Practical
1	EC2 Web Server with User Data
2	EC2 + S3 Access via IAM Role
3	EC2 Instance & EBS Volume Resize
4	Custom AMI Creation & Validation
5	EBS Snapshot Cross-Region Disaster Recovery
6	Elastic IP
7	EC2 Operations using AWS CLI
8	EC2 Cost Estimation
9	EC2 Automation with Boto3

Lab 1 — EC2 Web Server with User Data

What I Did

Launched an Amazon Linux 2023 EC2 instance using t2.micro.
Created a security group:
- HTTP (80) → 0.0.0.0/0
- SSH (22) → My IP
Added a User Data script during launch.
The script installed Apache (httpd), started and enabled the service, retrieved the Instance ID and Availability Zone from EC2 metadata, and created a custom index.html.
Opened the instance's public IP in a browser.

Verification

The instance automatically configured itself during first boot and served a custom webpage containing its Instance ID and Availability Zone.

Apache was verified with:

sudo systemctl status httpd

The service was active, enabled, and listening on port 80.

What Failed

The first instance did not work because of setup issues, so it was recreated cleanly. SSH key-access problems were also encountered on Windows.

Troubleshooting Notes

User Data was added before launching because it normally runs during first boot.
The .pem file needed to be locally accessible and readable by SSH.
SSH keys stored in cloud-synced folders such as OneDrive can cause file-access or permission issues.
The correct Security Group needed to be attached.

The troubleshooting order that worked was:

Security Group → EC2 status → SSH → Apache → User Data

Lab 2 — EC2 + S3 Access via IAM Role

What I Did

The goal was to allow EC2 to access S3 without storing AWS access keys on the instance.

Created:

S3 bucket: lab-bucket-tejasshinkar
Test object: test.txt
IAM Role: EC2-S3-ReadOnly-Role
Trusted entity: AWS Service → EC2
Permission: AmazonS3ReadOnlyAccess

The role was attached to the EC2 instance through its IAM Instance Profile.

Verification

Listed the S3 bucket:

aws s3 ls

Downloaded the test object:

aws s3 cp s3://lab-bucket-tejasshinkar/test.txt .
cat test.txt

Both operations succeeded.

Then write access was tested:

aws s3 cp test.txt s3://lab-bucket-tejasshinkar/write-test.txt

Result:

AccessDenied

The error confirmed that s3:PutObject was not allowed.

What This Proved

EC2
 ↓
Instance Profile
 ↓
IAM Role
 ↓
AmazonS3ReadOnlyAccess
 ↓
S3

The instance could list and read S3 objects, but could not write to S3.

No aws configure was used and no hardcoded AWS access keys were stored on the instance.

Lab 3 — EC2 Instance & EBS Volume Resize

Part A — EC2 Instance Resize

Stopped the running t3.micro instance.
Changed the instance type to t3.small.
Started the instance again.
SSH'd into the instance and verified it was operational.

The instance type had to be changed while the instance was stopped.

Part B — EBS Volume Resize

The root EBS volume was increased:

8 GiB → 16 GiB

This was done through:

EC2 → Volumes → Modify Volume

The AWS Console showed 16 GiB, but the operating system initially reported only about 8 GiB.

Checked the disk layout:

lsblk

The result showed:

nvme0n1 16G
└─nvme0n1p1 8G /

The EBS volume had been expanded, but the partition/filesystem had not yet used the additional space.

Because Amazon Linux 2023 was using XFS:

sudo xfs_growfs /

Finally:

df -h

confirmed approximately 16 GB available to the root filesystem.

Important Troubleshooting

Before running disk commands, the actual device name and filesystem type were checked.

EBS Volume
     ↓
   16 GB
     ↓
Partition / Filesystem
     ↓
16 GB usable by OS

The EBS volume expansion and filesystem expansion are separate operations.

For filesystem expansion:

XFS   → xfs_growfs
ext4  → resize2fs

EBS volume expansion can be performed while the instance is running, unlike changing the EC2 instance type.

Lab 4 — Custom AMI Creation & Validation

What I Did

Launched an Amazon Linux 2023 EC2 instance.
Installed and configured Nginx and Git.
Created a custom webpage:

echo "<h1>Custom AMI - $(date)</h1>" | sudo tee /usr/share/nginx/html/index.html

Verified Nginx locally:

curl localhost

Created custom-nginx-ami.
Waited for the AMI to become Available.
Launched a new EC2 instance using the custom AMI.
SSH'd into the new instance.
Verified Nginx:

sudo systemctl status nginx
curl localhost

Opened the new instance's public IP and confirmed the webpage was served.

What I Proved

The new EC2 inherited:

Nginx installation
Nginx configuration/service state
Custom index.html

No User Data script was required on the new instance.

Configured EC2
      ↓
   Create AMI
      ↓
Reusable EC2 Template
      ↓
    New EC2
      ↓
Same configuration

Mistakes & Troubleshooting

1. Browser SSH failed

The Security Group allowed SSH only from my IP, so browser-based EC2 Instance Connect did not work. I used the SSH client from PowerShell instead.

2. Tried SSH as root

Amazon Linux required the default user:

ec2-user

3. Website initially did not load

The problem was using:

https://<public-ip>

instead of:

http://<public-ip>

Nginx was serving HTTP on port 80, not HTTPS on port 443.

Lab 5 — EBS Snapshot Cross-Region Disaster Recovery

What I Did

The objective was to create an EBS snapshot in Mumbai, copy it to Virginia, restore it as a new EBS volume, and verify the original data.

Source Region:

ap-south-1

Destination Region:

us-east-1

Created test data:

echo "Mumbai DR test - original data" > test-dr.txt

Then:

Created an EBS Snapshot.
Copied the snapshot from Mumbai to Virginia.
Created a new EBS volume from the copied snapshot in us-east-1.
Launched an EC2 instance in the destination Region.
Attached the restored EBS volume as a secondary volume.
Mounted the restored volume.
Verified the original file:

ls /data
cat /data/test-dr.txt

The original data was successfully restored.

Architecture

Mumbai (ap-south-1)
        │
       EC2
        │
   EBS Volume
        │
        ▼
     Snapshot
        │
        │ Cross-Region Copy
        ▼
Virginia (us-east-1)
        │
Copied Snapshot
        │
        ▼
 New EBS Volume
        │
        ▼
       EC2
        │
        ▼
 Original Data

Mistakes & Troubleshooting

Initially created a snapshot before adding the test file, so another snapshot had to be created with the required data.
Multiple snapshots and volumes made identification confusing, so Snapshot IDs needed to be checked carefully.
The destination EC2 already had its own root EBS volume; the restored volume was attached separately as a secondary disk.
When attaching an EBS volume, the EC2 instance and volume must be in the same Availability Zone.

Additional Practical Tasks

Practical Task 1 — Elastic IP

What I Did

Launched an EC2 instance and recorded its initial dynamic public IP.
Stopped and started the instance.
Confirmed that the public IP changed.
Allocated an Elastic IP from EC2 → Elastic IPs.
Associated the Elastic IP with the instance.
Stopped and started the instance again.
Confirmed that the Elastic IP remained unchanged.
Released the Elastic IP after completing the practical.

Verification

Before Elastic IP
EC2 → Stop → Start → Different Public IP ❌

After Elastic IP
EC2 → Stop → Start → Same Public IP ✅

Practical Task 2 — EC2 Operations Using AWS CLI

The objective was to manage EC2 using only AWS CLI rather than the AWS Console.

AWS CloudShell was used because it provides a pre-configured AWS CLI environment.

Find Running Instances

aws ec2 describe-instances \
  --filters "Name=instance-state-name,Values=running" \
  --query "Reservations[].Instances[]. [InstanceId,InstanceType,State.Name,PublicIpAddress]" \
  --output table

This returned the running instance:

Instance ID: i-0548517fe09f6eeeb
Type: t3.micro
State: running

Stop the Instance

aws ec2 stop-instances --instance-ids i-0548517fe09f6eeeb

The instance state was then verified using describe-instances.

The practical demonstrated the flow:

CloudShell
   ↓
AWS CLI
   ↓
EC2 API
   ↓
Describe / Stop / Start / Snapshot

Practical Task 3 — EC2 Cost Estimation

Used AWS Pricing Calculator to estimate the monthly cost of:

1 × t3.micro EC2
Linux
ap-south-1
Running 24/7
20 GB gp3 EBS
100 GB data transfer out
Comparison with a 1-year Reserved Instance, No Upfront

The configuration was compared as:

EC2 t3.micro
+ 20 GB gp3 EBS
+ 100 GB Data Transfer Out

        ↓

On-Demand Monthly Cost
        VS
1-Year Reserved Instance Monthly Cost

The Reserved Instance estimate was cheaper for the continuous workload.

The EBS and data-transfer costs were considered separately rather than assuming the compute discount applies to the entire AWS bill.

Practical Task 4 — EC2 Automation with Python + Boto3

The task was to list EC2 instances in ap-south-1, display their Instance ID, Instance Type and State, and automatically stop running instances tagged:

Environment: dev

Script

import boto3

ec2 = boto3.client("ec2", region_name="ap-south-1")

response = ec2.describe_instances()

for reservation in response["Reservations"]:
    for instance in reservation["Instances"]:
        instance_id = instance["InstanceId"]
        instance_type = instance["InstanceType"]
        state = instance["State"]["Name"]

        environment = None

        for tag in instance.get("Tags", []):
            if tag["Key"] == "Environment":
                environment = tag["Value"]

        print(
            f"ID: {instance_id} | "
            f"Type: {instance_type} | "
            f"State: {state}"
        )

        if state == "running" and environment == "dev":
            print(f"Stopping dev instance: {instance_id}")
            ec2.stop_instances(InstanceIds=[instance_id])

Automation Logic

Boto3
  ↓
Connect to EC2 (ap-south-1)
  ↓
List instances
  ↓
Print ID + Type + State
  ↓
Check Environment tag
  ↓
Environment = dev?
  ↓
Running?
  ↓
STOP

The important condition was:

if state == "running" and environment == "dev":

So the script does not stop:

Production instances
Stopped dev instances
Testing instances

Only running instances tagged Environment=dev are stopped.

Practical Summary

EC2 + User Data: Automated first-boot configuration and Apache setup.

EC2 + IAM Role: Accessed S3 without storing AWS credentials on the instance and verified least-privilege read-only access.

EC2 + EBS: Resized compute and storage, including expanding the filesystem after increasing the EBS volume.

Custom AMI: Created a reusable configured EC2 template and launched another instance from it.

EBS Snapshot + Cross-Region Copy: Restored data in another AWS Region as a basic disaster-recovery workflow.

Elastic IP: Verified the difference between dynamic public IPs and a persistent Elastic IP.

AWS CLI: Performed EC2 operations from CloudShell without using the Console.

Boto3: Used Python to inspect EC2 instances and automate stopping only running development instances.

AWS Hands-On Practicals — EC2 | Cloud + DevOps learning journey

AWS Highly Available Web Application — ALB, Auto Scaling & Self-Healing

Tejas Shinkar — Thu, 20 Aug 2026 11:50:56 +0000

AWS Highly Available Web Application

ALB + EC2 + Auto Scaling + Multi-AZ + Self-Healing

A hands-on AWS project that deploys an Apache-based web application across multiple Availability Zones and demonstrates load balancing, health checks, dynamic scaling, and automatic instance replacement.

1. Project Overview

The application runs on Amazon EC2 instances managed by an Auto Scaling Group.

An internet-facing Application Load Balancer receives HTTP traffic and forwards requests to healthy EC2 instances through a Target Group.

The EC2 instances are created from a Launch Template, which uses User Data to automatically install and configure Apache and generate a webpage containing instance-specific information.

The environment was tested under normal operation, scale-in, and deliberate instance termination to verify automatic replacement and recovery.

Architecture

                         Internet
                            │
                         HTTP :80
                            │
                            ▼
                  ┌───────────────────┐
                  │ Application Load  │
                  │ Balancer (ALB)    │
                  │   web-app-alb     │
                  └─────────┬─────────┘
                            │
                            ▼
                  ┌───────────────────┐
                  │   Target Group    │
                  │   web-server-tg   │
                  │   HTTP Health     │
                  │      Checks       │
                  └─────────┬─────────┘
                            │
                  ┌─────────┴─────────┐
                  ▼                   ▼
          ┌─────────────┐     ┌─────────────┐
          │    EC2      │     │    EC2      │
          │ ap-south-1a │     │ ap-south-1b │
          └─────────────┘     └─────────────┘
                  ▲                   ▲
                  └─────────┬─────────┘
                            │
                  ┌───────────────────┐
                  │ Auto Scaling Group │
                  │    web-app-asg     │
                  └─────────┬─────────┘
                            │
                  ┌───────────────────┐
                  │  Launch Template  │
                  │   web-server-lt   │
                  └───────────────────┘
                            │
                         User Data
                            │
                      Apache + HTML

2. AWS Components

Component	Configuration / Role
Amazon EC2	Runs Apache and the web application
Launch Template	Reusable EC2 configuration blueprint
User Data	Installs/configures Apache and generates the application page
Security Groups	Controls ALB and EC2 traffic
Target Group	Maintains EC2 targets and performs HTTP health checks
Application Load Balancer	Public HTTP entry point and traffic distribution
Auto Scaling Group	Maintains and replaces EC2 capacity
CloudWatch	Supplies CPU utilization metrics for Target Tracking
Availability Zones	`ap-south-1a` and `ap-south-1b`

3. Network & Security Configuration

Two Security Groups were used.

SG-ALB

Inbound:
HTTP :80 → 0.0.0.0/0

Outbound:
All traffic

SG-EC2

Inbound:
HTTP :80 → SG-ALB

Outbound:
All traffic

Application traffic therefore follows:

Internet
    │
    │ HTTP :80
    ▼
 SG-ALB
    │
    │ HTTP :80
    ▼
 SG-EC2
    │
    ▼
   EC2

This keeps HTTP access to the EC2 instances restricted to traffic originating from the ALB.

For instance-level testing, SSH was temporarily allowed:

SSH :22
Source: <public-IP>/32

The SSH rule was removed during cleanup.

4. Launch Template & User Data

Launch Template

Setting	Value
Name	`web-server-lt`
AMI	Amazon Linux 2023
Architecture	64-bit x86
Instance type	`t3.micro`
Key pair	`lab-ec2`
Security Group	`SG-EC2`

The initial Launch Template version had an AMI configuration issue. Version 2 was created with the correct Amazon Linux 2023 AMI and made the default version used by the ASG.

User Data

User Data performs the initial EC2 configuration automatically:

Launch EC2
    │
    ├── Update packages
    ├── Install httpd
    ├── Start httpd
    ├── Enable httpd
    ├── Retrieve instance metadata
    │      ├── Instance ID
    │      ├── Availability Zone
    │      └── Region
    │
    └── Generate application HTML

The application displays:

AWS Highly Available Web Application

Built by Tejas Shinkar

ELB + Auto Scaling + EC2 + CloudWatch

Instance ID: ...
Availability Zone: ...
Region: ap-south-1

The instance-specific information makes backend changes visible when requests are served through the ALB.

5. Target Group & Health Checks

Target Group

Setting	Value
Name	`web-server-tg`
Target type	Instances
Protocol	HTTP
Port	80
Health check protocol	HTTP
Path	`/`
Port	Traffic port
Interval	30 seconds
Timeout	5 seconds
Healthy threshold	2
Unhealthy threshold	2
Success code	200

The Target Group was initially created with 0 targets intentionally. Targets were not manually registered because the Auto Scaling Group was configured to register its instances automatically.

Health check flow

Target Group
      │
      │ HTTP GET /
      ▼
   EC2 :80
      │
      │ HTTP 200
      ▼
   Healthy

An EC2 instance being in a running state does not by itself make it a healthy application target. The Target Group checks the HTTP application response.

6. Application Load Balancer

ALB Configuration

Setting	Value
Name	`web-app-alb`
Type	Application Load Balancer
Scheme	Internet-facing
IP address type	IPv4
VPC	Default VPC
Availability Zones	`ap-south-1a`, `ap-south-1b`
Security Group	`SG-ALB`

Listener

Protocol: HTTP
Port: 80
Default action: Forward to web-server-tg

Request flow

Browser
   │
   ▼
ALB :80
   │
   ▼
web-server-tg
   │
   ▼
Healthy EC2 :80
   │
   ▼
Apache
   │
   ▼
Application response

With two healthy instances, repeated requests to the ALB endpoint displayed different Instance IDs and Availability Zones, confirming that requests were reaching different healthy backends.

7. Auto Scaling Group

Configuration

Setting	Value
Name	`web-app-asg`
Launch Template	`web-server-lt`
Desired capacity	2
Minimum capacity	1
Maximum capacity	4
Availability Zones	`ap-south-1a`, `ap-south-1b`
Target Group	`web-server-tg`
Health check grace period	300 seconds

The ASG launches instances using the Launch Template and automatically registers them with the Target Group.

The 300-second grace period allows a new instance to boot, execute User Data, install/start Apache, and become ready for application health checks.

Initial deployment

Auto Scaling Group
       │
       ├──────────────┐
       ▼              ▼
     EC2 #1         EC2 #2
   ap-south-1a    ap-south-1b
       │              │
       └──────┬───────┘
              ▼
        Target Group
              │
              ▼
             ALB

8. Target Tracking & Scale-In

A Target Tracking policy was configured after the initial deployment was verified.

Setting	Value
Policy	Target Tracking
Metric	Average CPU Utilization
Target	50%
Instance warmup	300 seconds
Scale in	Enabled
Scale out	Enabled

EC2 CPU Utilization
        │
        ▼
    CloudWatch
        │
        ▼
Target Tracking
        │
        ▼
Auto Scaling Group
      /   \
     ▼     ▼
Scale Out  Scale In
   │          │
 Add EC2    Remove EC2

Observed scale-in

The workload remained low after the policy was enabled, so the ASG scaled in.

The instance being removed entered the Target Group's Draining state before termination.

Healthy EC2
     │
     ▼
ASG decides to scale in
     │
     ▼
Target Group → Draining
     │
     ▼
Instance terminates

The environment eventually reached:

Desired capacity: 1
Healthy targets: 1

At this point, repeated ALB refreshes naturally showed the same Instance ID because only one healthy backend remained.

9. Application Verification

EC2 verification

SSH was temporarily enabled for instance-level verification.

Apache was checked with:

sudo systemctl status httpd

Result:

active (running)

The application was then verified locally:

curl localhost

The command returned the custom HTML page.

ALB verification

The application was accessed through the ALB DNS endpoint.

The page displayed:

Instance ID
Availability Zone
Region

This verified the complete path:

Browser
  ↓
ALB
  ↓
Target Group
  ↓
EC2
  ↓
Apache
  ↓
Application

Load-balancing verification

When two backend instances were healthy, repeated ALB requests displayed different Instance IDs/AZs.

This provided a simple visual verification that the ALB was distributing traffic across healthy targets.

10. Failure Simulation & Self-Healing

An active EC2 instance managed by the ASG was deliberately terminated.

Recovery flow

Active EC2
    │
    ▼
Manual termination
    │
    ▼
ASG detects capacity loss
    │
    ▼
Replacement EC2 launched
    │
    ▼
Launch Template applied
    │
    ▼
User Data executes
    │
    ▼
Apache configured
    │
    ▼
Target Group registration
    │
    ▼
HTTP health check
    │
    ▼
Healthy
    │
    ▼
ALB can route traffic

The replacement instance was launched automatically using the Launch Template.

Because User Data was part of the launch configuration, the replacement instance automatically received the Apache configuration and application page.

The replacement was then automatically registered with web-server-tg and became Healthy.

This verified the self-healing behavior of the architecture.

11. Troubleshooting

Launch Template AMI configuration

The initial Launch Template version did not contain the expected AMI configuration.

Fix: Created Version 2 with Amazon Linux 2023 and made it the default version.

SSH connectivity

The initial SSH connection failed because port 22 was not allowed by SG-EC2.

Fix: Temporarily added:

SSH :22
Source: <public-IP>/32

SSH then worked and allowed:

sudo systemctl status httpd
curl localhost

The temporary SSH rule was removed during cleanup.

Same Instance ID after refresh

After Target Tracking was enabled, low workload caused the ASG to scale from 2 instances down to 1.

Therefore, subsequent ALB refreshes showed the same Instance ID.

The correct troubleshooting path was to check:

ASG capacity
      +
Target Group health
      +
Registered targets

before assuming the ALB was not distributing traffic.

12. Production Considerations

This project intentionally used a simplified setup for demonstration.

For a production deployment, the architecture would typically be extended with:

HTTPS/TLS on the ALB instead of HTTP.
A deliberately designed VPC and subnet architecture instead of the Default VPC.
IAM roles for EC2 rather than relying on credentials.
CloudWatch alarms and operational monitoring.
More deliberate access controls and administrative access patterns.
A real application instead of the demonstration HTML page.

13. Cleanup

After testing, all project-specific resources were removed.

The cleanup sequence was:

1. Auto Scaling Group
        ↓
2. EC2 instances
        ↓
3. Application Load Balancer
        ↓
4. Target Group
        ↓
5. Launch Template
        ↓
6. Security Groups
        ↓
7. Final resource verification

Final project resources:

web-app-asg  → Deleted
EC2          → Terminated
web-app-alb  → Deleted
web-server-tg → Deleted
web-server-lt → Deleted
SG-ALB       → Deleted
SG-EC2       → Deleted

No project-specific running resources were left behind after cleanup.

14. Project Result

The completed project demonstrated the following end-to-end behavior:

                         ┌─────────────────────┐
                         │      Internet       │
                         └──────────┬──────────┘
                                    │
                                    ▼
                         ┌─────────────────────┐
                         │        ALB          │
                         └──────────┬──────────┘
                                    │
                                    ▼
                         ┌─────────────────────┐
                         │    Target Group     │
                         │   Health Checks     │
                         └──────────┬──────────┘
                                    │
                         ┌──────────┴──────────┐
                         ▼                     ▼
                    ┌──────────┐          ┌──────────┐
                    │   EC2    │          │   EC2    │
                    │   AZ-A   │          │   AZ-B   │
                    └────┬─────┘          └────┬─────┘
                         ▲                     ▲
                         └──────────┬──────────┘
                                    │
                         ┌─────────────────────┐
                         │    Auto Scaling     │
                         │       Group         │
                         └──────────┬──────────┘
                                    │
                         ┌─────────────────────┐
                         │  Launch Template    │
                         │  + User Data        │
                         └─────────────────────┘

                  CloudWatch → Target Tracking

Verified

Multi-AZ EC2 deployment.
ALB-based application access.
Target Group health checks.
Automatic target registration through ASG.
User Data-based server configuration.
CPU Target Tracking.
Scale-in and target draining.
Automatic replacement after EC2 termination.
Replacement target becoming healthy.
ALB routing to the recovered backend.
Complete resource cleanup.

Github repository: https://github.com/tejasshinkar/aws-highly-available-web-app

AWS Cloud + DevOps — Project: Highly Available Web Application

Network Protocols — ARP, FTP, SMTP, HTTP, HTTPS, DNS and DHCP

Tejas Shinkar — Tue, 18 Aug 2026 03:05:10 +0000

🌐 Network Protocols

ARP, FTP, SMTP, HTTP, HTTPS, DNS and DHCP

Series: Networking Fundamentals for Cloud & DevOps — Part 6 of 6

We have already seen how hosts, switches, routers, MAC addresses and IP addresses work together. Now we move one level higher and look at the protocols that define how different types of communication happen.

1. Protocol

A protocol is a set of rules/messages that devices follow to communicate.

Think of a protocol as a common language with defined rules.

For example:

If two devices need to resolve an IP address to a MAC address → ARP
If a client needs to transfer a file → FTP
If a client needs to send an email → SMTP
If a browser needs web communication → HTTP/HTTPS
If a host needs to convert a domain name into an IP address → DNS
If a host needs its network configuration automatically → DHCP

A quick overview:

Protocol	Full Form	Main Job
ARP	Address Resolution Protocol	IP → MAC
FTP	File Transfer Protocol	Transfer files
SMTP	Simple Mail Transfer Protocol	Send email
HTTP	HyperText Transfer Protocol	Web communication
HTTPS	HTTP Secure	Secure HTTP using TLS
DNS	Domain Name System	Domain name → IP
DHCP	Dynamic Host Configuration Protocol	Automatically gives network configuration

The important thing is not just memorizing the names.

Each protocol solves a different communication problem.

2. ARP

ARP = Address Resolution Protocol. ARP solves this problem:

"I know the IP address, but what is the MAC address?"

Suppose a host knows 9.1.1.22 but needs the corresponding MAC address. It broadcasts an ARP Request:

Host 9.1.1.11: "Who has 9.1.1.22?"

The host with that IP responds:

Host 9.1.1.22: "My MAC is b3b3."

So the mapping becomes 9.1.1.22 → b3b3.

ARP = IP → MAC mapping

This connects directly to what we learned earlier:

IP address → ARP → MAC address → Ethernet frame

ARP is mainly concerned with finding the Layer 2 address needed for communication on the local network.

3. FTP

FTP = File Transfer Protocol. FTP is used for file transfer — a client sends a request to the FTP server for a file, e.g. file.pdf.

Client ── RETR file.pdf ──→ FTP Server
Client ←──── file.pdf ───── FTP Server

RETR essentially means:

Retrieve/download this file.

FTP → File transfer

4. SMTP

SMTP = Simple Mail Transfer Protocol. SMTP is used for sending email. Conceptually:

Client ── HELLO ───────→ SMTP Server
Client ── email ───────→ SMTP Server

The protocol defines how the email-sending communication takes place between the client and mail server, and between mail servers.

SMTP = sending mail

5. HTTP

HTTP = HyperText Transfer Protocol. HTTP is used for web communication. A browser/client can request a resource from a web server:

Client ── GET /index.html ──→ Web Server
Client ←────── 200 OK ─────── Web Server

The client asks for a resource, and the web server responds. This is the basic protocol underneath the request/response model used by the web.

HTTP → Web communication

6. HTTPS

HTTPS = HTTP + TLS security. HTTPS is HTTP communication protected using TLS.

HTTP + TLS security → HTTPS

TLS provides security mechanisms such as encryption and authentication for the communication. Conceptually, the connection between client and server is encrypted end-to-end (Client ═══ encrypted HTTPS connection ═══→ Web Server).

HTTP = Web communication. HTTPS = Secure web communication.

HTTPS is what allows a browser to communicate with a web server while protecting the connection against unauthorized observation or modification.

7. DNS

DNS = Domain Name System. DNS solves a different problem from ARP.

ARP asks:

"I know the IP. What is the MAC?"

DNS asks:

"I know the website/domain name. What is its IP?"

Suppose you want to access site.com. Your computer needs the server's IP address, so it asks a DNS server:

"What is the IP address of site.com?"

The DNS server responds: site.com → 160.8.23.154.

DNS = Domain name → IP

The key distinction is:

ARP: IP → MAC. DNS: Name → IP.

8. Four Things a Host Needs for Internet Connectivity

A host generally needs four important pieces of network configuration:

IP Address
Subnet Mask
Default Gateway
DNS Server

For example:

IP: 9.1.1.11
Mask: 255.255.255.0
Gateway: 9.1.1.1
DNS: 8.8.8.8

Each one has a different job.

IP Address

Identifies the host on the network.

IP → Which host am I?

Subnet Mask

Determines which addresses belong to the local network.

Subnet Mask → Which network am I in?

Default Gateway

Provides the router used to reach other networks.

Default Gateway → How do I reach other networks?

DNS

Converts domain names into IP addresses.

DNS → What IP belongs to this domain name?

So:

IP → identifies the host
Subnet mask → determines the local network
Default gateway → router used to reach other networks
DNS → converts names into IP addresses

These four pieces work together whenever a host needs to communicate beyond its immediate network.

9. DHCP

DHCP = Dynamic Host Configuration Protocol. DHCP automatically provides network configuration to a client.

Without DHCP, a host would need to be manually configured with values such as:

IP = 9.1.1.11
Subnet = 255.255.255.0
Gateway = 9.1.1.1
DNS = 8.8.8.8

With DHCP, the client can obtain these settings automatically from a DHCP server. The basic process is:

Client → DHCP Discover
Server → DHCP Offer
Client → DHCP Request
Server → DHCP ACK

D → O → R → A — Discover, Offer, Request, ACK

The result is that the host receives its network configuration automatically.

DHCP automatically gives a host its network configuration.

Quick Revision

ARP → IP → MAC
DNS → Name → IP
DHCP → IP + Mask + Gateway + DNS
HTTP → Web communication
HTTPS → Secure web communication
FTP → File transfer
SMTP → Send email

The Big Picture

These protocols solve different problems, but they work together when a host accesses something on the Internet. A simplified sequence could look like:

DHCP → get IP + subnet mask + gateway + DNS
DNS → convert domain name → destination IP
Routing → decide whether destination is local or remote
ARP → resolve the next-hop IP → MAC
HTTP / HTTPS → communicate with the web server

So don't think of these protocols as isolated terms. They are different pieces of the same networking process:

DHCP → network configuration · DNS → find destination IP · Routing → find next hop · ARP → find next-hop MAC · HTTP/HTTPS → exchange web data

And other protocols solve other application-level problems:

FTP → file transfer. SMTP → email.

The important mental model is:

Protocols exist because different parts of communication need different rules.

Part 6 of 6 — Networking Fundamentals for Cloud & DevOps
This completes the current networking fundamentals sequence and provides the protocol foundation needed for deeper AWS networking concepts.

DEV Community: Tejas Shinkar

So… If AI Can Write Code, What Are Software Engineers Getting Paid For? 👀

I Thought Knowing AWS Was The Goal. I Was Wrong.

I Thought Caching Was Just About Speed. Then I Found the Security Trap. 💀

I broke my own AWS infrastructure… without touching the infrastructure.

AWS CloudFront — Complete Understanding, Production Patterns & Practice

📋 Topics Covered

What CloudFront Actually Is

Why Do We Need a CDN?

Important CloudFront Terminology

Distribution

Origin

Edge Location / Point of Presence (PoP)

Complete Request Flow

Cache HIT / Cache MISS

Per-Edge Caching

Cache Key

Why Cache-Key Design Matters

The Golden Cache-Key Rule

Personalization and Caching

Cache Policy vs Origin Request Policy

Example — origin needs information but cache doesn't

Warning with Authorization headers

Query Strings, Cookies & Headers in the Cache Key

TTL — Minimum, Default, Maximum

Cache-Control & Invalidation

Versioned Assets

Cache Behaviors

Real-World Caching Scenarios

Request Collapsing & Cache Hit Ratio

Production Cache Hierarchy

CloudFront + S3 (OAC)

CloudFront + ALB + EC2

CloudFront + WAF

Production Techniques & Common Mistakes

Production Mini Architecture

Practice Labs

Practice Lab 1 — Basic CloudFront + S3

Practice Lab 2 — Observe HIT/MISS

Practice Lab 3 — Query-String Cache Key

Practice Lab 4 — Personalized Response Safety

Practice Lab 5 — Cache Policy vs Origin Request Policy

Practice Lab 6 — TTL Experiment

Practice Lab 7 — Cache Behaviors

Practice Lab 8 — Private S3 + OAC

Practice Lab 9 — CloudFront + ALB + EC2

Practice Lab 10 — Production-Style Mini Project

Interview Revision

Final Mental Model

AWS EFS Hands-On Lab

Problem Statement

Lab Architecture

Lab Setup — Two EC2 Instances + One Shared EFS

Shared-File Test — EC2-1 Writes, EC2-2 Reads

Failure Test — Remove NFS Access and Break the EFS Write

Lifecycle Management — Configure 7-Day IA Transition

Persistent Mount Test — /etc/fstab

EC2-1 — Manual Mount Only

EC2-2 — Persistent Mount

EBS vs EFS Copy Experiment

Initial /tmp Problem

Create the 1 GiB Test File on EBS

EBS → EBS

EBS → EFS

Benchmark Interpretation

Problems Encountered and Fixes

Evidence Collected

Production-Relevant Takeaways

Lab Completion

Your Torrent Isn't Downloading From "The Internet"

We've All Used Torrents. But Who's Actually Sending You the File? 👀

AWS VPC Hands-On Lab

📚 VPC Theory & Concepts

1.Building a VPC from Scratch with Public and Private Subnets

Objective

Architecture

What We Built

Implementation

VPC

Subnets

Persistent Mount Test — `/etc/fstab`

Initial `/tmp` Problem