DEV Community: Anupa Supul

How Do Large Applications Process Work in the Background? Understanding Amazon SQS, SNS & EventBridge

Anupa Supul — Tue, 20 Oct 2026 05:08:16 +0000

When I started learning AWS, I focused mainly on deploying applications, building APIs, and connecting databases.

But then I started thinking about something interesting.

What happens when an application needs to perform several tasks without making the user wait for everything to finish?

Imagine ordering a product from an online store. After placing the order, the system might need to update inventory, send a confirmation email, generate an invoice, and notify the delivery team.

Should the customer wait until every single task is completed?

Probably not.

This is where event-driven architecture and messaging services become useful.

In this article, I'll explore three AWS services that help applications communicate and process work more efficiently:

Amazon SQS — queues messages for processing.
Amazon SNS — distributes messages to multiple subscribers.
Amazon EventBridge — routes events to targets based on rules.

1. Why Do Applications Need Background Processing?

Consider a simple application where every operation happens inside one request.

Customer places an order
          |
          v
      Save order
          |
          v
   Update inventory
          |
          v
   Send confirmation
          |
          v
   Generate invoice
          |
          v
   Return response

If each operation runs sequentially, the customer may need to wait for all of them. If an external email service is slow, the entire request could take longer.

It also creates tight coupling: a problem in one dependency can affect the main request.

Instead, we can separate the immediate operation from background work.

Customer places an order
          |
          v
       Save order
          |
          v
   Queue background work
          |
          v
  Return response to customer
          |
          v
   Process tasks separately

Now the application can respond after completing the necessary synchronous work, while other operations happen independently.

This is one of the main ideas behind asynchronous processing.

2. Amazon SQS — Let Messages Wait in a Queue

Amazon Simple Queue Service (SQS) is a managed message queuing service.

Think of it as a waiting line for tasks. A producer sends a message to the queue, and a consumer retrieves and processes it.

For example:

Order API
    |
    v
Amazon SQS Queue
    |
    v
Order Processing Worker
    |
    v
Update Inventory

The API doesn't need to wait for the worker to finish processing the message.

Why is SQS useful?

It helps decouple application components.
Messages can wait until consumers are ready to process them.
It helps absorb temporary traffic spikes.
Failed messages can be retried.
Dead-letter queues can isolate messages that repeatedly fail.

SQS offers Standard queues and FIFO queues. Standard queues provide high throughput with at-least-once delivery, so consumers should be designed to handle duplicate messages. FIFO queues support ordering within message groups and deduplication features, with their own throughput and configuration considerations.

One important concept is the visibility timeout. When a consumer receives a message, the message becomes temporarily invisible to other consumers. If processing succeeds, the consumer deletes it. If the message isn't deleted before the visibility timeout expires, it can become available again.

That is why reliable message processing needs error handling and idempotency.

3. Amazon SNS — Send One Message to Multiple Subscribers

Amazon Simple Notification Service (SNS) is a managed publish/subscribe messaging service.

Instead of placing work in a queue for one consumer pattern, SNS can distribute a published message to multiple subscribers.

Imagine an order has been placed. Several systems need to react:

A notification service sends a confirmation.
An analytics service records the event.
Another service starts a fulfilment workflow.

SNS can help distribute the notification to these subscribers.

              Order Service
                    |
                    v
                 Amazon SNS
                    |
          +---------+---------+
          |         |         |
          v         v         v
        SQS A     Lambda    Email
      Queue       Function  Endpoint

SNS supports different subscription types, including SQS queues, Lambda functions, HTTP/S endpoints, and email, depending on the configuration and use case.

Why is SNS useful?

SNS is a good fit when multiple subscribers need to receive a notification about the same event.

For example, an order service can publish an OrderPlaced notification without needing to know the internal implementation of every subscriber.

That reduces direct dependencies between services.

A key distinction: SNS distributes messages; it is not a general-purpose durable work queue in the same way SQS is. Combining SNS with SQS can be useful when multiple consumers need their own queues and independent processing.

4. Amazon EventBridge — Route Events Based on Rules

Amazon EventBridge is a serverless event bus service that routes events to targets using rules.

An event describes something that happened, such as:

An order being placed.
A payment succeeding.
A file being uploaded to Amazon S3.
A scheduled task reaching its execution time.

EventBridge can inspect an event and route it to the targets whose rules match.

For example:

       OrderPlaced Event
               |
               v
       Amazon EventBridge
               |
        +------+------+
        |             |
        v             v
   Order Processor  Analytics
       Lambda        Target

You can define rules that match event attributes and send matching events to supported targets.

EventBridge is particularly useful for reacting to events from AWS services, your own applications, and supported SaaS integrations.

It also supports scheduled and time-based event patterns through its scheduling capabilities.

When should you use EventBridge?

Use it when you want to route events based on what happened, without tightly connecting the event producer to every downstream service.

For example, an application could react differently to OrderPlaced, PaymentFailed, and OrderCancelled events.

5. SQS vs SNS vs EventBridge

These services can work together, but they solve different problems.

Feature	Amazon SQS	Amazon SNS	Amazon EventBridge
Main purpose	Queue messages for processing	Distribute messages to subscribers	Route events using rules
Core pattern	Queue and consumer	Publish/subscribe	Event bus and targets
Typical use	Background jobs	Notifications to multiple subscribers	Reacting to application or AWS events
Message handling	Consumers poll or use supported integrations	Pushes notifications to subscribers	Routes matching events to targets
Useful feature	Visibility timeout and dead-letter queues	Multiple subscribers and filtering	Event pattern matching and routing

Here's the simple mental model I use:

SQS: "Process this work when you're ready."
SNS: "Send this notification to these subscribers."
EventBridge: "When this event happens, route it to the appropriate target."

6. How They Work Together in a Real Application

Let's return to our online store.

After a customer places an order, the API saves the order and publishes an event. Different services can then handle different responsibilities.

An e-commerce application can use queues for background processing, notifications for multiple subscribers, and event rules to route events.

Here is a simplified architecture diagram in text form:

React Frontend
      |
      v
API Gateway
      |
      v
AWS Lambda
      |
      v
Save Order in DynamoDB
      |
      v
Publish OrderPlaced Event
      |
      v
Amazon EventBridge
      |
      +-------------------+
      |                   |
      v                   v
Order Processing       Analytics
      |
      v
Amazon SQS
      |
      v
Background Worker
      |
      v
Update Inventory / Fulfil Order

A separate notification workflow could use SNS to distribute an order notification to multiple subscribers.

The exact architecture depends on the requirements. For example, an SQS queue can be an EventBridge target, allowing the event to be routed to a durable queue for later processing.

This architecture separates the order API from slower background tasks. It also lets different components scale and fail independently to a degree, provided that retries, error handling, and downstream dependencies are designed correctly.

What happens when the order is placed?

The React frontend sends a request to API Gateway.
Lambda validates the request and saves the order in DynamoDB.
The application publishes an order event.
EventBridge evaluates its rules and routes matching events to configured targets.
Background processing runs independently, potentially through SQS and a worker.
SNS can distribute notifications to subscribers when needed.

One important production consideration: don't assume that saving an order and publishing an event are automatically one atomic operation. If the database write succeeds but event publishing fails, the system may need a reliable publishing strategy, such as the transactional outbox pattern, to avoid losing important events.

7. Reliability: Retries, Dead-Letter Queues, and Duplicate Messages

Asynchronous systems are powerful, but they introduce their own challenges.

What happens if a worker crashes while processing a message? What if a message is delivered more than once?

These are normal distributed-system problems.

Retries

Temporary failures may be resolved by retrying the operation. However, retries should be configured carefully to avoid overwhelming a failing dependency.

Dead-letter queues

A dead-letter queue (DLQ) can hold messages that exceed the configured retry or receive-count threshold. This gives you a way to investigate and recover messages that repeatedly fail.

Idempotency

An operation is idempotent when repeating it produces the same intended result rather than performing the effect multiple times.

For example, if a worker receives the same order-processing message twice, it should not deduct inventory twice unintentionally.

SQS Standard queues can deliver duplicate messages, and other event-driven workflows can also retry delivery. Design consumers with this possibility in mind.

Monitoring

CloudWatch metrics, logs, and alarms can help identify growing queue backlogs, processing failures, throttling, and other problems.

The goal isn't to assume messages will never fail. It's to design the application so failures can be detected and handled safely.

8. When Should You Use Each Service?

Choose the service based on the communication problem you're trying to solve.

Use Amazon SQS when:

Work needs to wait until a consumer is ready.
You want to separate request handling from background processing.
You need to buffer traffic spikes.
Consumers need to retry processing failed messages.

Use Amazon SNS when:

One message needs to reach multiple subscribers.
You want a publish/subscribe communication pattern.
Multiple systems need to react to the same notification.

Use Amazon EventBridge when:

Events need to be routed based on their content.
AWS services or applications generate events.
Different event types need different processing workflows.
You want event-driven integration without hardcoding every destination into the producer.

These are common patterns, not strict boundaries. Real architectures often combine these services.

9. My Main Takeaways

As I continue learning AWS, I'm realizing that building a reliable application involves more than deploying a backend and connecting a database.

It also means understanding how different components communicate, how failures are handled, and how background work can be processed without slowing down the user experience.

Here is the mental model I use:

Amazon SQS queues work for consumers.
Amazon SNS distributes messages to subscribers.
Amazon EventBridge routes events using rules.
AWS Lambda can execute event-driven processing.
Amazon DynamoDB can store application data.
Amazon CloudWatch helps monitor and troubleshoot the system.

The key lesson is that these services solve different communication problems. Choosing the right one depends on whether you need a queue, a notification mechanism, or event-based routing.

And in production, retries, duplicate handling, permissions, monitoring, and cost still matter.

I'm documenting my AWS learning journey one concept at a time. If you're learning cloud computing too, I'd love to hear what you're building and which AWS services you're exploring.

Can You Build a Backend Without Managing Servers? Understanding Amazon API Gateway, AWS Lambda & DynamoDB

Anupa Supul — Tue, 20 Oct 2026 04:56:12 +0000

When I first started learning AWS, I thought deploying a backend meant creating an EC2 instance, installing Node.js, configuring a web server, and keeping everything running.

That approach works. But what if you could build an API without managing the underlying servers yourself?

That is where serverless architecture comes in.

In this article, I’ll explore three AWS services that work well together to build serverless backends:

Amazon API Gateway — receives and manages API requests.
AWS Lambda — executes backend code.
Amazon DynamoDB — stores and retrieves data.

We’ll also look at how these services connect, explore a small Node.js example, and discuss when serverless makes sense in a real application.

1. What Does Serverless Actually Mean?

Serverless does not mean there are no servers.

Servers still exist. The difference is that AWS manages much of the underlying infrastructure, including server provisioning and infrastructure availability for these services.

Instead of managing a server yourself, you focus more on writing application code and defining how the services should work together.

For example, imagine you are building a task management application.

Users should be able to:

Create a task.
View their tasks.
Update a task.
Delete a task.

With a traditional backend, you might deploy a Node.js application on EC2 or ECS.

With a serverless approach, you could use API Gateway, Lambda, and DynamoDB.

The basic flow looks like this:

User / React Frontend
         |
         v
   API Gateway
         |
         v
      Lambda
         |
         v
    DynamoDB
         |
         v
    JSON Response

Each service has a specific responsibility. This separation makes the architecture easier to understand and lets you scale different parts of the application according to their needs.

2. Amazon API Gateway — The Entry Point for Your API

Imagine your React frontend needs to request a list of tasks from your backend.

It might send a request such as:

GET /tasks

But where should that request go?

Amazon API Gateway provides an API entry point that clients can call. It can route requests to backend services such as Lambda functions.

It also supports capabilities such as authorization, request validation, throttling, and monitoring integrations, depending on the API type and configuration.

For example, you might define these routes:

HTTP Method	Endpoint	Purpose
GET	`/tasks`	Retrieve tasks
GET	`/tasks/{id}`	Retrieve one task
POST	`/tasks`	Create a task
PUT	`/tasks/{id}`	Update a task
DELETE	`/tasks/{id}`	Delete a task

Each route can be connected to a suitable Lambda function.

For example:

GET /tasks
    |
    v
getTasks Lambda

POST /tasks
    |
    v
createTask Lambda

API Gateway can also help protect an API from excessive requests through throttling and integrate with supported authorization mechanisms.

One important point: API Gateway is not your database, and it does not automatically make every API secure. Authentication, authorization, validation, and appropriate access policies still need to be configured.

3. AWS Lambda — Run Backend Code Without Managing Servers

Now let's look at the service that executes our backend logic.

AWS Lambda runs code in response to events. An API Gateway request can trigger a Lambda function, which processes the request and returns a response.

You don't need to manage an EC2 instance just to keep this function available.

For example, a createTask Lambda function might:

Receive the incoming request.
Read the task details.
Validate the input.
Store the task in DynamoDB.
Return a response to the client.

A simplified flow looks like this:

API Gateway
     |
     v
Lambda Function
     |
     +--> Validate request
     |
     +--> Execute business logic
     |
     +--> Save data in DynamoDB
     |
     v
Return JSON response

Lambda supports multiple runtimes, including Node.js and Python.

A simple Node.js Lambda example

Here is a minimal Lambda handler that returns a JSON response.

export const handler = async (event) => {
  return {
    statusCode: 200,
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      message: "Hello from AWS Lambda!",
      method: event.requestContext?.http?.method,
    }),
  };
};

This example assumes an API Gateway HTTP API using its payload format that exposes the HTTP method under requestContext.http.method.

When API Gateway invokes the function, Lambda returns a response that API Gateway can send back to the client.

This is only a demonstration. A production API would also need suitable error handling, input validation, authentication, and logging.

What happens when traffic increases?

Suppose your application normally receives 20 requests per minute, but suddenly receives thousands.

Lambda can scale the number of concurrent function executions in response to incoming demand, subject to account quotas, concurrency settings, and other service limits.

You don't manually create more EC2 instances just to handle every increase in traffic.

However, scaling is not unlimited or instantaneous in every situation. Concurrency quotas, downstream database capacity, throttling, and cold starts can affect performance.

That's why serverless applications still require thoughtful design.

4. Amazon DynamoDB — Store Application Data

Our backend now has an entry point and a way to execute code. But where should the application store its data?

For this example, we'll use Amazon DynamoDB.

DynamoDB is a managed NoSQL database service that supports key-value and document data models.

Unlike a traditional relational database, DynamoDB does not require you to design every relationship around SQL tables and joins. Instead, you design tables and indexes around the access patterns your application needs.

Imagine storing tasks with fields such as:

{
  "userId": "user-123",
  "taskId": "task-001",
  "title": "Learn AWS Lambda",
  "completed": false
}

A DynamoDB table could use:

userId as the partition key.
taskId as the sort key.

This key design makes it possible to retrieve tasks belonging to a particular user using a query on that user's partition key.

The actual design should depend on the application's access patterns. If you need to search tasks by status, due date, or another attribute, you may need additional indexes or a different key design.

Why choose DynamoDB?

Some useful characteristics include:

Managed infrastructure.
Flexible scaling options.
Low-latency access for suitable key-based access patterns.
Integration with Lambda and other AWS services.
On-demand capacity options for workloads where usage is difficult to predict.

But DynamoDB is not automatically the best database for every application.

If your application relies heavily on relational joins, complex SQL queries, or relational constraints, Amazon RDS or Amazon Aurora may be more suitable.

The important lesson is to choose a database based on your data model and access patterns, not simply because it is serverless.

5. Connecting the Architecture

Let's put the three services together.

Architecture overview: the frontend sends an API request, API Gateway routes it to Lambda, and Lambda interacts with DynamoDB before returning a response.

Here is the same architecture as a text diagram:

                 USER
                  |
                  v
            REACT FRONTEND
                  |
             HTTPS Request
                  |
                  v
           AMAZON API GATEWAY
                  |
           Route to Lambda
                  |
                  v
             AWS LAMBDA
                  |
          Validate and Process
                  |
                  v
             AMAZON DYNAMODB
                  |
             Query / Write
                  |
                  v
             AWS LAMBDA
                  |
            JSON Response
                  |
                  v
            API GATEWAY
                  |
                  v
            REACT FRONTEND

Let's walk through a request to create a task.

The user fills out a form in the React application.
The frontend sends a POST /tasks request.
API Gateway receives the request and invokes the configured Lambda function.
Lambda validates the input and executes the business logic.
Lambda writes the task to DynamoDB using the AWS SDK.
Lambda returns a response.
API Gateway sends the response to the frontend.
React updates the interface to show the result.

This architecture separates the frontend, API entry point, application logic, and database responsibilities.

For a small application, it can be a straightforward way to build and deploy a backend. As the application grows, you'll need to think more carefully about permissions, observability, data access patterns, and failure handling.

6. A Practical Node.js Example: Saving a Task to DynamoDB

Let's make the example a little more realistic.

Suppose we have a DynamoDB table named Tasks with userId as its partition key and taskId as its sort key.

The Lambda function below creates a task using the AWS SDK for JavaScript v3.

Install the required package in your project:

npm install @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

Create a file named index.mjs:

import { randomUUID } from "node:crypto";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import {
  DynamoDBDocumentClient,
  PutCommand,
} from "@aws-sdk/lib-dynamodb";

const client = new DynamoDBClient({});
const dynamoDB = DynamoDBDocumentClient.from(client);

const TABLE_NAME = process.env.TABLE_NAME;

export const handler = async (event) => {
  try {
    if (!TABLE_NAME) {
      throw new Error("TABLE_NAME is not configured");
    }

    const body = JSON.parse(event.body || "{}");
    const { userId, title } = body;

    if (
      typeof userId !== "string" ||
      !userId.trim() ||
      typeof title !== "string" ||
      !title.trim()
    ) {
      return {
        statusCode: 400,
        body: JSON.stringify({
          message: "userId and title are required",
        }),
      };
    }

    const task = {
      userId: userId.trim(),
      taskId: randomUUID(),
      title: title.trim(),
      completed: false,
      createdAt: new Date().toISOString(),
    };

    await dynamoDB.send(
      new PutCommand({
        TableName: TABLE_NAME,
        Item: task,
        ConditionExpression:
          "attribute_not_exists(userId) AND attribute_not_exists(taskId)",
      })
    );

    return {
      statusCode: 201,
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify(task),
    };
  } catch (error) {
    console.error("Failed to create task:", error);

    return {
      statusCode: 500,
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        message: "Unable to create task",
      }),
    };
  }
};

This example demonstrates the basic flow, but there are a few important details to understand.

First, the table name comes from an environment variable. Configure TABLE_NAME in the Lambda function's environment settings rather than hardcoding the table name throughout your code.

Second, the function uses the AWS SDK's default credential provider chain. When deployed to Lambda, it can obtain temporary credentials from the function's execution role. You generally should not put permanent AWS access keys in your application code.

Third, the conditional expression prevents overwriting an item with the same composite primary key. Because this example generates a UUID for each new task, a collision is extremely unlikely, but the condition still makes the intended behavior explicit.

Fourth, the example is simplified. It accepts a userId from the request body for demonstration. In a real multi-user application, derive the user's identity from a verified authentication mechanism rather than trusting a client-supplied user ID. Otherwise, a user might attempt to create or access another user's data.

Also, the catch block returns a generic error to the client while logging the underlying error for troubleshooting. Production applications should distinguish expected client errors from unexpected server failures and avoid exposing internal details.

Important: configure IAM permissions

The Lambda execution role needs permission to write to the intended DynamoDB table.

A tightly scoped policy should grant only the required action, such as dynamodb:PutItem, on the specific table ARN. If the function later needs to query or update tasks, grant only those additional actions.

Avoid giving the function broad permissions such as dynamodb:* on every table in the account.

Important: configure the DynamoDB table

The table must exist before this function can use it. Its primary key must match the attributes in the item:

Partition key: userId (String).
Sort key: taskId (String).

Configure the TABLE_NAME environment variable with the table's actual name.

This code is a handler example, not a complete deployment tutorial. You still need to create the table, configure the Lambda runtime and execution role, connect the function to API Gateway, and test the request and response formats.

7. What About Authentication and Security?

A serverless backend still needs security. AWS managing the infrastructure does not automatically secure your application's business logic or data.

Here are some services and controls that may be useful.

AWS IAM

IAM controls which AWS resources and actions an identity can access.

For this architecture, the Lambda execution role should grant only the permissions the function needs, such as writing to one DynamoDB table.

Amazon Cognito or another authentication provider

If users need to sign in, you need an authentication mechanism.

Amazon Cognito can provide user sign-up, sign-in, and token-based authentication capabilities. API Gateway can be configured to validate supported authorization tokens, depending on the API type and setup.

Authentication identifies the caller. Your application must still enforce authorization: a signed-in user should not automatically have access to every other user's tasks.

AWS Secrets Manager

If the application needs external API credentials or other secrets, Secrets Manager can store and manage them.

DynamoDB access through the Lambda execution role generally does not require a database password. Avoid adding unnecessary credentials simply because other backend architectures use them.

Amazon CloudWatch

CloudWatch can collect Lambda logs, metrics, and alarms. These help you investigate errors, monitor latency, and detect unusual invocation or throttling patterns.

A simple security model

Frontend
   |
   v
API Gateway
   |
   +--> Validate authentication
   |
   v
Lambda Execution Role
   |
   +--> Only required DynamoDB actions
   |
   v
DynamoDB Table

The exact configuration depends on the application's requirements, but the core principle stays the same: grant only the permissions required, and verify who is allowed to access each user's data.

8. What Happens When Traffic Increases?

One reason serverless architecture is attractive is that it can reduce the amount of infrastructure management required as demand changes.

However, each component has its own limits and scaling behavior.

API Gateway

API Gateway can handle API traffic and supports configurable throttling and quotas. Your API may still receive throttling responses if configured limits or service quotas are exceeded.

Lambda

Lambda can increase concurrent executions as demand grows, within applicable concurrency quotas and account limits.

But more concurrent Lambda executions can also increase pressure on downstream services. If a function triggers expensive database operations, scaling the function alone will not solve the underlying bottleneck.

DynamoDB

DynamoDB offers on-demand and provisioned capacity modes. The appropriate choice depends on traffic patterns, predictability, and cost requirements.

Your key design and indexes also matter. A poorly designed access pattern can create hot partitions or inefficient queries even when the database itself is managed.

What about cold starts?

A cold start can occur when Lambda needs to initialize a new execution environment before processing a request. This may add latency.

The effect depends on factors such as runtime, initialization work, dependencies, memory configuration, and the execution environment.

For many applications, occasional cold starts are acceptable. For latency-sensitive workloads, measure actual performance and evaluate options such as reducing initialization work or using provisioned concurrency where appropriate.

Serverless makes infrastructure management easier in many cases, but it doesn't remove the need for performance testing.

9. Serverless Is Not Always Cheaper

This is an important lesson: serverless is a deployment model, not a guarantee of lower cost.

With Lambda, costs can depend on request volume, execution duration, configured memory, and other applicable charges.

API Gateway pricing depends on the API type and usage. DynamoDB costs depend on the capacity mode, requests, storage, and additional features.

For a small application with occasional requests, a serverless design can be cost-effective because you don't need to maintain an always-running application server for every request.

For a high-volume application with predictable, sustained workloads, another architecture might be more economical. You need to compare actual usage and current pricing rather than assume one model always wins.

Remember to consider supporting services, logging volume, data transfer, provisioned capacity, and other configuration choices when estimating costs.

10. Lambda vs EC2 vs ECS/Fargate

Since I've been learning EC2, Docker, Amazon ECR, ECS, and Fargate, I find it useful to compare these approaches.

Feature	AWS Lambda	EC2	ECS with Fargate
Infrastructure management	AWS manages the execution infrastructure	You manage the instances and operating system	AWS manages the underlying compute infrastructure
Application model	Event-driven functions	Applications or services on virtual machines	Containerized applications
Scaling	Based on invocations and concurrency settings	Configure scaling for instances and workloads	Configure service/task scaling
Long-running processes	Subject to function execution limits	Suitable	Suitable, subject to task and service configuration
Runtime control	Supported Lambda runtimes and execution model	High control	Container-level control
Operational effort	Often lower for suitable workloads	Often higher	Moderate, depending on the application
Good fit	APIs, event handlers, scheduled tasks	Custom server environments and broad workloads	Containerized APIs, workers, and services

Lambda is especially useful when work can be broken into event-driven executions.

EC2 is useful when you need control over the virtual machine and its environment.

ECS with Fargate is useful when you want to run containers without managing the underlying EC2 instances yourself.

These services are not mutually exclusive. A real application can use Lambda for event-driven operations and ECS/Fargate for a long-running API or worker.

Choose based on the workload, operational requirements, performance expectations, and cost—not just which service sounds newer.

11. What I Would Consider Before Deploying to Production

Before deploying a serverless API for real users, I would check the following:

[ ] Configure API routes and appropriate authorization.
[ ] Validate request bodies and query parameters.
[ ] Derive user identity from verified authentication rather than trusting client-supplied identity fields.
[ ] Use a least-privilege Lambda execution role.
[ ] Design DynamoDB keys and indexes around real access patterns.
[ ] Configure environment variables and manage any required secrets safely.
[ ] Add useful logs, metrics, alarms, and tracing where appropriate.
[ ] Handle errors and retries carefully.
[ ] Make operations safe to retry where duplicate processing is possible.
[ ] Set suitable timeouts, concurrency limits, and throttling.
[ ] Test under realistic traffic and estimate costs.
[ ] Deploy through a repeatable infrastructure-as-code and CI/CD process.

One detail worth emphasizing is idempotency. Distributed systems can retry requests or events after failures. If a repeated operation would create duplicate records, charge a customer twice, or trigger the same action repeatedly, the application needs a strategy to prevent that.

For asynchronous workflows, services such as Amazon SQS can help decouple components. But queue-based processing also requires careful retry handling, dead-letter queues where appropriate, and idempotent consumers.

12. When Should You Use This Architecture?

A serverless API built with API Gateway, Lambda, and DynamoDB is worth considering when:

You are building a small or medium-sized API.
Traffic is variable or difficult to predict.
You want to reduce server administration.
Your application fits DynamoDB's data model and access patterns.
You want to integrate backend operations with events from other AWS services.
You prefer managed services for the initial deployment.

It may be less suitable when:

Your application depends heavily on relational queries and complex joins.
You need sustained, long-running processing beyond Lambda's execution limits.
You require very predictable low latency and have not accounted for initialization behavior.
Your workload has steady high utilization and another compute model may offer better economics.
You need operating-system or runtime control that the chosen serverless service does not provide.

These are trade-offs, not absolute rules. The best architecture depends on the actual requirements.

13. My Main Takeaways

After learning about API Gateway, Lambda, and DynamoDB, I think the biggest benefit of serverless is the change in focus.

Instead of spending as much time managing servers, you can spend more time designing APIs, writing business logic, securing data access, and understanding how application components interact.

Here is the mental model I use:

API Gateway receives and routes API requests.
Lambda executes backend logic.
DynamoDB stores and retrieves data.
IAM controls AWS permissions.
Cognito or another identity provider can support authentication.
CloudWatch helps monitor and troubleshoot the system.
SQS can decouple asynchronous work when the application needs it.

But serverless does not mean architecture disappears. You still need to design for security, failures, retries, performance, and cost.

That is what makes learning AWS interesting: the goal isn't simply to learn what each service does. It's to understand how to combine the services to solve a real problem.

What's next?

I'm continuing my AWS learning journey by exploring how these services fit into real application architectures, from containerized backends to serverless APIs and automated deployments.

If you're also learning AWS, I'd love to hear what you're building and which services you're exploring.

Stop Hardcoding Secrets: How Production Applications Manage Credentials on AWS

Anupa Supul — Mon, 12 Oct 2026 05:50:35 +0000

Imagine you're building a Node.js backend using Express and MongoDB.

During development, you might have a .env file containing:

PORT=5000
MONGO_URI=mongodb+srv://...
JWT_SECRET=...
API_KEY=...

Your application works perfectly.

But now imagine pushing the project to GitHub, deploying it through a CI/CD pipeline, and running it on AWS.

Where should those secrets live?

Should we put them directly inside the source code? Add them to the Docker image? Store them in GitHub? Or let every developer manage their own copies?

These questions become important when an application moves from a local development environment to production.

In this article, we'll explore how AWS Secrets Manager, Systems Manager Parameter Store, and IAM help us manage application configuration and credentials more securely.

1. Why Hardcoding Secrets Is Dangerous

Let's start with a simple example.

const dbPassword = "my-super-secret-password";
const jwtSecret = "my-jwt-secret";

This may look convenient, but it creates a serious security problem.

Anyone who can access the source code may also be able to access the credentials.

If we push these values to a public repository, the secrets could be exposed. Even a private repository can have too many people or automated systems with access.

There is another problem: what happens when a password changes?

We may need to modify the code, rebuild the Docker image, and deploy the application again just to update a credential.

Application code and sensitive configuration should be managed separately.

That is the principle behind secure secrets management.

2. What About the `.env` File?

A .env file is useful during local development.

For example:

PORT=5000
MONGO_URI=your_database_connection_string
JWT_SECRET=your_jwt_secret

Your Node.js application can load these values using a package such as dotenv.

require("dotenv").config();

const port = process.env.PORT;
const mongoUri = process.env.MONGO_URI;

This keeps configuration out of the source code itself.

However, a .env file is not automatically a secure secrets management system.

For local development, we should keep it out of Git using .gitignore.

.env
.env.*
!.env.example

An .env.example file can document the required variable names without containing real credentials.

For production, we need a controlled way to provide those values to our running application.

That's where AWS services come in.

3. AWS Secrets Manager — Store Sensitive Information

AWS Secrets Manager is designed to store, retrieve, and manage sensitive information such as database credentials, API keys, and other application secrets.

For example, we could store a secret containing:

Secret name: production/my-backend

MONGO_URI: ...
JWT_SECRET: ...
THIRD_PARTY_API_KEY: ...

The application retrieves the required secret when it needs it, provided it has the appropriate IAM permissions.

A simplified architecture looks like this:

Developer
    |
    v
AWS Secrets Manager
    |
    | Authorized access
    v
Application on ECS / Fargate
    |
    v
Connect to Database / External API

The important point is that the sensitive values do not need to be hardcoded into our application source code or baked into the Docker image.

Secrets Manager also supports secret rotation workflows, including integrations and rotation configurations for supported services.

Rotation still needs to be configured correctly; storing a secret in Secrets Manager does not automatically mean every secret will rotate itself.

4. Systems Manager Parameter Store — Manage Configuration

Not every configuration value is a secret.

Consider these examples:

APP_ENV=production
LOG_LEVEL=info
MAX_RETRIES=3
API_BASE_URL=https://api.example.com

These values may need to change between development, testing, and production, but they are not necessarily confidential.

AWS Systems Manager Parameter Store provides a central place to store application configuration and parameters.

It supports different parameter types, including:

String — ordinary configuration values.
StringList — comma-separated string values.
SecureString — sensitive values encrypted using AWS KMS.

For example:

/my-app/production/LOG_LEVEL
/my-app/production/API_BASE_URL
/my-app/production/DB_PASSWORD

Parameter Store can help organize configuration by application and environment.

A SecureString can be useful for encrypted parameters, while Secrets Manager is often a better fit when managed secret lifecycle and rotation features are important.

The choice depends on the requirements rather than one service being universally better.

5. Secrets Manager vs Parameter Store

This is where many developers get confused.

Both services can store sensitive configuration, but their strengths differ.

Feature	Secrets Manager	Parameter Store
Main purpose	Secret lifecycle management	Configuration and parameter management
Database credentials	Yes	Can store encrypted values
Encrypted values	Yes	Yes, using `SecureString`
Secret rotation	Built-in rotation capabilities and integrations	Usually requires a separate rotation workflow
Ordinary configuration	Possible	A natural fit
Cost considerations	Generally charged for secrets and API usage	Standard parameters have a free tier; advanced features can incur charges

For example, I might choose Secrets Manager for database credentials that require a managed rotation workflow and Parameter Store for ordinary application settings.

Both can be useful in the same application.

6. IAM — Who Is Allowed to Read the Secrets?

Storing secrets securely is only half of the problem.

We also need to decide who or what is allowed to access them.

This is where AWS Identity and Access Management (IAM) comes in.

IAM controls access to AWS resources through identities, policies, and roles.

Imagine our Node.js backend is running as an ECS task on Fargate.

We want it to retrieve one database secret from Secrets Manager.

We should not give the application unrestricted access to every secret in the AWS account.

Instead, we can grant the task permission to retrieve only the required secret.

ECS Task
   |
   v
Task Role
   |
   v
IAM Policy
   |
   v
Allowed Secret Only

This follows the principle of least privilege: give each component only the permissions it needs.

For ECS, there are two roles worth distinguishing:

Task execution role: Used by ECS/Fargate for operations such as pulling private images from ECR and sending logs to supported destinations.
Task role: Provides AWS permissions to the application code running inside the container.

If the application itself calls Secrets Manager, its task role generally needs the appropriate permission.

If ECS retrieves secrets and injects them into a container as environment variables, the task execution role needs the relevant permissions instead.

Getting this distinction right helps avoid unnecessary permissions and deployment failures.

7. How Does This Work with Docker and ECS?

Let's connect secrets management to the previous article about Docker, ECR, ECS, and Fargate.

A common mistake is putting production credentials directly into the Dockerfile.

For example, this is a bad idea:

ENV JWT_SECRET=my-production-secret

Why?

Because the value can become part of the image configuration and may be exposed to anyone who can inspect the image.

Instead, the image should contain the application, while production configuration is supplied separately.

                 Docker Build
                     |
                     v
              Application Image
                     |
                     v
                 Amazon ECR
                     |
                     v
                ECS / Fargate
                     |
                     v
           Runtime Configuration
                     |
           +---------+----------+
           |                    |
           v                    v
     Secrets Manager       Parameter Store

For example, an ECS task definition can reference a secret stored in Secrets Manager and make it available to the container at startup.

The application can then read the value through its environment.

One important detail: when ECS injects a secret into an environment variable, the value is retrieved when the container starts. If the secret changes later, an already-running container does not automatically receive the updated value. A new task deployment or restart may be needed.

Also, environment variables are convenient, but they can be exposed through debugging tools, process inspection, or logs if handled carelessly. They should never be printed unnecessarily.

8. Example: A Node.js Backend on AWS

Imagine we deploy a Node.js application using:

GitHub Actions for CI/CD.
Docker for packaging.
Amazon ECR for storing the image.
Amazon ECS with Fargate for running the backend.
Amazon RDS for the database.
AWS Secrets Manager for database credentials.
IAM for controlling access.

The application flow looks like this:

Users
  |
  v
Application Load Balancer
  |
  v
ECS / Fargate
  |
  +-------> Secrets Manager
  |          Database Credentials
  |
  +-------> Amazon RDS
              Application Data

The backend needs permission to retrieve the secret and network access to connect to the database.

The database should normally be placed in private subnets, with security group rules that allow connections only from the appropriate application tasks.

Secrets Manager handles secret storage, IAM controls authorization, and the VPC and security groups control network access.

These are separate security layers working together.

9. What About GitHub Actions and CI/CD?

Our previous article explained how GitHub Actions can build Docker images, push them to ECR, and deploy them to ECS.

The pipeline also needs secure access to AWS.

A recommended approach is to configure OpenID Connect (OIDC) so GitHub Actions can assume an IAM role and obtain temporary AWS credentials.

GitHub Actions
       |
       v
      OIDC
       |
       v
    AWS IAM Role
       |
       v
     Amazon ECR
       |
       v
     Amazon ECS

The workflow role should have only the permissions required for its deployment responsibilities.

This is different from the IAM task role used by the application after deployment.

We should also avoid printing secret values in CI logs or passing credentials through command-line arguments unnecessarily.

The goal is to secure both the deployment process and the application runtime.

10. What Should We Choose?

Here is a practical way to decide.

Use Secrets Manager when:

You need centralized storage for sensitive credentials.
You need secret rotation capabilities.
Your application needs controlled access to database passwords or API keys.

Use Parameter Store when:

You need centralized application configuration.
You want to organize settings by environment.
You need encrypted parameters and don't require the full Secrets Manager lifecycle features.

Use IAM when:

You need to control which users, services, or workloads can access AWS resources.
You want applications to use roles instead of hardcoded AWS access keys.
You want to enforce least-privilege permissions.

These services are complementary, not direct replacements for one another.

Quick Cheat Sheet

Tool	Main Responsibility
`.env`	Local application configuration
`.gitignore`	Prevents local environment files from being committed
AWS Secrets Manager	Stores and manages sensitive secrets
Systems Manager Parameter Store	Stores configuration and parameters
`SecureString`	Encrypted Parameter Store value
IAM Role	Grants AWS permissions to an identity or workload
ECS Task Role	Permissions used by application code
ECS Task Execution Role	Permissions used by ECS/Fargate for task startup and related operations
GitHub OIDC	Provides a way for GitHub Actions to obtain temporary AWS credentials
AWS KMS	Manages encryption keys used by supported AWS services

Final Thoughts

Moving an application to AWS is not just about getting it to run.

We also need to think about where its credentials live, who can access them, how they are updated, and what happens when those credentials change.

A local .env file can be useful during development, but production applications need a more controlled approach.

Secrets Manager, Parameter Store, and IAM help separate application code from sensitive configuration and enforce controlled access.

The key lesson is simple:

Don't hardcode secrets. Don't give every component unrestricted access. Keep configuration, permissions, and application code separate.

As I continue learning AWS, I'm finding that production engineering is not only about deploying applications. It's also about making sure they remain secure, maintainable, and reliable after deployment.

What Happens When You Push Code to Production? Understanding AWS CI/CD with GitHub Actions, Docker, ECR & ECS

Anupa Supul — Mon, 12 Oct 2026 05:43:14 +0000

Imagine you have finished building a Node.js backend. Everything works on your laptop, and you are ready to deploy it.

Traditionally, you might connect to a server, pull the latest code, install dependencies, restart the application, and hope nothing breaks.

But what happens when you need to do this every week? Every day? Or several times a day?

Manually repeating those steps becomes slow, error-prone, and difficult to manage.

This is where CI/CD becomes important.

After learning Docker, Amazon ECR, ECS, and Fargate, the next question is how to connect them into an automated deployment pipeline.

Let's follow a simple example: deploying a Node.js application to AWS whenever new code is pushed to GitHub.

1. First, What Is CI/CD?

CI/CD is a set of practices that helps developers build, test, and deliver software more reliably.

Continuous Integration (CI) means regularly integrating code changes and automatically checking whether the application still works.

Continuous Delivery (CD) means keeping the application ready for release. With Continuous Deployment, changes that pass the required checks can be deployed automatically to production.

Think about a team working on the same backend.

One developer fixes a login bug. Another adds a new API endpoint. Someone else updates the database integration.

Without automation, every change needs manual checking and deployment.

With CI/CD, a pipeline can automatically test the changes, build the application, package it into a Docker image, and deploy a new version.

The basic idea is:

Write code → Test it → Build it → Deploy it

2. The Complete AWS CI/CD Architecture

Here is the architecture we will use throughout this article.

                 DEVELOPER
                     |
                     v
                   GitHub
                     |
                     v
              GitHub Actions
                     |
              +------+------+
              |             |
              v             v
          Install         Run Tests
         Dependencies       |
              |             |
              +------+------+
                     |
                     v
               Build Docker Image
                     |
                     v
                Amazon ECR
             (Image Registry)
                     |
                     v
                Amazon ECS
                     |
                     v
                AWS Fargate
                     |
                     v
               Running Tasks
                     |
                     v
               Production App

This pipeline separates the deployment process into clear responsibilities.

GitHub stores the source code.
GitHub Actions automates the workflow.
Docker packages the application.
Amazon ECR stores the container image.
Amazon ECS manages the deployment and running tasks.
AWS Fargate runs the containers without requiring us to manage the underlying EC2 servers.

Each component solves a different problem.

3. Step One: Push Code to GitHub

Let's say we update our backend.

We add a new endpoint, fix a bug, or improve an existing feature.

After committing the changes, we push them to GitHub.

Developer
    |
    v
Local Code Changes
    |
    v
Git Commit
    |
    v
Git Push
    |
    v
GitHub Repository

GitHub becomes the starting point for our deployment workflow.

But pushing code to GitHub does not automatically mean that the code is safe to deploy. We still need to check it.

That is the responsibility of the next stage.

4. Step Two: GitHub Actions Runs the Pipeline

GitHub Actions is a CI/CD automation platform built into GitHub.

We can define a workflow that runs when someone pushes code to a particular branch or opens a pull request.

For example:

Code pushed to main
        |
        v
GitHub Actions starts
        |
        v
Install dependencies
        |
        v
Run automated tests
        |
        v
Continue if checks pass

A workflow is defined in a YAML file inside the repository, usually under:

.github/workflows/

A simplified workflow might look like this:

name: Backend CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 22

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test

This example focuses on continuous integration. It checks the code and runs tests, but it does not yet build or deploy the Docker image.

If the tests fail, we can configure the pipeline to stop before deployment.

That is important because automation should prevent broken code from reaching production, not just deploy faster.

5. Step Three: Build the Docker Image

Once the required checks pass, the next stage is packaging the application.

This is where Docker connects with the pipeline.

Source Code
     |
     v
Dockerfile
     |
     v
Docker Build
     |
     v
Docker Image

The Docker image contains the application code, its dependencies, and the instructions needed to run it.

For example, a Node.js backend might use a Dockerfile like this:

FROM node:22

WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY . .

EXPOSE 3000

CMD ["npm", "start"]

This is a simplified example. A production Dockerfile should also consider security, image size, dependency management, and which files should be excluded using .dockerignore.

The important point is that the pipeline can build a new image whenever the application changes.

Instead of manually preparing a server for every release, we produce a consistent application package.

6. Step Four: Push the Image to Amazon ECR

Now we have a Docker image. Where should it go?

Amazon Elastic Container Registry (ECR) is AWS's managed container image registry.

GitHub Actions can authenticate with AWS, build the image, tag it, and push it to an ECR repository.

GitHub Actions
      |
      v
Docker Image
      |
      v
Amazon ECR
      |
      v
Versioned Image

For example, we might tag images using the Git commit SHA:

my-backend:a1b2c3d
my-backend:d4e5f6a

Each tag identifies a particular build. This makes it easier to identify which version is running and to deploy a previous version when needed.

For production, avoid relying only on the mutable latest tag. Use explicit version tags and, where appropriate, immutable image digests.

One important security detail: GitHub Actions needs permission to access ECR. A recommended approach is to use AWS IAM roles with GitHub Actions OpenID Connect (OIDC), so the workflow can obtain temporary AWS credentials without storing long-lived AWS access keys in GitHub secrets.

7. Step Five: Deploy the New Image to Amazon ECS

The image is now stored in ECR, but it is not running yet.

We need ECS to deploy it.

Amazon ECS manages our container workloads. An ECS task definition describes how a container should run, including its image, CPU, memory, ports, and configuration.

When we deploy a new application version, the pipeline can register or reference an updated task definition and ask the ECS service to deploy it.

New Docker Image
       |
       v
Amazon ECR
       |
       v
Updated Task Definition
       |
       v
Amazon ECS Service
       |
       v
New Running Tasks

ECS then manages the deployment according to the service's configuration.

If we use Fargate, AWS manages the underlying compute infrastructure for the tasks.

We still need to configure the task's networking, IAM permissions, resource requirements, logging, and other application settings.

The benefit is that we no longer need to manually log into an EC2 instance to install the application and restart the process after every release.

8. Step Six: How Does the New Version Reach Users?

Imagine our backend currently has three running tasks:

ECS SERVICE — VERSION 1

Task 1  ── Version 1
Task 2  ── Version 1
Task 3  ── Version 1

Now we deploy Version 2.

ECS can use a rolling deployment to gradually replace the old tasks with new ones, according to the service's deployment settings.

DEPLOYMENT IN PROGRESS

Task 1  ── Version 2
Task 2  ── Version 1
Task 3  ── Version 1

Then:

Task 1  ── Version 2
Task 2  ── Version 2
Task 3  ── Version 1

Finally:

DEPLOYMENT COMPLETE

Task 1  ── Version 2
Task 2  ── Version 2
Task 3  ── Version 2

An Application Load Balancer can route requests to healthy registered targets while the deployment progresses.

However, a rolling deployment does not automatically guarantee zero downtime. That depends on settings such as health checks, minimum healthy capacity, maximum deployment capacity, and whether the new application version is compatible with existing clients and data.

This is why deployment configuration matters just as much as the pipeline itself.

9. What If the New Deployment Fails?

Suppose Version 2 contains a bug.

The application starts, but an API endpoint returns errors.

A reliable deployment process should not assume that a successful Docker build means the application is healthy.

We need checks at different stages:

Automated tests before building or deploying.
Container startup and health checks.
ECS service and deployment status.
Application logs and metrics.
Post-deployment smoke tests that verify important endpoints.

ECS can replace unhealthy tasks, but replacing a task is not the same as detecting every application bug. A task might be running while its business logic is broken.

For deployments that fail, ECS deployment circuit breaker settings can help detect failures and optionally roll back to a previous deployment. The pipeline can also stop or trigger further action when post-deployment checks fail.

The goal is to detect problems quickly and reduce the impact on users.

10. CI/CD Is More Than Just Deployment

A common misconception is that CI/CD simply means automatically uploading code to a server.

A good pipeline should help us deliver changes safely and consistently.

A more complete workflow might look like this:

Developer Pushes Code
          |
          v
   Automated Tests
          |
          v
   Code / Security Checks
          |
          v
    Build Docker Image
          |
          v
     Push Image to ECR
          |
          v
    Deploy to ECS
          |
          v
   Health / Smoke Tests
          |
          v
      Production

Depending on the project, we might also include dependency vulnerability scanning, infrastructure validation, manual approval before production, and notifications when a deployment succeeds or fails.

Not every project needs the same pipeline. The right setup depends on the application's risk, team size, compliance requirements, and deployment frequency.

11. What About Secrets and Permissions?

Automation needs access to AWS resources, but giving a pipeline unrestricted access is dangerous.

For example, GitHub Actions may need to push images to ECR and update an ECS service.

It should not automatically receive permission to modify every resource in the AWS account.

A safer design uses:

IAM: Grants only the permissions the workflow needs.
GitHub OIDC: Allows GitHub Actions to assume an AWS role using temporary credentials.
Secrets Manager or Parameter Store: Stores application secrets and configuration where appropriate.
ECS task roles: Allow the running application to access specific AWS services.
Security Groups and VPC networking: Restrict network communication between the load balancer, application tasks, and database.

It is also important to distinguish the deployment pipeline's permissions from the permissions of the running application. They serve different purposes and should be configured separately.

12. The Complete Deployment Architecture

Let's connect the pieces.

                    DEVELOPER
                        |
                        v
                      GitHub
                        |
                        v
                 GitHub Actions
                        |
              +---------+---------+
              |                   |
              v                   v
          Run Tests          Build Image
                                  |
                                  v
                             Amazon ECR
                                  |
                                  v
                             Amazon ECS
                                  |
                                  v
                             AWS Fargate
                                  |
                                  v
                          Running ECS Tasks
                                  |
                                  v
                                  ALB
                                  |
                                  v
                               USERS

This diagram shows the deployment pipeline and the running application, but they are two different flows.

Deployment flow:

Code → GitHub Actions → Docker → ECR → ECS

Application request flow:

Users → ALB → ECS Tasks → RDS / S3 / Other Services

In a complete web application, Route 53 and CloudFront can sit in front of the application, while networking, security, monitoring, and scaling support the system.

The CI/CD pipeline changes how we deliver application versions. It doesn't replace the architecture that serves user requests.

13. GitHub Actions vs AWS CodePipeline

One question worth asking is whether we must use AWS CodePipeline.

No.

GitHub Actions can handle CI/CD workflows directly from a GitHub repository. It can run tests, build Docker images, push them to ECR, and trigger ECS deployments.

AWS CodePipeline is another option for orchestrating release workflows using AWS services.

The choice depends on the existing development workflow and operational requirements.

GitHub Actions	AWS CodePipeline
Integrated with GitHub repositories	Integrated with AWS deployment workflows
Uses workflow YAML files	Uses pipeline stages and actions
Can build and deploy to AWS	Can orchestrate AWS and supported external actions
Convenient when source code is already on GitHub	Useful when a team wants an AWS-centered pipeline

Neither is automatically the better choice. A team should consider its existing tools, access controls, deployment process, and maintenance needs.

The Mental Model I Keep Now

Before learning CI/CD, my deployment mental model was:

Write Code
    |
    v
Deploy Manually
    |
    v
Restart Application

Now I think about the process as:

Code
 |
 v
GitHub
 |
 v
Automated Checks
 |
 v
Docker Image
 |
 v
Amazon ECR
 |
 v
Amazon ECS
 |
 v
AWS Fargate
 |
 v
Production

Each stage has a responsibility.

GitHub stores the code. GitHub Actions automates the workflow. Docker packages the application. ECR stores the image. ECS manages the deployment. Fargate runs the containers.

That is the connection between development and production.

Quick Cheat Sheet

Technology	Main Responsibility
GitHub	Source code management
GitHub Actions	CI/CD workflow automation
CI	Integrate changes and run checks
CD	Prepare or deliver changes to production
Docker	Package the application
ECR	Store container images
ECS	Manage container workloads and deployments
Task Definition	Describes how a task should run
ECS Service	Maintains desired tasks and manages deployments
Fargate	Runs containers without managing EC2 hosts
ALB	Routes requests to healthy targets
CloudWatch	Logs, metrics, dashboards, and alarms
IAM + OIDC	Control pipeline access to AWS

Final Thoughts

The biggest lesson for me is that deploying an application shouldn't depend on someone remembering a long list of manual steps.

A well-designed CI/CD pipeline makes those steps repeatable, testable, and easier to monitor.

It also connects the tools developers use every day with the AWS services that run their applications in production.

Docker taught me how to package an application. ECR gave that package a place to live. ECS and Fargate showed me how to run it on AWS.

CI/CD connects all of them into a repeatable delivery process.

And that's an important step toward understanding not just cloud infrastructure, but how modern software is delivered and operated.

The next time you push code to GitHub, think about everything that could happen automatically before that code reaches your users.

Code → Test → Build → Package → Deploy → Verify.

I Stopped Managing Servers Manually — Understanding Docker, Amazon ECR, ECS & Fargate

Anupa Supul — Tue, 15 Sep 2026 04:59:23 +0000

I Stopped Managing Servers Manually — Understanding Docker, Amazon ECR, ECS & Fargate

I had already learned how a web application can be built on AWS using EC2, Load Balancers, Auto Scaling, RDS, S3, SQS, and other services.

But there was still one question I kept coming back to:

What if I don't want to manage the server itself?

With EC2, I am still responsible for things like the operating system, installed software, runtime environment, and server configuration.

But modern applications are increasingly built around containers.

That led me to the next part of my AWS learning journey:

Docker → Amazon ECR → Amazon ECS → AWS Fargate

And once I understood how these four pieces connect, container deployment on AWS started making much more sense.

First, What Problem Are Containers Solving?

Imagine I build a Node.js application on my laptop.

It works perfectly.

Node.js version: 22
Express: installed
Dependencies: installed
Environment: configured
Application: running

Then I give the application to another developer.

Suddenly:

“It works on my machine.”

Maybe they have a different Node.js version.

Maybe a dependency is missing.

Maybe an environment variable isn't configured.

Maybe the operating system behaves differently.

This is one of the problems containers help solve.

Instead of only shipping our application code, we can package the application together with the environment it needs to run.

That's where Docker comes in.

1. Docker — Package the Application

Docker allows us to package an application into a container image.

For a Node.js backend, the image can contain:

Node.js
Application code
Dependencies
Configuration needed to run

A simplified flow looks like this:

Application Code
       │
       ▼
   Dockerfile
       │
       ▼
 Docker Image
       │
       ▼
   Container

The important distinction is:

Image = packaged blueprint

Container = running instance of that image

For example:

Docker Image
     │
     ├── Container 1
     ├── Container 2
     └── Container 3

This makes it much easier to run the same application consistently across different environments.

2. Dockerfile — The Instructions for the Image

A Docker image is usually created from a Dockerfile.

A simplified Node.js example could look like:

FROM node:22

WORKDIR /app

COPY package*.json ./
RUN npm install

COPY . .

EXPOSE 3000

CMD ["npm", "start"]

I don't need to manually install Node.js on every server.

The Docker image already defines what the application needs.

So instead of saying:

“Please configure a server exactly like my development machine.”

I can say:

“Run this container.”

That is a much more repeatable deployment model.

3. But Where Does the Docker Image Go?

Now we have a Docker image.

But AWS needs somewhere to store it.

This is where Amazon Elastic Container Registry (ECR) comes in.

ECR is AWS's managed container image registry.

The flow becomes:

Developer
    │
    ▼
Dockerfile
    │
    ▼
Docker Build
    │
    ▼
Docker Image
    │
    ▼
Amazon ECR

Think of ECR as a private warehouse for container images.

For example, we might have:

my-backend
my-worker
my-payment-service

inside an ECR repository.

When we build a new version of the application, we can push a new image to ECR.

4. ECR Doesn't Run Our Application

This distinction is important.

ECR stores the image.

It doesn't run the application.

For example:

ECR
 │
 │ stores
 ▼
Docker Image

We still need a service that can take that image and actually run it.

That's where Amazon ECS comes in.

5. Amazon ECS — Running Containers on AWS

Amazon Elastic Container Service, or ECS, is a container orchestration service.

In simple terms:

ECS manages running containers for our application.

The architecture now starts looking like:

Code
  │
  ▼
Docker Image
  │
  ▼
ECR
  │
  ▼
ECS
  │
  ▼
Running Container

But ECS introduces a few concepts that are worth understanding.

6. ECS Cluster

An ECS Cluster is a logical grouping of resources used to run our ECS workloads.

Think of it as the environment where our services run.

For example:

Production Cluster
       │
       ├── Backend Service
       ├── Worker Service
       └── Payment Service

A cluster itself isn't simply “one server.”

It's more useful to think of it as a logical home for our container workloads.

7. ECS Task Definition

Now ECS needs to know how our container should run.

That's where a Task Definition comes in.

It can describe things such as:

Which Docker image?
How much CPU?
How much memory?
Which port?
Environment variables?
IAM permissions?
Logging configuration?

For example:

Task Definition
       │
       ├── Image → ECR
       ├── CPU
       ├── Memory
       ├── Port
       └── Environment

This is similar to giving ECS a set of instructions:

“This is the container I want you to run, and this is how it should run.”

8. ECS Service — Keeping Containers Running

A task definition tells ECS how to run a container.

But what if I want three copies of my backend running continuously?

That's where an ECS Service comes in.

For example:

ECS Service
     │
     ├── Task 1
     ├── Task 2
     └── Task 3

The service maintains the desired number of tasks.

If a running task fails, ECS can replace it.

This is one of the important differences between simply running a container and having a container orchestration system manage it.

9. Where Does the Container Actually Run?

This is where AWS Fargate becomes interesting.

ECS can run containers using different compute options.

One option is managing the underlying EC2 instances yourself.

Another option is AWS Fargate.

Fargate is a serverless compute engine for containers.

With Fargate, AWS manages the underlying infrastructure required to run the containers.

So instead of thinking:

ECS
 ↓
EC2
 ↓
Operating System
 ↓
Container

we can use:

ECS
 ↓
Fargate
 ↓
Container

We still define the container's CPU, memory, networking, and other configuration, but we don't have to manage the underlying EC2 servers ourselves.

10. ECS + Fargate Architecture

Now our application starts looking like this:

                         USERS
                           │
                           ▼
                       Route 53
                           │
                           ▼
                      CloudFront
                           │
                           ▼
                          ALB
                           │
                           ▼
                     ECS Service
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
          Fargate       Fargate      Fargate
           Task          Task          Task
              │            │            │
              └────────────┼────────────┘
                           │
                           ▼
                           RDS

The ALB distributes requests between the running ECS tasks.

If one task becomes unhealthy, the service can replace it.

If traffic increases, we can increase the desired number of tasks.

11. What Happens When We Deploy a New Version?

This is where the whole model becomes really powerful.

Suppose we update our Node.js application.

We make some changes:

Version 1
    ↓
Code changes
    ↓
Version 2

We then build a new Docker image.

Source Code
     │
     ▼
Docker Build
     │
     ▼
New Docker Image
     │
     ▼
Amazon ECR

ECS can then deploy the new version.

The simplified flow becomes:

Developer
    │
    ▼
GitHub
    │
    ▼
Build Docker Image
    │
    ▼
Amazon ECR
    │
    ▼
ECS Service
    │
    ▼
New Tasks
    │
    ▼
Production

This is the foundation for modern container-based CI/CD pipelines.

12. What About the Database?

Containers are designed to be replaceable.

That means we generally shouldn't treat the container itself as the permanent place for important application data.

Instead:

ECS / Fargate
      │
      ├── Application
      │
      ▼
     RDS
      │
      ▼
Persistent Data

If a container disappears and ECS starts a new one, the application data remains in RDS.

The same principle applies to files.

Instead of storing uploaded files inside the container:

Container ❌

we can use:

S3 ✅

So:

Application
    │
    ├── Database → RDS
    ├── Files → S3
    ├── Cache → ElastiCache
    └── Jobs → SQS

This separation is extremely important when designing scalable applications.

13. Security Still Matters

Moving from EC2 to containers doesn't mean security disappears.

We still need to think about:

IAM

ECS tasks can use IAM roles to access AWS services without hardcoding AWS credentials into the application.

Security Groups

We can control network communication between:

ALB
 ↓
ECS
 ↓
RDS

Secrets Manager

Sensitive values such as database credentials and API secrets can be stored securely instead of being hardcoded into the image.

VPC

Our ECS tasks can run inside private subnets while the ALB remains publicly reachable.

So the architecture can still follow a secure pattern:

Internet
    │
    ▼
   ALB
    │
    ▼
Private ECS Tasks
    │
    ├── RDS
    ├── S3
    └── Other AWS Services

14. What Happens When Traffic Increases?

Let's say our application normally runs three tasks:

ECS Service

Task 1
Task 2
Task 3

Suddenly, traffic increases.

We can configure ECS Service Auto Scaling to increase the number of running tasks.

Normal Traffic

Task 1
Task 2
Task 3

Then:

High Traffic

Task 1
Task 2
Task 3
Task 4
Task 5
Task 6

The ALB distributes traffic across the available healthy tasks.

This gives us a scalable container architecture without manually logging into servers and starting applications ourselves.

15. ECS vs EC2 — What's the Difference?

This was one of the comparisons that helped me understand the purpose of ECS.

EC2	ECS + Fargate
Manage virtual servers	Manage containers
OS management required	AWS manages underlying infrastructure with Fargate
Install runtime manually	Runtime packaged in container
Server-focused	Container-focused
More infrastructure control	Less infrastructure management
Scaling servers	Scaling container tasks

Neither is automatically “better.”

The right choice depends on the application's requirements, team experience, control requirements, and operational needs.

16. ECS vs Lambda

Another useful comparison is ECS vs Lambda.

Lambda is excellent for event-driven, short-lived workloads where we don't want to manage servers or containers directly.

ECS is useful when we want more control over a containerized application and its runtime.

A simplified comparison:

ECS	Lambda
Container-based	Function-based
Long-running services are common	Event-driven execution is common
More runtime control	Less infrastructure management
Good for containerized APIs/workers	Good for many event-driven workloads

Again, neither service replaces the other.

The architecture should determine the choice.

17. The Complete Flow

Now we can connect everything together.

                         USER
                           │
                           ▼
                       Route 53
                           │
                           ▼
                      CloudFront
                           │
                           ▼
                          ALB
                           │
                           ▼
                    ECS Service
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
          Fargate       Fargate      Fargate
           Task          Task          Task
              │            │            │
              └────────────┼────────────┘
                           │
             ┌─────────────┼─────────────┐
             ▼             ▼             ▼
            RDS           S3        ElastiCache
         Database      File Storage      Cache
                           │
                           │
                           ▼
                          SQS
                           │
                           ▼
                     Worker Tasks

And the deployment side looks like:

Developer
    │
    ▼
GitHub
    │
    ▼
CI/CD Pipeline
    │
    ▼
Docker Build
    │
    ▼
Amazon ECR
    │
    ▼
Amazon ECS
    │
    ▼
Fargate
    │
    ▼
Production

Now we have two important flows:

Application flow

User → ALB → ECS → AWS Services

Deployment flow

Code → Docker → ECR → ECS → Fargate

That distinction made the whole architecture much easier for me to understand.

The Mental Model I Keep Now

I don't think of ECS, ECR, and Fargate as three unrelated AWS services anymore.

I think of them as a chain:

Docker
   │
   │ Packages
   ▼
ECR
   │
   │ Stores
   ▼
ECS
   │
   │ Manages
   ▼
Fargate
   │
   │ Runs
   ▼
Container

In simple words:

Docker → Package my application

ECR → Store my image

ECS → Manage my containers

Fargate → Run those containers without managing the underlying servers

That mental model is much easier to remember than four separate definitions.

What Changed My Perspective?

Before learning containers, my mental model was mostly:

Code
 ↓
EC2
 ↓
Run application

Now it looks more like:

Code
 ↓
Docker
 ↓
Container Image
 ↓
ECR
 ↓
ECS
 ↓
Fargate
 ↓
Production

And when we connect that with the architecture from my previous AWS article:

Users
  ↓
Route 53
  ↓
CloudFront
  ↓
ALB
  ↓
ECS / Fargate
  ↓
RDS / S3 / SQS / ElastiCache

AWS starts looking less like a collection of services and more like a complete platform for running applications.

Quick Cheat Sheet

Technology / Service	Main Job
Docker	Packages applications into containers
Dockerfile	Defines how an image is built
Docker Image	Packaged application blueprint
Container	Running instance of an image
ECR	Stores container images
ECS	Orchestrates containers
Task Definition	Defines how containers should run
ECS Service	Maintains desired running tasks
Fargate	Runs containers without managing servers
ALB	Distributes traffic
RDS	Stores relational data
S3	Stores files and objects
SQS	Handles asynchronous jobs
ElastiCache	Caches frequently accessed data
IAM	Controls permissions
VPC	Provides network isolation

Final Thoughts

The biggest shift in my AWS learning wasn't memorizing more services.

It was learning to ask:

“What problem is this service solving, and where does it fit in the architecture?”

Once I started thinking that way, AWS stopped feeling like a huge list of unrelated services.

Instead, it started looking like a set of building blocks.

And when those building blocks are combined properly, we can build applications that are scalable, secure, highly available, and easier to operate.

That, for me, is where learning AWS started becoming much more interesting.

How a Real Website Works on AWS — Complete AWS Architecture

Anupa Supul — Fri, 04 Sep 2026 09:16:17 +0000

Let’s imagine we are building a MERN web application with a React frontend, Node.js/Express backend, database, file uploads, authentication, and thousands of users.

This is what the architecture could look like.

The Big Picture

                         USERS
                           │
                           ▼
                       Route 53
                           │
                           ▼
                      CloudFront
                       /       \
                      /         \
                     ▼           ▼
                  S3             ALB
             React Frontend      │
                                 ▼
                           Auto Scaling
                                 │
                    ┌────────────┼────────────┐
                    ▼            ▼            ▼
                   EC2          EC2          EC2
                    │            │            │
                    └────────────┼────────────┘
                                 │
                    ┌────────────┼────────────┐
                    ▼            ▼            ▼
                   RDS       ElastiCache      SQS
                Database        Cache        Queue
                    │                         │
                    │                         ▼
                    │                       Worker
                    │
                    ▼
                   S3
              File Storage

At first, this diagram can look complicated.

But the important thing is that each service has a specific responsibility.

Instead of thinking about AWS as one giant system, think of it as a collection of specialized building blocks.

1. What Happens When You Open a Website?

Let's say a user types:

www.mywebsite.com

into their browser.

The request doesn't simply jump directly to an EC2 server.

It goes through several layers.

First, Route 53 helps the browser find where the domain should go. The request can then reach CloudFront, AWS's content delivery network.

CloudFront decides where the request should be served from.

If the user requests a React file such as JavaScript, CSS, or an image, CloudFront can retrieve it from Amazon S3.

If the request is something like:

GET /api/users

CloudFront can send it toward the backend through an Application Load Balancer (ALB).

So even before our backend receives the request, AWS has already handled DNS, content delivery, and traffic routing.

2. Route 53 — Finding Our Website

Amazon Route 53 is AWS's DNS service.

DNS is basically the system that translates a human-friendly domain name into a destination.

For example:

www.mywebsite.com
        ↓
     DNS lookup
        ↓
   CloudFront

Without DNS, users would need to remember an IP address instead of a domain name.

Route 53 therefore acts like the phone book of our website.

It doesn't run our frontend or backend. Its job is mainly to help users find the correct destination.

3. CloudFront — Bringing Content Closer to Users

Once Route 53 points the request toward CloudFront, CloudFront can handle the delivery of content.

CloudFront is a CDN (Content Delivery Network).

Imagine our application is hosted in a region far away from a user. Sending every image, JavaScript file, and CSS file all the way from our origin would add unnecessary latency.

CloudFront solves this by caching content at locations around the world called edge locations.

So a simplified flow looks like:

User
  │
  ▼
CloudFront Edge Location
  │
  ├── Cached content → Return immediately
  │
  └── Not cached → Ask the origin

This is especially useful for static content such as:

React JavaScript files
CSS
Images
Videos
Documents

It can also sit in front of our API and work with services such as AWS WAF.

4. React Frontend — Where Users Interact

Our React application is the part users actually see.

The frontend contains things like:

Login page
Dashboard
Profile
Products
Buttons
Forms
Images

Because a React production build consists largely of static files, we don't necessarily need an EC2 server just to serve those files.

We can store the built frontend in Amazon S3.

For example:

React Project
     │
     ▼
npm run build
     │
     ▼
HTML + CSS + JS
     │
     ▼
Amazon S3
     │
     ▼
CloudFront
     │
     ▼
Users

This is one of the first things that changed how I looked at AWS.

Not everything needs a traditional server.

5. Application Load Balancer — Sending Users to the Right Server

Now suppose the frontend calls our backend:

POST /api/login
GET /api/products
POST /api/orders

We don't want every user connecting directly to one EC2 instance.

Instead, requests can go through an Application Load Balancer (ALB).

                  ALB
                   │
        ┌──────────┼──────────┐
        ▼          ▼          ▼
       EC2        EC2        EC2

The ALB distributes incoming requests across healthy backend instances.

It also performs health checks.

For example, if one EC2 instance stops responding:

EC2 #1 → Healthy
EC2 #2 → Healthy
EC2 #3 → Unhealthy

The ALB can stop sending new requests to the unhealthy instance.

This is much more reliable than depending on a single server.

6. EC2 + Auto Scaling — Running the Backend

Our Node.js and Express backend can run on Amazon EC2.

For example:

React
  │
  ▼
API Request
  │
  ▼
ALB
  │
  ├── EC2 #1
  ├── EC2 #2
  └── EC2 #3

But what happens when traffic suddenly increases?

This is where EC2 Auto Scaling becomes important.

During normal traffic:

2 EC2 instances

During heavy traffic:

4 EC2 instances

During extremely high traffic:

6 EC2 instances

The exact numbers depend on how we configure the scaling policy.

The important idea is that Auto Scaling adjusts computing capacity based on demand, while the ALB distributes traffic between the available instances.

These two services solve different problems:

Service	Main Responsibility
ALB	Distributes requests
Auto Scaling	Adjusts number of EC2 instances
EC2	Runs our application

7. RDS — Where Our Application Data Lives

Our backend needs somewhere to store permanent data.

For example:

Users
Orders
Products
Payments
Messages
Profiles

This is where Amazon RDS can be used.

Instead of installing and maintaining a database manually on an EC2 server, RDS provides a managed relational database environment.

The backend might perform:

React
  ↓
ALB
  ↓
EC2
  ↓
RDS

For example, when a user creates an account:

User submits signup form
        ↓
React sends API request
        ↓
Backend receives request
        ↓
Backend validates data
        ↓
Backend writes user to RDS

The database should normally be placed in private subnets, rather than being directly exposed to the public internet.

8. S3 — Not Just for the Frontend

One thing that can easily confuse beginners is S3.

S3 is object storage.

It can store much more than React files.

Our application might allow users to upload:

Profile pictures
PDF files
Documents
Product images
Videos
Reports

Instead of putting these large files inside the database, we can store them in S3.

The database can then store information such as:

User ID
File name
S3 object key
Upload date

So we can think of it like:

RDS → Application data
S3  → Files / Objects

This separation makes the architecture easier to scale.

9. SQS — When Everything Doesn't Need to Happen Immediately

Some operations don't need to happen while the user is waiting.

Imagine a user uploads a video.

Our application might need to:

Save the file
Generate thumbnails
Process the video
Send an email
Update another system

Doing everything inside the original request could make the application slow.

Instead, we can use Amazon SQS.

User
 │
 ▼
Backend
 │
 ▼
SQS Queue
 │
 ▼
Worker
 │
 ├── Process file
 ├── Send email
 └── Perform background task

The backend can put a message into the queue and respond to the user.

A worker can process the job separately.

This creates decoupling between different parts of the system.

10. ElastiCache — Making Frequent Requests Faster

Not every piece of data needs to be fetched from the database every time.

Imagine thousands of users are requesting the same popular information.

Instead of repeatedly doing:

EC2 → RDS → Return data

we can cache frequently accessed data.

EC2
 │
 ▼
ElastiCache
 │
 ├── Data found → Return quickly
 │
 └── Data missing → Query RDS

ElastiCache can provide an in-memory cache using technologies such as Redis/Valkey or Memcached.

The important idea is:

Database = persistent source of data

Cache = fast temporary copy of frequently used data

Caching is an optimization, so it isn't required for every application.

11. Security — Protecting the Architecture

Now we have a working architecture, but putting everything on the internet would be a terrible idea.

Security needs to exist at multiple layers.

IAM

AWS IAM controls who or what can access AWS resources.

For example, instead of putting AWS access keys directly inside our Node.js code, we can give an EC2 instance an appropriate IAM role.

EC2
 │
 ▼
IAM Role
 │
 ▼
Allowed AWS Services

The application gets only the permissions it actually needs.

Security Groups

Security Groups act like virtual firewalls around AWS resources.

For example:

Internet
   │
   ▼
  ALB
   │
   ▼
  EC2
   │
   ▼
  RDS

We can configure rules such as:

Internet → ALB       ✅
ALB → EC2             ✅
EC2 → RDS             ✅
Internet → RDS        ❌

This is much safer than exposing the database directly to the internet.

AWS WAF

AWS WAF protects web applications by inspecting incoming HTTP/HTTPS requests.

It can help block things such as malicious or unwanted web requests based on configured rules.

A simplified security flow becomes:

Internet
   │
   ▼
  WAF
   │
   ▼
CloudFront
   │
   ▼
  ALB
   │
   ▼
  EC2

Secrets Manager

Our application also needs sensitive information such as:

Database password
API keys
Application secrets

These shouldn't simply be written directly inside source code.

AWS Secrets Manager provides a safer place to store and retrieve secrets.

12. VPC — The Network Behind Everything

All of these services don't just randomly exist on the internet.

Our backend architecture can be placed inside an Amazon VPC (Virtual Private Cloud).

A simple design could look like:

                    VPC
                     │
          ┌──────────┴──────────┐
          │                     │
    Public Subnets        Private Subnets
          │                     │
         ALB              EC2 / RDS / Cache

The public and private subnet design helps control which resources can be reached from the internet.

For a production architecture, we would normally distribute resources across multiple Availability Zones to improve availability.

13. Monitoring — Knowing What Is Happening

Building the infrastructure isn't enough.

We also need to know:

Is the application healthy?

Are servers running out of resources?

Are requests failing?

Is traffic increasing?

This is where Amazon CloudWatch becomes useful.

CloudWatch can provide metrics, logs, dashboards, and alarms.

For example:

CPU usage > threshold
        ↓
CloudWatch Alarm
        ↓
Auto Scaling / Notification

For auditing AWS API activity, CloudTrail records account and API activity.

So:

CloudWatch → Monitoring & Logs
CloudTrail  → AWS Activity & Auditing

14. HTTPS — Encrypting Communication

When users access:

https://www.mywebsite.com

their communication should be encrypted.

AWS Certificate Manager (ACM) can provide and manage TLS certificates for supported AWS services.

So instead of sending sensitive information over plain HTTP, we use HTTPS.

This is especially important for:

Login credentials
Personal information
Payments
API requests
Session information

15. Putting Everything Together

Now the individual services should make much more sense.

The complete request path can be summarized like this:

                         USER
                           │
                           ▼
                       Route 53
                           │
                           ▼
                      CloudFront
                       /       \
                      /         \
                     ▼           ▼
                  S3             ALB
             React Frontend      │
                                 ▼
                           Auto Scaling
                                 │
                    ┌────────────┼────────────┐
                    ▼            ▼            ▼
                   EC2          EC2          EC2
                    │            │            │
                    └────────────┼────────────┘
                                 │
                    ┌────────────┼────────────┐
                    ▼            ▼            ▼
                   RDS       ElastiCache      SQS
                Database        Cache        Queue
                    │                         │
                    │                         ▼
                    │                       Worker
                    │
                    ▼
                   S3
              File Storage

And around this architecture we have:

IAM
Security Groups
WAF
Secrets Manager
ACM
CloudWatch
CloudTrail
VPC

These aren't just extra AWS services added to make the diagram look impressive.

Each one solves a different operational problem.

The Most Important Thing I Learned

When I first started learning AWS, I looked at services individually.

EC2 = Server
S3 = Storage
RDS = Database
Route 53 = DNS
CloudFront = CDN

That is useful for getting started.

But real cloud architecture is about understanding how those services work together.

A real application isn't just:

User → EC2

It is closer to:

User
 ↓
DNS
 ↓
CDN
 ↓
Load Balancer
 ↓
Application Servers
 ↓
Database / Cache / Queue / Storage

And surrounding all of that are networking, security, monitoring, and scaling mechanisms.

That's when AWS started making much more sense to me.

Quick Cheat Sheet

Requirement	AWS Service
Domain / DNS	Route 53
CDN	CloudFront
React static files	S3
Backend	EC2
Load balancing	ALB
Automatic scaling	Auto Scaling
Relational database	RDS
File storage	S3
Background jobs	SQS
Caching	ElastiCache
Permissions	IAM
Network isolation	VPC
Firewall rules	Security Groups
Web protection	WAF
Secrets	Secrets Manager
HTTPS certificates	ACM
Monitoring	CloudWatch
AWS activity auditing	CloudTrail

Final Thoughts

The biggest shift in my AWS learning wasn't memorizing more services.

It was learning to ask:

“What problem is this service solving, and where does it fit in the architecture?”

Once I started thinking that way, AWS stopped feeling like a huge list of unrelated services.

Instead, it started looking like a set of building blocks.

And when those building blocks are combined properly, we can build applications that are scalable, secure, highly available, and easier to operate.

That, for me, is where learning AWS started becoming much more interesting.

Where Does My Website Actually Store Its Data? Understanding Amazon RDS

Anupa Supul — Mon, 17 Aug 2026 04:56:29 +0000

After learning how to run applications on EC2, I started thinking about something I had never really questioned before.

Where does all the data actually go?

When a user creates an account, where is that account stored?

When someone places an order, where does the order information live?

When I update my profile, how does the application remember that change the next time I log in?

As a MERN developer, I already understood the idea of a database.

I had worked with MongoDB and Mongoose.

But AWS made me think about the bigger question:

How do you run and manage a database reliably in the cloud?

That's when I started learning about Amazon RDS.

The Problem I Started With

Imagine I deploy my backend on an EC2 instance.

My application is running perfectly.

A user signs up.

Their information needs to be saved somewhere.

So my application needs a database.

The basic idea looks like this:

User
  │
  ▼
Backend
  │
  ▼
Database

At first, this seems simple.

But then I started thinking about everything that comes with managing a real database.

Someone has to:

Install the database
Configure it
Manage updates
Take backups
Monitor it
Handle failures
Protect it
Recover it when something goes wrong

That's a lot of responsibility.

And this is where managed database services become useful.

So What Is Amazon RDS?

The simplest way I understand Amazon RDS is:

Amazon RDS is a managed relational database service from AWS.

Instead of manually setting up and maintaining a relational database server yourself, AWS manages much of the underlying database infrastructure for you.

RDS supports database engines such as:

MySQL
PostgreSQL
MariaDB
Oracle
SQL Server
Amazon Aurora

So instead of spending most of my time managing the database server itself, I can focus more on my application and the data.

Running a Database Yourself vs RDS

This comparison helped me understand the difference.

If I Manage Everything Myself

Imagine I launch an EC2 instance and install MySQL manually.

EC2
 │
 ├── Operating System
 ├── MySQL
 ├── Configuration
 ├── Updates
 ├── Backups
 └── Monitoring

Now I'm responsible for all of it.

With Amazon RDS

With RDS, the architecture becomes much simpler:

Application
     │
     ▼
Amazon RDS
     │
     ▼
Relational Database

AWS handles much of the underlying infrastructure.

That doesn't mean I have no responsibility.

I still need to think about:

Database design
Queries
Users and permissions
Security
Application configuration
Cost

But AWS takes care of many infrastructure-level tasks.

A Real-World Example

Imagine I build an online store.

A customer creates an account.

Then they add a product to their cart.

Finally, they place an order.

The application needs to store information such as:

Customer
    │
    ├── Account
    │
    ├── Orders
    │
    ├── Products
    │
    └── Inventory

A relational database is useful for this kind of structured data.

The application architecture could look like:

User
  │
  ▼
Frontend
  │
  ▼
Backend API
  │
  ▼
Amazon RDS

The frontend doesn't directly communicate with the database.

The backend handles that communication.

Why Shouldn't Users Connect Directly to the Database?

This was another thing that became clear to me while learning AWS.

Imagine this:

Internet Users
      │
      ▼
   Database

That would be a terrible architecture.

You don't want random internet users connecting directly to your database.

Instead, we want:

Internet Users
      │
      ▼
   Frontend
      │
      ▼
  Backend API
      │
      ▼
  Amazon RDS

The backend controls how the application communicates with the database.

Security Groups can also be configured so the database only accepts connections from the backend resources that actually need access.

That creates a much safer architecture.

What Happens If the Database Fails?

This is where managing databases gets interesting.

Imagine my application has thousands of users.

Everything is working.

Then suddenly, the database server experiences a failure.

What happens to the application?

If I manually manage the database, I have to think about how to recover it.

RDS provides features designed to improve database availability and recovery.

One important feature is Multi-AZ deployment.

Multi-AZ

With Multi-AZ, AWS can maintain a standby database instance in another Availability Zone.

A simplified example:

                 AWS Region
                     │
          ┌──────────┴──────────┐
          │                     │
          ▼                     ▼
 Availability Zone A      Availability Zone B
          │                     │
          ▼                     ▼
     Primary RDS            Standby RDS

The application normally communicates with the primary database.

If a failure occurs, AWS can fail over to the standby.

What I found interesting here is that high availability isn't simply about having a backup file.

It's about designing the system so it can continue operating when something goes wrong.

What About Backups?

Another thing I started appreciating about managed databases is backups.

Imagine I accidentally delete important data.

Without a backup strategy, recovering it could become a nightmare.

RDS provides automated backup capabilities and supports database snapshots.

So instead of thinking:

"I'll configure backups later."

I can make backups part of the infrastructure from the beginning.

That was another important mindset shift for me.

What If My Application Gets Bigger?

Let's say my application starts with:

100 users

Then it grows to:

10,000 users

And eventually:

1,000,000 users

The database now has much more work to handle.

Eventually, database scaling becomes its own engineering problem.

Depending on the workload, there are different ways to improve database performance and capacity.

For example:

Increasing the database instance size
Using read replicas
Adding caching
Optimizing queries
Choosing the right database architecture

So scaling isn't only about EC2.

The database needs to scale too.

RDS With Everything I've Learned So Far

This was probably the most useful part for me.

I've already learned about:

Route 53
CloudFront
Elastic Load Balancer
EC2
Auto Scaling
Security Groups

Now I can finally see where the database fits into the bigger picture.

A simplified AWS architecture could look like this:

Users
  │
  ▼
Route 53
  │
  ▼
CloudFront
  │
  ▼
Load Balancer
  │
  ▼
Auto Scaling Group
  │
  ▼
EC2 Instances
  │
  ▼
Amazon RDS

Each service has a different responsibility.

Route 53 helps users find the application.

CloudFront helps deliver content efficiently.

Load Balancer distributes incoming traffic.

Auto Scaling adjusts the number of EC2 instances.

EC2 runs the application.

RDS stores the relational data.

Seeing all these pieces together made AWS feel much less like a list of services and more like an actual system.

One Thing I Finally Understood

Before learning AWS, I mostly thought about databases from the application developer's perspective.

I would:

Create Database
      ↓
Connect Backend
      ↓
Write Queries
      ↓
Done

But production systems are different.

Now I have to ask:

What happens if it fails?
        ↓
How are backups handled?
        ↓
Who can access it?
        ↓
Can it scale?
        ↓
How do I keep it available?

Those questions become increasingly important as an application grows.

The Bigger Picture

After learning EC2, ELB, Auto Scaling, and RDS, I started seeing cloud architecture differently.

It's not about memorizing:

"EC2 does this."

"RDS does that."

It's about understanding how the pieces work together.

For example:

                    Users
                      │
                      ▼
                  Route 53
                      │
                      ▼
                 CloudFront
                      │
                      ▼
              Load Balancer
                      │
                      ▼
              Auto Scaling
                 │      │
                 ▼      ▼
               EC2     EC2
                 │      │
                 └──┬───┘
                    │
                    ▼
                Amazon RDS

Now the architecture actually tells a story.

Users arrive.

Traffic is routed.

Content is delivered.

Requests are distributed.

Servers scale.

The application runs.

Data is stored.

That's when AWS started making much more sense to me.

What Changed My Perspective

Before learning AWS, I thought about databases mainly as something my application connects to.

Now I understand that the database itself is an important part of the architecture.

Questions like:

What happens if the database fails?

How do I protect it?

How do I back it up?

How will it handle more users?

How do I make it highly available?

These aren't just database questions.

They're system design questions.

And that's probably the biggest thing RDS taught me.

Final Thoughts

When I first started learning AWS, EC2 was exciting because I could actually see my application running on a cloud server.

But RDS taught me something different.

Running an application is only one part of the system.

The application also needs somewhere reliable to store the information that makes it useful.

And managing that database involves much more than simply creating a database and connecting to it.

That's where managed services like Amazon RDS become valuable.

For me, the biggest lesson was simple:

A database isn't just where data lives. It's part of the architecture that needs to be designed for security, availability, backups, and growth.

The more I learn AWS, the more I realize that building software isn't only about making the application work.

It's about making sure the entire system can keep working when real users start depending on it.

🚀 What I'm Learning Next

The more AWS services I learn, the more I realize that each one solves a different piece of the same puzzle.

EC2 taught me where applications can run.

ELB taught me how traffic can be distributed.

Auto Scaling taught me how applications can adapt to demand.

And RDS taught me where the application's important data can live reliably.

The next step is understanding how these pieces work together in a real application.

I'm continuing to document my AWS learning journey by taking the concepts I learn and explaining them in the simplest way I can.

If you're learning AWS too, I hope this made Amazon RDS a little easier to understand.

Happy learning! 🚀

Why Doesn't My Website Crash When Thousands of People Visit? Understanding Amazon EC2

Anupa Supul — Fri, 31 Jul 2026 10:16:11 +0000

When I first started learning AWS, I assumed that if a website became popular, the solution was simple.

Buy a bigger server.

More CPU.

More RAM.

More storage.

Problem solved.

But then I started thinking about websites like Netflix, Amazon, and Instagram.

Millions of people visit them every day.

Sometimes millions of users are online at exactly the same time.

Surely they don't keep buying one giant server every time traffic increases.

So how do they handle sudden spikes in visitors without crashing?

That question introduced me to Amazon EC2 Auto Scaling.

The Problem

Imagine I build my own portfolio website.

Normally, around 100 people visit it each day.

Everything works perfectly.

Then one day, someone with a huge social media following shares my website.

Within minutes...

10,000 people try to open it.

What happens?

If I only have one EC2 instance, that single server now has to handle every request.

Soon it becomes overloaded.

Pages load slowly.

Some users get errors.

Eventually...

The website crashes.

flowchart LR

Users --> EC2

style EC2 fill:#ff9999

Clearly, one server isn't enough.

My First Thought

My first solution was:

Just buy a bigger server.

For example:

Upgrade from

2 GB RAM

16 GB RAM

This is called Vertical Scaling.

flowchart LR

SmallEC2["Small EC2"]

-->LargeEC2["Bigger EC2"]

This works...

But only up to a point.

Eventually, even the biggest server has limits.

AWS Had a Better Idea

Instead of buying one huge server...

Why not create more servers whenever they're needed?

That's exactly what Auto Scaling does.

Instead of this:

flowchart LR

Users --> OneServer

AWS can do this:

flowchart LR

Users

-->ELB

ELB-->EC21

ELB-->EC22

ELB-->EC23

ELB-->EC24

Now the workload is shared across multiple EC2 instances.

Each server handles only part of the traffic.

What Is Amazon EC2 Auto Scaling?

Amazon EC2 Auto Scaling automatically launches or removes EC2 instances depending on your application's traffic.

Simply put:

More visitors ➜ More servers
Fewer visitors ➜ Fewer servers

No manual work required.

Scale Out

Suppose your website suddenly becomes popular.

Traffic increases.

Auto Scaling detects the increase.

It automatically launches additional EC2 instances.

flowchart LR

Traffic

-->AutoScaling

AutoScaling

-->EC21

AutoScaling

-->EC22

AutoScaling

-->EC23

AutoScaling

-->EC24

This process is called Scale Out.

Scale In

Now imagine it's midnight.

Most users leave.

You no longer need four servers.

Keeping them running would waste money.

Auto Scaling automatically terminates the extra EC2 instances.

flowchart LR

TrafficDrops

-->AutoScaling

AutoScaling

-->EC21

This is called Scale In.

You only pay for what you actually use.

Auto Scaling Group

Auto Scaling doesn't create random servers.

Instead, it manages them inside something called an Auto Scaling Group (ASG).

An ASG follows rules that you define.

For example:

Minimum Instances → 2
Desired Instances → 3
Maximum Instances → 10

If traffic increases, AWS can grow from:

3 → 4 → 5 → 6 → ... → 10

If traffic decreases, it can shrink back down.

Everything happens automatically.

Auto Scaling + Elastic Load Balancer

One thing finally clicked for me.

Auto Scaling creates servers.

But...

How do users know which server to use?

That's where the Elastic Load Balancer (ELB) comes in.

flowchart LR

Users

-->ELB

ELB-->EC21

ELB-->EC22

ELB-->EC23

ELB-->EC24

The Load Balancer distributes traffic.

Auto Scaling creates or removes servers.

Together, they make applications highly available.

Real-World Examples

Auto Scaling is useful almost everywhere.

Imagine:

Black Friday shopping
Concert ticket sales
Online gaming events
Netflix releasing a new season
Viral social media applications

Traffic suddenly increases.

Instead of crashing...

AWS automatically adds more servers.

Once traffic drops, it removes the extra servers.

Why This Saves Money

Without Auto Scaling, you might keep ten servers running all day.

Even when only one server is needed.

That's expensive.

With Auto Scaling:

Busy hours:

8 Servers

Quiet hours:

2 Servers

AWS adjusts automatically.

You only pay for the servers you actually need.

Benefits of Auto Scaling

Handles sudden traffic spikes

Improves application availability

Prevents server overload

Automatically adds or removes EC2 instances

Reduces infrastructure costs

Works seamlessly with Elastic Load Balancers

What Changed My Perspective

Before learning Auto Scaling, I thought handling more users meant buying a more powerful server.

Now I understand that's only one approach.

Sometimes the better solution isn't:

A bigger server.

It's:

More servers working together.

That small mindset shift completely changed how I think about scalable applications.

Final Thoughts

When I first heard the term Amazon EC2 Auto Scaling, I imagined it was just another AWS feature.

Now I realize it's one of the key reasons modern cloud applications stay online during sudden traffic spikes.

Instead of guessing how many servers you'll need...

AWS watches your traffic for you.

When demand increases, it automatically adds more servers.

When demand decreases, it removes the extras.

That's one of the biggest advantages of cloud computing.

You don't have to build for your busiest day.

AWS scales with you.

Key Takeaways

Auto Scaling automatically launches and removes EC2 instances.
Scale Out adds more servers during high traffic.
Scale In removes servers when traffic decreases.
Auto Scaling Groups manage EC2 instances using rules.
Elastic Load Balancer distributes traffic across all instances.
Together, ELB and Auto Scaling build highly available applications.

*Thanks for reading! *

How Does AWS Know Which Website I'm Trying to Visit? Understanding Amazon Route 53

Anupa Supul — Fri, 31 Jul 2026 09:57:57 +0000

When I first started learning web development, opening a website felt almost magical.

I would type something like:

www.amazon.com

Press Enter.

A few seconds later, the website appeared.

I never stopped to think about what actually happened.

How did my computer know where Amazon's servers were?

How did it find the correct server among millions connected to the internet?

And why don't we type an IP address like:

54.239.28.85

instead of simply typing:

amazon.com

Those questions eventually led me to one of the first networking services in AWS:

Amazon Route 53

The Question That Made Me Curious

Imagine I build my own MERN application.

I deploy it on an EC2 instance.

AWS gives me a public IP address like:

18.142.120.54

Technically, people can visit my application using that IP.

But let's be honest...

Nobody wants to remember numbers like that.

It's much easier to visit:

myportfolio.com

myapp.com

That made me wonder:

How does my computer know that "myapp.com" actually means "18.142.120.54"?

The Phone Contact Analogy

This example made everything click for me.

Imagine your best friend has this phone number.

+94 71 123 4567

Do you memorize the number?

Probably not.

Instead, you save it as:

John

Now whenever you want to call him, you simply tap:

John

Your phone automatically finds the correct phone number.

You don't need to remember it.

The internet works almost the same way.

Instead of remembering:

18.142.120.54

we simply remember:

mywebsite.com

Something translates that easy-to-read name into the server's IP address.

That "something" is called DNS (Domain Name System).

What Is Amazon Route 53?

Amazon Route 53 is AWS's managed DNS service.

Its job is simple:

Convert human-friendly domain names into IP addresses that computers understand.

Instead of this:

18.142.120.54

We simply type:

mywebsite.com

Route 53 performs the translation and sends us to the correct server.

What Actually Happens?

Let's say I open:

www.myportfolio.com

Here's what happens behind the scenes.

flowchart LR

A[User Types myportfolio.com]

-->B[Browser]

-->C[Amazon Route53 DNS Lookup]

-->D[Returns IP Address]

-->E[EC2 Server]

-->F[Website Loads]

The entire process usually takes only milliseconds.

A Real AWS Example

Suppose I host my MERN application on an EC2 instance in Singapore.

Its public IP address is:

18.xxx.xxx.xxx

Without Route 53, users would have to visit:

18.xxx.xxx.xxx

With Route 53:

flowchart LR

A[myportfolio.com]

-->B[Amazon Route53]

-->C[18.xxx.xxx.xxx]

-->D[EC2 Instance]

Much easier.

Why Is It Called Route 53?

One thing that caught my attention was the number 53.

It's actually named after Port 53, the standard network port used for DNS communication.

A small detail, but one I found interesting.

Route 53 Does More Than DNS

Initially, I thought Route 53 only converted domain names into IP addresses.

But it can also:

Route traffic globally
Perform Health Checks
Fail over to healthy resources
Route traffic intelligently
Improve application availability

Where Route 53 Fits in AWS

As I learned more AWS services, I realized Route 53 is usually the very first AWS service involved when someone opens a website.

flowchart LR

User

-->Route53

-->CloudFront

-->LoadBalancer

-->EC2

-->Database

Every service has its own responsibility.

Route 53 simply helps users find the right destination.

Why Route 53 Matters

Without DNS:

Users must remember IP addresses.

With Route 53:

Easy-to-remember domain names

Highly available DNS

Faster routing

Health monitoring

Traffic routing policies

Works with almost every AWS service

What Changed My Perspective

Before learning AWS, I thought typing a website name directly connected me to the server.

Now I understand there's an important translation happening first.

Someone has to convert:

mywebsite.com

into

18.xxx.xxx.xxx

That's exactly what DNS does.

And in AWS, that job belongs to Amazon Route 53.

Final Thoughts

When I first heard the name Route 53, I had no idea what it actually did.

Now I think of it much more simply.

It's like the contact list on my phone.

I don't remember phone numbers.

I remember names.

Similarly, my browser doesn't magically know where a website lives.

Route 53 translates a human-friendly domain name into the server's actual address.

Sometimes the most important AWS services are the ones quietly working behind the scenes.

And Route 53 is definitely one of them.

Key Takeaways

Route 53 is AWS's managed DNS service.
It converts domain names into IP addresses.
It makes websites easier to access.
It supports health checks and intelligent routing.
It's usually the first AWS service involved when visiting a website.

*Thanks for reading! *

Why Is Netflix So Fast Everywhere? Understanding Amazon CloudFront

Anupa Supul — Wed, 22 Jul 2026 10:27:13 +0000

When I first started learning AWS, I thought website speed mostly depended on one thing:

A powerful server.

More CPU.

More memory.

Better hardware.

Problem solved.

Or at least that's what I thought.

Then a question hit me.

Netflix loads quickly.

YouTube loads quickly.

Amazon loads quickly.

But their servers aren't sitting in my room.

In fact, many of them are located thousands of kilometers away.

So why doesn't every click feel slow?

That question eventually led me to Amazon CloudFront.

The Thought That Started It

Imagine I host a website on AWS in Singapore.

Now imagine someone opens that website from Sri Lanka.

Every image.

Every CSS file.

Every JavaScript file.

Every request.

Has to travel all the way to Singapore and back.

flowchart LR
    USER[User in Sri Lanka]
    SERVER[AWS Server in Singapore]

    USER --> SERVER

That doesn't sound very efficient.

And it gets even worse when thousands or millions of users are involved.

So I started wondering:

How do global companies deliver content so quickly to people all around the world?

The Concept That Finally Clicked

The simplest way I understand CloudFront is this:

Imagine there is only one library in an entire country.

Every student has to travel there to borrow a book.

It works.

But it's slow.

Now imagine small branches of that library are opened across different cities.

Students can get popular books from a nearby branch instead of traveling to the main library.

That's basically what CloudFront does.

Instead of forcing every user to communicate with the original server, AWS places copies of content closer to users.

What Is Amazon CloudFront?

Amazon CloudFront is AWS's Content Delivery Network (CDN).

Its job is simple:

Deliver content from the location closest to the user whenever possible.

Rather than always requesting files from the original server, CloudFront stores frequently requested content in locations around the world called Edge Locations.

When a user requests that content again, it can be delivered from a nearby Edge Location instead of the original AWS Region.

A Simple Example

Suppose I host a portfolio website.

The website contains:

Images
CSS files
JavaScript files

Without CloudFront:

flowchart LR
    USER[User]
    SERVER[Singapore Server]

    USER --> SERVER

Every request makes the full journey.

With CloudFront:

flowchart LR
    USER[User]

    EDGE[CloudFront Edge Location]

    SERVER[Singapore Server]

    USER --> EDGE
    EDGE --> SERVER

If the content is already cached, CloudFront delivers it immediately.

Fewer long-distance trips.

Faster loading times.

Better experience.

What Are Edge Locations?

While learning AWS Global Infrastructure, I came across three important concepts:

Regions
Availability Zones
Edge Locations

CloudFront uses Edge Locations to serve content closer to users.

Think of them as small delivery centers spread around the world.

flowchart TD
    USER[User]

    EDGE[Nearest Edge Location]

    ORIGIN[Origin Server]

    USER --> EDGE
    EDGE --> ORIGIN

When content is already available at the Edge Location, users get it much faster.

Why This Matters

Before learning CloudFront, I thought performance was mainly about the server.

Now I realize distance is part of the equation too.

Even a powerful server can't overcome physics.

Data still needs to travel.

CloudFront helps reduce that distance from the user's perspective.

And when you're serving users across different countries, that makes a huge difference.

CloudFront + S3

One of the most common AWS architectures combines CloudFront with Amazon S3.

flowchart LR
    USER[Users]

    CF[CloudFront]

    S3[Amazon S3 Bucket]

    USER --> CF
    CF --> S3

This setup is commonly used for:

Static websites
Images
Downloads
Documentation
Portfolio sites

CloudFront + Modern Web Applications

CloudFront can also work with dynamic applications.

flowchart LR
    USER[Users]

    CF[CloudFront]

    ELB[Load Balancer]

    EC2[EC2 Instances]

    USER --> CF
    CF --> ELB
    ELB --> EC2

Seeing architectures like this helped me realize that AWS services aren't isolated tools.

They work together to solve larger problems.

Why Is Netflix So Fast?

This was the original question that made me curious.

Streaming platforms serve enormous amounts of content every second.

Imagine if every user had to fetch videos directly from a single server.

The experience would be terrible.

Instead, content is distributed closer to users.

Popular files are cached.

Travel distance is reduced.

Latency drops.

The result is a much smoother experience.

While Netflix has a far more complex infrastructure than this simplified example, the underlying idea is very similar:

Bring content closer to users.

Benefits of Amazon CloudFront

After learning about CloudFront, these are the biggest benefits that stood out to me:

✅ Faster content delivery

✅ Lower latency

✅ Better user experience

✅ Reduced load on origin servers

✅ Global content distribution

✅ Improved scalability

What Changed My Perspective

Before learning AWS, I thought website performance was mostly about powerful servers.

Then I learned about:

Load Balancers
Auto Scaling
CloudFront

And I realized something.

Sometimes the fastest solution isn't a bigger server.

It's bringing content closer to the user.

That simple idea is what made CloudFront click for me.

Final Thoughts

When I first heard about Amazon CloudFront, it sounded like just another AWS service.

Now I see it differently.

It's AWS's answer to a simple problem:

How do you make a website feel fast for users all around the world?

Not by making people travel farther.

By bringing content closer to them.

And honestly, that's not something I ever thought about when I first started learning web development.

ops

How AWS Security Groups Finally Made Sense to Me

Anupa Supul — Wed, 22 Jul 2026 10:11:43 +0000

🔐 How AWS Security Groups Finally Made Sense to Me

The AWS feature that taught me security isn't just about passwords and encryption.

When I launched my first EC2 instance, AWS asked me to configure something called a Security Group.

At the time, I didn't think much about it.

I simply opened the ports AWS recommended, launched the server, and moved on.

The application worked.

The website loaded.

Everything seemed fine.

But as I continued learning AWS, I kept seeing Security Groups appear everywhere.

EC2 instances had them.

Databases had them.

Load Balancers used them.

That's when I started wondering:

What exactly are Security Groups, and why are they so important?

The Mistake Most Beginners Make

When I first heard the word security, I immediately thought about:

Passwords
Encryption
Authentication
Antivirus software

Security Groups are different.

They answer a much simpler question:

Who is allowed to communicate with my AWS resources?

And surprisingly, that single question is incredibly important.

A Real Scenario: Hosting a Website

Imagine I deploy a MERN application on an EC2 instance.

Users need to access the website through their browsers.

For that to happen, the server must allow:

HTTP (Port 80)
HTTPS (Port 443)

flowchart TD
    U[User]
    SG[Security Group]
    EC2[EC2 Instance]

    U --> SG
    SG --> EC2

Without those ports being open, nobody can access the application.

Even if the server is running perfectly, it remains unreachable.

This was my first realization:

Security Groups don't make resources public.

They decide whether traffic is allowed to reach them.

So What Is a Security Group?

The simplest explanation I found is:

A Security Group acts as a virtual firewall for AWS resources.

It controls two types of traffic:

Inbound Rules

Traffic coming into a resource.

Examples:

Users visiting a website
Developers connecting through SSH
Applications calling APIs

Outbound Rules

Traffic leaving a resource.

Examples:

Calling external APIs
Downloading updates
Connecting to databases

Every request is checked against these rules.

If a rule allows it, traffic passes.

If not, AWS blocks it automatically.

The SSH Example

This is where Security Groups became very practical for me.

Suppose I want to connect to my EC2 instance using SSH.

SSH uses:

Port 22

flowchart TD
    DEV[Developer]
    SSH[SSH Port 22]
    SG[Security Group]
    EC2[EC2 Instance]

    DEV --> SSH
    SSH --> SG
    SG --> EC2

A common beginner mistake is allowing:

0.0.0.0/0

Which means:

Anyone on the internet can attempt to connect.

It works.

But it's not ideal.

A safer approach is allowing only your own public IP address.

That way, only you can access the server.

The Database Scenario That Made Everything Click

This was the example that finally made Security Groups click for me.

Imagine a simple application architecture:

flowchart TD
    USER[Users]

    FE[Frontend]

    API[Backend API]

    DB[(Database)]

    USER --> FE
    FE --> API
    API --> DB

Now ask yourself:

Should internet users connect directly to the database?

Definitely not.

The database should only accept requests from the backend server.

Instead of this:

flowchart TD
    INTERNET[Internet Users]
    DB[(Database)]

    INTERNET -. Blocked ❌ .-> DB

We want this:

flowchart TD
    FE[Frontend]
    API[Backend API]
    DB[(Database)]

    FE --> API
    API --> DB

Security Groups make this possible by controlling exactly who can communicate with what.

Why Security Groups Are Stateful

One AWS concept that surprised me was that Security Groups are stateful.

Imagine a user sends a request to an EC2 instance.

flowchart LR
    USER[User]

    SG[Security Group]

    EC2[EC2 Instance]

    USER --> SG
    SG --> EC2

    EC2 --> USER

If the inbound request is allowed, the response is automatically allowed back.

You don't need to create a separate return rule.

AWS handles that automatically.

This makes Security Groups much easier to manage.

A Real-World AWS Architecture

A simple production setup might look like this:

flowchart TD
    USER[Users]

    SG1[Web Security Group]

    EC2[Backend Server]

    SG2[Database Security Group]

    DB[(Database)]

    USER --> SG1
    SG1 --> EC2

    EC2 --> SG2
    SG2 --> DB

In this setup:

Users can access the backend server.
The backend server can access the database.
Users cannot access the database directly.

This is one of the most common security patterns you'll see in AWS architectures.

Security Is Not Just About Applications

Before learning AWS, I thought security mostly happened inside the application.

Things like:

User authentication
Password hashing
JWT tokens
Role-based access control

Those are still important.

But AWS taught me something else:

Security also exists at the infrastructure level.

Even a perfectly written application can become vulnerable if network access isn't configured properly.

Security Groups provide one of the first layers of defense.

What Changed My Perspective

The first time I configured a Security Group, it felt like just another setup step before launching an instance.

Now I see it differently.

Every rule is answering a simple question:

Who should be allowed to communicate with this resource?

The more I learn AWS, the more I realize that cloud security isn't about blocking everything.

It's about allowing only what is necessary.

And Security Groups are one of the simplest and most effective tools AWS provides for doing exactly that.

Final Thoughts

Security Groups looked complicated when I first encountered them.

Today, I think of them much more simply.

They're the gatekeepers of AWS resources.

They decide:

Who can enter
What traffic is allowed
What traffic should be blocked

And understanding that concept helped me appreciate why Security Groups appear in almost every AWS architecture diagram.

Sometimes the most important AWS services aren't the most exciting ones.

They're the ones quietly protecting everything behind the scenes.

Why Doesn't Everyone Connect to the Same Server? Understanding AWS Elastic Load Balancer (ELB)

Anupa Supul — Mon, 20 Jul 2026 05:51:01 +0000

While learning AWS, I noticed something interesting.

In almost every AWS architecture diagram, there was a service sitting in front of multiple servers.

It looked something like this:

Users
   │
   ▼
Load Balancer
   │
 ┌─┴─┐
 ▼   ▼
EC2 EC2

At first, I didn't understand why it was there.

I kept thinking:

"Why can't users just connect directly to the server?"

After all, if the server is running my application, shouldn't users talk to it directly?

The more I learned, the more I realized that this simple service solves a huge problem.

And that's how I discovered AWS Elastic Load Balancer (ELB).

The Question That Made Me Curious

Imagine I build a website.

On a normal day, maybe 100 people visit it.

One server is enough.

Everything works smoothly.

Now imagine my website suddenly goes viral.

Thousands of people start visiting at the same time.

What happens?

Every request goes to the same server.

Eventually, that server becomes overwhelmed.

Pages become slow
Users start waiting
Requests fail
Visitors leave

That's when I realized:

The problem isn't just having servers.

It's deciding how traffic should be shared between them.

The Coffee Shop Analogy

This was the example that made everything click for me.

Imagine a coffee shop with only one cashier.

500 Customers
      │
      ▼
  1 Cashier

The line becomes huge.

Customers wait.

Some leave before getting served.

Now imagine adding three more cashiers.

500 Customers
      │
      ▼
 ┌─────────┐
 │ Manager │
 └─────────┘
  │  │  │  │
  ▼  ▼  ▼  ▼
 C1 C2 C3 C4

The manager directs customers to whichever cashier is available.

Nobody has to choose a cashier manually.

The workload gets distributed automatically.

That manager is basically what a Load Balancer does.

So... What Is an Elastic Load Balancer?

The simplest way I understand it is:

An Elastic Load Balancer automatically distributes incoming traffic across multiple servers.

Instead of sending every user to one server, AWS spreads the requests across several servers.

Benefits include:

Improved performance
Higher availability
Better fault tolerance
Handling traffic spikes

Without a Load Balancer

10,000 Users
      │
      ▼
     EC2

One server has to handle everything.

If it fails, the application becomes unavailable.

With a Load Balancer

10,000 Users
       │
       ▼
 Load Balancer
   │    │    │
   ▼    ▼    ▼
 EC2  EC2  EC2

Traffic gets shared.

Each server handles a smaller amount of work.

The application stays responsive.

What Happens If a Server Fails?

Without a Load Balancer:

Users
  │
  ▼
Failed Server ❌

Website unavailable.

With a Load Balancer:

Users
  │
  ▼
Load Balancer
 │     │     │
 ▼     ▼     ▼
❌    EC2   EC2

Traffic automatically moves to healthy servers.

Most users never notice anything happened.

How ELB Works With Auto Scaling

Internet Users
       │
       ▼
 Load Balancer
       │
 ┌─────┼─────┐
 ▼     ▼     ▼
EC2   EC2   EC2
       │
       ▼
 Auto Scaling

Auto Scaling decides how many servers exist.
ELB decides where traffic goes.

Together, they keep applications scalable and highly available.

Final Thoughts

When I first saw Elastic Load Balancer in AWS diagrams, it looked like just another service with a complicated name.

Now I see it much more simply.

"If thousands of users arrive at the same time, who decides where each request should go?"

That's exactly the problem ELB solves.

And for me, it was another reminder that cloud computing isn't just about servers.

It's about designing systems that continue working even when demand grows.

DEV Community: Anupa Supul

How Do Large Applications Process Work in the Background? Understanding Amazon SQS, SNS & EventBridge

1. Why Do Applications Need Background Processing?

2. Amazon SQS — Let Messages Wait in a Queue

Why is SQS useful?

3. Amazon SNS — Send One Message to Multiple Subscribers

Why is SNS useful?

4. Amazon EventBridge — Route Events Based on Rules

When should you use EventBridge?

5. SQS vs SNS vs EventBridge

6. How They Work Together in a Real Application

What happens when the order is placed?

7. Reliability: Retries, Dead-Letter Queues, and Duplicate Messages

Retries

Dead-letter queues

Idempotency

Monitoring

8. When Should You Use Each Service?

9. My Main Takeaways

Can You Build a Backend Without Managing Servers? Understanding Amazon API Gateway, AWS Lambda & DynamoDB

1. What Does Serverless Actually Mean?

2. Amazon API Gateway — The Entry Point for Your API

3. AWS Lambda — Run Backend Code Without Managing Servers

A simple Node.js Lambda example

What happens when traffic increases?

4. Amazon DynamoDB — Store Application Data

Why choose DynamoDB?

5. Connecting the Architecture

6. A Practical Node.js Example: Saving a Task to DynamoDB

Important: configure IAM permissions

Important: configure the DynamoDB table

7. What About Authentication and Security?

AWS IAM

Amazon Cognito or another authentication provider

AWS Secrets Manager

Amazon CloudWatch

A simple security model

8. What Happens When Traffic Increases?

API Gateway

Lambda

DynamoDB

What about cold starts?

9. Serverless Is Not Always Cheaper

10. Lambda vs EC2 vs ECS/Fargate

11. What I Would Consider Before Deploying to Production

12. When Should You Use This Architecture?

13. My Main Takeaways

What's next?

Stop Hardcoding Secrets: How Production Applications Manage Credentials on AWS

1. Why Hardcoding Secrets Is Dangerous

2. What About the .env File?

3. AWS Secrets Manager — Store Sensitive Information

4. Systems Manager Parameter Store — Manage Configuration

5. Secrets Manager vs Parameter Store

6. IAM — Who Is Allowed to Read the Secrets?

7. How Does This Work with Docker and ECS?

8. Example: A Node.js Backend on AWS

9. What About GitHub Actions and CI/CD?

10. What Should We Choose?

Quick Cheat Sheet

Final Thoughts

What Happens When You Push Code to Production? Understanding AWS CI/CD with GitHub Actions, Docker, ECR & ECS

1. First, What Is CI/CD?

2. The Complete AWS CI/CD Architecture

3. Step One: Push Code to GitHub

4. Step Two: GitHub Actions Runs the Pipeline

5. Step Three: Build the Docker Image

6. Step Four: Push the Image to Amazon ECR

7. Step Five: Deploy the New Image to Amazon ECS

8. Step Six: How Does the New Version Reach Users?

9. What If the New Deployment Fails?

10. CI/CD Is More Than Just Deployment

11. What About Secrets and Permissions?

12. The Complete Deployment Architecture

13. GitHub Actions vs AWS CodePipeline

The Mental Model I Keep Now

Quick Cheat Sheet

Final Thoughts

I Stopped Managing Servers Manually — Understanding Docker, Amazon ECR, ECS & Fargate

I Stopped Managing Servers Manually — Understanding Docker, Amazon ECR, ECS & Fargate

2. What About the `.env` File?