A customer opens an online store, browses a product, and clicks Place Order.
From the customer's perspective, it is a simple interaction. Behind the scenes, however, several components may work together: containers run the application, orchestration platforms manage those containers, databases store information, caches improve response times, queues handle background work, and monitoring tools help engineers understand what is happening in production.
The interesting part is not memorizing the names of these technologies. It is understanding which problem each one solves, how they connect, and what trade-offs they introduce.
Let's follow one order through a typical online application and explore the technologies along the way.
The architecture in this article is an illustrative example, not a required stack. A real application should use only the components that solve problems it actually has.
1. Docker: Package the application consistently
A familiar problem for developers is:
It works on my machine, but it doesn't work in production.
An application may run with one runtime version on a developer's laptop, while the production server has another. A dependency may be missing, or a library version may be different.
Docker helps reduce this environment mismatch by packaging an application and its required runtime dependencies into an image.
A useful way to think about the two main terms:
- Image: the packaged template.
- Container: a running instance created from that image.
The same image can be used in development, testing, staging, and production, helping those environments behave more consistently.
There is an important qualification: a container image does not make every machine identical. CPU architecture and operating-system compatibility still matter. For example, an image built for one processor architecture may not run natively on a machine with another architecture. Multi-platform image builds can address this when needed.
Docker solves the packaging problem. It does not, by itself, manage a fleet of containers.
2. ECS and Kubernetes: Run and manage containers
Once an application is containerized, we need a way to run it, scale it, recover from failures, and manage its deployment.
Two common options are Amazon ECS and Kubernetes.
Amazon ECS
If an organization is already using AWS, Amazon Elastic Container Service (ECS) can be a practical choice. It is an AWS-managed container orchestration service, and AWS handles much of the underlying orchestration infrastructure.
Kubernetes
Kubernetes is an open-source container orchestration platform. It can run across different cloud providers and in on-premises environments, offering flexibility and control.
That flexibility comes with operational responsibility. A managed Kubernetes service such as Amazon EKS or Google Kubernetes Engine manages important parts of the control plane, but teams still need to understand and manage areas such as worker nodes, networking, add-ons, workloads, and cluster configuration.
So which one is better?
Neither is universally better. The right choice depends on the team's needs and operational capacity:
- A smaller team operating primarily on AWS may prefer ECS for its simpler operational model.
- A team that needs Kubernetes portability, its ecosystem, or more control over orchestration may choose Kubernetes.
The key question is not which tool is more popular. It is how much complexity the team is prepared to operate.
3. Terraform: Define infrastructure as code
Containers need infrastructure: compute resources, networks, access rules, databases, and other cloud resources.
One approach is to create these resources manually through a cloud console. The problem appears when the same setup must be repeated for development, staging, and production.
Over time, environments can become different. Someone changes a security rule in staging but forgets to make the corresponding change in production. A bug then appears in production but cannot be reproduced in staging.
This is known as configuration drift.
Terraform addresses this by allowing infrastructure to be described as code.
Instead of relying on a sequence of manual console operations, teams define resources in configuration files and keep those files in version control.
That brings several benefits:
- Infrastructure changes can be reviewed through pull requests.
- Environments can be created more consistently.
- Git history helps explain what changed and when.
- Rebuilding an environment becomes more repeatable.
Infrastructure as code does not eliminate the need for careful review. A reviewed configuration can still contain a mistake. It does, however, make infrastructure changes more visible, repeatable, and manageable.
4. DNS and load balancing: Route incoming requests
Now imagine the application is deployed and a customer opens shop.com.
The browser needs to find the server responsible for that domain. This is where DNS (Domain Name System) comes in: it resolves a domain name to the information needed to reach the service, commonly an IP address.
The request then reaches a load balancer.
A production application may run across multiple backend instances rather than on one server. The load balancer distributes incoming requests among those instances.
It can also perform health checks. If an instance stops responding or fails its health check, the load balancer can stop sending new traffic to it and route requests to healthy instances instead.
This helps improve availability and prevents one server from becoming the only point handling all incoming traffic.
5. PostgreSQL: Store the source-of-truth data
The backend receives the order request, validates it, and saves the order.
A relational database such as PostgreSQL can store the application's core business data:
- Customers
- Products
- Orders
- Payments and order status
- Other related records
For many applications, the relational database is the authoritative source of truth.
But as traffic grows, repeatedly running the same database queries can become expensive. A popular product page, for example, might be requested thousands of times.
That leads to the next component.
6. Redis: Reduce repeated database work with caching
Redis is commonly used as an in-memory data store and cache.
Suppose many customers open the same product page. If every request reads the same product information from PostgreSQL, the database has to repeat work that may not need to be repeated.
A cache can help:
- The application checks Redis for the product data.
- If the data is present, the application returns it from the cache.
- If it is missing, the application reads it from PostgreSQL.
- The application can then place a copy in Redis for later requests.
This is a common cache-aside pattern.
Redis does not automatically replace the main database. PostgreSQL can remain the source of truth while Redis stores temporary or reusable copies.
The challenge: cache invalidation
What happens if a product's price changes in PostgreSQL while Redis still contains the old price?
The application could return stale information.
The cache therefore needs an invalidation or update strategy. Depending on the use case, that might involve deleting the cached entry, updating it after a database change, or allowing it to expire after a defined time.
Caching can improve performance, but it introduces a consistency problem that needs to be designed deliberately.
7. OpenSearch: Handle more advanced search requirements
A customer searches for wireless headphones. They might make a typo, use a slightly different phrase, or expect the most relevant products to appear first.
A relational database can support text search, but more advanced requirements—such as relevance ranking, typo tolerance, and large-scale search—may call for a dedicated search engine.
OpenSearch is designed for search and analytics use cases.
An application can index its product catalog in OpenSearch and use it to handle product-search queries. PostgreSQL can still remain the authoritative source of product data.
This distinction matters:
- PostgreSQL stores the core business records.
- OpenSearch provides search capabilities over indexed data.
The search index is typically a derived representation of the source data, so the application needs a process to keep it reasonably up to date.
A dedicated search engine is useful when the product's search requirements justify the extra infrastructure. It is not something every application needs from day one.
8. Amazon S3: Store files outside the relational database
Product images and profile photos are files, not ordinary relational records.
Amazon S3 is an object storage service designed to store files and other unstructured data.
A typical flow looks like this:
- The application uploads an image to S3.
- S3 stores the object.
- The application stores the object's key or reference in PostgreSQL.
- When the image is needed, the application or browser retrieves it from S3, subject to the configured access controls.
This separates responsibilities:
- PostgreSQL stores business data and file references.
- S3 stores the actual file objects.
The file reference should not be treated as automatically public. Access can be controlled using bucket policies, application authorization, or time-limited pre-signed URLs, depending on the design.
9. Message queues: Move slow work out of the request
Saving an order may trigger several additional tasks:
- Send an order confirmation email.
- Generate an invoice.
- Notify the warehouse.
- Record an analytics event.
Should the customer wait for every task to finish before receiving a response?
Usually, that would make the request slower than necessary.
A message queue allows the backend to place work onto a queue and return a response without waiting for every background task to complete. Background workers consume the messages and perform the work independently.
Common options include Amazon SQS and RabbitMQ.
Acknowledgements and retries
A worker generally acknowledges a message after successfully processing it. If the worker crashes before acknowledging it, the queue may make the message available for processing again.
That means a message can sometimes be delivered more than once.
Workers should therefore be designed to handle duplicate delivery safely. This is called idempotency: processing the same operation more than once should not produce an unintended duplicate effect.
For example, an email worker could record that a confirmation has already been sent for a particular order and avoid sending it again when the same message is retried.
Queues improve responsiveness and decouple background work from the main request, but they introduce concerns such as retries, duplicate messages, failure handling, and monitoring.
10. Kafka: Distribute events to multiple consumers
A queue is useful when an application needs to hand off tasks to background workers. But sometimes an event needs to be made available to several independent systems.
For example, when an order is placed, different consumers may need to react:
- An analytics service records the purchase.
- An inventory service updates stock.
- A notification service sends a message.
- A reporting pipeline processes the event.
Apache Kafka is a distributed event-streaming platform that can support this kind of event-driven architecture.
Rather than thinking of every event as a one-time task handed to a single worker, Kafka lets producers publish events to topics. Multiple consumer applications can read those events independently, according to their own processing needs.
Kafka is especially useful when event volume, independent consumers, replay, or durable event streams are important requirements.
It is not automatically a replacement for every queue. A simpler queue may be easier to operate when the requirement is simply to distribute background tasks. Choose based on the messaging problem, not just the technology's popularity.
11. CI/CD and Amazon ECR: Deliver code changes reliably
The application is running, but developers continue to fix bugs and add features. How does a code change reach production?
With a manual deployment process, someone might connect to each server, pull the latest code, and restart the application. If one server is missed, different instances may run different versions.
A CI/CD pipeline automates much of this delivery process.
A typical container-based pipeline might:
- Start when a developer pushes code.
- Build the application and run automated tests.
- Build a Docker image if the required checks pass.
- Push the image to a container registry.
- Deploy the selected image version to the target environment.
On AWS, Amazon Elastic Container Registry (ECR) can store Docker and other OCI container images.
Think of ECR as a registry for container images: it stores the packaged application versions that deployment systems can pull.
CI/CD makes delivery more repeatable, but the quality of the pipeline still depends on the checks, approvals, and deployment safeguards configured by the team.
12. Helm: Manage Kubernetes application configuration
When deploying to Kubernetes, teams need configuration describing how an application should run.
Development, staging, and production may share most of the same deployment structure while differing in image tags, replica counts, resource limits, or environment-specific settings.
Maintaining separate copies of nearly identical configuration files can lead to drift.
Helm helps manage Kubernetes applications through charts and configurable values.
A chart defines reusable Kubernetes templates. Different values files can provide environment-specific settings.
This reduces duplication and makes it easier to keep deployment configuration consistent across environments.
Helm is a Kubernetes package manager; it does not replace Kubernetes itself. It helps organize and render the configuration that Kubernetes uses.
13. Argo CD: Use GitOps to keep deployments aligned
Argo CD is a declarative GitOps continuous delivery tool for Kubernetes.
The central idea behind GitOps is that the desired state of the application and cluster configuration is stored in Git.
Argo CD watches the configured Git repository and compares the desired state there with the state currently running in the Kubernetes cluster.
If they differ, Argo CD can report the difference and, depending on the configured synchronization policy, apply the desired configuration.
For example, if someone manually changes a deployment in the cluster, Argo CD can detect that the live state no longer matches the configuration in Git. With automated synchronization and self-healing configured, it can bring the cluster back toward the declared state.
This gives teams a traceable workflow:
- Configuration changes are committed to Git.
- Changes can be reviewed.
- The cluster is reconciled against the declared configuration.
- Git history provides a record of changes and supports rollback workflows.
GitOps does not make every deployment safe automatically. If an incorrect configuration is merged, the system may faithfully deploy that incorrect configuration. Reviews, tests, and rollout safeguards remain important.
14. Prometheus and Grafana: Understand application health
The application is live. Then an order endpoint that normally responds in one second starts taking five seconds.
Where should the team look?
Prometheus is a monitoring system that collects and stores metrics. Applications and infrastructure can expose measurements such as:
- Request count
- Request latency
- Error rate
- CPU and memory usage
- Queue depth
Prometheus commonly uses a pull-based model: it periodically scrapes metrics from configured targets.
If a target becomes unavailable, Prometheus can report that it is no longer being scraped successfully. That signal can help teams detect failures.
Grafana is commonly used to visualize metrics through dashboards. Engineers can inspect trends, identify when a latency spike began, and compare measurements across services.
Metrics are useful for showing that something is wrong. But a graph showing that an endpoint is slow may not explain which dependency caused the delay.
That is where tracing becomes useful.
15. OpenTelemetry: Instrument and follow requests across services
In a distributed application, one customer request may travel through several components:
- The order service receives the request.
- It calls the inventory service.
- The inventory service calls another dependency.
- The order service communicates with a payment service or external API.
If the overall request takes five seconds, which step consumed the time?
Distributed tracing follows a request as it moves across services. A trace is made up of spans, which represent timed operations and their relationships. Looking at those spans helps engineers identify where time was spent.
OpenTelemetry provides vendor-neutral APIs, SDKs, and instrumentation for generating and collecting telemetry, including traces, metrics, and logs.
An application needs to be instrumented for useful telemetry to appear. That can involve adding SDK instrumentation to the code or using supported automatic instrumentation.
The OpenTelemetry Collector can receive, process, and export telemetry to compatible observability backends.
One important point: installing OpenTelemetry alone does not magically reveal application behavior. The application and its dependencies must produce telemetry, and the collection and export pipeline must be configured.
Prometheus, Grafana, and OpenTelemetry serve related but different roles:
- Prometheus: collects and stores metrics.
- Grafana: visualizes metrics and other supported telemetry sources.
- OpenTelemetry: provides instrumentation and a standard way to generate, collect, and export telemetry.
Together, these tools help teams move from noticing a performance problem to investigating its cause.
16. Bringing the pieces together
Let's revisit the customer's order and group the technologies by the problems they solve.
| Area | Technologies | Main responsibility |
|---|---|---|
| Application packaging | Docker | Package the application and its dependencies |
| Container orchestration | ECS or Kubernetes | Run and manage containers |
| Infrastructure | Terraform | Define and provision infrastructure as code |
| Request routing | DNS and load balancer | Resolve the service address and distribute traffic |
| Primary data | PostgreSQL | Store core business records |
| Caching | Redis | Reduce repeated reads and improve response times |
| Search | OpenSearch | Support advanced search and relevance |
| File storage | Amazon S3 | Store images and other objects |
| Background processing | SQS or RabbitMQ | Decouple asynchronous tasks from requests |
| Event streaming | Kafka | Distribute event streams to independent consumers |
| Delivery | CI/CD and ECR | Test, package, store, and deliver application versions |
| Kubernetes configuration | Helm | Template and manage Kubernetes application configuration |
| GitOps deployment | Argo CD | Reconcile cluster state with configuration in Git |
| Monitoring and observability | Prometheus, Grafana, OpenTelemetry | Collect, visualize, and investigate telemetry |
A simplified flow looks like this:
DEVELOPER WORKFLOW
|
Code pushed to Git
|
CI/CD
Build + test + image
|
Amazon ECR
|
Helm configuration in Git
|
Argo CD
|
ECS / Kubernetes
|
Docker images
|
CUSTOMER --> DNS --> LOAD BALANCER --> BACKEND APPLICATION
|
+---------------------+----------------------+
| | |
PostgreSQL Redis OpenSearch
source of truth cache product search
|
File references
|
Amazon S3
Order created --> Queue --> Background workers
|
+------> Kafka events --> Independent consumers
Prometheus --> Metrics
Grafana ----> Dashboards
OpenTelemetry --> Traces, metrics, and logs
This is a conceptual view. The exact deployment and telemetry architecture will vary by application and cloud environment.
17. Every technology solves a problem—and adds responsibility
It is tempting to think that adding more tools automatically makes an application more scalable. In reality, every new component introduces operational and design trade-offs.
Consider a few examples:
- Redis can reduce database load, but cached data can become stale.
- Queues keep slow work out of the request path, but messages may be retried or delivered more than once.
- Kafka supports durable event streams and multiple consumers, but it adds operational and architectural complexity.
- Kubernetes provides orchestration flexibility, but teams must manage more of the platform.
- Argo CD can reconcile a cluster to Git, but it can also apply a bad configuration if that configuration is approved and committed.
- OpenSearch improves search capabilities, but the search index must be maintained alongside the source data.
This leads to a more useful way to evaluate technology:
- What concrete problem are we trying to solve?
- Is the problem significant enough to justify another component?
- What new failure modes and operational responsibilities will this introduce?
- Can the team support and monitor it?
- Is there a simpler solution that is sufficient?
Conclusion
A scalable application is not built by collecting as many technologies as possible.
Docker, Kubernetes, Terraform, PostgreSQL, Redis, Kafka, Prometheus, and the other tools discussed here each address different concerns. Their value comes from how well they fit the application's actual requirements and how thoughtfully they are operated together.
Start with the simplest architecture that meets the need. Add caching when repeated reads become a bottleneck. Add queues when background work slows down user requests. Introduce a dedicated search engine when search requirements outgrow the database's capabilities. Adopt orchestration and GitOps practices when deployment complexity justifies them.
System design is not about remembering a list of tools. It is about understanding the problem that makes each tool worth introducing—and the trade-offs that come with it.
Further reading
- Docker documentation
- Amazon ECS documentation
- Kubernetes documentation
- Terraform documentation
- PostgreSQL documentation
- Redis documentation
- OpenSearch documentation
- Amazon S3 documentation
- Amazon SQS documentation
- RabbitMQ documentation
- Apache Kafka documentation
- Amazon ECR documentation
- Helm documentation
- Argo CD documentation
- Prometheus documentation
- Grafana documentation
- OpenTelemetry documentation
This article is based on the video How Docker, Kubernetes, Redis, Kafka & Prometheus Fit Together in One Application, with the concepts reorganized and explained in article form.
Top comments (0)