π Hello Access Managers!!
From setting secure roles to watching them in action β letβs move from IAM to real-time monitoring and logging! πβ‘οΈπ
π All code, docs, and resources are available in my GitHub repository:
madhurimarawat
/
Cloud-Computing
This repository focuses on cloud computing and demonstrates how to set up virtual machines, S3, and other services using LocalStack. It provides a comprehensive guide to simulating AWS services locally for development and testing purposes.
Cloud-Computing
This repository focuses on cloud computing and demonstrates how to set up virtual machines, S3, and other services using LocalStack. It provides a comprehensive guide to simulating AWS services locally for development and testing purposes.
Tools and Technologies βοΈπ»
1. AWS CLI
AWS Command Line Interface (CLI) is a powerful tool that allows users to interact with AWS services directly from the terminal. It simplifies managing cloud resources by providing commands for a wide range of AWS services, enabling tasks such as provisioning, managing, and automating workflows with ease.
2. LocalStack
LocalStack is a fully functional, local testing environment for AWS services. It enables developers to simulate AWS services on their local machines, facilitating the development and testing of cloud-based applications without needing access to an actual AWS account.
3. Docker
Docker is a containerization platform that allows developers to build, share, and run applications in isolated environments calledβ¦
In the last post,

π₯ Cloud Identity and Access Management (IAM) π
Madhurima Rawat γ» May 17
we explored Cloud Identity and Access Management (IAM) and how to secure your cloud infrastructure with fine-grained access control.
Today, weβre gonna dive into something equally vital,
π Cloud Monitoring and Logging!
First, Iβll explain what monitoring and logging actually mean, why theyβre absolutely essential in any cloud setup, and share some real-life use cases that show their impact.
Then weβll move ahead with the implementation.
What is AWS CloudWatch?
AWS CloudWatch is a monitoring and observability service that collects and tracks metrics, logs, and events to provide insights into cloud resources and applications.
πΉ Key Features of CloudWatch:
- Metrics Monitoring π: Collects and analyzes performance metrics for AWS services.
- Log Management π: Stores and processes logs for applications and infrastructure.
- Alarms & Notifications π¨: Triggers alerts based on threshold values.
- Dashboards & Visualization π: Provides real-time monitoring dashboards.
- Event-driven Automation βοΈ: Automates actions based on events.
Understanding Log Management
Log management is a crucial part of cloud monitoring and involves the collection, storage, and analysis of logs.
π‘ Six Components of Log Management:
- Log Collection ποΈ: Logs are gathered from various sources (servers, applications, cloud services).
- Log Aggregation ποΈ: Logs are combined and stored in a centralized system like CloudWatch Logs.
- Log Storage π’: Logs are kept in CloudWatch Log Groups and Log Streams.
- Log Analysis π: Tools like AWS CloudWatch Insights analyze log data for patterns and trends.
- Log Monitoring & Alerts π¨: CloudWatch can send alerts based on predefined conditions.
- Compliance & Retention β : Logs are archived based on security and compliance requirements.
What Are Log Streams?
Log streams represent a sequence of log events from the same source (e.g., an application instance, EC2 instance, or microservice).
πΉ Example:
- A web server might have separate log streams for each instance it runs on.
- CloudWatch logs are organized in log groups, and each log group contains multiple log streams.
Real-life Use Case: CloudWatch Logs & Metrics
Scenario:
A company hosts its web application on AWS EC2 instances and wants to monitor CPU usage and analyze logs for errors.
Steps in AWS CloudWatch:
1οΈβ£ Enable CloudWatch Logs on EC2 instances to store application logs.
2οΈβ£ Create a log group /my/app/logs
and define log streams for different instances.
3οΈβ£ Send logs & metrics (e.g., CPU usage, errors, response time) to CloudWatch.
4οΈβ£ Set up alarms to notify the DevOps team when CPU usage exceeds 80%.
5οΈβ£ Analyze logs using CloudWatch Insights to detect anomalies.
πΌοΈ About the Cover Image:
It begins with a speedometer icon, representing metrics and performance monitoring β capturing real-time data like logs, usage stats, and resource health.
Next, a checkbox with a right tick symbolizes that all essential cloud services are being actively monitored and running smoothly β
.
Following that, a warning icon appears to indicate that if anything goes wrong, the system will trigger alerts and notifications β οΈ.
Finally, a heart with a heartbeat line represents the idea of continuous, real-time monitoring β ensuring your cloud stays alive, healthy, and responsive ππ‘.
AWS CLI Commands for LocalStack (CloudWatch and Logs)
1. Create a Log Group
aws --endpoint-url=http://localhost:4566 logs create-log-group
--log-group-name /my/app/logs
Description:
- Creates a log group named
/my/app/logs
in AWS CloudWatch Logs.
2. Create a Log Stream
aws --endpoint-url=http://localhost:4566 logs create-log-stream
--log-group-name /my/app/logs --log-stream-name my-stream
Description:
- Creates a log stream named
my-stream
within the/my/app/logs
log group.
3. Put Log Events
aws --endpoint-url=http://localhost:4566 logs put-log-events --
log-group-name /my/app/logs --log-stream-name my-stream --log-
events "[{\"timestamp\":1741348140000,\"message\":\"Test log entry\"}]"
Description:
- Adds a log entry with a timestamp (
1741348140000
) and message (Test log entry
) to themy-stream
log stream.
Output:
-----------------------------------------------------------------------------------
| PutLogEvents |
+--------------------+------------------------------------------------------------+
| nextSequenceToken | 00000000000000000000000000000000000000000000000000000001 |
+--------------------+------------------------------------------------------------+
|| rejectedLogEventsInfo ||
|+-------------------------------------------------------------------+-----------+|
|| tooNewLogEventStartIndex | 0 ||
|+-------------------------------------------------------------------+-----------+|
4. Put Metric Data
aws --endpoint-url=http://localhost:4566 cloudwatch put-metric-
data --namespace "MyApp" --metric-name "CPUUsage" --value 75
Description:
- Publishes a custom CloudWatch metric
CPUUsage
with a value of75
in theMyApp
namespace.
5. List CloudWatch Metrics
aws --endpoint-url=http://localhost:4566 cloudwatch
list-metrics --namespace "MyApp"
Description:
- Lists all CloudWatch metrics under the
MyApp
namespace.
Output:
-------------------------------
| ListMetrics |
+-----------------------------+
|| Metrics ||
|+-------------+-------------+|
|| MetricName | Namespace ||
|+-------------+-------------+|
|| CPUUsage | MyApp ||
|+-------------+-------------+|
6. Describe Log Groups
aws --endpoint-url=http://localhost:4566 logs describe-log-groups
Description:
- Retrieves details of all log groups available in CloudWatch Logs.
Output:
-----------------------------------------------------------------------------------------
| DescribeLogGroups |
+---------------------------------------------------------------------------------------+
|| logGroups ||
|+-------------------+-----------------------------------------------------------------+|
|| arn | arn:aws:logs:us-east-1:000000000000:log-group:/my/app/logs:* ||
|| creationTime | 1741322075726 ||
|| logGroupName | /my/app/logs ||
|| metricFilterCount| 0 ||
|| storedBytes | 0 ||
|+-------------------+-----------------------------------------------------------------+|
7. Retrieve Log Events
aws --endpoint-url=http://localhost:4566 logs get-log-events
--log-group-name /my/app/logs --log-stream-name my-stream
Description:
- Fetches the log events from
my-stream
under/my/app/logs
.
Output:
-------------------------------------------------------------------------------------
| GetLogEvents |
+--------------------+--------------------------------------------------------------+
| nextBackwardToken | b/00000000000000000000000000000000000000000000000000000000 |
| nextForwardToken | f/00000000000000000000000000000000000000000000000000000000 |
+--------------------+--------------------------------------------------------------+
Want to see how it all worked step by step? Check it out here:
π Experiment 9 Output (PDF)
π§ Curious about how each command runs and responds? See the detailed input-output flow here:
π₯οΈ Monitoring Setup Input-Output Flow (PDF)
π And thatβs a wrap on Cloud Monitoring and Logging!
π Found great monitoring tools, cheat sheets, or observability hacks? Drop them in the comments. Iβd love to check them out and share with the community!
π Stay tuned for the next article!
Up next, weβll dive into Setting Up a Cloud-based CI/CD Pipeline β where youβll learn how to automate deployment and streamline your cloud workflows like a pro. πβοΈπ»
Top comments (5)
Pretty cool seeing someone break it down like this - real hands-on beats reading docs any day.
Thanks! I also believe real hands-on is the best way to learn.
Hey
Hello, how was the article? Did you liked it?
Yup! Glad :)