DEV Community: Hyelngtil Isaac

Secure Secrets with Secrets Manager

Hyelngtil Isaac — Fri, 29 May 2026 07:46:07 +0000

Introducing Today's Project!

In this project, I will demonstrate how to identify and remove hard‑coded credentials from a configuration file, create and store those credentials in AWS Secrets Manager, update the application to fetch secrets at runtime, configure least‑privilege IAM access so the app can retrieve the secret, and rotate the secret while verifying the application continues to authenticate; I’m doing this project to show practical Secrets Manager workflows, secure secret handling that prevents committing credentials to source control, how to design minimal IAM policies, and how to test secret rotation and deployment changes so I can confidently deploy applications without exposing sensitive data.

Tools and concepts

Services I used were AWS Secrets Manager, AWS IAM, Amazon S3, the boto3 AWS SDK for Python, GitHub (with secret scanning), Git (interactive rebase and history rewrite), and Python tooling (virtualenv, FastAPI, uvicorn, requirements.txt); Key concepts demonstrated are: never hardcode secrets, use a managed secrets store to centralize and encrypt credentials, apply least‑privilege IAM policies, automating secret rotation, practice Git hygiene and removing secrets from history when leaks occur, fetch secrets at runtime instead of storing them in config files, and follow local‑dev best practices to isolate dependencies and keep sensitive data out of source control.

Project reflection

This project took me approximately an hour. The most challenging part was safely removing hardcoded credentials from the repository history and configuring Secrets Manager and IAM so the app could retrieve secrets at runtime. It was most rewarding to replace insecure config files with a managed secrets workflow and verify the app could access S3 using least‑privilege credentials.

I chose to do this project today because I wanted to demonstrate practical skills in securing application credentials and show I can move a demo app from insecure config files to a managed‑secrets workflow; something that would make learning with NextWork even better is more hands‑on, bite‑sized labs with step‑by‑step remediation (including sample IAM policies, Secrets Manager rotation examples, and CI/CD checks) plus instant feedback or badges to validate each completed task.

Hardcoding credentials

In this project, a sample web app is exposing AWS credentials and other secrets directly in source files config.py. Hardcoding credentials is unsafe because anyone with repo access or attackers who find the repo can steal keys to impersonate the app, access or delete data, run costly resources, or escalate privileges; secrets also persist in Git history. Use a managed secrets store, least‑privilege IAM, and rotate/revoke compromised keys.

I put placeholder credentials and config values into config.py to simulate an insecure app for the exercise: example AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, an AWS_REGION, a database connection string stub (e.g., DB_HOST, DB_USER, DB_PASS), and a DEBUG=True flag. These are fake/example values so you can reproduce the risk of hard‑coding without exposing real secrets; the goal is to demonstrate detection, removal, and replacement with a secure store (Secrets Manager), then rotate and revoke any real keys if they were used.

Using my own AWS credentials

As an extension for this project, I also decided to install fastapi==0.115.8 to build a lightweight and efficient web API, uvicorn==0.34.0 as the ASGI server to run FastAPI applications, boto3==1.36.20 to interact with AWS services like Secrets Manager, and python-multipart==0.0.5 to handle file uploads in API requests—all essential for securely managing secrets while building and deploying the application.

When I first ran the app, I ran into an error because the AWS Access Key ID I provided was invalid, meaning it didn’t match any credentials in AWS’s records. This happens when the key is mistyped, deleted, or not properly configured, and it prevents the app from authenticating to AWS to list the S3 buckets.

To resolve the InvalidAccessKeyId error, I updated the config.py file to include the correct AWS credentials. It now contains the proper AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION values, ensuring the app can authenticate with AWS and successfully list the S3 buckets.

Pushing Insecure Code to GitHub

Once I updated the web app code with credentials, I forked the repository because I wanted my own independent copy under my GitHub account where I could make changes without affecting the original project. A fork is different from a clone because a fork stays linked to the original repository on GitHub, allowing me to propose changes back through pull requests, while a clone is just a local copy on my machine with no direct connection to the original.

To connect my local repository to the forked one, I first realized git add was pointing to the original repo I didn’t have permission to write to because I initially cloned directly to my IDE. So I used git remote set-url origin https://github.com/hyelngtil/nextwork-security-secretsmanager.git to update the remote to my fork. Then I ran git remote -v to confirm the change, followed by git add and git commit to stage and save my updates locally. Finally, git push to upload those commits to my forked repository on GitHub.

GitHub blocked my push because there were hardcoded secrets in my config.py file, which triggered its secret scanning protection. This is a good security feature because it prevents sensitive credentials from being exposed publicly, protecting both my AWS account and anyone who might accidentally use those leaked keys.

Secrets Manager

Secrets Manager is an AWS service designed to securely store, manage, and automatically rotate sensitive information like credentials, API keys, and database passwords. I'm using it to store my AWS access keys so they aren’t hardcoded in my application files, which keeps them protected and retrievable only when needed. Other common use cases include managing database connection strings, third‑party API tokens, and encryption keys, all while ensuring secrets remain encrypted and access is tightly controlled.

Another feature in Secrets Manager is secret rotation, which means the service can automatically update and replace stored credentials (like AWS keys or database passwords) on a schedule without manual intervention. It's useful in situations where long‑lived credentials could become a security risk, such as database connections or API tokens, because rotation reduces exposure time and ensures that applications always use fresh, valid secrets.

Secrets Manager provides sample code in various languages, like Python, Java, and JavaScript, to show developers how to retrieve secrets directly from their applications. This is helpful because it makes integration straightforward, reduces the risk of exposing credentials in code, and ensures that applications can securely access secrets without manual handling.

Updating the web app code

I updated the config.py file to retrieve credentials directly from AWS Secrets Manager instead of hardcoding them. The get_secret() function will connect to Secrets Manager, fetch the stored secret containing my AWS access keys, and return those values securely at runtime. This way, the application uses credentials managed by AWS rather than exposing them in the source code.

I also added code to config.py to extract the individual values—like AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION from the JSON secret retrieved by get_secret(). This is important because the application needs those specific fields to authenticate with AWS services, and by parsing them out of the secret object, the app can use them securely at runtime without ever exposing the raw credentials in the source code.

Resetting the repository

Git reset is a Git command that lets you move your current branch pointer to a different commit and optionally change the state of your working directory and staging area. I used it to roll back the repository to a clean state before the commit that contained my hardcoded secrets. This was necessary because those secrets had already been staged and committed, and I needed to remove them from the commit history so they wouldn’t be pushed to GitHub or exposed publicly.

A merge conflict occurred during rebasing because rewriting the branch history left the working tree and the new base out of sync and both versions modified the same lines in config.py. I resolved it by checking git status and git diff, manually editing config.py to keep the Secrets Manager changes and remove the hardcoded keys, staging and committing the fix, and then updating the remote so the exposed credentials were removed.

Once the merge conflict was resolved, I verified that my hardcoded credentials were out of sight in the repository by checking the files with git status and git diff to confirm no sensitive lines remained, reviewing the commit history with git log to ensure the secrets were removed from past commits, and inspecting the remote repository on GitHub to confirm that only the updated config.py with Secrets Manager integration was present.

Key Takeaways

Zero-Trust Secret Handling: Refactored a FastAPI web application to remediate architectural vulnerabilities caused by hard-coded credentials in config.py. Implemented dynamic, runtime retrieval of AWS access keys and database strings via AWS Secrets Manager, mitigating the risk of source control credential leaks.
Git History Sanitization: Utilized git reset and interactive rebasing to purge compromised commits from the Git metadata. Successfully resolved merge conflicts to establish strict Git hygiene and bypass GitHub Secret Scanning blockers during remote pushes.
Least-Privilege & Automation: Designed minimal AWS IAM policies to grant the application restrictive access to secrets, while successfully configuring and validating automated secret rotation loops.

🤝Next in the series builds on this, which will be Build a Security Monitoring System

Threat Detection with GuardDuty

Hyelngtil Isaac — Thu, 02 Apr 2026 07:46:22 +0000

Introducing Today's Project!

I built a hands-on project where I wore two hats; attacker and defender, to demonstrate how SQL injection and command injection can escalate into a full cloud credential breach, and how AWS GuardDuty surfaces those behaviors in near real-time.

Tools & Concepts

Services used: Amazon GuardDuty, Amazon EC2, Amazon S3, AWS CloudFormation, AWS CloudShell, IAM Roles, Amazon CloudFront, VPC and networking components.

Key concepts: Threat detection with GuardDuty, SQL injection, command injection, Instance Metadata Service (IMDS) and credential exfiltration, simulating attacker behavior with CloudShell, S3 Malware Protection, and incident investigation workflows.

This project took approximately 1 hour. The most challenging part was tuning GuardDuty detections and IAM role permissions. The most rewarding moment was watching real threat findings surface during testing.

Project Setup

I deployed a CloudFormation template that provisions three functional pillars:

Web App Infrastructure — an Amazon EC2 instance inside a dedicated VPC (not the default), with its own Subnet, Internet Gateway, and Elastic Load Balancer for isolated networking.
S3 Storage — a bucket containing a protected important-information.txt file that the EC2 instance is authorized to access, simulating sensitive data.
GuardDuty Monitoring — automatically enabled as a security sentinel to monitor resources and detect threats.

The web app deployed is OWASP Juice Shop, a deliberately vulnerable application. My objective as the simulated attacker: gain access to the EC2 web server and read the sensitive file in S3.

What is GuardDuty?

GuardDuty is an intelligent threat detection service that continuously monitors AWS accounts and workloads for malicious activity. In this project, it analyzes:

VPC Flow Logs
CloudTrail management events
S3 data events

It uses machine learning and integrated threat intelligence to detect indicators of compromise — credential exfiltration, communication with known malicious IPs, and more. Findings trigger an automated remediation workflow via Amazon EventBridge.

Attack Phase 1: SQL Injection

SQL injection involves injecting malicious SQL code into an input field to manipulate backend database queries. It's dangerous because it allows attackers to bypass authentication and access sensitive data without authorization.

I entered the following into the email field of the Juice Shop login page:

' or 1=1;--

What this does:

1=1 is always true, so the database validates the login regardless of the password.
-- comments out the rest of the original query, neutralizing the intended security check.

Result: administrative access to the OWASP Juice Shop portal — no password required.

Attack Phase 2: Command Injection

Command injection is a vulnerability where an attacker executes arbitrary OS commands via a vulnerable application. Juice Shop is vulnerable because it fails to sanitize user input before passing it to a system shell.

I exploited the search field by injecting a Node.js payload that:

Queried the Instance Metadata Service (IMDSv2) to retrieve a session token.
Identified the IAM Role attached to the EC2 instance.
Fetched temporary security credentials — AccessKeyId, SecretAccessKey, and SessionToken.
Piped the JSON output to a publicly accessible path:

/frontend/dist/frontend/assets/public/credentials.json

A simple application flaw became a significant cloud infrastructure breach.

Attack Verification

I navigated to the public URL at /assets/public/credentials.json and confirmed the exfiltrated credentials — a structured JSON object containing the stolen IAM temporary credentials tied to the EC2 instance's role.

This proved the attacker now had everything needed to authenticate as a legitimate internal service and begin compromising additional AWS resources, including the S3 bucket.

Using CloudShell to Escalate the Attack

CloudShell provided a pre-authenticated environment with the AWS CLI pre-installed — perfect for simulating an attacker operating from an external machine using stolen credentials.

Steps taken:

# Download the exfiltrated credentials file
wget <public-url>/assets/public/credentials.json

# Extract the credential values
cat credentials.json | jq '.AccessKeyId, .SecretAccessKey, .SessionToken'

# Configure a new AWS CLI profile called "stolen"
aws configure --profile stolen

Using the stolen profile isolated the "hacker" identity from the default CloudShell credentials. I could now simulate unauthorized S3 access — the exact behavior GuardDuty would flag as anomalous.

GuardDuty's Findings

Within 15 minutes of executing the attack, GuardDuty generated a finding:

UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.InsideAWS
Severity: HIGH

What this means: GuardDuty detected that IAM credentials were exfiltrated and then used inside the AWS environment — indicating a likely credential compromise and unauthorized lateral movement within the account.

How it detected it:
GuardDuty models normal AWS behavior and flags deviations. It correlates telemetry from CloudTrail, VPC Flow Logs, and DNS logs to spot unusual patterns — atypical API calls, credential use from unexpected sources, sudden internal data access, or reconnaissance activity.

The detailed finding reported that credentials for the EC2 instance role were used from a remote AWS account, confirming the simulated exfiltration scenario.

S3 Malware Protection

To test GuardDuty's Malware Protection for S3, I uploaded the standard EICAR anti-malware test file — a harmless string that antivirus products are configured to recognize as a test signature.

GuardDuty instantly triggered a security alert, confirming that Malware Protection detected the uploaded object and generated a finding indicating potential malware.

Key Takeaways

A single unsanitized input field can escalate from a login bypass to full cloud credential theft.
IMDS is a high-value target; restricting it with IMDSv2 and tight IAM policies is critical.
GuardDuty's anomaly detection is effective, it flagged the credential misuse within 15 minutes with no manual configuration beyond enabling the service.
Simulating attacks in a controlled lab environment is one of the best ways to build intuition for both offensive techniques and defensive tooling.

🤝Next in the series builds on this, which will be "Secure Secrets with Secrets Manager"

Encrypt Data with AWS KMS

Hyelngtil Isaac — Sun, 15 Mar 2026 11:17:31 +0000

Introducing Today's Project!

In this project, I will demonstrate how to create AWS KMS encryption keys, use them to encrypt a DynamoDB table, add and retrieve data to verify the encryption, observe how AWS blocks unauthorized access, and grant a user the necessary encryption permission. The goal is to show end‑to‑end data protection in AWS by provisioning keys, applying encryption to a live database, validating that only authorized principals can read or write the data, and confirming that key policies and IAM controls effectively prevent unauthorized access.

Tools and concepts

I used AWS KMS to create and manage a customer‑managed key, Amazon DynamoDB to store and encrypt table data, and IAM to create a test user and attach scoped policies. I worked in the AWS console to edit key policies, add key users, and verify access from an incognito session.

I learned about encryption at rest, that KMS key policies are the ultimate authority, and how to enforce least privilege by separating DynamoDB permissions from KMS decrypt permissions. I practiced using grants for temporary access, testing permissions, and documenting changes for auditability and rollback.

Project reflection

This project took me less an hour to complete, including setup, testing, and documentation. It was rewarding to see least‑privilege controls work in practice: the test user initially received access denied errors, then, after a narrowly scoped policy change, could decrypt the DynamoDB items, and I captured the steps and evidence for an auditable, repeatable workflow.

I chose to do this project today because I wanted hands‑on experience with real AWS security controls, creating a customer‑managed KMS key, attaching it to a DynamoDB table, and testing least‑privilege in practice. Working through the console and verifying access as a restricted test user helped me connect theory to operational tasks I’ll face as a junior cloud engineer and gave me confidence in key policy mechanics and auditable workflows.

Encryption and KMS

Encryption is the process of converting plaintext into an unreadable format (ciphertext) using mathematical algorithms so that unauthorized parties cannot understand the data.

Companies and developers do this to protect sensitive data at rest and in transit, prevent data breaches, satisfy legal and regulatory obligations, and preserve customer trust by ensuring confidentiality and integrity.

Encryption keys are secret values used by encryption algorithms to lock (encrypt) and unlock (decrypt) data; proper key management (storage, rotation, access control, and auditing) is essential because weak key handling defeats encryption.

AWS KMS is a fully managed AWS service that creates, stores, and controls cryptographic keys, enforces key policies, integrates with other AWS services, and logs key usage for auditing; key management systems are important because they protect the secrets that secure your data, enforce least‑privilege access, provide auditable trails for compliance, simplify key rotation and lifecycle operations, and reduce the operational risk and complexity of managing encryption securely.

Encryption keys are broadly categorized as symmetric (one secret used for both encryption and decryption) and asymmetric (a public/private key pair where the public key encrypts and the private key decrypts); I set up a symmetric key because symmetric keys are the recommended and most efficient choice for encrypting data at rest in AWS services like DynamoDB, they offer better performance for bulk data operations, integrate seamlessly with AWS KMS and service‑side encryption, simplify access control and key lifecycle management, and keep the sensitive key material protected inside KMS while still allowing authorized AWS principals to read and write encrypted table data.

Encrypting Data

My encryption key will safeguard data in DynamoDB, which is a fully managed, serverless NoSQL database that stores items as key‑value or document records, delivers single‑digit millisecond performance at scale, supports transactions and global replication, and integrates with AWS KMS for server‑side encryption; by attaching a customer‑managed symmetric KMS key to the table, storage and backups are encrypted at rest, access is controlled through key policies and IAM permissions, and every use of the key is logged for auditability so only authorized principals can decrypt and read the plaintext.

The different encryption options in DynamoDB include AWS owned keys (fully managed by AWS with no customer control), AWS managed KMS keys (service‑managed CMKs that AWS creates and rotates but still surface usage in CloudTrail), and customer‑managed KMS keys (CMKs you create and control in AWS KMS).
Their differences are based on who controls the key material and lifecycle, how much policy and access control you can enforce, the granularity of audit and rotation capabilities, and how quickly you can revoke or disable access.
I selected a customer‑managed symmetric KMS key because it gives me full policy control, immediate revocation and rotation options, detailed auditability, and the performance and seamless integration needed for encrypting DynamoDB table data.

Data Visibility

Rather than controlling who has access to the key, KMS manages user permissions by requiring explicit, key‑level authorization through a key policy (the primary control), and by evaluating IAM policies, grants, and any explicit denies before allowing cryptographic operations such as Encrypt, Decrypt, ReEncrypt, GenerateDataKey, or DescribeKey. This means no principal has any KMS key permissions unless the key policy or an allowed IAM policy/grant gives them those permissions.

Despite encrypting my DynamoDB table, I could still see the table’s items because DynamoDB’s server‑side encryption with KMS is transparent to authorized clients: AWS encrypts data at rest and stores ciphertext, but when an IAM principal or service with the necessary permissions reads an item, DynamoDB requests KMS to decrypt the data and returns plaintext to the caller, so applications and users who hold the right IAM/KMS permissions see normal, readable items; this protects storage, snapshots, and backups from anyone who cannot obtain KMS decrypt rights, while if you need ciphertext visible to clients you must use client‑side encryption (where the application holds the keys) rather than KMS server‑side encryption.

Denying Access

I configured a new IAM user named nextwork-kms-user to act as a test account for DynamoDB work; I attached the AmazonDynamoDBFullAccess managed policy so the user can fully interact with DynamoDB and saved the login credentials from the Retrieve password page, but I did not grant any permissions to my KMS key (no KMS actions or key policy access), ensuring the user cannot manage or decrypt encrypted data.

After accessing the DynamoDB table as the test user, I encountered an error when attempting to view the encrypted item attributes because the test user lacked permissions to use the KMS key (the console returned an access denied / missing kms:Decrypt message). This confirmed that attaching AmazonDynamoDBFullAccess alone does not permit reading encrypted data and that explicit KMS key permissions are required to decrypt and view those items, validating the principle of least privilege.

Granting Access

To let my test user use the encryption key, I added maven-kms-user as a key user in the KMS console so the principal can perform cryptographic operations; my key's policy was updated in the policy view to include a narrowly scoped statement granting that user Encrypt, Decrypt, ReEncrypt, GenerateDataKey, and DescribeKey on the key (targeted to the user’s ARN) while explicitly omitting any key‑management actions so the user can decrypt and encrypt data but cannot manage or change the key.
Using the test user, I retried accessing the DynamoDB table and refreshed the Items view; I observed the previously encrypted attributes now displayed in plaintext and successful GetItem/Scan operations without any KMS access‑denied errors, which confirmed that adding the user to the key policy (granting kms:Decrypt and related use actions) allowed the test user to decrypt and view the encrypted data while still not granting key management permissions.

Encryption protects data by making it unreadable without keys, while access control restricts who can request or retrieve that data; use encryption when you need protection against compromised storage or cross‑boundary exposure, and combine it with access controls, IAM, network controls, and logging to enforce defense in depth

🤝Next in the series builds on this, which will be "Threat Detection with Amazon GuardDuty"

Query Data with DynamoDB

Hyelngtil Isaac — Thu, 05 Mar 2026 17:02:13 +0000

Introducing Today's Project!

What is Amazon DynamoDB?

Amazon DynamoDB is a fully managed, serverless NoSQL database service from AWS that provides fast, predictable performance and scales automatically. It is useful because it eliminates the need to manage servers, supports both key‑value and document data models, and ensures single‑digit millisecond response times even at massive scale.

How I used Amazon DynamoDB in this project

In today’s project, I used Amazon DynamoDB to practice querying and updating data across tables in a way that keeps everything consistent. I started by running get-item commands to retrieve specific records using partition keys and projection expressions, which allowed me to pull back only the attributes I needed. Then I explored how related tables can be updated together by running a transaction with transact-write-items, which let me insert a new comment into one table while simultaneously updating a counter in another. This showed me how DynamoDB ensures atomicity, both operations succeed or fail together making it really useful for handling connected data across multiple tables without risking mismatched updates.

One thing I didn’t expect in this project is how seamlessly DynamoDB handled transactions across multiple tables. I thought working with related data in separate tables would require a lot of manual coordination, but using transact-write-items made it surprisingly straightforward to insert a new record in one table while simultaneously updating another. It was eye‑opening to see how DynamoDB guarantees atomicity, either both operations succeed or neither does which really simplifies keeping related data consistent. This step showed me that DynamoDB isn’t just about speed and scalability, but also about reliability when managing complex relationships.

This project took me through the full cycle of working with DynamoDB from retrieving specific items with "get-item" and "projection expressions", to exploring how tables can be related, and finally running a transaction that updated two tables at once. It wasn’t just about learning commands; it was about seeing how DynamoDB ensures consistency and reliability when handling connected data

Querying DynamoDB Tables

A partition key is the primary attribute DynamoDB uses to distribute and retrieve data across its storage partitions. Every item in a DynamoDB table must include a partition key, and items with the same partition key value are grouped together. This key determines where the data is stored internally and is essential for efficient queries, since DynamoDB can quickly locate items based on that key rather than scanning the entire table.
A sort key is the secondary attribute in a DynamoDB table’s primary key schema that works alongside the partition key to uniquely identify items. While the partition key determines which partition the data belongs to, the sort key organizes items within that partition. This means multiple items can share the same partition key but be distinguished by different sort key values. Sort keys also enable powerful query patterns, such as retrieving items in a range (e.g., all comments after a certain date) or ordering results by the sort key.

Limits of Using DynamoDB

I ran into an error when I queried for items in the Comment table without providing a value for the partition key Id. This was because DynamoDB requires the partition key filter to be specified in every query, without it, the system doesn’t know which partition to look in, so the console flagged the input as invalid. In other words, the query failed because the partition key field was left empty, and DynamoDB cannot execute a query unless the full key schema is respected.
Insights we could extract from our Comment table include the ability to see which posts attract the most engagement, track how often specific users contribute comments, and identify time-based activity patterns such as peak commenting hours or days. We can also observe relationships between posts and their associated comments, giving us a clear picture of community interaction at a structural level.

Insights we can’t easily extract from the Comment table include deeper qualitative analysis, such as the sentiment or tone of comments, trending topics across multiple posts, or demographic-based engagement patterns. DynamoDB stores structured attributes but doesn’t analyze meaning or allow complex joins across tables, so extracting these kinds of insights would require additional tools or data sources.

Running Queries with CLI

A query I ran in CloudShell was

aws dynamodb get-item \
--table-name ContentCatalog \
--key '{"Id":{"N":"202"}}' \
--projection-expression "Title, ContentType, Services" \
--return-consumed-capacity TOTAL

This query will fetch the item in the ContentCatalog table with the partition key Id equal to 202, but instead of returning the entire record, DynamoDB will only return the attributes I specified in the projection expression — Title, ContentType, and Services. Alongside those values, the response will also include a Consumed-Capacity block that shows how many read capacity units (RCUs) were used, giving me both the filtered item data and a usage report in one result.

Query options I could add to my query affect how DynamoDB returns the data and what additional information I get back. Specifically:

--consistent-read
Ensures I always get the most up-to-date version of the item, rather than a possibly stale copy from a replicated node.

--projection-expression
Lets me specify which attributes to return, so instead of the full record I only get the fields I care about (Title, ContentType, and Services).

--return-consumed-capacity TOTAL
Adds a usage report to the response, showing how many read capacity units were consumed by the query.

Transactions

A transaction is a coordinated set of operations in DynamoDB that are executed together so they either all succeed or all fail. Based on the surrounding page content, the idea is that when you need to update related data across multiple tables or items, you can group those changes into a single transaction. DynamoDB then guarantees atomicity: if one part of the transaction cannot be completed, none of the changes are applied. This ensures consistency and prevents situations where one table is updated but another is left behind, keeping your data reliable and synchronized across different parts of your application.

I ran a transaction using the aws dynamodb transact-write-items command with a client request token called TRANSACTION1. This transaction did two things: first, it added a new item into the "Comment" table with details such as the event name, the date and time of the comment, the comment text, and the user who posted it. Second, it updated the "Forum" table by incrementing the Comments attribute for the Events item. By grouping these two operations together in a single transaction, DynamoDB ensured that both changes either succeeded or failed as one unit, keeping the data consistent across the related tables.

🤝This is the end of this Series.
Next Series will be Security

Load Data into a DynamoDB Table

Hyelngtil Isaac — Mon, 02 Feb 2026 22:20:35 +0000

Introducing Today's Project!

What is Amazon DynamoDB?

DynamoDB is useful because it combines fast performance, flexible data modeling, and effortless scaling, making it a strong choice for modern applications that need to handle large amounts of varied data reliably.

How I used Amazon DynamoDB in this project

In today's project, I used Amazon DynamoDB to create tables, load diverse data like projects and videos into the ContentCatalog, and then view and update those items, because DynamoDB’s flexible schema allowed me to store different
types of content side by side while still retrieving them quickly with partition keys.

One thing I didn't expect in this project was...

One thing I didn't expect in this project is how straightforward it was to set up and run DynamoDB compared to relational databases like RDS or Aurora, because DynamoDB doesn’t require configuring servers, managing connections,
or defining rigid schemas, it just lets you create a table and start loading items right away.

This project took me about an hour, taking me through the full cycle of working with Amazon DynamoDB, creating tables, loading diverse data like projects and videos, and then viewing and updating items, because it was designed to show
how DynamoDB’s flexibility and speed make it easier to manage different types of content compared to traditional relational databases.

Create a DynamoDB table

DynamoDB tables organize data using items, which are records made up of attributes that describe details about each item; unlike relational databases, items don’t need to share the same attributes, giving DynamoDB a flexible way to store varied information in one table.

An attribute is a single piece of data that describes an item in DynamoDB, for example, if the item is a student record, attributes could include the student’s name, age, or number of projects completed. Unlike traditional relational databases where every row must share the same set of columns, DynamoDB
items can each have different attributes, giving you flexibility to store varied information within the same table.

Read and Write Capacity

Read Capacity Units (RCUs) and Write Capacity Units (WCUs) are DynamoDB’s measures of throughput, where RCUs define how many reads per second a table can handle and WCUs define how many writes per second it can handle.

Amazon DynamoDB’s Free Tier provides 25 GB of storage, along with 25 Read Capacity Units (RCUs) and 25 Write Capacity Units (WCUs), which together allow upto 200million of requests per month at no cost. I turned off auto scaling because while it can automatically increase capacity in production to handle
spikes in demand, it could push usage beyond the Free Tier limits and lead to unexpected charges, so disabling it ensures my table stays within the free allowance while I safely experiment.

Using CLI and CloudShell

AWS CloudShell is a browser‑based command line environment provided by Amazon Web Services that let's you securely manage, explore, and interact with your AWS resources without needing to install or configure tools locally. It comes pre‑authenticated with your AWS account and includes popular developer tools, making it easy to run commands, scripts, and manage services like DynamoDB directly from your web browser.

AWS CLI is a command-line interface tool that lets you manage and interact with AWS services by typing commands instead of using the web console. It provides a unified way to automate tasks, run scripts, and control resources like DynamoDB, S3, or EC2 directly from your terminal, making it especially useful for developers and administrators who want efficiency and repeatability in managing their cloud infrastructure.

I ran a CLI command in AWS CloudShell that created a new
DynamoDB table, because CloudShell provides a ready‑to‑use, browser‑based terminal that’s already authenticated with my AWS account, making it simple to execute AWS CLI commands without installing or configuring anything locally. This step is part of learning how to provision DynamoDB resources directly from the command line, reinforcing the idea that you can manage AWS services not only through the console but also programmatically

Loading Data with CLI

I ran a CLI command in AWS CloudShell that created new DynamoDB tables, because CloudShell comes pre‑installed with the AWS CLI and is already authenticated with my AWS account, making it easy to provision resources directly from the browser without needing any local setup. This step shows how DynamoDB tables can be defined programmatically, reinforcing the flexibility of managing cloud databases through commands instead of the console.

Observing Item Attributes

I checked a ContentCatalog item, which had the following attributes:
Id (partition key, number), Title (string), URL (string), Authors (list), Price (number), Difficulty (string), Published (boolean), ProjectCategory (string), and ContentType (string).
I checked another ContentCatalog item, which had a different set of attributes:
Id (partition key, number), Title (string), URL (string), VideoType (string), Price (number), Services (list, sometimes included), and ContentType (string).

Benefits of DynamoDB

A benefit of DynamoDB over relational databases is flexibility, because it doesn’t require a fixed schema, items in the same table can have different sets of attributes and data types. This means you can store diverse records (like projects and videos in your ContentCatalog) side by side without redesigning the table, whereas relational databases enforce rigid column structures that must be consistent across all rows. This flexibility makes DynamoDB especially useful for applications where data models evolve quickly or vary widely.

Another benefit over relational databases is speed, because DynamoDB is designed for high‑performance at scale, using SSD storage and a distributed architecture that allows single‑digit millisecond response times. Unlike relational databases, which often need complex joins and indexing across rigid schemas, DynamoDB retrieves items directly by their keys, making lookups and writes much faster. This speed is especially valuable for applications like gaming, e‑commerce, or real‑time analytics, where quick responses are critical.

🤝Next in the series builds on this, which is "Query Data with DynamoDB"

Connect a Web App to Amazon Aurora

Hyelngtil Isaac — Tue, 13 Jan 2026 11:51:33 +0000

Introducing Today's Project!

What is Amazon Aurora?

Amazon Aurora is a fully managed relational database service from AWS that is compatible with MySQL. It is useful because it combines the familiarity of MySQL with the scalability, speed, and reliability of a cloud‑native service. Aurora
automatically handles tasks like backups, replication, and failover, which makes it easier to build web apps that need secure, high‑performance data storage without managing complex infrastructure yourself.

How I used Amazon Aurora in this project

In today’s project, I used Amazon Aurora to store and manage the data from my web app. By connecting my EC2‑hosted application to Aurora, I was able to capture user input through the web interface and save it securely in a relational
database. Aurora’s compatibility with MySQL made it easy to query and verify the data using the MySQL CLI, while its scalability and reliability ensured that the app could handle future growth without me having to manage complex infrastructure.

One thing I didn’t expect in this project was how quickly Amazon Aurora connected with my EC2 instance once the configuration details were set. I thought it might take longer or require a more complex setup, but the compatibility with MySQL and the php‑mysqli extension made the process smoother than I anticipated.
This showed me that cloud services can simplify tasks that would normally be more complicated to manage on my own.

Creating a Web App

To connect to my EC2 instance, I used SSH with my .pem key file because this provides secure, authenticated access to the server. By running the "ssh -i MavenAuroraApp.pem ec2-user@" command, I was able to log in remotely, and I'm ready to begin installing and configuring my web application.

To help me create my web app, I first connected to my EC2 instance through SSH and installed the necessary software, Apache, PHP, and the php‑mysqli extension, because these tools turn the EC2 instance into a functioning web server capable of running a dynamic application and communicating with my Aurora database.

Connecting my Web App to Aurora

I set up my EC2 instance's connection details to my database.

My Web App Upgrade

Next, I upgraded my web app by adding a new PHP script that connects to my Aurora database and displays a more user‑friendly web page. This upgrade allowed the app to move beyond a simple static page and start handling dynamic data, capturing user input, sending queries to Aurora, and showing results directly in the browser.

Testing my Web App

To make sure my web app was working correctly, I tested it in the browser by submitting data through the web page and then used the MySQL CLI on my EC2 instance to query the Aurora database. By checking that the new entries appeared in the database, I confirmed that the app was successfully sending and storing data in Aurora.

🤝Next in the series builds on this, which is "Load Data into DynamoDB"

Aurora Database with EC2

Hyelngtil Isaac — Mon, 05 Jan 2026 10:18:09 +0000

Connect a Web App to Amazon Aurora

Introducing Today's Project!

What is Amazon Aurora?

Amazon Aurora is a high‑performance, fully managed database engine that combines the speed and reliability of commercial databases with the simplicity and cost‑effectiveness of open‑source ones. It’s useful because it scales automatically, stays highly available, and integrates smoothly with your AWS environment.

How I used Amazon Aurora in this project

In today’s project, I used Amazon Aurora to set up a highly available relational database that integrates seamlessly with my EC2 instance. Aurora provided the database endpoint I needed to connect my application, while automatically handling scalability, replication, and fault tolerance. This allowed me to focus on building and testing my app without worrying about manual database management.

One thing I didn't expect in this project

One thing I didn’t expect in this project was how straightforward the setup turned out to be. I thought it would be much of a hassle to configure the database and connect it to my EC2 instance, but the AWS steps made the process surprisingly simple.

I completed the Aurora Database with EC2 project in about an hour, which was faster than I expected. The guided AWS setup made connecting my EC2 instance to Aurora straightforward and efficient.

In the first part of my project

Creating an Aurora Cluster
A relational database is one in which data are organized into tables, which are collections of rows and columns. It's called "relational" because the rows relate to the columns and vice versa.
Aurora is a good choice when we need something large-scale, with peak performance and uptime. This is because Aurora databases use clusters. Ordinary relational databases, like MySQL and Oracle, are more generic and cost-effective. They suit smaller databases and less demanding workloads.

Halfway through I stopped!

I stopped creating my Aurora database because I am trying to connect a web app server to my Aurora database. That is why I needed to set up an EC2 instance to serve as the web app server.

Features of my EC2 instance

I created a new key pair for my EC2 instance, because I need keys to access my EC2 instance if I want to add, change, or update how my EC2 instance is running.
When I created my EC2 instance, I took particular note of the "Public IPv4 DNS" and the "Key pair name." The Public IPv4 DNS is essentially the address of my EC2 instance on the internet, while the Key pair is a set of cryptographic keys (public and private) used to securely access the instance. The Key pair name identifies which keys are associated with it.

Then I could finish setting up my database

Aurora Database uses clusters because they work together, so your data is always available. Aurora is really good for the big jobs because of these clusters.
Each cluster consists of a primary instance (where all write operations occur) and multiple read replicas as back-ups. If your database's primary instance fails, one of the replicas can be promoted to primary automatically.

🤝Next in the series builds on this, which is "Connect a Web App with Aurora"

AWS Databases!

Hyelngtil Isaac — Sat, 03 Jan 2026 09:54:53 +0000

I'm exploring AWS Databases!

I'm building database solutions on AWS
In this AWS Databases series, I'm learning about AWS databases (Relational and NoSQL databases). By the end of these projects, I will have known how to Connect an Aurora Database to EC2, connect a Web App to Amazon Aurora, Load Data into a DynamoDB Table, Visualize a Relational Database, and Query Data with DynamoDB. I'm learning about cloud databases because I want to know how it works and then use it create impactful solutions.

I am excited to share my progress - explore AWS databases with me!
I will set aside few hours daily to work on these database projects. I will keep myself accountable by tracking my daily progress and sharing updates to stay consistent. My reward for completing this AWS Databases series will be more ability to manage cloud databases effectively and confidently in real projects, and the satisfaction of mastering practical cloud database skills.

What are databases?
Databases are organized systems for storing and managing information digitally. They allow data such as customer records, product details, and transactions to be kept in one central place, making it easy to access, update, and share securely across teams. Cloud engineers use databases to store, organize, and manage application data securely in the cloud. They rely on databases to connect applications to data, run queries for insights, ensure scalability, and maintain high availability across different services.

What do database professionals do?
Database professionals are responsible for setting up databases, modelling data, writing queries, and ensuring database security. They also handle performance tuning, backups, and connecting databases to applications. The most interesting part of their job is using queries to turn raw data into meaningful insights that support decision‑making.

Automated Deployment Bash Script: Deploying a Flask App to AWS

Hyelngtil Isaac — Sat, 25 Oct 2025 06:50:06 +0000

Hey DevOps folks! 👋 I've just wrapped up the DevOps Intern Stage 1 Task from HNG13, inspired by that dev.to challenge post. The mission? Build a single, robust Bash script to automate deploying a Dockerized app to a remote Linux server. I nailed it by deploying to an AWS EC2 instance with a simple Flask app that displays a success message and server time. This setup showcases real-world automation, idempotency, and reliability in DevOps workflows.

In this article, I'll share my deploy.sh script, explain the process, and how it all came together. Everything's based on my actual project files, feel free to check them out and adapt!

Task Overview

The script (deploy.sh) handles everything in one executable file:

Collect and validate user inputs (Git repo, PAT, branch, SSH details, app port).
Clone or update the repo.
Verify Docker files.
Test SSH and prepare the remote env (install Docker, Compose, Nginx).
Transfer files via rsync.
Deploy the app (build/run containers idempotently).
Set up Nginx reverse proxy.
Validate with health checks and curls.
Log everything, handle errors, and support cleanup with --cleanup.

I used AWS EC2 (Ubuntu 22.04) as the remote server. My app is a basic Flask site in Fapp.py, Dockerized via Dockerfile. Repo includes requirements.txt and a detailed README.md.

Prerequisites

AWS EC2 instance (e.g., t2.micro Ubuntu) with SSH key access. Open security group ports: 22 (SSH), 80 (HTTP), 8080 (app, direct testing).
Git repo with the app files: Fapp.py, Dockerfile, requirements.txt.
Local machine with Git, SSH, rsync.
PAT for GitHub repo access.

Here's a peek at the app files for context:

Fapp.py (Flask app serving HTML):

from flask import Flask
from datetime import datetime

app = Flask(__name__)

@app.route('/')
def home():
    return f'''
    <!DOCTYPE html>
    <html lang="en">
    <head>
      <meta charset="UTF-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>Stage1 HNG13 Deployment Successful!</title>
      <style>
        body {{
          font-family: Arial, sans-serif;
          display: flex;
          justify-content: center;
          align-items: center;
          height: 100vh;
          margin: 0;
          background: linear-gradient(135deg, #3f8bcd 0%, #2a629a 100%);
          color: white;
        }}
        .container {{
          text-align: center;
          padding: 2rem;
          background: rgba(255, 255, 255, 0.1);
          border-radius: 10px;
          backdrop-filter: blur(10px);
        }}
        h1 {{ margin-bottom: 1rem; }}
        .timestamp {{ font-size: 0.9em; opacity: 0.8; }}
      </style>
    </head>
    <body>
      <div class="container">
        <h1>🚀Stage1 HNG13 Deployment Successful!</h1>
        <p>Your automated deployment script is working!</p>
        <p class="timestamp">Server Time: {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}</p>
      </div>
    </body>
    </html>
    '''

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Dockerfile:

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8080

CMD ["python", "Fapp.py"]

requirements.txt:

Flask==3.0.0

The Bash Script: `deploy.sh`

This is the heart of it; POSIX-compliant, executable, and fully featured. Run chmod +x deploy.sh first.

#!/bin/bash

set -euo pipefail

# Create timestamped log file
LOG_FILE="deploy_$(date +%Y%m%d_%H%M%S).log"

# Log messages
log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE"
}

# Trap errors
trap 'log "ERROR: Script failed at line $LINENO"' ERR

# Read input with validation
read_input() {
    local prompt="$1"
    local var_name="$2"
    local default="${3:-}"

    read -p "$prompt: " value
    value="${value:-$default}"

    if [[ -z "$value" && -z "$default" ]]; then
        log "ERROR: $var_name cannot be empty"
        exit 1
    fi

    echo "$value"
}

# Gather inputs

GIT_REPO=$(read_input "Enter Git Repository URL" "GIT_REPO")
BRANCH=$(read_input "Enter branch name [main]" "BRANCH" "main")

# PAT: Silent read, validate non-empty
echo -n "PAT: "; stty -echo; read -r PAT; stty echo; echo
[[ -n "$PAT" ]] || { echo "Error: PAT required" >&2; exit 1; }

SSH_USER=$(read_input "Enter SSH username" "SSH_USER")
SERVER_IP=$(read_input "Enter server IP address" "SERVER_IP")
APP_PORT=$(read_input "Enter application port" "APP_PORT")
SSH_KEY=$(read_input "Enter SSH key path" "SSH_KEY")
#: Silent, validate file/permissions
#echo -n "SSH Key Path: "; stty -echo; read -r SSH_KEY; stty echo; echo
#[[ -f "$SSH_KEY" ]] || { echo "Error: Key invalid" >&2; exit 1; }
#chmod 400 "$SSH_KEY" || log "WARN: chmod 400 failed for $SSH_KEY (continuing)"

# Clone Git repository with authentication (not exposing PAT)
clone_repo() {
    local repo_url="$1" token="$2" branch="$3"
    REPO_NAME=$(basename "$repo_url" .git)
    export REPO_NAME

    # create a temporary GIT_ASKPASS helper that prints the PAT
    TMP_ASKPASS="$(mktemp)"
    cat > "$TMP_ASKPASS" <<'EOF'
#!/bin/sh
# Git calls this script to obtain a password. It expects the password on stdout.
echo "$GIT_PASSWORD"
EOF
    chmod +x "$TMP_ASKPASS"

    # Use GIT_ASKPASS to provide the token securely to git
    if [[ -d "$REPO_NAME" ]]; then
        log "Repository exists, updating to latest changes on branch '$branch'..."
        cd "$REPO_NAME" || { rm -f "$TMP_ASKPASS"; exit 2; }
        GIT_PASSWORD="$token" GIT_ASKPASS="$TMP_ASKPASS" git fetch origin || { rm -f "$TMP_ASKPASS"; exit 2; }
        GIT_PASSWORD="$token" GIT_ASKPASS="$TMP_ASKPASS" git checkout "$branch" || { rm -f "$TMP_ASKPASS"; exit 2; }
        GIT_PASSWORD="$token" GIT_ASKPASS="$TMP_ASKPASS" git pull origin "$branch" || { rm -f "$TMP_ASKPASS"; exit 2; }
    else
        log "Cloning repository on branch '$branch'..."
        GIT_PASSWORD="$token" GIT_ASKPASS="$TMP_ASKPASS" git clone -b "$branch" "$repo_url" || { rm -f "$TMP_ASKPASS"; exit 2; }
        cd "$REPO_NAME" || { rm -f "$TMP_ASKPASS"; exit 2; }
    fi

    rm -f "$TMP_ASKPASS"
    log "Successfully cloned/updated repository"
}

# Function to verify Docker configuration
verify_docker_config() {
    if [[ -f "Dockerfile" ]] || [[ -f "docker-compose.yml" ]]; then
        log "✓ Docker configuration found"
        return 0
    else
        log "✗ No Dockerfile or docker-compose.yml found"
        exit 3
    fi
}

# Function to test SSH connection
test_ssh() {
    local user="$1" ip="$2" key="$3"

    log "Testing SSH connection to $user@$ip..."

    if ssh -i "$key" -o ConnectTimeout=10 -o BatchMode=yes \
           "$user@$ip" "echo 'SSH connection successful'" &>/dev/null; then
        log "✓ SSH connection successful"
        return 0
    else
        log "✗ SSH connection failed"
        exit 4
    fi
}

# Function to setup remote environment
setup_remote_environment() {
    local user="$1" ip="$2" key="$3"

    log "Setting up remote environment..."

    # Execute commands on remote server
    ssh -i "$key" "$user@$ip" 'bash -s' << 'ENDSSH'
        set -e

        # Update packages
        sudo apt-get update -y

        # Install Docker
        if ! command -v docker &> /dev/null; then
            curl -fsSL https://get.docker.com -o get-docker.sh
            sudo sh get-docker.sh
            sudo usermod -aG docker $USER
        fi

        # Install Docker Compose
        if ! command -v docker-compose &> /dev/null; then
            sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" \
                -o /usr/local/bin/docker-compose
            sudo chmod +x /usr/local/bin/docker-compose
        fi

        # Install Nginx
        if ! command -v nginx &> /dev/null; then
            sudo apt-get install -y nginx
        fi

        # Start services
        sudo systemctl enable docker nginx
        sudo systemctl start docker nginx

        # Verify installations
        docker --version
        docker-compose --version
        nginx -v
ENDSSH

    log "✓ Remote environment ready"
}

# Deploy Docker application
deploy_application() {
    local user="$1" ip="$2" key="$3" app_port="$4"

    log "Deploying application..."

    ssh -i "$key" "$user@$ip" bash -s << ENDSSH || { log "✗ Deploy failed (check connection/logs)"; exit 1; }
        set -e
        mkdir -p ~/deployment
        cd ~/deployment/$REPO_NAME || { echo "Error: Repo dir not found" >&2; exit 1; }

        # Stop old containers
        docker-compose down 2>/dev/null || docker stop \$(docker ps -q) 2>/dev/null || true

        # Remove stopped containers to free names
        docker rm \$(docker ps -aq --filter "name=my-app") 2>/dev/null || true

        # Build and start
        if [[ -f "docker-compose.yml" ]]; then
            docker-compose up -d --build --force-recreate
        else
            docker build -t my-app .
            docker run -d -p $app_port:$app_port --name my-app my-app
        fi

        # Wait for container to be healthy
        sleep 5

        # Verify container is running
        if docker ps | grep -qE "my-app|$REPO_NAME"; then
            echo "✓ Containers running"
        else
            echo "✗ No running containers found" >&2
            exit 1
        fi
ENDSSH
    log "✓ Application deployed successfully"
}

# Function to transfer application files to the remote server
transfer_files() {
    local user="$1" ip="$2" key="$3" local_dir="$4"

    log "Transferring application files..."

    # Ensure the remote deployment directory exists
    ssh -i "$key" "$user@$ip" "mkdir -p ~/deployment" || exit 9

    # The REPO_NAME is globally available from the clone_repo call
    local REPO_TO_COPY="$local_dir/$REPO_NAME"
    if [[ ! -d "$REPO_TO_COPY" ]]; then
        log "ERROR: Local repository directory '$REPO_TO_COPY' not found."
        exit 10
    fi

    # Optional: Clean up existing remote repo dir to avoid conflicts/permissions issues
    #ssh -i "$key" "$user@$ip" "rm -rf ~/deployment/$REPO_NAME" || true

    # Transfer with rsync, excluding .git and logs
    rsync -avz -e "ssh -i '$key'" --exclude='.git/' --exclude='deploy_*.log' "$REPO_TO_COPY/" "$user@$ip:~/deployment/$REPO_NAME/" || exit 11


    log "✓ Files transferred successfully to ~/deployment/$REPO_NAME"
}

# Configure Nginx as reverse proxy
configure_nginx() {
    local user="$1" ip="$2" key="$3" app_port="$4"

    log "Configuring Nginx..."

    # Create Nginx config
    NGINX_CONFIG="
server {
    listen 80;
    server_name $ip;

    location / {
        proxy_pass http://localhost:$app_port;
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
        proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto \$scheme;
    }
}
"

    # Deploy config
    ssh -i "$key" "$user@$ip" bash -s << ENDSSH
        echo '$NGINX_CONFIG' | sudo tee /etc/nginx/sites-available/app.conf
        sudo ln -sf /etc/nginx/sites-available/app.conf /etc/nginx/sites-enabled/
        sudo nginx -t
        sudo systemctl reload nginx
ENDSSH

    log "✓ Nginx configured successfully"
}

validate_deployment() {
    local user="$1" ip="$2" key="$3"

    log "Validating deployment..."

    # Check container health with fallback if no HEALTHCHECK is defined
    CONTAINER_NAME="my-app"
    HEALTH_STATUS=$(ssh -i "$key" "$user@$ip" "docker inspect --format '{{.State.Health.Status}}' $CONTAINER_NAME 2>/dev/null || true")


    if [[ -n "$HEALTH_STATUS" ]]; then
        if [[ "$HEALTH_STATUS" != "healthy" ]]; then
            log "✗ Container $CONTAINER_NAME not healthy (status: $HEALTH_STATUS)"
            exit 6
        fi
    else
        # Fallback: ensure container exists and is running
        if ! ssh -i "$key" "$user@$ip" "docker ps --filter name=$CONTAINER_NAME --filter status=running --format '{{.Names}}' | grep -q ."; then
            log "✗ Container $CONTAINER_NAME not running"
            exit 6
        fi
    fi

    # App endpoint with retries
    MAX_RETRIES=3
    for i in $(seq 1 $MAX_RETRIES); do
        if curl -f -s "http://$ip/" > /dev/null; then
            log "✓ Application /health accessible"
            break
        fi
        [[ $i -eq $MAX_RETRIES ]] && { log "✗ /health failed after $MAX_RETRIES tries"; exit 8; }
        sleep $((i * 2))  # Backoff: 2s, 4s, 6s
    done
    log "✓ All validation checks passed"
}

cleanup() {
    local user="$1" ip="$2" key="$3"

    log "Cleaning up deployment..."

    ssh -i "$key" "$user@$ip" bash -s << 'ENDSSH' || exit 15
        # Stop and remove containers
        docker-compose down -v 2>/dev/null || true
        docker stop $(docker ps -aq --filter "name=^my-app") 2>/dev/null || true
        docker rm $(docker ps -aq --filter "name=^my-app") 2>/dev/null || true

        # Remove Nginx config
        sudo rm -f /etc/nginx/sites-enabled/app.conf
        sudo rm -f /etc/nginx/sites-available/app.conf
        sudo systemctl reload nginx

        # Remove deployment files
        rm -rf ~/deployment
ENDSSH

    log "✓ Cleanup completed"
}

# Check for cleanup flag
if [[ "${1:-}" == "--cleanup" ]]; then
    cleanup "$SSH_USER" "$SERVER_IP" "$SSH_KEY"
    exit 0
fi

main() {
    local original_dir="$(pwd)"  # Capture parent dir before any cd
    log "===== Starting Deployment ====="
    log "Repository: $GIT_REPO"
    log "Branch: $BRANCH"
    log "Target Server: $SERVER_IP"

    clone_repo "$GIT_REPO" "$PAT" "$BRANCH"
    verify_docker_config
    cd "$original_dir" || exit 12  # Reset to parent dir for correct transfer path
    test_ssh "$SSH_USER" "$SERVER_IP" "$SSH_KEY"
    setup_remote_environment "$SSH_USER" "$SERVER_IP" "$SSH_KEY"
    transfer_files "$SSH_USER" "$SERVER_IP" "$SSH_KEY" "$(pwd)"
    deploy_application "$SSH_USER" "$SERVER_IP" "$SSH_KEY" "$APP_PORT"
    configure_nginx "$SSH_USER" "$SERVER_IP" "$SSH_KEY" "$APP_PORT"
    validate_deployment "$SSH_USER" "$SERVER_IP" "$SSH_KEY"

    log "===== Deployment Completed Successfully ====="
}

main

How It Works: Step-by-Step

From the README.md:

Inputs: Secure prompts (PAT hidden), with defaults and validation.
Clone: Uses GIT_ASKPASS to handle PAT without exposure; pulls if exists.
Verify: Ensures Dockerfile is present.
SSH Test: Quick connectivity check with timeout.
Remote Setup: Installs Docker/Compose/Nginx if missing, starts services.
Transfer: Rsync for efficient, excluding unnecessary files.
Deploy: Idempotent stops/removes old containers, builds/runs new ones (uses docker build/run since no compose file).
Nginx: Dynamic config proxies 80 to app port (8080 here).
Validate: Checks container status, curls the endpoint with retries.
Logging/Cleanup: Timestamped logs; --cleanup tears down everything.

AWS-Specific Adaptations

EC2 Launch: Used Ubuntu 22.04 AMI, attached SSH key, configured security groups.
Permissions: SSH user (ubuntu) needs sudo; script handles Docker group add.
Testing: Ran locally, deployed to EC2—browser hit the IP showed the success page with timestamp. Re-runs worked without issues thanks to idempotency.
Security: No SSL yet (add Certbot for prod); ensure key is 400 perms.

Lessons Learned

Secure handling of secrets (like PAT) is crucial GIT_ASKPASS was a game-changer.
Idempotency via stop/rm commands prevents redeploy failures.
Remote exec with heredocs keeps things clean but watch for quoting.
Logging + traps make debugging remote issues easier.

Grab the full repo (including README) here: github.com/hyelngtil/hng13-stage1-devops. It's ready to fork and test!

What's your go-to automation trick in Bash? Drop it in the comments! 🚀

DevOps Challenge 1: Set Up a Web App Using AWS and VS Code

Hyelngtil Isaac — Mon, 06 Oct 2025 00:37:51 +0000

Introducing Today's Project!

In this project, I will demonstrate how to set up a remote SSH connection to an EC2 instance using VS Code. I'll install Maven and Java, generate a basic web app, and edit code without VS Code.

Key tools and concepts
Services I used were IAM, EC2, Security Groups, Key Pairs, VS Code, Maven, and Java. Key concepts I learnt include SSH for secure access, Maven project structure, dynamic vs static pages with JSP, and why IDEs like VS Code simplify cloud development.

Project reflection
One thing I didn't expect in this project was discovering that index.jsp isn’t just like HTML but can actually run Java code to make the page dynamic.

🔥This project took me approximately 2hours. The most challenging part was setting permissions using 'icalcs' and sustaining the SSH connection for an extended period of time. I was able to stay logged in for the period of the project. I'll have to re-establish the SSH connection intermittently. It was most rewarding to be able to successfully set the permission on Windows OS especially

This project is part one of a series of DevOps projects where I'm building a CI/CD pipeline! I'll be working on the next project in the next few days.

Launching an EC2 instance

I started this project by launching an EC2 instance because I needed a secure, cloud-based server to host my web app and practice DevOps workflows like remote development, CI/CD setup, and infrastructure management.

I also enabled SSH
SSH is a secure protocol that verifies your identity using a private key and encrypts data between your computer and a remote server. I enabled SSH so I could safely connect to my EC2 instance and use it.

Key pairs
A key pair is what let you securely access your EC2 instance. It’s made of two halves: a public key that AWS keeps, and a private key that you download.
Once I set up my key pair, AWS automatically downloaded a '.pem' file to my DevOps folder—my private key for secure EC2 access

Set up VS Code

VS Code, short for Visual Studio Code, is a powerful and widely used Integrated Development Environment (IDE) that helps developers write, edit, and manage code efficiently.
I installed it so I could securely connect to my EC2 instance and build my web app with a smooth coding experience

My first terminal commands

The first command I ran for this project was 'cd Desktop\DevOps' to navigate to the folder where my key file is located.

I also updated my private key's permissions by running: 'icacls' commands in the terminal as shown in the screenshot. This made the file readable only by me, securing it for SSH access to my EC2 instance.

SSH connection to EC2 instance

To connect to my EC2 instance, I ran the command >>ssh -i Maven-DevOps Keypair.pem ec2-user@ec2-3-80-80-185.compute-1.amazonaws.com<<

This command required an IPv4 address
A server's IPV4 DNS is the public address for your EC2 server that the internet uses to find and connect to it. The local computer you're using to do this project will find and connect to your EC2 instance through this IPv4 DNS.

Maven & Java

Apache Maven is a Project Builder and a Dependency manager. It gives a ready‑made structure for Java projects (folders, config files, etc.) and downloads the necessary extra code libraries needed by apps (e.g., logging tools, database connectors).

Maven is required in this project because I will use Maven to generate a web app project from a template (called an archetype).

Java is a popular programming language used to build different types of applications, from mobile apps to large enterprise systems.
Java is required in this project because it is used by Maven to operate. It will be used in generating and building the intended web app.

Create the Application

I generated a Java web app using the command

mvn archetype:generate \
-DgroupId=com.nextwork.app \
-DartifactId=nextwork-web-project \
-DarchetypeArtifactId=maven-archetype-webapp \
-DinteractiveMode=false

I installed Remote - SSH, which is an extension in VS Code that lets me connect directly via SSH to my EC2 instance securely over the internet. I installed it to use VS Code to work on files or run programs on my instance easily.

Configuration details required to set up a remote connection include

Host ec2-54-196-112-71.compute-1.amazonaws.com
HostName ec2-54-196-112-71.compute-1.amazonaws.com
IdentityFile C:\Users\hyeln\Desktop\DevOps\nextwork-keypair.pem
User ec2-user

Using VS Code's file explorer, I could see all the webapp project files, folders, and subfolders in their hierarchy!

Two of the project folders created by Maven are src (source) folder which holds all the source code files that define how your web app looks and works. Webapp is a special subfolder within src dedicated to the web-facing part of the app. HTML CSS JS.

Using Remote - SSH

The index.jsp is the starting page of your Java web app — like index.html in a static site, but with the added power of Java code to generate dynamic, changing content.

I edited index.jsp by navigating to the file via remote-SSH, the editing and saving.

Deploy an App Across Accounts

Hyelngtil Isaac — Thu, 04 Sep 2025 19:41:38 +0000

Introducing Today's Project!

Here, I am going Build a Docker container image and an Amazon ECR (Elastic Container Registry) to store the image securely.

What is Amazon ECR?
Amazon ECR is AWSʼs managed container registry for storing and sharing Docker images. In todayʼs project, we used it to push our app image and let our buddy pull and run it from their account.

One thing I didn't expect
My buddy was in 'us-east-1' and I was in 'af-south-1', so he couldn't authenticate to my ECR because ECR authentication is region specific. I resolved it by creating a matching repository in 'us-east-1'.
I later discovered that we needed to explicitly tell the AWS CLI to connect to the ECR service endpoints in the regions (af-south-1 & us-east-1) to get the correct token.

This project took us about an hour and half.

Creating a Docker Image

I set up a Dockerfile and an index.html in my local environment. Both files are needed because the Dockerfile defines how to build my custom container, and index.html provides the web content it serves.

My Docker file tells docker how to build my image and to use the index.html file I created as the web page that will be served.

I also set up an ECR repository
ECR stands for Elastic Container Registry. It is important because it makes it easy for one to store, manage, and deploy their container images.

Set Up AWS CLI Access

AWS CLI can let me run ECR commands
AWS CLI is a terminal tool to manage AWS services. The CLI asked for my credentials because browser logins arenʼt shared, so it needs its own access keys to authenticate.

To enable CLI access, I set up a new IAM user with AmazonEC2ContainerRegistryFullAccess permission. I also set up an access key for this user, which means the CLI can authenticate to AWS.

To pass my credentials to the AWS CLI, I ran the command aws configure. I had to provide my Access Key ID, Secret Access Key, the AWS region code for my repository, and optionally an output format.

Pushing My Image to ECR

Push commands in Amazon ECR (Elastic Container Registry) are the specific Docker commands you run to upload — or “push” — your container image from your local machine into an ECR repository in AWS.

There are three main push commands
To authenticate Docker with my ECR repo, I used the command 'aws ecr get-login password --region | docker login --username AWS --password-stdin .dkr.ecr..amazonaws.com'.

To push my container image, I ran the command 'docker push .dkr.ecr. .amazonaws.com/maven-cross-account-docker-app:latest'. Pushing means uploading my local image to Amazon ECR for others to pull.

When I built my image, I tagged it with the label latest. This means itʼs marked as the most current version, so anyone pulling latest will always get my newest build.

Resolving Permission Issues
When I first pulled my buddyʼs image, I got a 403 Forbidden error because their ECR repo is private and my AWS account didnʼt yet have permission. They had to update the repo policy to allow access.

To resolve each otherʼs pull errors, we updated our ECR repo policies to add each otherʼs IAM ARNs with permissions to pull images, enabling cross
account access to our private repositories.

Key Takeaways

🔥 The most challenging part of the project was setting permissions to pull an image directly from a private ECR repository in another account and another region.

🔥 While it was possible for me to do that, the best practice for deploying applications that span accounts and regions is to use Amazon ECR Private Image Replication.

You can configure a replication rule in the source account/region to automatically copy images to a destination ECR repository in the pulling account/region.

Benefits of Replication are:

Lower Latency
Cost Optimization
Resiliency

🌟 Something that stood out for me was the commands that look like Linux (bash) based succeeded on Windows PowerShell because the AWS CLI and Docker CLI are cross-platform tools, and PowerShell supports the same piping mechanism as Linux shells. As long as the required tools are installed and configured, this command is platform-agnostic and will work on both Windows and Linux.

🤝In the next project, I'm going to start a 7days DevOps series.

Deploy an App with Docker

Hyelngtil Isaac — Tue, 12 Aug 2025 14:19:40 +0000

Introducing Today's Project!

What is Docker?
Docker is a tool for building and running self-contained environments that package your app and its dependencies. I used Docker to create a custom container image that served a webpage, tested locally, and deployed it to AWS Elastic Beanstalk live.

One thing I didn't expect.
One thing I didn't expect in this project was that AWS Elastic Beanstalk would automatically create an S3 bucket and upload my zipped Docker project file to it. That behind-the-scenes setup made deployment smoother than I anticipated.

It took me about 90 minutes to complete the project, from installing Docker to deploying the app with Elastic Beanstalk.

Understanding Containers and Docker

Containers
Containers are packages that include an app and all its dependencies. They are useful because they ensure the app runs consistently anywhere.

A container image is a file that bundles an app with everything it needs to run like code, libraries, and settings so it behaves the same on any system.

Docker
Docker is a tool for creating, managing containers, and packages that hold everything an app needs to run. Docker Desktop is its user-friendly interface for local use.

The Docker daemon is the background service that builds, runs, and manages
containers, powering Docker commands and enabling app deployment

Running an Nginx Image

Nginx is a fast, lightweight web server used to serve web pages. In this project, it's the container image that helps you test Docker by instantly loading a webpage.

The command I ran to start a new container was 'docker run -d -p 80:80 nginx'. It launched an Nginx container in the background and mapped port 80 so I could view the webpage locally'

Creating a Custom Image

The Dockerfile is a text file with instructions to build a container image, like replacing Nginxʼs default page with your own and exposing port 80.

My Dockerfile tells Docker three things: use the official Nginx image, copy in my custom webpage to replace the default, and expose port 80 to serve it online.

The command I used to build a custom image with my Dockerfile was 'docker build -t my-web-app .' The '.' at the end of the command means Docker uses the current folder where the Dockerfile and 'index.html' live to build the image.

Running My Custom Image

There was an error when I ran my custom image because another container was already using port 80. I resolved this by stopping the active container in Docker Desktop and restarting mine.

In this example, the container image is the blueprint that defines what goes into the container, like the Nginx base and my custom webpage. The container is the running instance created from that image, serving my site locally.

Elastic Beanstalk

Elastic Beanstalk is an AWS service that helps you deploy containerized apps to the cloud easily, handling servers, scaling, and app health for you.

Deploying my custom image with Elastic Beanstalk took me just a few minutes 10 - 15, AWS handled setup, so my containerized app went live fast.

🤝In the next project, I'm going to demonstrate how to 'Deploy an App Across Accounts.'

DEV Community: Hyelngtil Isaac

Secure Secrets with Secrets Manager

Introducing Today's Project!

Tools and concepts

Project reflection

Hardcoding credentials

Using my own AWS credentials

Pushing Insecure Code to GitHub

Secrets Manager

Updating the web app code

Resetting the repository

Key Takeaways

🤝Next in the series builds on this, which will be Build a Security Monitoring System

Threat Detection with GuardDuty

Introducing Today's Project!

Tools & Concepts

Project Setup

What is GuardDuty?

Attack Phase 1: SQL Injection

Attack Phase 2: Command Injection

Attack Verification

Using CloudShell to Escalate the Attack

GuardDuty's Findings

S3 Malware Protection

Key Takeaways

Encrypt Data with AWS KMS

Introducing Today's Project!

Tools and concepts

Project reflection

Encryption and KMS

Encrypting Data

Data Visibility

Denying Access

Granting Access

Query Data with DynamoDB

Introducing Today's Project!

What is Amazon DynamoDB?

How I used Amazon DynamoDB in this project

Querying DynamoDB Tables

Limits of Using DynamoDB

Running Queries with CLI

Transactions

Load Data into a DynamoDB Table

Introducing Today's Project!

What is Amazon DynamoDB?

How I used Amazon DynamoDB in this project

One thing I didn't expect in this project was...

Create a DynamoDB table

Read and Write Capacity

Using CLI and CloudShell

Loading Data with CLI

Observing Item Attributes

Benefits of DynamoDB

Connect a Web App to Amazon Aurora

Introducing Today's Project!

What is Amazon Aurora?

How I used Amazon Aurora in this project

Creating a Web App

Connecting my Web App to Aurora

My Web App Upgrade

Testing my Web App

Aurora Database with EC2

Connect a Web App to Amazon Aurora

Introducing Today's Project!

What is Amazon Aurora?

How I used Amazon Aurora in this project

One thing I didn't expect in this project

In the first part of my project

Halfway through I stopped!

Features of my EC2 instance

Then I could finish setting up my database

AWS Databases!

I'm exploring AWS Databases!

Automated Deployment Bash Script: Deploying a Flask App to AWS

Task Overview

Prerequisites

The Bash Script: deploy.sh

How It Works: Step-by-Step

AWS-Specific Adaptations

Lessons Learned

The Bash Script: `deploy.sh`