DEV Community: Gus

Automating the Cloud Resume Challenge: Implementing CI/CD with GitHub Actions

Gus — Thu, 08 Aug 2024 17:34:15 +0000

Cloud Resume Challenge - Part 2

Introduction

In the first part of this series, we walked through building a cloud-native resume website using various AWS services. Now, we'll take our project to the next level by implementing Continuous Integration and Continuous Deployment (CI/CD) using GitHub Actions. This automation is crucial for maintaining and updating our cloud resume efficiently.

CI/CD is a modern software development practice that emphasizes automating the stages of app development. In the context of our Cloud Resume Challenge, it means we can update our resume or make changes to our backend code, push these changes to GitHub, and have them automatically deployed to our AWS infrastructure.

Why CI/CD Matters in Cloud Development

Before we dive into the implementation, let's discuss why CI/CD is so important:

Consistency: Automated deployments ensure that every change is applied consistently across your infrastructure.
Efficiency: Manual deployments are time-consuming and prone to human error. Automation saves time and reduces mistakes.
Rapid Iteration: With CI/CD, new features and bug fixes can be deployed quickly and safely.
Best Practices: Implementing CI/CD encourages good development practices like frequent commits, comprehensive testing, and code reviews.
Scalability: As your project grows, CI/CD pipelines can easily scale to accommodate more complex deployment processes.

Setting Up the GitHub Repositories

For this project, we'll use two separate repositories:

Frontend Repository: Contains the HTML, CSS, and JavaScript files for the static website.
Backend Repository: Houses the AWS CDK code for the Lambda function, API Gateway, and DynamoDB table.

This separation allows us to manage and version control our frontend and backend code independently.

Implementing CI/CD for the Frontend

Let's start by setting up a GitHub Actions workflow for our frontend. Create a new file in your frontend repository at .github/workflows/deploy-frontend.yml:

name: Deploy Frontend

on:
  push:
    branches: [ main ]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3

    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: us-east-1

    - name: Deploy to S3
      run: aws s3 sync . s3://${{ secrets.S3_BUCKET }} --delete

    - name: Invalidate CloudFront
      run: |
        aws cloudfront create-invalidation --distribution-id ${{ secrets.CLOUDFRONT_DISTRIBUTION_ID }} --paths "/*"

This workflow does the following:

Triggers on pushes to the main branch
Sets up AWS credentials (which we'll configure in GitHub secrets)
Syncs the repository contents to the S3 bucket
Invalidates the CloudFront cache to ensure the latest version is served

Implementing CI/CD for the Backend

For the backend, we'll create a similar workflow. Create a new file in your backend repository at .github/workflows/deploy-backend.yml:

name: Deploy Backend

on:
  push:
    branches: [ main ]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3

    - name: Set up Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '16'

    - name: Install dependencies
      run: npm ci

    - name: Run tests
      run: npm test

    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: us-east-1

    - name: Deploy to AWS
      run: npx cdk deploy --require-approval never

This workflow:

Triggers on pushes to the main branch
Sets up Node.js
Installs dependencies
Runs tests (which you should implement)
Sets up AWS credentials
Deploys the CDK stack

Managing Secrets

To keep our sensitive information secure, we'll use GitHub Secrets. Go to your repository settings, click on "Secrets and variables", then "Actions", and add the following secrets:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
S3_BUCKET
CLOUDFRONT_DISTRIBUTION_ID

These secrets are securely encrypted and only exposed to the GitHub Actions workflow during execution.

Best Practices for CI/CD in Cloud-Native Applications

Keep Secrets Secure: Never hard-code sensitive information. Always use environment variables or a secrets management service.
Implement Robust Testing: Include unit tests, integration tests, and end-to-end tests in your CI pipeline.
Use Infrastructure as Code: Define your infrastructure using tools like AWS CDK or CloudFormation. This ensures consistency and allows version control of your infrastructure.
Monitor Your Pipelines: Set up notifications for failed deployments and regularly review your CI/CD logs.
Implement Gradual Rollouts: Consider using techniques like blue-green deployments or canary releases for safer deployments.

Challenges and Lessons Learned

Implementing CI/CD wasn't without its challenges. Here are some lessons learned:

IAM Permissions: Ensure your AWS IAM user has the correct permissions for deployment. It may take some trial and error to get this right.
Dependency Management: Keep your dependencies up-to-date in the CI environment. Consider using tools like Dependabot to automate this process.
Testing is Crucial: Invest time in writing comprehensive tests. They will save you from deploying bugs to production.
Cost Management: Be aware of the costs associated with your CI/CD pipeline, especially if you're running extensive tests or deployments frequently.

Conclusion

Implementing CI/CD with GitHub Actions has significantly streamlined our development process for the Cloud Resume Challenge. It allows us to focus on writing code and making improvements, knowing that deployment is just a git push away.

This experience reinforces the importance of automation in cloud development and provides hands-on experience with industry-standard CI/CD practices. Whether you're working on a personal project or a large-scale application, investing time in setting up a robust CI/CD pipeline pays dividends in productivity and reliability.

Remember, CI/CD is not a one-time setup. Continue to refine your pipelines, add more tests, and optimize your workflows as your project evolves. Happy coding and deploying!

Mastering Cloud Security: My Journey Deploying OWASP Juice Shop on AWS ECS

Gus — Fri, 02 Aug 2024 18:12:27 +0000

In the dynamic world of cybersecurity, I've found that hands-on experience is crucial. That's why I embarked on a project to deploy OWASP Juice Shop, an intentionally vulnerable web application, on Amazon Web Services (AWS) using Elastic Container Service (ECS). In this post, I'll share why I chose this project, my reasons for selecting AWS and ECS, and what I've learned along the way.

The Learning Benefits of This Project

Real-world Application: OWASP Juice Shop isn't a simple "Hello World" app. It's a full-stack JavaScript web application mimicking a real e-commerce site, exposing me to vulnerabilities I might encounter in actual production environments.
Hands-on Experience: By deploying Juice Shop on AWS, I'm not just reading about cloud security - I'm actively implementing it. This practical experience has been invaluable for truly grasping the concepts and challenges involved.
Comprehensive Learning: This project has allowed me to touch on multiple aspects of IT and security, including cloud services, containerization, networking, and web application security. It's been a holistic learning experience bridging several crucial domains in modern tech stacks.
Safe Environment: Juice Shop provides me with a legal and safe environment to practice ethical hacking and security testing. I can explore vulnerabilities without the risks associated with probing production systems.

Why I Chose AWS

I selected Amazon Web Services as my cloud platform for several reasons:

Market Leader: As the largest cloud provider, experience with AWS is highly valued in the job market.
Comprehensive Services: AWS offers a vast array of services that allowed me to build a complete, production-like environment.
Robust Documentation: AWS's extensive documentation and learning resources made it easier for me to get started and deepen my knowledge.
Scalability: While my project starts small, AWS provides the capability to scale to enterprise-level deployments, allowing me to extrapolate my knowledge to larger scenarios.

My Decision to Use ECS (Elastic Container Service)

I chose ECS for container orchestration due to several advantages:

Simplified Orchestration: ECS abstracts away much of the complexity, allowing me to focus on deployment and security aspects.
Integration with AWS Services: ECS integrates seamlessly with other AWS services, providing a cohesive learning experience within the AWS ecosystem.
Fargate Option: Using ECS with Fargate allows for serverless container deployment, reducing the operational overhead and allowing me to focus on the application and its security.
Industry Relevance: Container orchestration is a highly sought-after skill, and experience with ECS provides me with valuable, transferable knowledge.

Why I Opted for Scripts Instead of Infrastructure as Code (IaC)

For this project, I decided to use shell scripts and JSON configuration files instead of IaC tools like CloudFormation, CDK, or Terraform. Here's why:

Learning Fundamentals: Using scripts allowed me to understand the basic AWS CLI commands and API interactions, providing a solid foundation before moving to more abstract IaC tools.
Simplicity: For a small-scale project like this, scripts offer a straightforward approach that's easy to understand and modify.
Direct Control: Writing scripts gave me direct control over each step of the deployment process, which was beneficial for learning how each AWS service works.
Gradual Learning Curve: This approach allowed me to gradually introduce AWS services and concepts without the additional complexity of learning an IaC tool simultaneously.

While IaC tools would be preferable for larger, production-grade deployments due to their state management and reproducibility features, using scripts has been an excellent starting point for my learning journey.

What I've Learned

This project has offered me a wealth of learning opportunities:

Cloud Architecture: I've gained understanding in designing and implementing a secure cloud architecture using VPCs, subnets, and security groups.
Container Deployment: I've learned how to deploy and manage containerized applications in a cloud environment.
Security Best Practices: I've implemented and understood AWS security best practices, including the principle of least privilege with IAM roles.
Networking in the Cloud: I've configured and managed networking in a cloud environment, including public and private subnets.
Scripting and CLI Usage: I've improved my scripting skills and become proficient with the AWS CLI.
Monitoring and Logging: I've set up and used CloudWatch for monitoring the application and infrastructure.
Web Application Security: I've gained hands-on experience with common web vulnerabilities by working through Juice Shop's challenges.
Cost Management: I've learned to understand AWS pricing models and optimize costs in cloud deployments.

Conclusion

Deploying OWASP Juice Shop on AWS ECS has been more than just a technical exercise - it's been a comprehensive learning journey touching on crucial aspects of modern application deployment and security. As an aspiring cybersecurity professional and cloud enthusiast, this project has provided me with valuable, hands-on experience that I'm sure will serve me well in my career.

I recognize that the cloud and cybersecurity landscapes are constantly evolving. This project has provided me with a solid foundation, but I know my learning doesn't stop here. I plan to use this as a springboard to dive deeper into areas that interest me, stay updated with the latest developments, and continue to practice and expand my skills.

Here's to continuous learning and staying secure!

Building a Cloud-Native Website with AWS: The Cloud Resume Challenge

Gus — Thu, 01 Aug 2024 21:10:49 +0000

AWS Cloud Resume Challenge Part 1

Introduction

As a tech enthusiast and cloud professional looking to expand my AWS skills, I recently took on the Cloud Resume Challenge. This project, created by Forrest Brazeal, is an excellent opportunity for anyone – from beginners to experienced professionals – to gain hands-on experience with a variety of AWS services.

The Cloud Resume Challenge is more than just a simple project; it's a comprehensive, real-world application of cloud technologies. Here's why it's such a valuable learning experience:

Practical Application: You're not just learning theory; you're building a functional, cloud-native application.
Service Integration: The challenge requires you to use multiple AWS services together, mimicking real-world scenarios.
Full-Stack Development: From frontend to backend, you'll touch every part of the application stack.
Best Practices: The project encourages the use of Infrastructure as Code (IaC) and CI/CD pipelines, essential skills in modern cloud development.
Portfolio Building: Upon completion, you have a tangible project to showcase to potential employers or clients.

In this blog post, I'll walk you through my journey of completing the Cloud Resume Challenge, detailing each step, the AWS services used, and the valuable insights gained along the way. Whether you're new to AWS or looking to deepen your expertise, I hope my experience will inspire and guide you in your own cloud learning journey.

Setting Up a Static Website with S3
Securing the Website with CloudFront
Configuring DNS with Route 53
Implementing a Visitor Counter
- DynamoDB for Data Storage
- Lambda for Backend Logic
- API Gateway for HTTP Endpoints
Frontend JavaScript Integration
Testing the JavaScript Code
Conclusion

Setting Up a Static Website with S3

Amazon S3 (Simple Storage Service) is the foundation of our cloud resume. It's a highly scalable object storage service that can host static websites efficiently and cost-effectively. Here's a step-by-step guide on how to set it up:

Create an S3 bucket:
- Log into the AWS Management Console and navigate to S3.
- Click "Create bucket" and choose a globally unique name (e.g., "my-cloud-resume-yourname").
- In the bucket settings, enable "Static website hosting".
- Set the index document to "index.html".
Upload your website files:
- Prepare your HTML resume. If you're not comfortable with HTML, you can use a template or convert your existing resume to HTML using online tools.
- Upload your HTML, CSS, and any JavaScript files to the bucket.
- Ensure the files have public read permissions.
Configure bucket policy:
- In the bucket permissions, add a bucket policy to allow public read access:



   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "PublicReadGetObject",
               "Effect": "Allow",
               "Principal": "*",
               "Action": "s3:GetObject",
               "Resource": "arn:aws:s3:::your-bucket-name/*"
           }
       ]
   }

Replace "your-bucket-name" with your actual bucket name.

Why S3 for static website hosting?

Scalability: S3 can handle any amount of traffic without you needing to manage servers.
Cost-effective: You only pay for the storage you use and the data transferred.
Simplicity: Easy to set up and requires minimal ongoing maintenance.

Securing the Website with CloudFront

While S3 provides a solid foundation, CloudFront provides a Content Delivery Network (CDN) service. This not only improves the performance of our website but also adds an extra layer of security. Here's how to set it up:

Create a CloudFront distribution:
- In the AWS Console, go to CloudFront and click "Create Distribution".
- Set the origin domain to your S3 bucket's website endpoint.
- Enable "Redirect HTTP to HTTPS" for added security.
Configure SSL/TLS certificate:
- Use AWS Certificate Manager (ACM) to create a free SSL/TLS certificate for your domain.
- In your CloudFront distribution settings, select this certificate.
Set up Origin Access Identity (OAI):
- Create an OAI and associate it with your distribution.
- Update your S3 bucket policy to only allow access from this OAI.

Why use CloudFront?

Improved Performance: CloudFront caches your content at edge locations worldwide, reducing latency for global visitors.
Enhanced Security: HTTPS encryption and the ability to configure security headers.
Cost Optimization: CloudFront can reduce your S3 data transfer costs.

Configuring DNS with Route 53

Now that we have our website hosted and a CDN set up, we need to give it a user-friendly domain name. This is where Route 53, AWS's Domain Name System (DNS) web service, comes into play. Here's how to set it up:

Create a hosted zone:
- In Route 53, create a hosted zone for your domain.
Update name servers:
- If your domain is registered elsewhere, update the name servers at your registrar to use Route 53's name servers.
Create record sets:
- Create an A record for your domain, aliasing it to your CloudFront distribution.
- If needed, create a CNAME record for www subdomain.

Why use Route 53?

Integration: Seamless integration with other AWS services.
Reliability: Backed by AWS's global infrastructure.
Advanced Features: Health checks, traffic flow, and domain registration in one place.

Implementing a Visitor Counter

Now comes the exciting part – adding dynamic functionality to our static website. We'll implement a visitor counter using a serverless architecture with DynamoDB, Lambda, and API Gateway. This showcases how even a simple static site can incorporate dynamic elements using cloud services.

DynamoDB for Data Storage

DynamoDB is AWS's fully managed NoSQL database service. It's perfect for our use case due to its scalability and low latency. Here's how to set it up:

Create a DynamoDB table:
- Name: "VisitorCount"
- Partition key: "id" (String)
- Add an item with id "visitors" and an attribute "count" set to 0.

Why DynamoDB?

Fully Managed: No servers to provision or manage.
Scalability: Automatically scales to handle any amount of traffic.
Performance: Single-digit millisecond latency at any scale.

Lambda for Backend Logic

AWS Lambda allows us to run code without provisioning or managing servers. Our Lambda function will update the visitor count in DynamoDB. Here's how to set it up:

Create a Lambda function:
- Runtime: Node.js 20.x
- Code:



   const { DynamoDBClient, UpdateItemCommand } = require('@aws-sdk/client-dynamodb');

   const client = new DynamoDBClient();

   exports.handler = async (event) => {
       const params = {
           TableName: process.env.TABLE_NAME,
           Key: { id: { S: 'visitors' } },
           UpdateExpression: 'ADD visitorCount :inc',
           ExpressionAttributeValues: { ':inc': { N: '1' } },
           ReturnValues: 'UPDATED_NEW'
       };

       try {
           const command = new UpdateItemCommand(params);
           const data = await client.send(command);

           const visitorCount = data.Attributes.visitorCount.N;

           return {
               statusCode: 200,
               headers: {
                   "Access-Control-Allow-Origin": "https://yourdomain.com",
                   "Access-Control-Allow-Headers": "Content-Type",
                   "Access-Control-Allow-Methods": "OPTIONS,POST"
               },
               body: JSON.stringify({ visitorCount: parseInt(visitorCount) })
           };
       } catch (error) {
           console.error('Error:', error);
           return {
               statusCode: 500,
               headers: {
                   "Access-Control-Allow-Origin": "https://yourdomain.com",
                   "Access-Control-Allow-Headers": "Content-Type",
                   "Access-Control-Allow-Methods": "OPTIONS,POST"
               },
               body: JSON.stringify({ error: 'Failed to update visitor count' })
           };
       }
   };

Set up environment variables:
- Add TABLE_NAME environment variable with your DynamoDB table name.
Configure permissions:
- Give the Lambda function permission to read/write to your DynamoDB table.

Why Lambda?

Pay per Use: You only pay for the compute time you consume.
No Server Management: AWS handles all the infrastructure management.
Automatic Scaling: Lambda automatically scales your application by running code in response to each trigger.

API Gateway for HTTP Endpoints

Amazon API Gateway acts as the "front door" for our Lambda function, allowing it to be triggered via HTTP requests. Here's how to set it up:

Create a new API:
- Choose HTTP API type for simplicity and cost-effectiveness.
Create a resource and method:
- Create a resource named "visitorcount".
- Add a POST method to this resource, integrating it with your Lambda function.
Enable CORS:
- In the API settings, enable CORS for your domain.
Deploy the API:
- Create a new stage (e.g., "prod") and deploy your API.

Why API Gateway?

Managed Service: No need to manage any infrastructure.
Scalability: Handles any number of API calls simultaneously.
Security: Provides authorization and access control.

Frontend JavaScript Integration

Now that our backend is set up, let's integrate it with our frontend:

Add HTML for the visitor counter:



   <div id="visitor-count">Visitors: <span id="count">0</span></div>

Add JavaScript to call our API:



   function updateVisitorCount() {
       fetch('https://your-api-gateway-url/visitorcount', {
           method: 'POST',
           headers: {
               'Content-Type': 'application/json'
           },
           mode: 'cors'
       })
       .then(response => response.json())
       .then(data => {
           document.getElementById('count').textContent = data.visitorCount;
       })
       .catch(error => console.error('Error:', error));
   }

   document.addEventListener('DOMContentLoaded', updateVisitorCount);

Replace 'https://your-api-gateway-url' with your actual API Gateway URL.

Testing the JavaScript Code

Testing is a crucial part of any development process. For our visitor counter, we can use Jest, a popular JavaScript testing framework. Here's how to set it up:

Install Jest: ```

npm install --save-dev jest


2. **Create a test file (e.g., `visitorCounter.test.js`):**

   ```javascript


   // Mock fetch function
   global.fetch = jest.fn(() =>
     Promise.resolve({
       json: () => Promise.resolve({ visitorCount: 5 }),
     })
   );

   // Import the function to test
   const { updateVisitorCount } = require('./visitorCounter');

   test('updates visitor count correctly', async () => {
     // Set up our document body
     document.body.innerHTML = '<div id="count">0</div>';

     // Call the function
     await updateVisitorCount();

     // Check that the count was updated
     expect(document.getElementById('count').textContent).toBe('5');
   });

Run the tests: ```

npx jest


This test mocks the fetch function to simulate an API call and checks if our updateVisitorCount function correctly updates the DOM with the returned count.

Additional testing examples could include:

- Testing error handling:
  ```javascript


  test('handles errors gracefully', async () => {
    global.fetch.mockImplementationOnce(() => Promise.reject("API is down"));

    console.error = jest.fn();

    await updateVisitorCount();

    expect(console.error).toHaveBeenCalled();
  });

Testing with different response values: ```javascript

test('handles large numbers correctly', async () => {
global.fetch.mockImplementationOnce(() =>
Promise.resolve({
json: () => Promise.resolve({ visitorCount: 1000000 }),
})
);

document.body.innerHTML = '<div id="count">0</div>';
await updateVisitorCount();
expect(document.getElementById('count').textContent).toBe('1000000');

});


These tests help ensure our visitor counter works correctly under various conditions, improving the reliability of our cloud resume.

## Conclusion

Completing the Cloud Resume Challenge is an excellent way to gain hands-on experience with a variety of AWS services. Throughout this project, we've touched on several key aspects of cloud computing:

1. **Static Web Hosting**: Using S3 to host our resume website.
2. **Content Delivery**: Implementing CloudFront to improve performance and security.
3. **DNS Management**: Configuring Route 53 for a custom domain.
4. **Serverless Computing**: Using Lambda for our backend logic.
5. **NoSQL Databases**: Utilizing DynamoDB for data storage.
6. **API Development**: Creating an API with API Gateway.
7. **Frontend Development**: Integrating our backend with JavaScript.
8. **Testing**: Implementing unit tests for our JavaScript code.

This challenge provides a comprehensive overview of building a cloud-native application, from frontend to backend. It's an excellent starting point for anyone looking to dive deeper into AWS services or to showcase their cloud skills to potential employers.

In the next part of this series, we'll explore how to automate the deployment of our cloud resume using CI/CD practices with GitHub Actions, further enhancing our cloud development skills.

Remember, the cloud is constantly evolving, and there's always more to learn. Keep exploring, keep building, and most importantly, keep challenging yourself!

DEV Community: Gus

Automating the Cloud Resume Challenge: Implementing CI/CD with GitHub Actions

Cloud Resume Challenge - Part 2

Introduction

Why CI/CD Matters in Cloud Development

Setting Up the GitHub Repositories

Implementing CI/CD for the Frontend

Implementing CI/CD for the Backend

Managing Secrets

Best Practices for CI/CD in Cloud-Native Applications

Challenges and Lessons Learned

Conclusion

Mastering Cloud Security: My Journey Deploying OWASP Juice Shop on AWS ECS

The Learning Benefits of This Project

Why I Chose AWS

My Decision to Use ECS (Elastic Container Service)

Why I Opted for Scripts Instead of Infrastructure as Code (IaC)

What I've Learned

Conclusion

Building a Cloud-Native Website with AWS: The Cloud Resume Challenge

AWS Cloud Resume Challenge Part 1

Introduction

Table of Contents

Setting Up a Static Website with S3

Securing the Website with CloudFront

Configuring DNS with Route 53

Implementing a Visitor Counter

DynamoDB for Data Storage

Lambda for Backend Logic

API Gateway for HTTP Endpoints

Frontend JavaScript Integration

Testing the JavaScript Code