DEV Community

Cover image for Automated End-to-End CI/CD Pipeline via AWS CodePipeline
Hyelngtil Isaac
Hyelngtil Isaac

Posted on Originally published at hyelngtil.awstech

Automated End-to-End CI/CD Pipeline via AWS CodePipeline

#cicd #aws #automation #github

Project Scope: A comprehensive implementation guide for building a fully automated continuous integration and continuous deployment (CI/CD) pipeline using AWS developer tools and GitHub: AWS CodeArtifact, AWS CodeBuild, and AWS CodeDeploy via AWS CodePipeline.

1. Executive Summary & Project Reflection

Overview:
This runbook details the end-to-end architecture and deployment of an automated CI/CD pipeline. The project is broken down into six distinct phases: provisioning a cloud-based development environment, integrating source control, securing package dependencies, configuring continuous integration, defining continuous deployment, and finally, orchestrating the entire workflow via an event-driven pipeline.

Key Tools and Concepts:
Services utilized include Amazon EC2, AWS IAM, GitHub, AWS CodeArtifact, Amazon S3, AWS CodeBuild, AWS CloudFormation, AWS CodeDeploy, and AWS CodePipeline. Key concepts mastered include Infrastructure as Code (IaC), event-driven triggers, immutable artifact transitions, least-privilege IAM enforcement, and disaster recovery via automated rollbacks.

Project Reflection:
This architecture requires approximately 3 hours to provision from scratch across all phases. The most challenging aspect was architecting fine-grained, cross-service IAM policies to enforce least-privilege access across pipeline stages. The most rewarding milestone was successfully troubleshooting permission errors, watching event-driven GitHub trigger automated runs, and seeing the entire end-to-end CI/CD workflow execute and deploy to the target EC2 fleet with zero manual intervention.

2. Prerequisites & Baseline Requirements

Before initializing the pipeline, the following baseline configurations must be established:

  1. AWS Account: Active AWS account with IAM Admin privileges (avoid using the Root user).
  2. Local IDE: Antigravity IDE or VS Code installed locally with the "Remote - SSH" extension enabled.
  3. GitHub Account: Active GitHub account for source control integration.

Part 1: Set Up a Web App in the Cloud

To ensure a consistent and isolated development environment, the project begins by provisioning a cloud-based IDE host to write and compile the application code.

1.1 Development EC2 Instance Provisioning

The development environment is hosted on an Amazon EC2 instance.

  • Instance Setup: Provisioned an Amazon Linux 2023 EC2 instance (t2.micro or t3.micro) depending on the Region.
  • Security & Access: Generated a new RSA .pem key pair (keypair.pem) for secure SSH access.
  • Network Inbound Rules: Restricted the Security Group inbound rules to allow SSH (Port 22) traffic strictly from the developer's local IP address (appending /32 for CIDR notation).

1.2 Remote SSH & Environment Bootstrapping

To interact with the EC2 instance securely, we configure an SSH tunnel from the local Antigravity IDE environment.

  1. Secure the Key Pair: Locally stored the .pem file in a dedicated DevOps directory and restricted file permissions to enforce read-only access for the current user. * Mac/Linux: chmod 400 keypair.pem * Windows (PowerShell): icacls "keypair.pem" /inheritance:r /grant:r "$($env:USERNAME):R"

  2. SSH Tunneling: Utilized the Antigravity IDE Remote - SSH extension to connect to the instance using the command:

    ssh -i keypair.pem ec2-user@<YOUR_EC2_PUBLIC_IPV4_DNS>

1.3 Toolchain Installation & Application Scaffolding

Once tunneled into the EC2 instance, the environment requires specific tools to build Java applications.

  1. Install Java & Maven:
    Bootstrapped the instance by installing Java 8 (Amazon Corretto) and Apache Maven 3.5.2 via the terminal. Export the $JAVA_HOME and $PATH environment variables within ~/.bashrc to ensure persistence across sessions.

    # Maven
wget https://archive.apache.org/dist/maven/maven-3/3.5.2/binaries/apache-maven-3.5.2-bin.tar.gz
sudo tar xzf apache-maven-3.5.2-bin.tar.gz -C /opt
echo 'export PATH=/opt/apache-maven-3.5.2/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

#Java
sudo dnf install -y java-1.8.0-amazon-corretto-devel
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64
export PATH=$JAVA_HOME/jre/bin/:$PATH

# Verification Commands
java -version
mvn -v

  2. Generate the Web App:
    Utilized Maven's archetype:generate command to scaffold a standard Java web application directory structure.

    mvn archetype:generate \
   -DgroupId=com.hye.app \
   -DartifactId=hye-web-app \
   -DarchetypeArtifactId=maven-archetype-webapp \
   -DinteractiveMode=false

Part 2: Connect a GitHub Repo with AWS

Version control is the foundation of any CI/CD pipeline. This phase establishes the upstream repository, ensuring the web application's codebase is securely tracked and centralized before being consumed by downstream CI/CD services.

2.1 Git Initialization & Upstream Connection

To bridge the local development environment on the EC2 instance with GitHub, we initialize a local repository and establish the remote origin.

  1. Local Git Setup:
    Within the Antigravity IDE terminal (connected to the EC2 instance via SSH), navigate to the root of the newly scaffolded Maven project (hye-web-app). Initialize the repository and stage the files:

    cd hye-web-app
git init
git add .
git commit -m "Initial commit"

  2. GitHub Repository Creation:
    Navigate to GitHub and create a new private repository (e.g., hye-web-app). This acts as the central source of truth for the CI/CD pipeline.

  3. Push to Upstream:
    Link the local EC2 directory to the GitHub repository and push the master branch using your secure GitHub credentials (Personal Access Token):

    git remote add origin [https://github.com/](https://github.com/)<your-username>/hye-web-app.git
git branch -M master
git push -u origin master

Part 3: Secure Packages with CodeArtifact

To maintain software supply chain security and ensure deterministic builds, external Maven dependencies must be proxied and cached through a secure, private repository rather than downloading them directly from the public internet during every build.

3.1 CodeArtifact Repository Setup

  1. Provisioning: In the AWS Console, navigate to AWS CodeArtifact and create a new domain (e.g., hye-cicd-domain) and a repository (e.g., hye-cicd-codeartifact-repo).
  2. Upstream Configuration: Ensure the repository is configured with maven-central-store as its upstream connection so it can successfully proxy public Java packages.

3.2 IAM Role & Policy Configuration

The EC2 instance requires explicit permissions to authenticate with CodeArtifact. We enforce least-privilege access using AWS IAM.

  1. Create IAM Policy: Create a custom JSON policy (hye-codeartifact-access-policy) granting token retrieval and read access:

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "codeartifact:GetAuthorizationToken",
                "codeartifact:GetRepositoryEndpoint",
                "codeartifact:ReadFromRepository"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "sts:GetServiceBearerToken",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "sts:AWSServiceName": "codeartifact.amazonaws.com"
                }
            }
        }
    ]
}

  2. Attach to EC2: Create an IAM Role (Hye-DevOps-Instance-CodeArtifact-Role) with EC2 as the trusted entity, attach the policy above, and map this role to the running development EC2 instance via the EC2 console (Actions > Security > Modify IAM role).

3.3 Maven Authentication & Configuration

With IAM permissions in place, Maven must be configured to route package requests through the CodeArtifact endpoint.

  1. Generate Auth Token:
    In the Antigravity IDE terminal (EC2 instance), use the AWS CLI to extract a temporary CodeArtifact authorization token and export it as an environment variable:

    export CODEARTIFACT_AUTH_TOKEN=`aws codeartifact get-authorization-token --domain hye-cicd-domain --domain-owner <YOUR_ACCOUNT_ID> --region <YOUR_REGION> --query authorizationToken --output text`

  2. Configure settings.xml:
    In the root of your project (hye-web-app), create a settings.xml file. Populate it with the CodeArtifact connection instructions (provided in the CodeArtifact console), ensuring the <password> tag references the environment variable: ${env.CODEARTIFACT_AUTH_TOKEN}.

  3. Compile & Verify:
    Execute a Maven compile command pointing to the custom settings file to force the application to download dependencies via the private CodeArtifact repository:

    mvn compile -s settings.xml

Part 4: Continuous Integration with CodeBuild

This phase automates the compilation, testing, and packaging of the web application into an immutable artifact. By utilizing AWS CodeBuild, we provision an isolated, stateless Linux container that pulls the source code, resolves dependencies securely via CodeArtifact, and compiles a deployable target.

4.1 S3 Artifact Store Provisioning

Before creating the build project, we need a centralized staging area to store the compiled output.

  1. Provision S3 Bucket: Navigate to the Amazon S3 console and create a new bucket (e.g., hye-codebuild-artifact-store).
  2. Configuration: Leave all settings as default (Block all public access enabled). This bucket will serve as the secure artifact transition mechanism between the Build and Deploy stages of the pipeline.

4.2 CodeBuild Project Initialization

  1. Create Project: Navigate to AWS CodeBuild and create a new build project named hye-cicd-codebuild-project.
  2. Source Control Integration:
    • Set the Source provider to GitHub.
    • Connect CodeBuild to your GitHub account using the GitHub App connection.
    • Select the private repository you created in Part 2 (hye-web-app).

  1. Environment Settings:
    • Environment image: Managed image
    • Operating system: Amazon Linux
    • Runtime: Standard
    • Image: Select the latest aws/codebuild/amazonlinux2-x86_64-standard:corretto8 image.
    • Service role: Select New service role (CodeBuild will automatically generate a least-privilege IAM role for this project).

4.3 IAM Role Modification (CodeArtifact Authorization)

The CodeBuild container must securely pull Maven dependencies from our private CodeArtifact repository. We must explicitly grant CodeBuild's newly created IAM service role access to CodeArtifact.

  1. Locate Service Role: In the IAM Console, find the role automatically generated by CodeBuild (e.g., Hye-cicd-CodeBuild-Project-Role).
  2. Attach Policy: Attach the hye-codeartifact-access-policy (created in Part 3) to this role. This ensures the build container has the exact same least-privilege token retrieval permissions as our development EC2 instance.

4.4 Define Build Directives (buildspec.yml)

AWS CodeBuild requires a declarative YAML file to execute build phases.

  1. Create Buildspec: In your local Antigravity IDE terminal, create a buildspec.yml file in the root directory of your project.

  2. Author Directives: Define the installation, pre-build (CodeArtifact authentication), build (Maven compile), and post-build (Artifact packaging) phases:

    version: 0.2

phases:
  install:
    runtime-versions:
      java: corretto8
  pre_build:
    commands:
      - echo Logging in to AWS CodeArtifact...
      - CODEARTIFACT_AUTH_TOKEN=`aws codeartifact get-authorization-token --domain hye-cicd-domain --domain-owner <YOUR_ACCOUNT_ID> --region <YOUR_REGION> --query authorizationToken --output text`
      - export CODEARTIFACT_AUTH_TOKEN
  build:
    commands:
      - echo Build started on `date`
      - mvn clean install -s settings.xml
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Packaging artifacts...
      - mvn package -s settings.xml
      - chmod +x scripts/*.sh
artifacts:
  files:
    - target/*.war
    - scripts/**/*
    - appspec.yml
  discard-paths: no

  3. Commit & Push: Add, commit, and push the buildspec.yml file to the GitHub master branch so CodeBuild can read the execution instructions.

4.5 Build Execution & Artifact Verification

  1. Trigger Build: In the AWS CodeBuild console, manually click Start build to validate the configuration.
  2. Monitor Logs: CodeBuild will provision the container, download the source from GitHub, authenticate with CodeArtifact, and execute the Maven build. Monitor the real-time Phase Details for success.
  3. Artifact Output: Upon a successful build phase, CodeBuild extracts the compiled .war file (defined in the artifacts block of the buildspec) and prepares it for downstream deployment.

Part 5: Deploy a Web App with CodeDeploy

This phase provisions the live production infrastructure and automates the continuous delivery process. Instead of manually copying files and restarting services, AWS CodeDeploy will orchestrate the deployment of our compiled .war artifact directly to the target web servers with zero downtime.

5.1 Infrastructure as Code (CloudFormation)

To ensure our production environment is reproducible and isolated from our development environment, we will provision the target EC2 fleet using Infrastructure as Code (IaC).

  1. Navigate to CloudFormation: Open the AWS CloudFormation console and click Create stack > With new resources (standard).
  2. Upload Template: Select Upload a template file and upload your declarative YAML configuration file hye-web-app.yaml. This template is pre-configured to:
    • Provision a production-ready Amazon Linux EC2 instance.
    • Automatically install the AWS CodeDeploy Agent via user data scripts.
    • Attach the necessary IAM Instance Profile allowing S3 read access (to fetch the artifact).
    • Configure Security Groups to allow inbound HTTP (Port 80) and SSH (Port 22) traffic.
  3. Configure Stack Details:
    • Stack name: hye-cicd-ec2-server-stack
    • MyIP: Enter your local IP address appended with /32 (e.g., 192.168.1.1/32) to secure SSH access.
  4. Execute Deployment: Acknowledge the IAM resource creation capabilities and click Submit. Wait for the stack status to reach CREATE_COMPLETE.

5.2 CodeDeploy Orchestration Setup

With the infrastructure running, we must configure CodeDeploy to target the new instances.

  1. Create Application:
    • Navigate to AWS CodeDeploy > Applications > Create application.
    • Application name: hye-cicd-codedeploy-application
    • Compute platform: EC2/On-premises

  1. Create Deployment Group:
    • Inside the application, click Create deployment group.
    • Deployment group name: hye-cicd-codedeploy-deployment-group
    • Service role: Select the IAM role granting CodeDeploy access to read target instances (e.g., Hye-cicd-CodeDeploy-Role).

*   **Environment configuration:** Select **Amazon EC2 instances**. Use Tag keys to target the instance provisioned by CloudFormation (e.g., Key: `role`, Value: `webserver`).
*   **Agent configuration:** Select `Never` (the CloudFormation template already installed the CodeDeploy agent).
*   **Load balancer:** Uncheck 'Enable load balancing' (we are using a single instance for this architecture).
Enter fullscreen mode Exit fullscreen mode

5.3 Define Deployment Directives (appspec.yml)

AWS CodeDeploy relies on an AppSpec file to understand exactly where to place files and which lifecycle scripts to run during the deployment.

  1. Create appspec.yml: In your local Antigravity IDE terminal, create an appspec.yml file at the absolute root of your project directory.

  2. Author Directives: Define the OS, files destination, and the lifecycle hooks.

    version: 0.0
os: linux
files:
  - source: /target/hye-web-app.war
    destination: /usr/share/tomcat/webapps/
hooks:
  BeforeInstall:
    - location: scripts/stop_server.sh
      timeout: 300
      runas: root
  ApplicationStart:
    - location: scripts/start_server.sh
      timeout: 300
      runas: root

  3. Create Lifecycle Scripts: Create a scripts folder and add the referenced bash files (stop_server.sh and start_server.sh). These scripts instruct the target instance to stop the Tomcat web server, clear the old cache, and start it back up once the new .war file is unpacked.

  4. Commit & Push: Add, commit, and push the appspec.yml and scripts/ directory to the GitHub master branch.

5.4 Manual Deployment Verification

Before automating the workflow, we must verify that CodeDeploy can successfully execute the AppSpec directives using the latest S3 artifact.

  1. Create Deployment: In the CodeDeploy console, navigate to your deployment group and click Create deployment.
  2. Revision Type: Select My application is stored in Amazon S3.
  3. Revision Location: Paste the S3 URI of the latest .zip artifact compiled by CodeBuild in Part 4.

  1. Deploy: Click Create deployment.
  2. Monitor Lifecycle Events: CodeDeploy will execute the AppSpec hooks. Monitor the stages (DownloadBundle, BeforeInstall, Install, ApplicationStart).
  3. Verify Live App: Once successful, navigate to the production EC2 instance's Public IPv4 DNS in your browser. You should see the Java web application rendering live.

Part 6: Build a CI/CD Pipeline with AWS

The final phase unites all previously configured components (GitHub, CodeArtifact, CodeBuild, CodeDeploy) into a single, fully automated, event-driven orchestration engine using AWS CodePipeline.

6.1 Pipeline Initialization & State Management

When creating the pipeline, specific execution and authentication parameters must be set to ensure deployment consistency and security.

  1. Create Pipeline: Navigate to AWS CodePipeline and select Create pipeline.
  2. Pipeline Settings:
    • Pipeline category: Buid custom pipeline 

*   **Pipeline name:** `hye-cicd-codepipeline`
*   **Execution mode:** `Superseded`. This mode ensures that newer pipeline executions automatically cancel and override in-flight runs, guaranteeing only the absolute latest commit is deployed.
*   **Service role:** Select **New service role** to generate least-privilege IAM permissions.
Enter fullscreen mode Exit fullscreen mode

6.2 Stage 1: Source Control Integration

The Source stage defines the trigger mechanism, monitoring the specified Git branch for commit events via webhooks.

  1. Source Provider: Select GitHub (Version 2) and authorize the AWS Connector for GitHub.
  2. Repository & Branch: Select the hye-web-app repository and the master branch.
  3. Output Artifact: Set the output artifact format to CodePipeline default (SourceArtifact).
  4. Webhook events: Enable the Start your pipeline on push and pull request events check-box.

6.3 Stage 2: Continuous Integration (Build)

The Build stage is responsible for transforming raw source code into a deployable package.

  1. Build Provider: Select AWS CodeBuild.
  2. Region & Project: Select your region and the hye-cicd-codebuild-project CodeBuild project created in Part 4.
  3. Input & Output:
    • Input artifacts: Map to SourceArtifact (the ZIP file generated by Stage 1).
    • Output artifacts: Define a new namespace, BuildArtifact (this represents the compiled .war file).

6.4 Stage 3: Continuous Deployment (Deploy)

The final stage provisions the compiled application artifact to the live EC2 infrastructure.

  1. Deploy Provider: Select AWS CodeDeploy.
  2. Application & Group: Select the hye-cicd-codedeploy-application application and the hye-cicd-codedeploy-deployment-group deployment group created in Part 5.
  3. Input Artifact: Map to BuildArtifact (the compiled package generated by Stage 2).

7. Execution, Testing, & Validation

To prove the pipeline's end-to-end automation, a live code change must be pushed to the upstream repository.

7.1 Triggering the Pipeline via Code Change

  1. Open the application source code in your local Antigravity IDE terminal.
  2. Navigate to the primary web page file: src/main/webapp/index.jsp.

  3. Inject a visible UI change, for example:

    <p>If you see this lines, that means your latest changes are automatically deployed into production by CodePipeline!</p>
<p>If you see this lines, that means your latest changes are automatically deployed into production by CodePipeline for the second time!</p>
<p>That's how easy it can get.</p>

  4. Commit and push the code to the tracked branch:

    git add .
git commit -m "Update index.jsp to verify automated CI/CD deployment."
git push

7.2 Monitoring Automated Orchestration

Immediately upon pushing the code, navigate to the AWS CodePipeline console.

  • The GitHub webhook will have dispatched a payload, initiating a new execution graph.
  • The commit message under the Source stage will reflect your recent git commit.
  • Monitor cross-stage transitions as CodeBuild compiles the WAR package and CodeDeploy initiates the in-place deployment to the EC2 fleet.

7.3 Live Production Verification

  1. Locate the Public IPv4 DNS of your production EC2 instance.
  2. Navigate to this address in a standard web browser.
  3. The updated application revision (featuring the newly added HTML paragraph tag) will render live, validating zero-touch, zero-downtime release orchestration.

8. Disaster Recovery: Automated Rollback Validation

A critical component of a robust CI/CD pipeline is the ability to recover from bad deployments instantly. To validate disaster recovery protocols, a manual rollback is initiated to simulate an emergency reversion.

8.1 Executing the Rollback

  1. In the CodePipeline console, locate the Deploy stage.
  2. Click the options menu (three dots) on the Deploy stage card and select Start rollback.
  3. Select the previous stable execution ID from the dropdown menu and confirm the rollback.

8.2 Validating State Isolation and Recovery

  • Result: CodeDeploy successfully restores the last known good deployment artifact to the EC2 instance. Navigating to the EC2 instance's Public IPv4 DNS confirms the web page displays the previous version (the test paragraph tag is removed).
  • State Isolation Proof: The Source and Build stages are entirely unaffected. They retain the latest Git commit SHA and message, while only the Deploy stage reverts its execution state to the previous stable revision. This proves that CodeDeploy rollback actions are isolated to the deployment environment and do not destructively invalidate S3 build artifacts or overwrite the upstream GitHub repository state.

Congratulations! You now have a fully functional end-to-end CI/CD pipeline for a Java web application hosted on AWS!

Top comments (0)