DEV Community: Emmanuel K

Simplifying GitHub Actions Workflows Through Typescript

Emmanuel K — Wed, 30 Aug 2023 12:09:46 +0000

Quick Summary

Harness the power of Typescript for designing your GitHub Actions Workflows. Check out github-actions-workflow-ts

The Challenge with YAML

Despite its widespread use in DevOps and CI/CD pipelines—think Kubernetes, Jenkins, CircleCI, Bitbucket Pipelines, CloudFormation, and of course, GitHub Actions—YAML comes with its own set of complexities. Issues often arise from poor indentation, block referencing, and the tedious nature of code reuse.

Why Not YAML for GitHub Actions?

While YAML is somewhat versatile, it's not the most readable or maintainable choice for composing GitHub Actions Workflows. Attempting to implement complex functionality can make your YAML files convoluted, making the process of maintaining, updating or even remembering their function painful.

The Epiphany

Enter my introduction to AWS CDK a few years ago — a tool that lets you write infrastructure code in your preferred language and then compiles it into Cloudformation YAML. This sparked an idea: why not apply the same principle to GitHub Actions Workflows?

The Inspiration

Initially, I stumbled upon webiny/github-actions-wac, an incredible package that did exactly what I was envisioning. While it's a remarkable tool, I encountered some limitations concerning type exports, file naming, and directory placement. These led me down a path of extensive modifications and enhancements.

Introducing github-actions-workflow-ts

Instead of merely tweaking the existing package, I found myself reimagining many of its core functionalities. Thus, I introduce github-actions-workflow-ts — a package heavily inspired by the original but with expanded capabilities. While it's still in its beta stage (version 0.X.X), I'm close to releasing a stable 1.0.0 version. However, the current API is robust and already implemented in several of my projects.

Get Started Now

Curious to see it in action? Explore these Replit examples:

About the Author

I'm Emmanuel, passionate about software, AWS, and DevOps. If you found this article valuable, feel free to connect with me on LinkedIn or follow me on Twitter.

How to monitor long running ALTERS in AWS RDS MySQL 5.7+ using the Performance Schema

Emmanuel K — Fri, 07 Oct 2022 20:02:19 +0000

You've probably landed on this page because you have a MySQL RDS database that has one/many massive tables and anytime you run an ALTER it takes hours, maybe even days. Unfortunately, MySQL does not provide an easy way to monitor this and neither does RDS monitoring or RDS Performance Insights on AWS.

Luckily, since MySQL 5.7 there has been support for Progress information and that is implemented with the Performance Schema using the stage events. I recommend you read this excellent article for a detailed explanation (and also a better way to do the below in MySQL 8). In this article I will focus on MySQL 5.7

So how do you do it on RDS?

Set performance_schema parameter to 1 in your database's parameter group (⚠Requires instance reboot so that parameter is applied)

Once the parameter has been applied. Run the following queries

-- Enable events_stages_current to monitor threads
UPDATE performance_schema.setup_consumers 
SET ENABLED = 'YES' 
WHERE NAME = 'events_stages_current';

-- Check results of below to see that ENABLED 
-- and EnabledWithHierarchy are both "YES"
SELECT 
    NAME, 
    ENABLED, 
    sys.ps_is_consumer_enabled(NAME) AS EnabledWithHierarchy 
FROM performance_schema.setup_consumers 
WHERE NAME = 'events_stages_current';

If the above works then you are good to go. You can now start an alter and execute this query to see its progress

SELECT 
    stmt.THREAD_ID, 
    stmt.SQL_TEXT, 
    stage.EVENT_NAME AS State,
        stage.WORK_COMPLETED, 
    stage.WORK_ESTIMATED,
  ROUND(100*stage.WORK_COMPLETED/stage.WORK_ESTIMATED, 2) AS CompletedPct
FROM performance_schema.events_statements_current stmt
INNER JOIN performance_schema.events_stages_current stage
ON stage.THREAD_ID = stmt.THREAD_ID
AND stage.NESTING_EVENT_ID = stmt.EVENT_ID

-- ALTERNATIVELY
----------------
-- SELECT 
--  thd_id, 
--     conn_id, 
--     db, 
--     command, 
--     state, 
--     current_statement,
--     statement_latency, 
--     progress, 
--     current_memory, 
--     program_name
-- FROM sys.session
-- WHERE progress IS NOT NULL;

You should see something like this (where WORK_COMPLETED is how many rows have been altered and WORK_ESTIMATED is how many rows are left to alter and CompletedPct is the completion percent). Run this query everytine you need to see the updated values.

And that's it -- hopefully now you can monitor long running queries on an RDS database!

Hi I'm Emmanuel! I write about software, AWS and DevOps.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

How to get old LinkedIn app in China

Emmanuel K — Wed, 22 Dec 2021 20:41:34 +0000

TLDR -- The China-localised LinkedIn app has been replaced with InJobs app. If you are unfortunate like me, that localised LinkedIn app is not easily replaceable with a non-China-localised version on your phone (e.g. on most Samsung phones) -- which means you simply cannot use the LinkedIn app on your phone anymore because if you open it, you are prompted to download InJobs app instead. Scroll to 'The solution' section to see how to fix this.

The Problem

Recently, the LinkedIn mobile app exited the China market -- more specifically, they sunsetted the normal LinkedIn app and replaced it with a new application called InJobs specifically for China region.

So as the new InJob app rollout is underway, users of the old China-localised LinkedIn app are now blocked from logging in and are instead prompted to download the new InJobs app. This happened to me yesterday.

This would usually be fine because it means all you have to do is uninstall the China-localised LinkedIn application and install a non-China-localised LinkedIn application from the Google Playstore to continue using the LinkedIn app you are used to.

But... What happens when you cannot remove it? Unfortunately for most Samsung users, apps like Facebook (HK, TW, Rest of the world), LinkedIn (Mainland, HK, TW, Rest of the world) and many Microsoft apps are in most cases pre-installed bloatware. This means they cannot be uninstalled easily -- only disabled.

These are apps I use (or at least did not mind being bloatware) so this hasn't been an issue for me -- until now. Unfortunately for users who got their Samsung (and probably other phones) from China region (Mainland, HK, Taiwan) the uninstallable LinkedIn app is the now discontinued China-localised version.

Since I'm no longer in the China region (where I bought my phone), the InJobs app is useless to me. And even if I wasn't I would probably still want to use the regular LinkedIn app and not a stripped down version.

The Solution

Fortunately, there is a work-around. It is quite technical so I hope you know how to use the terminal.

NOTE: Don't attempt this if you don't know what you are doing or have never used the terminal. I am not responsible for you bricking yor phone.

Pre-requisites

Computer
Data wire
Your mobile phone
Internet

Initial Steps

Enable USB debugging on your Phone
https://www.verizon.com/support/knowledge-base-223020/
Download Android SDK platform tools for your operating system (I use Linux and it is what I will show from now on)
https://developer.android.com/studio/releases/platform-tools
Unzip platform tools you just downloaded
Enter the platform tools folder and open your terminal there
Connect your phone to computer and allow it to share data if prompted
Download the latest version of LinkedIn APK from APKPure. Download here. We need this non-China-localised version to overwrite the existing China localised version -- Don't worry its safe to install. Also, we only need it for a few minutes and all future versions will be from your trusted Google Playstore. Copy the location of the downloaded LinkedIn APK
Prepare your phone -- Set the screen timeout to be at least 10 minutes. You don't want your screen going odd during this process. Make sure you do not disconnect the wire during any of the following steps.

Next steps

Open your terminal and do the following

./adb devices

You should see your device connected.

Run the following command to ensure you grant permissions to run commands on your device

./adb shell pm list packages | grep linkedin

You should see that linkedin is installed

Install the LinkedIn apk you just downloaded

./adb install -r -d ~/Downloads/LinkedIn\ Jobs\ Business\ News_v4.1.649.1_apkpure.com.apk

In this command, -r will force the APK to update with the new APK and -d will force the phone to ignore the possibly lower LinkedIn version you want to install

If prompted to grant permission on your phone, ensure you grant it. If the command fails because of this, just run it again.
When it is done (may take a while -- give it up to 3 min) -- You should see success

That's it. You can now disconnect your phone from your computer and now you should have the correct non-China localised LinkedIn application installed on your phone.

If the version you downloaded from APKPure is lower than the latest version on Google Play, you will see a prompt to update the app in Google Play. Update it.

That's all.

Hi I'm Emmanuel! I write about software, devOps and other random technical stuff.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

Saving and Retrieving Well-known Text (WKT) in MySQL or Postgres with TypeORM

Emmanuel K — Wed, 09 Jun 2021 19:50:49 +0000

Sometimes in TypeORM you want to save location/coordinates in a type that can take advantage of GIS functions like distance, closeness etc MySQL and Postgres have these functions that get this information when you have geometry data column types.

This was surprisingly difficult to find information on so here's my implementation.

I assume you already have a TypeORM project set up.

Install these packages:

npm install wkx geojson 
npm install @types/geojson --save-dev

Then add this custom transformer that converts GeoJSON to WKT and WKT back to GeoJSON

Example GeoJSON:

const location = {
   type: 'point',
   coordinates: [-74.534, 39.123]
}

Example WKT (like you would use in MySQL query):

POINT(-74.534, 39.123)

This would be saved as BLOB type in MySQL

Files

First you need to add the transformer class

// ./src/lib/transformers.ts

import * as wkx from 'wkx'
import { Geometry } from 'geojson'
import { ValueTransformer } from 'typeorm/decorator/options/ValueTransformer'

/**
 * TypeORM transformer to convert GeoJSON to MySQL WKT (Well Known Text) e.g. POINT(LAT, LON) and back
 */
export class GeometryTransformer implements ValueTransformer {
    to(geojson: Geometry): string {
        return wkx.Geometry.parseGeoJSON(geojson).toWkt()
    }

    from(wkb: string): Record<string, any> | undefined {
        if(!wkb) return

        return wkx.Geometry.parse(wkb).toGeoJSON()
    }
}

Then inside your target entity class:

// ./src/entities/user.ts

import {
    Entity,
    PrimaryGeneratedColumn,
    Column,
    BaseEntity
} from 'typeorm'
import { GeometryTransformer } from '../transformers'
import { Geometry } from 'geojson'


@Entity({ name: 'user' })
export class User extends BaseEntity {
    @PrimaryGeneratedColumn()
    idUser!: number

    @Column({
        type: 'geometry',
        spatialFeatureType: 'Point',
        srid: 4326, // WGS84 reference system
        transformer: new GeometryTransformer(),
    })
    location?: Geometry
}

Insert GeoJSON

And then finally if you want to insert location data, insert geoJSON:

// ./src/services/user.service.ts

import { Geometry } from 'geojson'
import { getRepository } from 'typeorm'
import { User } from '../entities/user'

export const createUser = async (): Promise<User|undefined> => {

    const location: Geometry = {
        type: 'Point',
        coordinates: [-74.534, 39.123]
     }
    // you should validate the geojson here
    // https://www.npmjs.com/package/geojson-validation

    // if(!validGeoJson(location))throw new Error('invalid GeoJSON')

    return getRepository(User)
        .create({ location })
        .save()
}

Retrieve GeoJSON

And then when you query the object you will get location as type Geometry.

That's it!

Hi I'm Emmanuel! I write about Software and DevOps.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

Part 4 - Wordpress EC2 instance in ASG with RDS database and ALB- Awesome AWS CDK

Emmanuel K — Mon, 24 May 2021 10:48:41 +0000

Introduction
Abbreviations
Setup
- Setup AWS configuration and CDK
- Create a new project
- Project structure
- Config
VPC
- Setup
RDS
- Setup
Application Load Balancer
- Important Load Balancer concepts
- Setup
EC2 via Autoscaling Group
- Setup
- User script
- EC2 instance in ASG
Deployment
- Local machine
- Github Actions
- Destroying the stack
Final result
Conclusion
- Debugging
- Homework Assignments

Introduction

Hot off the heels (more than one month ago lol) of the last post, we're going to provision better infrastructure for our Wordpress powered website using the AWS CDK!

In this tutorial, you will learn how to:

setup a Wordpress EC2 instance in an Autoscaling Group (ASG) that is attached to an Application Load Balancer (ALB)
setup AWS RDSMySQL database using the CDK
place your RDS MySQL database in an isolated subnet in a custom VPC for better security
SSH into the Wordpress instance using AWS SSM and IAM credentials
add CI/CD to your deployment so that you can deploy to multiple environments via Github actions

All of this will be done in the CDK without needing you to even open the AWS web console. How awesome is that?

The repository of this project is emmanuelnk/wordpress-ec2-rds. You can find all the final code files here.

Alright, let's go!

Abbreviations

I may use certain abbreviations throughout this tutorial. When using AWS generally, these abbreviations are quite common due to the insanely verbose names AWS gives its services

Setup

Setup AWS configuration and CDK

As usual, you're going to have to ensure you have your AWS dev environment set up (crendentials and config).
See Awesome AWS CDK - Part 2 - Setting up AWS CDK for how to do this.

Create a new project

Make a new directory and init a new TS project:

mkdir wordpress-ec2-rds && cd wordpress-ec2-rds
cdk init --language typescript

NOTE: Sometimes it so happens that the cdk init installs an old verison of cdk-core. A version mismatch between @aws-cdk/core and @aws-cdk/aws-SOME_SERVICE can cause errors in Typescript.

If you get any weird TS errors while writing CDK code, try checking package.json to ensure you're using the latest version of @aws-cdk/core and that that version matches exactly with other CDK packages e.g. below aws-core and other aws-cdk packages are all on version 1.102.0

    // package.json

    "dependencies":  {
        "@aws-cdk/aws-ec2":  "^1.102.0",
        "@aws-cdk/aws-rds":  "^1.102.0",
        "@aws-cdk/aws-secretsmanager":  "^1.102.0",
        "@aws-cdk/core":  "^1.102.0",
        "dotenv":  "^8.4.0",
        "source-map-support":  "^0.5.16"
    }

To be on the safe side, everytime you start a new CDK project, just run:
```
npm install @aws-cdk/core@latest
```

Project structure

In the last tutorial, we put all of our infrastructure in one file. This is fine if you're provisioning a few resources. However it's always better to split your resources logically into different files to make the project easier to manage.

This time, we will create a folder called constructs (a construct is just a particular class that is created from a base service class) i.e. we will then create the following files:

    mkdir lib/constructs
    touch lib/constructs/ec2.ts
    touch lib/constructs/alb.ts
    touch lib/constructs/rds.ts
    touch lib/constructs/vpc.ts
    touch lib/config.ts
    touch .env

where each construct file will contains a specfic AWS service. These constructs will then be imported into the base infrastructure file that the CDK created for us (lib/wordpress-ec2-rds-stack.ts)

We also create a config.ts file to hold our configurations and a .env file to contain our envrionment variables.

(Optional) add a CI file for Github Actions. At the end of the tutorial I will show you how to deploy to either or both prod and dev using Github Actions

    mkdir -p .github/workflows/deploy.yml

These are thus the files we will concentrate on:

├── bin
│   └── wordpress-ec2-rds.ts       # entry file
├── lib
│   ├── config.ts                  # stack and account config
│   ├── constructs
│   │   └── ec2.ts
│   │   └── rds.ts
│   │   └── alb.ts
│   │   └── vpc.ts
│   └── wordpress-ec2-rds-stack.ts # where we import the constructs
├── test
│   └── wordpress-ec2-rds.test.ts  # where we test our infrastructure
├── .github
│   ├── workflows
│   │   └── deploy.yml             # CI/CD config file (Github Actions)
└── .env                           # environment variables

Config

Let's install some required dependencies first
```
npm install dotenv
```

Let's write our configuration file (lib/config.ts):

// lib/config.ts
import  *  as  dotenv  from  'dotenv'

// load our environment variables from .env
dotenv.config()

// this will be used in resource names
const  stage  =  process.env.STAGE  ||  'dev'

export  const  config  =  {
    // we'll use this prefix to help name our provisioned resources
    projectName:  `wordpress-ec2-rds-${stage}`,
    stage,
    stack:  {
        account:  process.env.AWS_ACCOUNT_NUMBER,
        region:  process.env.AWS_REGION  ||  'us-west-2'
    },
    deployedBy:  process.env.DEPLOYED_BY  ||  process.env.USER,
}

Let's create our .env file. The variables below are used to aid deployment from localhost as you will see later.
```
STAGE=dev
AWS_ACCOUNT_NUMBER=XXXXXXXXXXXXX
AWS_REGION=us-west-2
DEPLOYED_BY=john.doe
```

Let's modify our entry file (bin/wordpress-ec2-rds.ts) from this

// bin/wordpress-ec2-rds.ts
#!/usr/bin/env node
import  'source-map-support/register';
import  *  as  cdk  from  '@aws-cdk/core';
import  {  WordpressEc2RdsStack  }  from  '../lib/wordpress-ec2-rds-stack';

const  app  =  new  cdk.App();

new  WordpressEc2RdsStack(app,  'WordpressEc2RdsStack',  {

/* If you don't specify 'env', this stack will be environment-agnostic.
* Account/Region-dependent features and context lookups will not work,
* but a single synthesized template can be deployed anywhere. */

/* Uncomment the next line to specialize this stack for the AWS Account
* and Region that are implied by the current CLI configuration. */

// env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },

/* Uncomment the next line if you know exactly what Account and Region you
* want to deploy the stack to. */
// env: { account: '123456789012', region: 'us-east-1' },

/* For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html */
});

to this:

// bin/wordpress-ec2-rds.ts
import  'source-map-support/register';
import  *  as  cdk  from  '@aws-cdk/core';
import  {  WordpressEc2RdsStack  }  from  '../lib/wordpress-ec2-rds-stack';
import  {  config  }  from  '../lib/config';

const  app  =  new  cdk.App();

new  WordpressEc2RdsStack(app,  'WordpressEc2RdsStack',  {
    env:  config.env,
    description:  'Deploys resources for RDS and S3 powered Wordpress infrastructure',
    tags:  {  Project:  config.projectName,  Deployedby:  config.deployedBy  }
});

Note: It's very important to let the CDK know what account and region you will deploy the stack to. This is why we explicitly set the env object (which contains account and region from your environment variables)

Our configuration files and entry files are nopw mostly set up. Let's get to creating our construct files.

VPC

In short, a VPC (Virtual Private Cloud) is virtual local network (VLAN) that you create on your cloud service provider. You can then add resources to this network and rest assured this network is separated (isolated) from other people's resources on the cloud provider. This has many advantages, the biggest one being security.

A VPC, essentially being a virtual LAN looks like this: 172.25.0.0/16. To understand what I just wrote, its important to understand:

Additionally, here is a great explanation from Amazon on VPCs.

Setup

mkdir -p lib/constructs/vpc.ts
You can either create a new VPC or import an existing one.
Since I do not have a VPC with isolated subnets, I will create a new VPC
We need to install the ec2 cdk module to get access to VPC creation functions
```
npm install @aws-cdk/aws-ec2
```

Here we create the new VPC

// lib/constructs/vpc.ts

import * as cdk from '@aws-cdk/core'
import * as ec2 from '@aws-cdk/aws-ec2'

interface StackProps {
  prefix: string
  cidr: string
}

/**
 * Creates a new custom VPC
 *
 * @param  {cdk.Construct} scope stack application scope
 * @param  {StackProps} props props needed to create the resource
 *
 */
export class CustomVPC {
  // we export the vpc we just created so other resources can use it
  public readonly vpc: ec2.IVpc

  constructor(scope: cdk.Construct, props: StackProps) {
    this.vpc = new ec2.Vpc(scope, `${props.prefix}-vpc`, {
      maxAzs: 2, // RDS requires at least 2 availability zones
      cidr: props.cidr, // the ip address block of the vpc e.g. '172.22.0.0/16'
      enableDnsHostnames: true,
      enableDnsSupport: true,
      // expensive -- we don't need that yet (we have no PRIVATE subnets)
      natGateways: 0, 
      subnetConfiguration: [
        {
          cidrMask: 22,
          name: `${props.prefix}-public-`,
          subnetType: ec2.SubnetType.PUBLIC, // for WP instance
        },
        {
          cidrMask: 22,
          name: `${props.prefix}-isolated-`,
          subnetType: ec2.SubnetType.ISOLATED, // for RDS DB
        },
      ],
    })
  }
}

Here it's important to note that scope represents the base infrastructure class constructor that the new DefaultVPC instance will be created in. It comes from lib/wordpress-ec2-rds-stack.ts
Thus let's fetch our custom VPC inside the lib/wordpress-ec2-rds-stack.ts contructor.

Change lib/wordpress-ec2-rds-stack.ts from:

// lib/wordpress-ec2-rds-stack.ts

import  *  as  cdk  from  '@aws-cdk/core';

export  class  WordpressEc2RdsStack  extends  cdk.Stack  {
    constructor(scope:  cdk.Construct,  id:  string,  props?:  cdk.StackProps)  {
        super(scope,  id,  props);
        // The code that defines your stack goes here

    }
}

to this:

// lib/wordpress-ec2-rds-stack.ts

import  *  as  cdk  from  '@aws-cdk/core';
import  {  CustomVPC  }  from  './constructs/vpc'

export  class  WordpressEc2RdsStack  extends  cdk.Stack  {
    constructor(scope:  cdk.Construct,  id:  string,  props?:  cdk.StackProps)  {
        super(scope,  id,  props);
        // The code that defines your stack goes here

        // VPC -- fetch the custom VPC
        const  customVPC  =  new  CustomVPC(this,  {
            prefix:  config.projectName,
            cidr:  '172.22.0.0/16',
        })
    }
}

If you're not sure about what all these configurations mean, you can read more about configuring VPCs in AWS in this tutorial
The important takeaway for this tutorial is that your custom VPC has both PUBLIC and ISOLATED (or PRIVATE) subnets

RDS

From the horse's mouth:

Amazon Relational Database Service (Amazon RDS) makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while automating time-consuming administration tasks such as hardware provisioning, database setup, patching and backups. It frees you to focus on your applications so you can give them the fast performance, high availability, security and compatibility they need.

Amazon RDS is available on several database instance types - optimized for memory, performance or I/O - and provides you with six familiar database engines to choose from, including Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle Database, and SQL Server. You can use the AWS Database Migration Service to easily migrate or replicate your existing databases to Amazon RDS.

In the last tutorial, we installed and ran the MySQL database on the same instance running Wordpress. You usually want to have your database on a separate (and managed) instance so that you can easily configure it, scale or resize it, back it up, secure it behind a private subnet etc AWS RDS is thus perfect for our use case

Setup

Create the cdk construct file
```
mkdir -p lib/constructs/rds.ts
```

Let's install the cdk modules that we need

npm install @aws-cdk/aws-rds @aws-cdk/aws-secretsmanager

Let's import what we need

// lib/constructs/rds.ts

import * as cdk from '@aws-cdk/core'
import * as ec2 from '@aws-cdk/aws-ec2'
import * as rds from '@aws-cdk/aws-rds'

// this is where we will keep our database credentials e.g. user, password, host etc
import * as secrets from '@aws-cdk/aws-secretsmanager'

Since the RDS construct is initialized from the base infrastructure class, when we create a new instance we can pass some needed properties

For example, we need to know the vpc we want to use, the database port or user we would like to use, the name of the secret in secretsmanager that will keep our db secrets etc

interface StackProps {
  // this is useful when deploying to multiple environments e.g. prod, dev
  prefix: string
  vpc: ec2.Vpc
  user: string
  port: number
  database: string
  secretName: string
}

/**
 * Creates a MySQL DB on AWS RDS
 * 
 * @param  {cdk.Construct} scope stack application scope
 * @param  {StackProps} props props needed to create the resource
 * 
 */
export class MySQLRdsInstance {
  public readonly databaseSecretName: string

  constructor(scope: cdk.Construct, props: StackProps){
    // this is where all the following code will go
  }
}

Eveything that follows is inside the constructor of MySQLRdsInstance i.e. right below the // this is where all the following code will go comment:

Let's get our vpc from the props we just created

    // use the vpc we expoted from lib/constructs/vpc.ts
    const customVPC = props.vpc

Create a security group to allow connections only from inside the vpc

    // create the security group for RDS instance
    const ingressSecurityGroup = new ec2.SecurityGroup(scope, `${props.prefix}-rds-ingress`,{ 
        vpc: customVPC,
        securityGroupName: `${props.prefix}-rds-ingress-sg`,
      })

    ingressSecurityGroup.addIngressRule(
      // defaultVPC.vpcCidrBlock refers to all the IP addresses in defaultVPC
      ec2.Peer.ipv4(defaultVPC.vpcCidrBlock),
      ec2.Port.tcp(props.port || 3306),
      'Allows only local resources inside VPC to access this MySQL port (default -- 3306)'
    )

Generate the database secrets using Secrets Manager

    // Dynamically generate the username and password, then store in secrets manager
    const databaseCredentialsSecret = new secrets.Secret(
      scope,
     `${props.prefix}-MySQLCredentialsSecret`,
      {
        secretName: props.secretName,
        description: 'Credentials to access Wordpress MYSQL Database on RDS',
        generateSecretString: {
          secretStringTemplate: JSON.stringify({ username: props.user }),
          excludePunctuation: true,
          includeSpace: false,
          generateStringKey: 'password',
        },
      }
    )

Now let's create our RDS MySQL instance

    const mysqlRDSInstance = new rds.DatabaseInstance(scope, `${props.prefix}-MySqlRDSInstance`, {
      credentials: rds.Credentials.fromSecret(databaseCredentialsSecret),
      engine: rds.DatabaseInstanceEngine.mysql({
        version: rds.MysqlEngineVersion.VER_5_7_31
      }),
      port: props.port,
      allocatedStorage: 100,
      storageType: rds.StorageType.GP2,
      backupRetention: cdk.Duration.days(7),
      // t2.micro is free tier so we use it  
      instanceType: ec2.InstanceType.of(
        ec2.InstanceClass.T2,
        ec2.InstanceSize.MICRO
      ),
      vpc: customVPC,
      // we chose to place our database in an isolated subnet of our VPC
      vpcSubnets: { subnetType: ec2.SubnetType.ISOLATED },
      // if we destroy our database, AWS will take a snapshot of the database instance before terminating it
      removalPolicy: cdk.RemovalPolicy.SNAPSHOT,
      // accidental deletion protection -- you need to manually disable this in AWS web console to delete the database
      deletionProtection: true,
      securityGroups: [ingressSecurityGroup],
    })

View the final file here

Then let's import the RDS construct in our lib/wordpress-ec2-rds-stack.ts file:

    // lib/wordpress-ec2-rds-stack.ts
    import * as cdk from '@aws-cdk/core';
    import { DefaultVPC } from './constructs/vpc'
    import { MySQLRdsInstance } from './constructs/rds'
    import { config } from './config'

    export class WordpressEc2RdsStack extends cdk.Stack {
        constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
            super(scope, id, props);

            // The code that defines your stack goes here

            // VPC -- fetch the default VPC
            const defaultVPC = new DefaultVPC(this)

            // RDS -- create the mysql database
            new MySQLRdsInstance(this, {
                prefix: config.projectName,
                vpc: defaultVPC.vpc, // deploy the database in the default VPC!
                user: 'wordpress_admin',
                database: 'awesome-wp-site-db',
                port: 3306,
                // DB credentials will be saved under this pathname in AWS Secrets Manager
                secretName: `${config.projectName}/rds/mysql/credentials`, // secret pathname
            })
        }
    }

Application Load Balancer

From the horse's mouth:

Elastic Load Balancing automatically distributes your incoming traffic across multiple targets, such as EC2 instances, containers, and IP addresses, in one or more Availability Zones. It monitors the health of its registered targets, and routes traffic only to the healthy targets. Elastic Load Balancing scales your load balancer as your incoming traffic changes over time. It can automatically scale to the vast majority of workloads.
Elastic Load Balancing supports the following load balancers: Application Load Balancers (ALBs), Network Load Balancers (NLBs), Gateway Load Balancers (GLBs), and Classic Load Balancers (CLBs).

Important Load Balancer concepts

A load balancer serves as the single point of contact for clients. The load balancer distributes incoming application traffic across multiple targets, such as EC2 instances, in multiple Availability Zones. This increases the availability of your application. You add one or more listeners to your load balancer.
A listener checks for connection requests from clients, using the protocol and port that you configure. The rules that you define for a listener determine how the load balancer routes requests to its registered targets. Each rule consists of a priority, one or more actions, and one or more conditions. When the conditions for a rule are met, then its actions are performed. You must define a default rule for each listener, and you can optionally define additional rules.
Each target group routes requests to one or more registered targets, such as EC2 instances, using the protocol and port number that you specify. You can register a target with multiple target groups. You can configure health checks on a per target group basis. Health checks are performed on all targets registered to a target group that is specified in a listener rule for your load balancer.

Setup

For this tutorial we will be setting up an ALB.
You can read about the difference between ALB, NLB and CLB here

Create a new file to define the ALB in and install required CDK dependency

touch lib/constructs/alb.ts
npm install @aws-cdk/aws-elasticloadbalancingv2

In the newly created file:

// lib/constructs/alb.ts

import * as cdk from '@aws-cdk/core'
import * as ec2 from '@aws-cdk/aws-ec2'
import * as elbv2 from '@aws-cdk/aws-elasticloadbalancingv2'

interface StackProps {
  prefix: string
  vpc: ec2.IVpc
}

/**
 * Creates an Application Load Balancer for our Wordpress stack
 *
 * @param  {cdk.Construct} scope stack application scope
 * @param  {StackProps} props props needed to create the resource
 *
 */
export class WordpressApplicationLoadBalancer {
  // export the DNS name of the load balancer for WP install 
  public readonly loadBalancerDnsName: string
  // export ALB listener so we can attach autoscaling group
  listener: elbv2.IApplicationListener

  constructor(scope: cdk.Construct, props: StackProps) {
    const alb = new elbv2.ApplicationLoadBalancer(
      scope,
      `${props.prefix}-alb`,
      {
        loadBalancerName: `${props.prefix}-alb`,
        vpc: props.vpc,
        internetFacing: true,
      }
    )

    // we need to expose the dns name of the load balancer
    // so we can use it when installing Wordpress later
    this.loadBalancerDnsName = alb.loadBalancerDnsName

    // we will  need the listener to add our autoscaling group later
    this.listener = alb.addListener(`${props.prefix}-alb-listener`, {
      port: 80,
      open: true,
    })

    // print out the dns name of the alb
    new cdk.CfnOutput(scope, `${props.prefix}-alb-dns-name`, {
      value: alb.loadBalancerDnsName,
    })
  }
}

Then let’s import the ALB construct in our lib/wordpress-ec2-rds-stack.ts file:

// lib/wordpress-ec2-rds-stack.ts
import * as cdk from '@aws-cdk/core';
import { DefaultVPC } from './constructs/vpc'
import { MySQLRdsInstance } from './constructs/rds'
import { WordpressApplicationLoadBalancer } from './constructs/alb'
import { config } from './config'

export class WordpressEc2RdsStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // The code that defines your stack goes here

    // VPC -- fetch the default VPC
    const defaultVPC = new DefaultVPC(this)

    // RDS -- create the mysql database
    new MySQLRdsInstance(this, {
      prefix: config.projectName,
      vpc: defaultVPC.vpc, // deploy the database in the default VPC!
      user: 'wordpress_admin',
      database: 'awesome-wp-site-db',
      port: 3306,
      // DB credentials will be saved under this pathname in AWS Secrets Manager
      secretName: `${config.projectName}/rds/mysql/credentials`, // secret pathname
    })

    // ALB -- for our single instance
    const { loadBalancerDnsName, listener } = new WordpressApplicationLoadBalancer(this, {
      prefix: config.projectName,
      vpc: customVPC.vpc,
    })
  }
}

EC2 via Autoscaling Group

In the last tutorial, we used the CDK to launch a single EC2 instance. This time, we will do the same thing, however, we will launch the instance using an ASG.

From the horse's mouth:

An Auto Scaling group contains a collection of Amazon EC2 instances that are treated as a logical grouping for the purposes of automatic scaling and management. An Auto Scaling group also enables you to use Amazon EC2 Auto Scaling features such as health check replacements and scaling policies. Both maintaining the number of instances in an Auto Scaling group and automatic scaling are the core functionality of the Amazon EC2 Auto Scaling service.

The size of an Auto Scaling group depends on the number of instances that you set as the desired capacity. You can adjust its size to meet demand, either manually or by using automatic scaling.

An Auto Scaling group starts by launching enough instances to meet its desired capacity. It maintains this number of instances by performing periodic health checks on the instances in the group. The Auto Scaling group continues to maintain a fixed number of instances even if an instance becomes unhealthy. If an instance becomes unhealthy, the group terminates the unhealthy instance and launches another instance to replace it. For more information, see Health checks for Auto Scaling instances.

In our use case, we will use just one instance. This is because Wordpress is installed to the volume attached to this singular instance. If we launched more than two instances, then we would be redirecting users to two seperate WP installations (that may end up having different uploaded content, plugins etc because these files are installed to the file volumes).
This is desirable in certain cases e.g. if you want to serve users in different geographic regions different website content (without changing the url).
For most users however, the main use case of autoscaling is to spread server load horizonatally across instances.
A work-around that I did not implement in this tutorial is to sync uploaded content to S3 and use a shared EFS file system for the wordpress installation (something which I may look at in detail in another tutorial or at the ednof this one)
What then is the purpose of using an ASG with just one desired instance? Good question. For this tutorial, it is mostly used to warm you up to the concept of using ASGs and Load Balancers. In later tutorials, I will write applications that are better at utilizing ASGs than Wordpress. Also, if you instance should get accidentally terminated somehow, the ASG will immediately spin up another one. Awesome!

Setup

Create the file to hold our EC2 constructs and install dependecies

touch lib/constructs/ec2/ts
npm install @aws-cdk/aws-secretsmanager @aws-cdk/aws-autoscaling

This time, instead of using an SSH key, we will use aws-ssm and IAM to access our instances via SSH
We will place our instance in an autoscaling group with a maximum and minimum capacity of 1
We are also going to define a helper function to help do some string replacement later on. We will be using this function to insert envrionment secrets into our user script.

touch lib/utils.ts

// lib/utils.ts

/**
 * Replaces all given substring in a text with new substring e.g.
 *
 * @example
 *  const text = 'The woman and man and woman and man'
 *  const wordsArray = [{ man: 'boy' }, { woman: 'girl' }]
 *
 *  console.log(replaceAllSubstrings(wordsArray, text))
 *  // The girl and boy and girl and boy
 *
 * @param  {Array<Record<string, string>>} wordsArray the words to be substituted and the words to substitute them with
 * @param  {string} text the text from which to substitute the given sub strings
 * @returns the altered text
 */
export const replaceAllSubstrings = (
  wordsArray: Array<Record<string, string>>,
  text: string
) =>
  wordsArray.reduce(
    (f, s) =>
      `${f}`.replace(new RegExp(Object.keys(s)[0], 'g'), s[Object.keys(s)[0]]),
    text
  )

Inside lib/constructs/ec2.ts, we start by getting our imports

import  *  as  fs  from  'fs'
import  *  as  cdk  from  '@aws-cdk/core'
import  *  as  ec2  from  '@aws-cdk/aws-ec2'
import  *  as  secrets  from  '@aws-cdk/aws-secretsmanager'
import  *  as  autoscaling  from  '@aws-cdk/aws-autoscaling'
import  *  as  iam  from  '@aws-cdk/aws-iam'
import  {  config  }  from  '../config'
import  {  replaceAllSubstrings  }  from  '../utils'

We then define the props for the EC2 instance class we are defining

interface  StackProps  {
    prefix:  string
    vpc:  ec2.IVpc
    /* the dns name of the ALB */
    dnsName:  string
    /* the path of the db access secret in AWS SM */
    dbSecretName: string 
    /* the path of the wp admin secret in AWS SM */
    wpSecretName: string
}

We then define a new class to hold our construct code

/**
 * Creates the Wordpress EC2 AutoscalingGroup
 *
 * @param  {cdk.Construct} scope stack application scope
 * @param  {StackProps} props props needed to create the resource
 *
 */
export class WordpressAutoScalingGroup {
  // export our newly created instance
  public readonly asg: autoscaling.AutoScalingGroup

  constructor(scope: cdk.Construct, props: StackProps) {
    // all the code will go here
  }
}

Inside the constructor, we can start by importing the custom vpc, defining a role for ur ec2 instance and creating the necessary security groups:

    // use the vpc we just created
    const customVPC = props.vpc

    // define a role for the wordpress instances
    const role = new iam.Role(scope, `${props.prefix}-instance-role`, {
      assumedBy: new iam.CompositePrincipal(
        new iam.ServicePrincipal('ec2.amazonaws.com'),
        new iam.ServicePrincipal('ssm.amazonaws.com')
      ),
      managedPolicies: [
        // allows us to access instance via SSH using IAM and SSM
        iam.ManagedPolicy.fromAwsManagedPolicyName(
          'AmazonSSMManagedInstanceCore'
        ),
        // allows ec2 instance to access secrets maanger and retrieve secrets
        iam.ManagedPolicy.fromAwsManagedPolicyName('SecretsManagerReadWrite'),
      ],
    })

    // lets create a security group for the wordpress instance
    const securityGroup = new ec2.SecurityGroup(
      scope,
      'wordpress-instances-sg',
      {
        vpc: customVPC,
        allowAllOutbound: true,
        securityGroupName: 'wordpress-instances-sg',
      }
    )
    // NB: the WP instance will not be exposed to the public Internet this time
    // the Internet can access it through the ALB only
    // the admin can access it (the console) via SSM
    securityGroup.addIngressRule(
      ec2.Peer.ipv4(customVPC.vpcCidrBlock),
      ec2.Port.tcp(80),
      'Allows HTTP access from resources inside our VPC (like the ALB)'
    )

Since we want to automate the entire installation of WP on our instance, we will need to automate the famous Wordpress 5 minute install (See step 9 in this article).
In order to do this, we need to define secrets for our WP Admin

Inside lib/config.ts, we can add a new object wordpress to keep some of those secrets:

// lib/config.ts
...
...
export const config = {
  projectName: `wordpress-ec2-rds-${stage}`, 
  ...
  wordpress: {
    admin: {
      username: process.env.WP_ADMIN_USER || 'admin',
      email: process.env.WP_ADMIN_EMAIL || 'admin@whatever.com'
    },
    site: {
      // name of the database WP will use (cannot have hyphens) 
      databaseName: process.env.WP_DB_NAME || 'awesome_wp_site_db',
      // the name of our WP  website
      title: process.env.WP_SITE_TITLE || 'awesome-wp-site',
      // where we will install WP on our instance volume
      installPath: process.env.WP_SITE_INSTALL_PATH || '/var/www/html/',
    }
  }
}

Make sure you add those variables in your .env file:

STAGE=dev
AWS_ACCOUNT_NUMBER=XXXXXXXXXXXXX
AWS_REGION=us-west-2
DEPLOYED_BY=john.doe

# Wordpress Site Variables
WP_DB_NAME='awesome_wp_site_db'
WP_SITE_TITLE='awesome_wp_site'
WP_SITE_INSTALL_PATH=/var/www/html/

# Wordpress Admin Secrets
WP_ADMIN_EMAIL=admin@awesomewpsite.com
WP_ADMIN_USER=admin

Let's add the code to add our WP admin secrets in AWS SM for Wordpress Admin as well as generate a new WP Admin password inside the WordpressAutoScalingGroup constructor

    ...
    ...

    // secrets for wp admin
    new secrets.Secret(scope, 'WordpressAdminSecrets', {
      secretName: props.wpSecretName,
      description: 'Admin credentials to access Wordpress',
      generateSecretString: {
        secretStringTemplate: JSON.stringify({
          username: config.wordpress.admin.username,
          email: config.wordpress.admin.email,
        }),
        // will generate a random password under the object key 'password'
        generateStringKey: 'password',
      },
    })

User script

We now need to write our user script to configure Wordpress on our instance and automate the famous 5 minute WP install
In summary, we need to first install the dependencies (e.g. Apache, PHP and Wordpress etc) on the instance
Then we need to wait for the RDS MySQL database to initialize and be in a ready state (very important). If the database is not ready, then WP installation will fail because you won't be able to create the necessary tables in the database.
When the database is ready, we install Wordpress and that's it!

Read the comments in the script to understand what's going on

mkdir lib/scripts && touch lib/scripts/wordpress_install.sh

```bash
#! /bin/bash

# lib/scripts/wordpress_install.sh

#------------------ 0.USEFUL FUNCTIONS

# Checks to see if an env is defined (not null) in the bash session
is_defined () {
    for var in "$@" ; do
        if [ ! -z "${!var}" ] & [ "${!var}" != "null" ]; then
            echo "$var is set to ${!var}"
        else
            echo "$var is not set"
            return 1
        fi
    done
}

# Checks if desired db secrets in secrets manager are ready
# Db secrets are only fully ready when the RDS DB is ready
db_secrets_ready () {
    if ! is_defined "AWS_REGION" "DB_SECRETS_PATH";then
        return 0
    fi

    echo "Retrieving secrets..." 
    DB_SECRETS_JSON=$(aws secretsmanager get-secret-value --secret-id $DB_SECRETS_PATH --region $AWS_REGION | jq -r '.SecretString')

    echo "Retrieved secrets." 
    DB_USER=$(echo $DB_SECRETS_JSON | jq -r '.username')
    DB_PASS=$(echo $DB_SECRETS_JSON | jq -r '.password')
    DB_HOST=$(echo $DB_SECRETS_JSON | jq -r '.host')
    DB_PORT=$(echo $DB_SECRETS_JSON | jq -r '.port')

    echo "Checking secrets..." 
    if ! is_defined "DB_USER" "DB_PASS" "DB_HOST" "DB_PORT";then
        echo "Secrets are not ready." 
        return 1
    fi

    echo "Secrets are ready." 
    return 0

}

#------------------  1.INSTALL DEPENDECIES
# update dependencies
yum -y update

# Install Apache
yum -y install httpd

# Start Apache
service httpd start

# Install PHP, PHP CLI, JQ, MySQL
yum -y install php php-cli php-mysql jq mysql mysqladmin

# PHP7 needed for latest wordpress
amazon-linux-extras install php7.4 -y 

# Restart Apache
service httpd restart

# Install the Wordpress CLI which will help us install Wordpress correctly
curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
chmod +x wp-cli.phar
mv wp-cli.phar /usr/local/bin/wp

#------------------  2.SET SCRIPT GLOBAL VARIABLES

# AWS and Wordpress variables to replace
# We will replace these variables in the CDK ec2 construct file
# before using the script to launch an ec2 instance
DB_SECRETS_PATH=_DB_SECRETS_PATH_
WP_SECRETS_PATH=_WP_SECRETS_PATH_
AWS_REGION=_AWS_REGION_
WP_DB_NAME=_WP_DB_NAME_
WP_SITE_TITLE=_WP_SITE_TITLE_
WP_SITE_INSTALL_PATH=_WP_SITE_INSTALL_PATH_
WP_SITE_BASE_DOMAIN=_WP_SITE_BASE_DOMAIN_

# Wait for Secrets Manager to have RDS secret ready
# Certain database secrets (e.g host, port) won't be ready until the database is ready
echo "Waiting up to 20 minutes for Secrets Manager to be ready with Secrets";
for i in {1..240}; do
    echo "try count: $i"
    db_secrets_ready && break;
    # retry every 30 seconds
    sleep 30s; 
done
echo "Secrets Manager is ready with Secrets";

# Use the AWS CLI to get secrets from Secrets Manager
DB_SECRETS_JSON=$(aws secretsmanager get-secret-value --secret-id $DB_SECRETS_PATH --region $AWS_REGION | jq -r '.SecretString')
WP_SECRETS_JSON=$(aws secretsmanager get-secret-value --secret-id $WP_SECRETS_PATH --region $AWS_REGION | jq -r '.SecretString')

# Parse secrets from JSON response using the useful jq
DB_USER=$(echo $DB_SECRETS_JSON | jq -r '.username')
DB_PASS=$(echo $DB_SECRETS_JSON | jq -r '.password')
DB_HOST=$(echo $DB_SECRETS_JSON | jq -r '.host')
DB_PORT=$(echo $DB_SECRETS_JSON | jq -r '.port')
WP_ADMIN_USER=$(echo $WP_SECRETS_JSON | jq -r '.username')
WP_ADMIN_PASSWORD=$(echo $WP_SECRETS_JSON | jq -r '.password')
WP_ADMIN_EMAIL=$(echo $WP_SECRETS_JSON | jq -r '.email')

# If some ENV is not defined, stop the script
if ! is_defined \
"DB_SECRETS_PATH" \
"WP_SECRETS_PATH" \
"AWS_REGION" \
"WP_DB_NAME" \
"WP_SITE_TITLE" \
"WP_SITE_INSTALL_PATH" \
"WP_SITE_BASE_DOMAIN" \
"DB_USER" \
"DB_PASS" \
"DB_HOST" \
"DB_PORT" \
"WP_ADMIN_USER" \
"WP_ADMIN_PASSWORD" \
"WP_ADMIN_EMAIL" \
; then
    echo "Exiting WP installation script because some variables were undefined"
    exit 0
fi

#------------------  3.CREATE WORDPRESS MYSQL DATABASE

# Wait for the database to be ready
# Usually this should only run once because of the DB secrets in AWS SM are ready
# then it means the database is likely ready as well
for i in {1..30}; do
    echo "try count: $i"
    mysqladmin ping -h "$DB_HOST" -u$DB_USER -p$DB_PASS -P $DB_PORT --silent && break;
    # retry every 30s
    sleep 30s
done

# Create the database.
echo "Creating the database $WP_DB_NAME..."
mysql -h $DB_HOST -u$DB_USER -p$DB_PASS -P $DB_PORT -e"CREATE DATABASE $WP_DB_NAME"

#------------------  4.SETUP WORDPRESS INSTALLATION

# Download WP Core.
/usr/local/bin/wp core download --path=$WP_SITE_INSTALL_PATH

# Generate the wp-config.php file
/usr/local/bin/wp core config \
--path=$WP_SITE_INSTALL_PATH \
--dbname=$WP_DB_NAME \
--dbuser=$DB_USER \
--dbpass=$DB_PASS \
--dbhost=$DB_HOST \
--extra-php <<PHP
define('WP_DEBUG', true);
define('WP_DEBUG_LOG', true);
define('WP_DEBUG_DISPLAY', true);
define('WP_MEMORY_LIMIT', '256M');
PHP

# Install the WordPress database.
/usr/local/bin/wp core install \
--path=$WP_SITE_INSTALL_PATH \
--url=$WP_SITE_BASE_DOMAIN \
--title=$WP_SITE_TITLE \
--admin_user=$WP_ADMIN_USER \
--admin_password=$WP_ADMIN_PASSWORD \
--admin_email=$WP_ADMIN_EMAIL

# Restart Apache
service httpd restart

# Wordpress is now installed!
```

If all goes well, when yuo navigate to the loadbalancer DNS name, the wordpress website should just pop up without any need for any configuration
Let's now add the user script and insert some required environment variables in the CDK file

In the WordpressAutoScalingGroup constructor

    ...
    ...

    // Fetch the user script from file system as a string
    const userScript = fs.readFileSync(
      'lib/scripts/wordpress_install.sh',
      'utf8'
    )

    // Replace the following variable substrings in the userScript
    const modifiedUserScript = replaceAllSubstrings(
      [
        { _DB_SECRETS_PATH_: props.dbSecretName },
        { _WP_SECRETS_PATH_: props.wpSecretName },
        { _AWS_REGION_: config.env.region },
        { _WP_DB_NAME_: config.wordpress.site.databaseName },
        { _WP_SITE_TITLE_: config.wordpress.site.title },
        { _WP_SITE_INSTALL_PATH_: config.wordpress.site.installPath },
        { _WP_SITE_BASE_DOMAIN_: props.dnsName }, // our load balancer dns name
      ],
      userScript
    )

EC2 instance in ASG

Finally, we can create our ec2 instance in an ASG

...
...
// finally create and export out autoscaling group
this.asg = new autoscaling.AutoScalingGroup(scope, `${props.prefix}-asg`, {
  vpc: customVPC,
  // add the role we created (needs access to AWS SM)
  role,
  instanceType: ec2.InstanceType.of(
    ec2.InstanceClass.T2,
    ec2.InstanceSize.MICRO
  ),
  machineImage: ec2.MachineImage.latestAmazonLinux({
    generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
  }),
  // add our modified user script to launch with instances in this ASG
  userData: ec2.UserData.custom(modifiedUserScript),
  // we only want one instance in our ASG
  minCapacity: 1,
  maxCapacity: 1,
  associatePublicIpAddress: true,
  vpcSubnets: { subnetType: ec2.SubnetType.PUBLIC },
})

The final file should look like this
We then need to attach the ASG we just created as a target to the the listener of the ALB we created earlier, using port 80
This will redirect all requests for WP at the loadblancer to port 80 of our WP instance.

Right below the EC2 construct in the WordpressEc2RdsStack constructor, add:

    // lets add our autoscaling group to our load balancer
    listener.addTargets(`${config.projectName}-wp-asg-targets`, {
      port: 80,
      targets: [asg]
    })

The final lib/wordpress-ec2-rds-stack.ts should look like:

import * as cdk from '@aws-cdk/core'
import { CustomVPC } from './constructs/vpc'
import { MySQLRdsInstance } from './constructs/rds'
import { WordpressAutoScalingGroup } from './constructs/ec2'
import { WordpressApplicationLoadBalancer } from './constructs/alb'
import { config } from './config'

export class WordpressEc2RdsStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props)
    // The code that defines your stack goes here

    // VPC -- fetch the custom VPC
    const customVPC = new CustomVPC(this, {
      prefix: config.projectName,
      cidr: '172.22.0.0/16',
    })

    // RDS -- create the mysql database
    new MySQLRdsInstance(this, {
      prefix: config.projectName,
      vpc: customVPC.vpc,
      user: 'wordpress_admin',
      database: 'awesome-wp-site-db',
      port: 3306,
      // DB credentials will be saved under this pathname in AWS Secrets Manager
      secretName: `${config.projectName}/rds/mysql/credentials`, // secret pathname
    })

    // Application Loadbalancer -- for our single instance
    const { loadBalancerDnsName, listener } = new WordpressApplicationLoadBalancer(this, {
      prefix: config.projectName,
      vpc: customVPC.vpc,
    })

    // EC2 -- create the Wordpress instance in an autoscaling group
    const { asg } = new WordpressAutoScalingGroup(this, {
      prefix: config.projectName,
      vpc: customVPC.vpc,
      dnsName: loadBalancerDnsName,
      dbSecretName: `${config.projectName}/rds/mysql/credentials`,
      wpSecretName: `${config.projectName}/wordpress/admin/credentials`,
    })

    // lets add our autoscaling group to our load balancer
    listener.addTargets(`${config.projectName}-wp-asg-targets`, {
      port: 80,
      targets: [asg]
    })
  }
}

At this point, all the resources are ready to be deployed!

Deployment

Local machine
To deploy from your local machine, make sure all your aws credentials and configurations are correctly setup

Ensure you have the .env file with the following environment variables setup:

STAGE=dev
AWS_ACCOUNT_NUMBER=XXXXXXXXXXXXXXX
AWS_REGION=us-west-2
DEPLOYED_BY=john.doe

# Wordpress Variables
WP_DB_NAME='awesome_wp_site_db'
WP_SITE_TITLE='awesome_wp_site'
WP_SITE_INSTALL_PATH=/var/www/html/

# Wordpress Admin Secrets
WP_ADMIN_EMAIL=admin@awesomewpsite.com
WP_ADMIN_USER=admin

You can then first try to synthesize the stack and make sure everything is correct
```
cdk synth --profile default
```
If the CDK outputs a YAML file to the console without any issue, then you can deploy
```
cdk deploy --profile default
```
You will be asked to approve of IAM changes before the deployment can continue

Github Actions

Deploying from your local machine is not recommended in a production environment.
Fortunately, Github provides a free CI/CD service called Github Actions
All we have to do in this instance is create a simple YAML configuration file that can help us deploy to different stages using GA (for the purposes of this tutorial, just dev and prod)

From the root of your project

mkdir -p .github/workflows && touch .github/workflows/deploy.yml

Open up .github/workflows/deploy.yml and add the following code

name: Deploy

on:
  push:
    branches:
      - master
      - dev
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v2
      - uses: nelonoel/branch-name@v1.0.1
      - name: cdk deploy
        uses: youyo/aws-cdk-github-actions@v1
        with:
          cdk_subcommand: 'deploy'
          cdk_args: '--require-approval never'
        env:
          STAGE: ${{ env.BRANCH_NAME == 'master' && 'prod' || 'dev'  }}
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
          AWS_ACCOUNT_NUMBER: ${{ secrets.AWS_ACCOUNT_NUMBER }}
          DEPLOYED_BY: ${{ secrets.DEPLOYED_BY }}

What's going on?
- GA will only run this script when a push/merge to the branches dev or master happens
- GA will create an Ubuntu container, checkout the repository and run CDK deploy
- A useful GA action called branch-name@v1.0.1 gets the name fo the branch we are working on and deploys to the correct stage based on that branch (it sets the STAGE env variable)
- GA action aws-cdk-github-actions@v1 runs cdk deploy and deploys the resources to AWS
In order for this to work, you have to to go the settings of your repository on Github and add the follwowing secrets:
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_DEFAULT_REGION
- AWS_ACCOUNT_NUMBER
- DEPLOYED_BY
For AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, you can create a new user in IAM in the AWS console called github.actions.bot that has administrator privileges and only PROGRAMMATIC ACCESS and paste those values in Github settings as secrets

Destroying the stack

To destroy the stack from your local machine using the CDK, just run
```
cdk destroy --profile default
```
To destroy the stack from AWS console
- Log in to the AWS console
- Go to Cloudformation
- Find your stack > Destroy/Delete

Final result

Whether you deploy from your local machine or via Github actions, if all goes well, after about 20 minutes (RDS databases take a while to initialize) you should see this output:

In your browser, navigate to the DNS name printed in the output of your local machine console or the GA console(replace the asteriks with the region you deployed in).
Alternatively, you can find the DNS name of the ALB via EC2 console > Load balancers
You should then see:

Conclusion

In this tutorial I tried to keep things simple and did not add many improvements that I would likely add in a real production scenario (See the Homework Assignments section for improvements that can make your WP installation Production ready on AWS)

Debugging

To access the ec2 instance your ASG just spn up, you need to get the instance id from the AWS EC2 console

Once you have the instance id, to log into your instance, just do:

aws ssm start-session --target "i-0191364267ad972a2" --profile default

where i-0191364267ad972a2 is the id of the instance as seen in the AWS console. Cool right? No need for SSH keys!
Of course, if you still want to use regular SSH then in the CDK you have to open port 22 and create an SSH key pair on your machine and upload the public key to the ec2 instance.
If Wordpress does not load correctly when you navigate to the ALB DNS name, then it is likely something with the configuration of the database, or access to secrets manager went wrong.

You can check the user script logs in the ec2 instance

# list all log files related to ec2 start up scripts
find / -name "*cloud*log"

Homework Assignments

There are many improvements you can make to this CDK stack to make a better WP installation. I'll leave them as homework

Create an S3 bucket using the CDK and use this Wordpress plugin to sync Wordpress uploads to AWS S3. Hint: You install this plugin via the user script.
Use the CDK to create an EFS filesystem to install Wordpress to instead of installing Wordpress to var/www/html on the default attached volume (which limits WP to one instance). The key here is the same EFS filesystem can be used by many instances.
The above two improvements combined will allow you to use more than one instance in the WP ASG and make your Wordpress Installation much more highly available and resilient.
Create a cloudfront distribution to serve the WP content from the S3 bucket you just created
Buy a cheap 5 dollar .link domain on AWS Route53 and use it in the CDK instead of the ALB dnsName. That way, your site can be accessed via something like mywebsite.link
Add CloudWatchAgentServerPolicy to Wordpress instance role and in the user script, install Cloudwatch Agent so that you can collect logs and internal metrics from the instance in AWS Cloudwatch. See this tutorial.

Hi I'm Emmanuel! I write about Software and DevOps.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

Quickly install software on a new Ubuntu installation with GMUAR

Emmanuel K — Fri, 19 Mar 2021 08:47:21 +0000

GMUAR -- Get Me Up And Running -- is a pure bash command-line utility program to install a host of common software on Ubuntu/Debian. This helps get software onto fresh Ubuntu installs fast.

I was inspired to create this by all the sudo apt install commands I would have to get through just to get a new development machine up and running. Some software also wasn't easily installable via apt and need the addition of repositories, curling scripts etc.

This is my solution to all of that.

Simply clone the repository and run ./gmuar.sh

The program runs through several categories of software to install:

setup (e.g. curl or pip3)
utilities (e.g. net-tools)
desktop (e.g. spotify)
customization (e.g zsh)
development (e.g. node.js )

It has the ability to check via apt, PATH or even filename to determine whether a program has been installed (since not all programs are installed via apt or snap).

GMUAR can be extended by adding your own scripts to add your own desired software. Instructions are in the repository README

Check it out now!

Hi I'm Emmanuel! I write about Software and DevOps.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

Part 3 - Simple EC2 instance - Awesome AWS CDK

Emmanuel K — Thu, 18 Mar 2021 11:06:56 +0000

Introduction
Bootstrapping a new CDK Application
Structure and Setup of a CDK application
- Structure
- Setup
Writing the infrastructure code
Creating a new Key Pair in the AWS Console
Deploy
Accessing the instance
Updating the deployed stack
Testing the stack
- Test requirements
Destroying the stack
Notes
Conclusion
Up next

Introduction

Most tutorials on AWS try to get you to deploy the world famous t2.micro instance (a small virtual server under the free usage tier on AWS) using the AWS console. You then go on to install something like Wordpress through SSH or through a user script (a script that runs when an instance is created) that you define in the AWS console.

So that's exactly what we're going to deploy. But we're going to use the AWS CDK instead.

This is what you'll learn in this tutorial:

Bootstrapping a new CDK application
The structure and setup of a CDK application
How to provision and setup an ec2 instance
How to setup terminal access to the instance via SSH key.
How to update the ec2 instance with a user script and update the deployed cdk stack
How to write tests for the cdk application
How to destroy the deployed stack

Let's go!

Bootstrapping a new CDK Application

Using your terminal, create a new directory simple-ec2 and cd into it:



mkdir simple-ec2 && cd simple-ec2

We previously setup the AWS CDK cli globally. If you haven't done that see Part 2 in this series.

Bootstrap a new cdk project template that uses Typescript:



cdk init --language=typescript

This will initialize a new cdk project for you in Typescript.

Run npm update to ensure you're using the latest version of the CDK.
If you have version conflicts between @aws-cdk/core and other @aws-cdk sub-packages, then you'll come across some weird errors. @aws-cdk/core and every other imported @aws-cdk/PACKAGE should have the same version.

Structure and Setup of a CDK application

Structure


 bash
# tree -I 'node_modules'
.
├── bin
│   └── simple-ec2.ts       # entry point
├── cdk.json
├── jest.config.js          # for tests
├── lib                     # where the infrastructure code you write will go
│   └── simple-ec2-stack.ts 
├── package.json
├── package-lock.json
├── README.md
├── test                    # test folder
│   └── simple-ec2.test.ts
└── tsconfig.json

./bin/simple-ec2.ts is the entry point file used by the cdk. This is where you define your stack(s).
The IaC that provisions the resources will be inside the lib folder and is required by ./bin/simple-ec2.ts during synth and deploy actions. I'll explain both commands later.
./test/simple-ec2.test.ts contains the template code to test your CDK application

Setup

In ./bin/simple-ec2.ts a new App() is defined and this represents a single stack.

We can add a description for our stack in this file.
This description will be visible in the Cloudformation console: ```ts

// ./bin/simple-ec2.ts

import 'source-map-support/register';
import * as cdk from '@aws-cdk/core';
import { SimpleEc2Stack } from '../lib/simple-ec2-stack';

const app = new cdk.App();
new SimpleEc2Stack(app, 'SimpleEc2Stack', {
description: 'This is a simple EC2 stack'
});


Let's head over to `./lib/simple-ec2-stack.ts` and see what the CDK boostrapped for us:

```ts


// ./lib/simple-ec2-stack.ts

import * as cdk from '@aws-cdk/core';

export class SimpleEc2Stack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // The code that defines your stack goes here
  }
}

As you can see, the cdk init command bootstrapped a nice template for us to get to writing our IaC fast. The base class cdk.Stack gives us the ability to create a new Cloudformation stack.

We also need to create a .env file to keep our AWS Account number and the region we will use. In the root of the project create a file called .env and add the following. Replace the xxXxXxxXXxXxx with your AWS account number and use the region you want.



AWS_ACCOUNT_NUMBER=xxXxXxxXXxXxx
AWS_ACCOUNT_REGION=us-west-2

Writing the infrastructure code

We want to create an ec2 instance so we will need the @aws-cdk/ec2 library. (This is the same process for any other AWS service you need to provision resources). We will also need @aws-cdk/aws-iam library to give permissions to our instance to do stuff. We also want to be able to read our .env file so lets also install dotenv package



npm install @aws-cdk/aws-ec2 @aws-cdk/aws-iam dotenv

Remember, since the CDK is written in Typescript and is typed excellently, while typing you can access intellisense and see the various properties of CDK resources e.g. in the image below I can see what properties an instance of ec2.Instance() class has.

Here goes our first iteration:



import * as cdk from '@aws-cdk/core'
import * as ec2 from '@aws-cdk/aws-ec2' // import ec2 library 
import * as iam from '@aws-cdk/aws-iam' // import iam library for permissions

require('dotenv').config()

const config = {
  env: {
    account: process.env.AWS_ACCOUNT_NUMBER,
    region: process.env.AWS_REGION
  }
}

export class SimpleEc2Stack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    // its important to add our env config here otherwise CDK won't know our AWS account number
    super(scope, id, { ...props, env: config.env })

    // Get the default VPC. This is the network where your instance will be provisioned
    // All activated regions in AWS have a default vpc. 
    // You can create your own of course as well. https://aws.amazon.com/vpc/
    const defaultVpc = ec2.Vpc.fromLookup(this, 'VPC', { isDefault: true })

    // Lets create a role for the instance
    // You can attach permissions to a role and determine what your
    // instance can or can not do
      const role = new iam.Role(
        this,
        'simple-instance-1-role', // this is a unique id that will represent this resource in a Cloudformation template
        { assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com') }
      )

    // lets create a security group for our instance
    // A security group acts as a virtual firewall for your instance to control inbound and outbound traffic.
    const securityGroup = new ec2.SecurityGroup(
      this,
      'simple-instance-1-sg',
      {
        vpc: defaultVpc,
        allowAllOutbound: true, // will let your instance send outboud traffic
        securityGroupName: 'simple-instance-1-sg',
      }
    )

    // lets use the security group to allow inbound traffic on specific ports
    securityGroup.addIngressRule(
      ec2.Peer.anyIpv4(),
      ec2.Port.tcp(22),
      'Allows SSH access from Internet'
    )

    securityGroup.addIngressRule(
      ec2.Peer.anyIpv4(),
      ec2.Port.tcp(80),
      'Allows HTTP access from Internet'
    )

    securityGroup.addIngressRule(
      ec2.Peer.anyIpv4(),
      ec2.Port.tcp(443),
      'Allows HTTPS access from Internet'
    )

    // Finally lets provision our ec2 instance
    const instance = new ec2.Instance(this, 'simple-instance-1', {
      vpc: defaultVpc,
      role: role,
      securityGroup: securityGroup,
      instanceName: 'simple-instance-1',
      instanceType: ec2.InstanceType.of( // t2.micro has free tier usage in aws
        ec2.InstanceClass.T2,
        ec2.InstanceSize.MICRO
      ),
      machineImage: ec2.MachineImage.latestAmazonLinux({
        generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
      }),

      keyName: 'simple-instance-1-key', // we will create this in the console before we deploy
    })

    // cdk lets us output prperties of the resources we create after they are created
    // we want the ip address of this new instance so we can ssh into it later
    new cdk.CfnOutput(this, 'simple-instance-1-output', {
      value: instance.instancePublicIp
    })
  }
}

Creating a new Key Pair in the AWS Console

Before we try to deploy our newly created instance, we need to go to the AWS console and create a key pair that we will use to access the instance called simple-instance-1-key

Log into the AWS console.
Go to EC2 dashboard.
Go to Key Pairs and click create Key Pair
Enter key name as simple-instance-1-key and click create
Your new key pair will be created and your browser will automatically download a new .pem file called simple-instance-1-key.pem
this is the key file you'll use to gain access to your instance via SSH

Create a new directory under .aws/ called pems ```bash

mkdir ~./aws/pems/

- move the newly downloaded file to this directory and give it the necessary permissions
```bash


mv ~/Downloads/simple-instance-1-key.pem ~/.aws/pems

# important step or your key file won't work
chmod 400 ~/.aws/pems/simple-instance-1-key.pem

Now that we have our key file properly setup, lets deploy our instance!

Deploy

Remember we set up our aws profiles and crednetials in ~/.aws/config and ~/.aws/credentials back in part 2

I will be deploying to my default profile which is linked to my personal AWS account with the region us-west-2.

If you have another profile you want to use then in the commands below use that profile name instead of default.

In your terminal:



cdk synth --profile default

This command will synthesize your stack.

When CDK apps are executed, they produce (or “synthesize”, in CDK parlance) an AWS CloudFormation template for each stack defined in your application.

Essentially it will print the cloudformation template for your stack to your console.
It's a good way to check that there's nothing wrong with your stack before trying to deploy since cdk synth will verify the resources you are trying to provision can actually be provisioned.
You should something like this:



❯ cdk synth --profile default
Resources:
  simpleinstance1role9EEDA67C:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action: sts:AssumeRole
            Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
        Version: "2012-10-17"
   ...
   ...
   ...

If the entire stack prints without error then you're okay to go.

Now we can deploy:



cdk deploy --profile default

You should get a prompt accessing for Cloudformation to allow the creation of resources that need approval. Type y and press ENTER to continue with the deployment:

You'll start seeing the output from Cloudformation in your console as the stack is being created.

When the stack has been successfully deployed, you should see:

Notice the output which we defined at the end of the stack. The CDK printed the public ip address of the newly created instance for us because we told it to, awesome! You can use this method to print out any value when a stack has successfully deployed.

You can alternatively get this information from the ec2 console by checking on the instances dashboard.

Accessing the instance

Let's ssh into our newly created instance with our key file and the public ip address.

Note: that the default user for AMAZON LINUX images is ec2-user



ssh -i ~/.aws/pems/simple-instance-1-key.pem ec2-user@34.220.79.175

You should now be able to log into your instance!

Unfortunately, we have an instance that isn't running anything on it.

Let's fix that!

Adding User script

Let's create a new file under ./lib/ directory called user_script.sh. Paste this code into that file.
This code will deploy Apache, Wordpress, Mysql server on this intance. In this script, the database password will be pl55w0rd.
It's very insecure to add passwords to scripts but in this case I'm doing this just for demonstration purposes.
In production you should first of all never use such a weak password and secondly, not inside such a script.
Rather, you should deploy the Mysql database on AWS RDS and setup credentials for that database using AWS Secrets Manager.

Here's the setup file:



#! /bin/bash
# become root user
sudo su

# update dependencies
yum -y update

# we'll install 'expect' to input keystrokes/y/n/passwords
yum -y install expect 

# Install Apache
yum -y install httpd

# Start Apache
service httpd start

# Install PHP
yum -y install php php-mysql
# php 7 needed for latest wordpress
amazon-linux-extras -y install php7.2 

# Restart Apache
service httpd restart

# Install MySQL
wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
rpm -ivh mysql-community-release-el7-5.noarch.rpm

yum -y update 
yum -y install mysql-server

# Start MySQL
service mysqld start

# Create a database named blog
mysqladmin -uroot create blog

# Secure database
# non interactive mysql_secure_installation with a little help from expect.

SECURE_MYSQL=$(expect -c "

set timeout 10
spawn mysql_secure_installation

expect \"Enter current password for root (enter for none):\"
send \"\r\"

expect \"Change the root password?\"
send \"y\r\"
expect \"New password:\"
send \"pl55w0rd\r\"
expect \"Re-enter new password:\"
send \"pl55w0rd\r\"
expect \"Remove anonymous users?\"
send \"y\r\"

expect \"Disallow root login remotely?\"
send \"y\r\"

expect \"Remove test database and access to it?\"
send \"y\r\"

expect \"Reload privilege tables now?\"
send \"y\r\"

expect eof
")

echo "$SECURE_MYSQL"

# Change directory to web root
cd /var/www/html

# Download Wordpress
wget http://wordpress.org/latest.tar.gz

# Extract Wordpress
tar -xzvf latest.tar.gz

# Rename wordpress directory to blog
mv wordpress blog

# Change directory to blog
cd /var/www/html/blog/

# Create a WordPress config file 
mv wp-config-sample.php wp-config.php

#set database details with perl find and replace
sed -i "s/database_name_here/blog/g" /var/www/html/blog/wp-config.php
sed -i "s/username_here/root/g" /var/www/html/blog/wp-config.php
sed -i "s/password_here/pl55w0rd/g" /var/www/html/blog/wp-config.php

# create uploads folder and set permissions
mkdir wp-content/uploads
chmod 777 wp-content/uploads

#remove wp file
rm -rf /var/www/html/latest.tar.gz

We will need to add fs module at the top of our ./lib/simple-ec2-stack.ts file since instance.addUserData() needs to access the file system during deployment.



import * as cdk from '@aws-cdk/core'
import * as ec2 from '@aws-cdk/aws-ec2' // import ec2 library 
import * as iam from '@aws-cdk/aws-iam' // import iam library for permissions

// lets include fs module
import * as fs from 'fs'

Then we can the function instance.addUserData() right before our output function:



...

    // add user script to instance
    // this script runs when the instance is started 
    instance.addUserData(
      fs.readFileSync('lib/user_script.sh', 'utf8')
    )

    // cdk lets us output prperties of the resources we create after they are created
    // we want the ip address of this new instance so we can ssh into it later
    new cdk.CfnOutput(this, 'simple-instance-1-output', {
      value: instance.instancePublicIp
    })
...

Update the deployed stack

Let's re-synthesize to check everything is okay:



cdk --synth --profile default

You should now see the user script commands in the synth output
Let's deploy our new changes.
Cloudformation will only update resources that are being updated.
In this case, only ec2 instance is being updated.
Other things like roles and security groups will remain as they are since there are no changes to them in the updated stack.



cdk --deploy --profile default

Take note: Since we are not using an elastic IP, its highly likely that the public ip address of the instance has changed.

Let's use the outputted IP address and check to see that Wordpress, Mysql and PHP were installed correctly:

In your browser, navigate to http:///blog
You should then see:

and then you can complete the installation of Wordpress!
Remember your database credentials root and pl55w0rd as defined in the script

Testing the stack

Well we know our CDK code works and can provision an ec2 instance to run our Wordpress server and database. Good.

But how can we make sure that changes to the CDK code do not do what we don't want it to do?

This is where tests come in.

Test requirements

I do not want any instance other t2.micro to be used as my server instance type because I always want to remain under AWS free tier usage for EC2. Let's ensure that.
I want to ensure that my instance uses the SSH key with the name simple-instance-1-key

To accomplish this, we change the code inside the file ./test/simple-ec2.test.ts to:



import { expect as expectCDK, haveResourceLike } from '@aws-cdk/assert';
import * as cdk from '@aws-cdk/core';
import * as SimpleEc2 from '../lib/simple-ec2-stack';

test('Check InstanceType and SSH KeyName', () => {
    const app = new cdk.App();
    const stack = new SimpleEc2.SimpleEc2Stack(app, 'MyTestStack');

    expectCDK(stack).to(
      haveResourceLike('AWS::EC2::Instance', {
        InstanceType: 't2.micro',
        KeyName: 'simple-instance-1-key'
      })
    )
});

As you can see from the test code, the test will check the generated Cloudformation template generated by the CDK.

In our case we want to check that the instance is a t2.micro and that it uses the SSH key simple-instance-1-key. These are two crucial properties to us.

You can read more about testing infrastructure with the CDK here here

Run the test:



npm test

All good!

And now your code should be able to run a test before deploying your infrastructure! Fantastic!



npm test && cdk deploy --profile default

Destroying the stack

If you would like to destroy the infrastructure you just provisioned, it's as simple as:



cdk destroy --profile default

And Cloudformation will remove your entire stack!

Notes:

It's not advisable to run mysql on the same instance as your Wordpress server. You can instead use AWS managed Database service RDS to deploy the database that Wordpress will use.
Don't put sensitive information like passwords in user scripts since in many cases they are committed to source control or their output is visible in a CI/CD console or instance terminal history

Conclusion

The AWS CDK makes writing IaC, provisioning, deploying, updating and destroying infrastructure very painless. You can write tests to make sure you do not deploy the wrong things.

This was a simple example and may seem quite a lot just to deploy an ec2 instance. However, as we progress through the series, you will realize how its very beneficial to complex infrastructure.

Up next

In part 4, using the CDK, we will make our Wordpress server more more production ready. We will:

create AWS RDS Mysql database instead of running the database on the ec2 instance
provision this database in an isolated subnet to keep it secure from the public Internet
use AWS SSM to access our instance instead of an SSH key and gain all the benefits of IAM permissions/roles
deploy an Application Load Balancer
Create our EC2 instance with better/more advanced script
Place the Wordpress instance in an AutoScaling Group

Hi I'm Emmanuel! I write about Software and DevOps.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

Awesome AWS CDK - Part 2 - Setting up AWS CDK

Emmanuel K — Sun, 14 Mar 2021 09:39:04 +0000

What you will learn in this part

In this tutorial, you will learn:

How to install AWS CDK

Let's go!

Pre-requisites

You are going to need:

Setup

Make sure you have Node.js setup
Make sure your aws credentials are properly configured.

Here's an example of my AWS configured access files where I have my own personal aws account as the default profile and two other aws accounts for my work company -- acme-corp👨🏿‍💻.

AWS credentials file:

# Linux and MacOS: ~/.aws/credentials
# Windows: %USERPROFILE%\.aws\credentials

[default]
aws_access_key_id=xXXxxxxXXXXxxxxxxx
aws_secret_access_key=xxXXXXxxxXXxXxx

[acme-corp-prod-acc]
aws_access_key_id=xXXxxxxXXXXxxxxxxx
aws_secret_access_key=xxXXXXxxxXXxXxx

[acme-corp-dev-acc]
aws_access_key_id=xXXxxxxXXXXxxxxxxx
aws_secret_access_key=xxXXXXxxxXXxXxx

AWS config file:

# Linux and MacOS: ~/.aws/config
# Windows: %USERPROFILE%\.aws\config

[default]
region=us-west-2
output=json

[profile acme-corp-prod-acc]
region=us-west-2
output=json

[profile acme-corp-dev-acc]
region=us-west-2
output=json

The AWS CDK will look for these files when it needs access to whatever AWS accounts it needs to deploy resources to.
It helps that the IAM credentials used have administrator privileges when deploying resources as this will reduce a lot of headache regarding permissions.
You can filter out permissions and privileges if you have a good understanding of AWS IAM and the permission needs of your deployments.

Install the AWS CDK globally (so that you can bootstrap a new CDK project in your language choice) in any folder

npm install -g aws-cdk

Check that its installed

cdk --version

That's it!

In the next part, we will set up a project with the cdk and deploy it!

Hi I'm Emmanuel! I write about Software and DevOps.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

Awesome AWS CDK - Part 1 - A series on common and practical deployments with the AWS CDK

Emmanuel K — Sun, 14 Mar 2021 09:24:34 +0000

PART 1

Introduction

This is going to be the first in a series of articles about using the AWS CDK (Cloud Development Kit) for common and practical deployment of cloud resources on AWS. This is to help those who are already familiar with AWS to deploy cloud resources using this awesome IaC (Infrastructure as Code) library from AWS.

As you may have figured, the AWS CDK is targeted at the deployment of AWS cloud infrastructure only. So if you are afraid of AWS vendor lock-in for your cloud deployments, then maybe this will not be the series for you. There are other tools that can do multi-vendor cloud deployments and provisioning (like Terraform, Pulumi, Serverless).

However, if you feel like you use AWS a lot like many DevOps/Solutions Architects/Developers do then this is the series to show you practical examples of using the AWS CDK to deploy resources.

Background

In the beginning...

You can manually create resources on AWS using the console, command line interface or other IaC tools. And this is what people did when AWS was new. But it became evident pretty fast that this mode of provisioning cloud infrastructure was error prone and not sustainable, especially for massive and complex stacks of resources. These frustrations led to the birth of Cloudformation in 2011.

Cloudformation

AWS Cloudformation is AWS' managed service for provisioning cloud infrastructure and services. You do this using a JSON or YAML template file that defines what resources you would like AWS to Create, Update or Destroy. Cloudformation consumes file and makes the necessary changes to your deployed stack of resources.

Cloudformation is the backbone service of automatically deployed resources on AWS. It ensures consistency of your deployed resources and can keep track of drifts (when resources are created, updated or destroyed outside of the Cloudformation template file).

The problem is Cloudformation template files are ridiculously hard to write, read, update and manage. After all they are basic YAML/JSON. Its fine if you are provisioning a few resources. But it is completely unmanageable with human effort once you exceed 20 resources.

This is where the CDK comes in.

Cloud Development Kit

The AWS CDK is a library of functions that will generate a Cloudformation template and use it to provision resources in AWS. It allows you to write code to define in your favorite language to create these Cloudformation templates.

With the CDK, the ability to have access to intellisense from your code editor, ability to break up resources into logical parts, type checking, enums etc means that the tendency to make errors when configuring resources dramatically reduces. You can also write tests for the infrastructure you want to deploy to ensure that whatever changes made in the IaC are not going to break your deployed stack of resources or cost you millions when you mistakenly provision beyond your means.

Its been a joy working with CDK for the past year for me and it is hard to go back to other solutions like Terraform, Serverless, SAM for AWS resource provisioning.

It's not all roses yet though

No project is perfect. The AWS CDK, as great as it is, is also a recent project (launched in July 2019) so it doesn't yet cover everything possible to deploy on AWS (Even Cloudformation doesn't cover everything possible in AWS yet). Some things will need manual configuration via the console or command line interface. There are also minor bugs, fast changing interfaces, deprecations etc as the CDK development tries to catch up to and keep up with the pace at which AWS releases new services and updates. Maybe within a year many of my own code examples from this series will have some deprecations. But the core API and concepts will remain the same. Also, for most common services, the CDK will have you covered.

Conclusion

Note:

I will focus on simple and common deployments.
I will use github actions as my CI/CD tool.

There are other projects out there that focus on patterns like the awesome cdk patterns project which concentrates on serverless deployments with the cdk and AWS' own aws-cdk-examples project which has tons of great examples on using the cdk. I drew tons of inspiration and learning from these projects.

My series will focus on simple common deployments with a heavy leaning on practicality. This means I will take you through some best practices as well as well as my own experience.

I will use Typescript because:

The CDK API is essentially the same in Typescript, Python, .Net, Java. The language itself isn't too important.
The CDK is also written in Typescript and then wrapped for use in other languages.
It's awesome!

Hope you can stick with me throughout. See you in part 2!

Hi I'm Emmanuel! I write about Software and DevOps.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

Is the AWS Solutions Architect Certification worth it? A review

Emmanuel K — Wed, 17 Feb 2021 07:53:39 +0000

TLDR: Yes it is! But not for the reasons you may think.

Intro

AWS has very many services. And every year when re:Invent rolls around, they add even more. Most people don't even dare venture into any other services beyond EC2, S3, Lambda and RDS. These are the primary services you'd use to setup a simple web server, for example. If you're a little more involved you may use Route53 to manage domains, VPC to manage your resource networks and subnets and Cloudfront as a CDN. Most people would find little use for SQS, Kinesis, DynamoDB, Step Functions etc

Additionally, many DevOps engineers like to keep the deployed cloud infrastructure as high level as possible to prevent over-reliance on managed AWS services and avoid vendor lock-in (the concern of which is usually overblown). However even if you are unnecessarily afraid of vendor lock-in, content with managing cloud infrastructure yourself or you really have legitimate reasons to avoid using AWS managed services, getting (or at least studying for) an AWS Solutions Architect certificate is still worth it.

This is because the AWS Solutions Architect exam (both Associate and Professional), doesn't just challenge you on your knowledge of AWS services, it challenges how you think about cloud infrastructure altogether in order to... yes you guessed it -- architect solutions.

A Solutions Architect with bad solutions?

Nothing is worse than a mechanic with a box full of tools who doesn't really know how to fix things. The AWS Solutions Architect Associate exam (the one I did) is truly a test of understanding the world of managed cloud services and how one can deliver practical and reliable solutions. You WILL need to know and understand not only many AWS services but also how they work together to produce the best possible results according to various scenarios.

Because of this, this is not an exam you can study for on a weekend and pass on Monday. Even though some people may be able to memorize common solutions, the questions can and will be quite challenging and even seasoned AWS pros need to study for it.

I had used AWS for about 3 years before doing the AWS Solutions Architect Associate exam in July 2020. I had deployed instances, serverless functions, message brokers and queues, databases etc But I only started to realize the many gaps in my knowledge once I started reading for the exam. I realized certain things I would do were either not best practice, impractical, not secure or just plain ridiculous. And a lot of it had nothing to do specifically with AWS services, but more to do with doing DevOps and cloud infrastrcture design in a correct way. I ended up reading many of the famous AWS whitepapers like Well Architected Framework and AWS Security Checklist and gained a wealth of knowledge I wouldn't have obtained otherwise because I thought I knew what I was doing and I would never have taken the time to dive deeper into AWS.

A new understanding

Within one week of starting my revision, I knew several things:

I wasn't ready to do the exam anytime soon
I had deployed poor or insecure architecture
I was eager to apply my new found knowledge and understanding almost immediately

Around this time I started to see the real value in what I was training myself for. See, my original intention was to get the certification just to look good on my resume and I really thought I'd only need about two weeks to prepare for the test (after all, I had used AWS for quite a while). I ended up using about 5 weeks in total for preparation (usually more than 7 hours a day). I also took two video courses to supplement my knowledge (one from ACloud Guru and one from Stephane Maarek)

This is not to frighten/discourage you. Most people who attempt the test with good preparation WILL pass. But the point of this post is to shed light on the value studying for this kind of exam will bring in your DevOps/Cloud journey beyond the certification itself. A lot of the knowledge I gained from SAA-C02 can easily be applied to other cloud providers such as Azure and GCP. Their services may have different names and may be slightly different in their capabilities and functionality but the core principles of deploying robust, highly scalable, highly available, fault tolerant, disaster tolerant and manageable cloud infrastructure still apply.

It's been about 8 months since I did my test and while it has helped my profile gain more notice in the DevOps hiring space, the greater gift is the expanded knowledge that I'm utilizing in my daily DevOps life of writing good Infrastructure as Code and managing existing cloud infrastructure. It makes me much more confident in whatever I'm doing inside AWS.

Conclusion: Bite the bullet

So if you are a:

DevOps Engineer
Software Engineer
Full-stack Developer
Web Developer
CTO
Principal Tech Lead
Curious noob who wants to be any of the above
or even an AWS employee who has to interact with AWS services a lot

then the certification is definitely worth studying for and obtaining.

But don't do it for the wrong reasons i.e. its a hot commodity that everyone talks about constantly, so you must have it. The certification itself is a nice to have and does look good on your profile/resume/CV and will open more doors for you, but the deeper understanding of how AWS and similar cloud providers works is much more valuable. You will forego years of trial and error experience by learning so many essentials in one fell swoop.

If you would like more information about how to start this journey towards certification or any other general questions, send me a message here, on Twitter or on LinkedIn .

Hi I'm Emmanuel! I write about Software and DevOps.

I will be writing a series on AWS CDK, Terraform and other great deployment tools and what many lessons I've learned from them.

If you liked this article and want to see more, add me on LinkedIn or follow me on Twitter

Using sudo without password prompt as non-root docker user

Emmanuel K — Tue, 17 Nov 2020 06:50:43 +0000

Follow me on twitter: emmanuelnk

WARNING: This is generally considered NOT SECURE and thus do not use the methods in this article in a production container.

I have a particular use case that justifies this usage. Now you can read on.

Recently I wrote a pure bash menu program that has install scripts for various Ubuntu software I use. This is to allow me to reproduce my dev environment as fast as possible on a new Ubuntu installation.

I want to release this program as an open source tool and for that reason it needs testing and CI. Docker is perfect for this. Except for one thing. Most docker images use user root to execute commands. This is fine for most intent and purposes. But to appropriately test my program, I would need to be a non-root user inside the docker container.

Changing the root user

This is trivial and actually quite common in Dockerfiles. By default, most docker images, including ubuntu:latest have USER set to root. This can be changed by creating a new user in a Dockerfile by:

RUN useradd -ms /bin/bash newuser
# where
# -m -> Create the user's home directory
# -s /bin/bash -> Set as the user's 
# default shell
USER newuser

This will create a newuser without root privileges to run commands in the container.
NB: You can add && echo 'pa55w0rd' | chpasswd right after the useradd to set a password.

For my use case, I need the password disabled and I need to NOT be prompted for a password when using sudo command. Now you may be asking, why would someone want to do this?

Well if you're using docker in CI and need to test certain commands being run as a regular user then this is the way. For example, on my machine, I am the USER=emmanuel. I don't have root privileges and when I need to install something, I do sudo apt-get install and enter my password to give me su access.

For my project, I'm trying to test install scripts as a regular user and thus these scripts use sudo and variables such as $HOME a lot ($HOME for root is /root). Hence using the default that most docker images have, programs would not install in the correct locations or not install correctly at all. This is not good.

Anyway, enought talking. This is what my Dockerfile looks like to accomplish this. Explanations in the comments.

# Get latest official Ubuntu image
FROM ubuntu

# ubuntu:latest does not have sudo
# fetch it and install it
RUN apt-get update && apt-get install -y sudo

# Create new user `docker` and disable 
# password and gecos for later
# --gecos explained well here:
# https://askubuntu.com/a/1195288/635348
RUN adduser --disabled-password \
--gecos '' docker

#  Add new user docker to sudo group
RUN adduser docker sudo

# Ensure sudo group users are not 
# asked for a password when using 
# sudo command by ammending sudoers file
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> \
/etc/sudoers

# now we can set USER to the 
# user we just created
USER docker

# we can now run sudo commands 
# as non-root user `docker` without
# password prompt
RUN sudo apt-get update 

WORKDIR /home/docker/src

COPY . .

That's it. Thanks!

If you have suggestions, improvements or want to correct me, let me know in the comments!

If this helped you out, follow me on twitter: emmanuelnk

Local Python Module imports when using virtualenv

Emmanuel K — Sun, 15 Nov 2020 07:49:44 +0000

Importing modules in python3 can be unnecessarily confusing, especially when using virtualenv

Suppose you have this file structure:

.env
test.py
cool_modules
    |--__init__.py
    |--cool_module1.py
    |--cool_module2.py

To prevent import errors while using virtualenv .env:

cool_module1.py:

class CoolModule1():
    ...

cool_module2.py:

class CoolModule2():
    ...

Here's the critical part.
init.py:

# The part to note here is you have to use `parent_folder.child_file`
from cool_modules.cool_module1 import CoolModule1
from cool_modules.cool_module2 import CoolModule2

test.py:

from cool_modules import CoolModule1, CoolModule2

That's it!

If this helped you out, follow me on Twitter: emmanuel_n_k

DEV Community: Emmanuel K

Simplifying GitHub Actions Workflows Through Typescript

Quick Summary

The Challenge with YAML

Why Not YAML for GitHub Actions?

The Epiphany

The Inspiration

Introducing github-actions-workflow-ts

Get Started Now

About the Author

How to monitor long running ALTERS in AWS RDS MySQL 5.7+ using the Performance Schema

How to get old LinkedIn app in China

The Problem

The Solution

Pre-requisites

Initial Steps

Next steps

Saving and Retrieving Well-known Text (WKT) in MySQL or Postgres with TypeORM

Files

Insert GeoJSON

Retrieve GeoJSON

Part 4 - Wordpress EC2 instance in ASG with RDS database and ALB- Awesome AWS CDK

Introduction

Abbreviations

Setup

Setup AWS configuration and CDK

Create a new project

Project structure

Config

VPC

Setup

RDS

Setup

Application Load Balancer

Important Load Balancer concepts

Setup

EC2 via Autoscaling Group

Setup

User script

EC2 instance in ASG

Deployment

Local machine

Github Actions

Destroying the stack

Final result

Conclusion

Debugging

Homework Assignments

Quickly install software on a new Ubuntu installation with GMUAR

Part 3 - Simple EC2 instance - Awesome AWS CDK

Introduction

Bootstrapping a new CDK Application

Structure and Setup of a CDK application

Structure

Setup

Writing the infrastructure code

Creating a new Key Pair in the AWS Console

Deploy

Accessing the instance

Adding User script

Update the deployed stack

Testing the stack

Test requirements

Destroying the stack

Notes:

Conclusion

Up next

Awesome AWS CDK - Part 2 - Setting up AWS CDK

What you will learn in this part

Pre-requisites

Setup

Install the AWS CDK globally (so that you can bootstrap a new CDK project in your language choice) in any folder

Awesome AWS CDK - Part 1 - A series on common and practical deployments with the AWS CDK

PART 1

Introduction

Background

In the beginning...

Cloudformation

Cloud Development Kit

It's not all roses yet though