DEV Community: Colin Duggan

A CLI tool to visualize AWS cost and usage data

Colin Duggan — Mon, 27 Feb 2023 11:58:36 +0000

ccExplorer is a CLI tool I created to make it easier to surface AWS cost and usage data in human readable formats (like table, csv, chart, and HTML)and which can be used to easily sort results in descending order by cost or date.

It doesn't replace the cost explorer capabilities provided by the AWS CLI but instead focuses on providing more convenient output formats which enables the user to quickly reason about data without having to traverse a YAML or JSON tree.

CostAndUsage Queries

Where possible ccExplorer uses the same nomenclature as the AWS CLI when naming command line options. Queries use the groupBy or filterby flags to specify the dimensions and/or tags used when fetching cost and usage data and when outputting results.

By default results are sorted by cost in descending order. The following command exports results to stdout and groups by SERVICE and OPERATION. The -l flag is used to exclude discount, credit and refund information to get a true sense of costs.

ccExplorer get aws -g DIMENSION=SERVICE,DIMENSION=OPERATION -p stdout -m MONTHLY -l

Cost Allocation Tags

ccExplorer makes it easy to surface costs associated with custom cost allocation tags. For example:

Surface S3 Bucket Costs ccexplorer get aws -g DIMENSION=SERVICE,TAG=BucketName -f SERVICE="Amazon Simple Storage Service" -l
Surface DynamoDB Table Costs ccexplorer get aws -g DIMENSION=SERVICE,TAG=DynamoTableName -f SERVICE="Amazon DynamoDB" -l

Print Formats

ccExplorer supports output formats including CSV, chart, and HTML. HTML reports are generated using OpenAI and the GPT-3 completions API. In order to use this feature the end user must supply an OPEN AI API KEY. The resulting HTML report includes the top 10 costs in descending order. Results also include cost optimization hints for each of the highest costing resources. Note: The HTML output type does not support queries which include the LINKED_ACCOUNT dimension type, which would require sending sensitive account information to GPT-3. The HTML output type is still very experimental and subject to changes.

Conclusion

At the outset I was hoping to build something that could quickly surface high level data about the resources I was using. I quickly discovered that ccExplorer was also deepening my knowledge across other facets of AWS including usage and operation types. I was able to pinpoint the cost implication of specific resources including S3 buckets and DynamoDB tables. This has helped me to better reason about the workloads I manage and ultimately eliminate inefficiencies.

ccExplorer is still very much a work in progress but I hope you find it useful. Suggestions and feedback are welcome.

Using AWS EventBridge to Handle Stripe Webhook Events

Colin Duggan — Fri, 25 Mar 2022 18:09:12 +0000

In this article, I want to walk through the process of integrating a Stripe asynchronous subscription event with AWS EventBridge.

In a nutshell, EventBridge is a serverless event bus which hugely simplifies the process of building event-driven applications. This simplification is made possible with AWS taking care of event ingestion, delivery, security, authorization, and error handling.

What about Stripe? It is a fully integrated payments platform offering a rich set of APIs, a client library available in a range of languages, webhooks and a CLI which greatly simplifies life for a developer.

There are still several moving parts to consider with this serverless example With AWS CDK much of the complexity can be offloaded (GitHub link for sample project included later). The AWS stack includes:

An API Gateway endpoint acting as a target for the Stripe webhook event with integration request type of LAMBDA_PROXY.
A Lambda function which verifies the Stripe webhook request signature before programmatically sending the EventBridge event. The handler function accepts events of type APIGatewayProxyRequest.
The EventBridge event bus with supporting rule. When EventBridge receives an event matching the pattern defined by the rule it will send to the source supplied in the event pattern.
A second Lambda function which acts as the target for your EventBridge rule. The handler function accepts events of type CloudWatchEvent.
And finally, a DynamoDB table updated as a consequence of the target functions being invoked.

Step 1 Creating the Event Bus

You have options when it comes to creating an event bus. Use the default event bus which handles all events emitted by AWS services, create a partner event bus described as being suitable for integrating with SaaS partners, or finally the option I have chosen which is to create a custom event bus which will receive events from the services and applications we create. AWS CDK reduces setup to a few lines of code(using Python CDK).

L1 Create event bus. L3 Create rule and associate with event bus, the default event bus will be used if one is not specified. L9 Define rule event pattern. L14 Add the target EventBridge will send the event to when a rule is emitted matching the pattern you have defined.
In this situation, we are not attempting to capture events emitted by the native AWS services in our account. If that was the case the default event bus must be used. When creating our rule we also have the option of choosing a target from one of the almost 20 AWS resources EventBridge can emit events to ranging from SQS to RedShift. We can also specify a Dead-Letter queue which EventBridge can use to send unsuccessfully delivered events.

Step 2 Creating a Stripe Webhook

I will be using the Stripe developer dashboard to create a test webhook, you can also do all this through the API. Through the web dashboard, use the API Gateway endpoint along with the customer.subscription.created event type to complete setup.

Step 3 Creating Lambda Handlers

The first Lambda function we create will handle the webhook request via our API Gateway endpoint. Some code is excluded from the snippet below however the main thrust of this function is to verify the incoming Stripe request signature. The signature is included in the incoming requests Stripe-Signature header and allows you to verify the request without requiring a third party. Signature verification is done by first downloading the endpoint's secret which we subsequently place in AWS Secrets Manager. Once verification is successful, the function creates an event and emits it to EventBridge.

The second Lambda function will act as the target for the event bus rule we have defined earlier and which will subsequently write to DynamoDB. It accepts events of type CloudWatchEvent. Details of the new customer subscription are extracted from the CloudWatchEvent object and written to the database table.

Step 4 Invoke the Test Webhook

Through the Stripe developer dashboard invoke the new webhook endpoint selecting the customer.subscription.created event type.

Step 5 Verify Rule Metrics and Database Update

CloudWatch metrics for your event bus rule allow you to quickly sanity check whether the rule was invoked successfully or not.

Finally, verify the database has been updated accordingly with the new customer ID.

Conclusion
It has been suggested that AWS EventBridge is the biggest thing to happen for serverless since Lambda. This is only my first foray with EventBridge so I won't wax lyrical just yet however I can certainly see where much of the excitement stems from. With an ever-growing number of integration options both AWS native services and SaaS providers, advanced monitoring and analytics, and a simple API for generating and emitting custom events it really does offer a management and coordination layer to tie all of your serverless modules together. I'm looking forward to watching this technology grow and hopefully have the opportunity to share more insights.
A full code example can be found over on GitHub: https://github.com/cdugga/eventbridge-stripe-go

Compiling a Go Program

Colin Duggan — Mon, 07 Mar 2022 21:38:22 +0000

The Go command provides a number of options allowing you to run, build or install your Go program. The following diagram illustrates these options at a high-level.

Running a Go Program

Go is a compiled language. Source code belonging to a Go project must be run through a compiler. The compiler generates a binary which can then be used to run the program on a target OS.

Phases of the compiler

Scanner: Converts source code to tokens for use by the parser.
Parser: Converts tokens into an abstract syntax tree to be used by code generation
Code generation: Compiles and install the packages to $GOPATH/bin

Options for Compile and Run

Module	Description
`go run`	A command used for convenience allowing you to compile and run a program. While it does generate a temporary binary to run the program, that binary is deleted once execution completes.
`go build`	A command which compiles the packages named by the import paths, along with their dependencies, into a binary in the current working directory. It does not install the resulting binary.
`go install`	A command which compiles the command in a temporary directory and moves the generated binary to the default target `$GOPATH/bin` alongside third-party dependencies. If a value has been provided for the $GOBIN environment variable, then it will install to that location instead.

Related Environment Variables

Name	Description
`GOBIN`	If the `GOBIN` environment variable is set, commands are installed to the directory it names instead of the default `$GOPATH/bin` folder
`GOPATH`	Required to resolve import statements. Defaults to the user's home director, `$HOME/go` on Unix or `%USERPROFILE%/go` on Windows

The `GOPATH` directory structure

GOPATH is a list of directory trees containing source code. It is consulted to resolve imports that cannot be found in the standard Go tree.

src - Holds source code. The path below src indicates the import path.
pkg - Package directory holds installed package objects. Each target OS and architecture pair has its own subdirectory
bin - Holds compiled commands. Each command is named from its source directory, but only the final element and not the entire path. The command prefix is stripped, so you can add $GOPATH/bin to your path to get all installed commands.

Useful Commands

Name	Description
`go clean -cache`	Removes entire go build cache.
`go clean --modcache`	Removes entire module download cache. Including unpacked source code of versioned dependencies
`go tool fix`	Runs the `go fix` command on packages named by the import paths
`go mod vendor`	Copies all third party dependencies to vendor directory in your project root. The go command loads packages from vendor directory instead of downloading modules from their sources into the module cache
`go env`	Location of cached files from `go build`command
`go mod tidy`	Loads packages in main module and transitive dependencies recursively. Does not consider packages in the main module in directories named `testdata`or with names that start with '.' or '_' unless those packages are explicitly imported by other packages.

References

A Quick Guide to Go's Assembler

Middleware in Go

Colin Duggan — Wed, 16 Feb 2022 19:31:13 +0000

Middleware is a powerful idea that allows us to decompose complex systems into layers of abstractions. When we discuss middleware in the context of Go we are referring to the ability of some self-contained code (third-party or other) to hook into a server's request/response processing before or after its handler is invoked. This is a powerful concept which allows us to separate cross-cutting concerns such as:

Observability
Logging
Authentication

Middleware code typically does one thing and then passes the request to the next layer or final processing before returning to the client. This results in code that is more modular, easier to maintain and reusable.

How It Works

HTTP request processing in Go is handled by two things, ServeMux and handlers. The ServeMux, a request multiplexer, simplifies the association between URLs and handlers by matching the URL of incoming requests against predefined patterns and then forwarding to the appropriate handler. Using middleware we can interrupt this flow and instruct our middleware function to act before or after the handler function has executed. We can also dictate whether to apply the middleware to all request or just those matching a particular pattern.

The following snippet represents a typical example of how we would use a ServerMux to associate an incoming request URL with a specific handler.

func main() {
  mux := http.NewServeMux()
  mux.Handle("/users", http.HandlerFunc(usersList))
}
func usersList(w http.ResponseWriter, req *http.Request){
  // fetch users
}

Our handler of type func(w http.ResponseWriter, req *http.Request) doesn't satisfy the http.Handler interface and therefore cannot be passed directly to mux.Handle.

To resolve this issue and satisfy the http.Handler contract we use the http.HandlerFunc(usersList) expression. Note this is not a function call but instead a conversion. The HandlerFunc type has methods and satisfies the http.Handler interface. Its ServeHTTP method calls our underlying function, therefore acting as an adapter. We can now use our usersList function as a http handler now that we are satisfying the http.Handler interface.

// The HandlerFunc type is an adapter to allow the use of
// ordinary functions as HTTP handlers. If f is a function
// with the appropriate signature, HandlerFunc(f) is a
// Handler that calls f.
type HandlerFunc func(ResponseWriter, *Request)
  // ServeHTTP calls f(w, r).
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
  f(w, r)
}

Adding Middleware

Armed with the knowledge of how a typical request handler is constructed we can now quite easily extend our example to include a middleware capability. Once again we are relying on the http.HandlerFunc adapter type to allow us wrap our middleware function so it implements the http.Handler interface. This time we also want the ability to chain handlers together.

func middlewareFunc(next http.Handler) http.Handler {
 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
  // Some logic …
  next.ServeHTTP(w, r)
 }
}

The anonymous inner function closes over the next variable allowing us to effectively chain or transfer control from this handler to the next by calling ServeHTTP. Because the inner function has the signature func(rw http.ResponseWriter, r *http.Request) we can convert it to a HandlerFunc type using the http.HandlerFunc adapter.

The signature of our middleware function can be replicated by other middlewares to create arbitrarily long chains. Control can also be stopped at any time by simply issuing a return from the middleware handler instead of calling the next handler.

Package Management in Go

Colin Duggan — Wed, 16 Feb 2022 09:18:15 +0000

Go Package Management

Go takes a semi-decentralized approach to package management, allowing any git repo to be used as a module package system. Modules can be downloaded directly from their source control servers or alternatively through a module proxy which performs a simple forwarding of requests to the appropriate source. The following diagram is a high-level take on what happens when go get attempts to resolve a package or module.

Module Proxy

The module proxy is an HTTP server which implements the module proxy protocol. It responds to GET requests matching a particular pattern, See examples below. The module proxy ensures changes or downtime to a module origin server will not bubble up and impact your build process. The module proxy can also act as a gatekeeper, restricting access to modules which have unsuitable licences or contain security vulnerabilities. Module proxies can also provide faster dependency resolution by caching previously fetched modules, which can then be used to serve future requests. A module's dependencies are defined in the following files:

Module	Description
`go.mod`	Defines the module name, the version of Go with which to build the project and the list of dependencies.
`go.sum`	Contains cryptographic hashes of the module's direct and indirect dependencies.

Module Proxy Protocol

Since version 1.13, proxy.golang.org is the default module proxy for fetching modules.

It's possible to build your own module proxy, which consists of an HTTP server and REST API that satisfies the GOPROXY protocol.

Go Get Command

The go get command follows this protocol when attempting to resolve a package path:

Search modules in the current project build list (go.mod) including all build lists of transitive dependencies, looking for modules with paths which are prefixes of the requested package path.
If one module in the build list contains the required package, then that module will be used.
If none of the modules in the searched build lists provide the package or if two or more modules provide the requested package, then go get will report an error.

When go get is resolving a new module for a package path , it uses the GOPROXY environment variable to determine whether modules will be downloaded directly, through a module proxy or some other private mirror. GOPROXY accepts a list of proxy URLs separated by either commas or pipes. The separator used in this list determines how the go get command will fallback in different scenarios:

`GOPROXY` Separator Type	Behaviour
List separated by commas	`go get` will fallback to the next URL in the event of a 404 or 410 HTTP response. All other response codes are treated as terminal.
List separated by pipes	`go get` will fallback to the next URL in the proxy list in the event of any HTTP or non-HTTP error.

The go get command visits each proxy in the list sequentially until it receives a successful response or error.

GoProxy Environment Variable Options

The default configuration for the GOPROXY environment variable is GOPROXY=proxy.golang.org,direct which effectively tells the go get command to first attempt module retrieval using the module mirror and if that approach is unsuccessful, fallback to the approach of connecting directly to the module origin repository. A number of other environment variables can also impact how go get will attempt to resolve dependencies. These variables are listed here.

`GPROXY` keywords	Description
direct	`go get` will download directly from version control repo
off	Disallows downloading from any source

Go Module Proxy Request Patterns

A GET request for a particular module will follow the pattern: https://$base/$module/@v/$version.zip.

Where:

Request portion	Description	Sample value
$base	The path portion of the proxy URL	proxy.golang.org
$module	Module path	go.uber.org/zap
$version	Version	v1.21.0

The resulting GET request will download the module and dependencies from: https://proxy.golang.org/go.uber.org/zap/@v/v1.21.0.zip. The 'go get' and 'go mod tidy' commands abstract the details of this from the developer.

Go Module Proxy Response Codes

The type of separator used in the GOPROXY environment variable will determine if the go get command will fall back to the next option in the list or fail completely. See section on 'go get'.

Proxy Response Code	Description
200	OK
300	Redirects are followed
404	Not found
403	Forbidden - Private proxies can choose to prevent access to modules based on unsuitable licenses or security vulnerabilities
410	Module not available on the proxy server but might be available elsewhere
4xx & 5xx	Treated as errors

Go Environment Variables Which Impact Module Retrieval

If set with specific values, GOPRIVATE and GONOPROXY can impact the behaviour of GOPROXY, by preventing specific modules from being downloaded through the proxy.

Environment Variable	Description
`GOPRIVATE`	Comma separated list of module prefixes considered private. If set, this acts as a default value for `GONOSUMDB`
`GONOPROXY`	Comma separated list of module prefixes, which should always be downloaded directly from source. If set, this acts as a default value for `GOPRIVATE`
`GONOSUMDB`	Comma separated list of module prefixes which the `go` command should not verify checksums using the checksum database.

Verifying Modules

After downloading a .mod or .zip file, go get computes a cryptographic hash and checks that it matches a hash in the main modules go.sum. If the hash is not present in go.sum then the go command retrieves it from the checksum database. The checksum database, located at sum.golang.org, allows for global consistency and reliability for all publicly available modules. It also ensures bits associated with specific module versions do not change from one day to the next.

Module Caching

The 'go get' command will cache modules downloaded from both module proxies and directly from a module's source repository in $GOPATH/pkg/mod/cache/download. This local cache layout is identical to the proxy URL space.

Module Graph

Module graph pruning was introduced in version 1.17. See go.dev for more information. Module pruning uses Minimal version selection to select a set of module versions to use when building packages. It operates on a directed graph of modules, specified in go.mod. Each vertex in the graph represents a module version. Each edge represents a minimum required version of a dependency, specified using a require directive. The graph may be modified by exclude and replace directives in go.mod. MVS produces the build list as an output.

Command	Description
`go list -m all`	Prints a modules build list
`go mod graph`	Prints the edges of the graph, one per line. Each line contains a module version and one of its dependencies
`go mod graph -go <some-version>`	Identical to previous command however it prints based on how the selected go version would interrupt the graph
`go mod verify`	Checks dependencies of the main module stored in module cache have not been modified since they were downloaded
`go mod why`	Shows the shortest path in the import graph from the main module to each of the listed packages

Full details can be found at understanding-go-get

Working with EKS: Using IAM and native K8s service accounts to access AWS S3

Colin Duggan — Sat, 05 Feb 2022 13:40:32 +0000

This post looks at how a native K8s service account can be used along with an IAM role as a strategy for managing AWS credentials for applications running in EKS.

This use case arises when an application running in an EKS pod wants to use the AWS SDK to call some AWS service. Requests must be signed with an AWS credential. The SDK will use the default credential provider chain to identify a suitable credential to use. In this scenario, we want to leverage the AWS_WEB_IDENTITY_TOKEN_FILE credential. The following paragraphs look at the steps required to surface that credential, so we can connect to and read from an S3 bucket.

We will take advantage of EKS's built-in support for using AWS IAM user and roles as entities for authenticating against a cluster. The first step is to create an IAM OIDC Identity provider using the OpenID Connect provider URL, which is automatically provided during cluster creation.

Create IAM OIDC provider for the EKS cluster

The OpenID Connect provider URL is available under the EKS dashboard under tabs: Configuration - Details
The provider URL can be copied and used to create a new Identity Provider in IAM.

Provider Type - choose OpenID Connect
Provider URL - paste the OIDC URL from your cluster
Audience - sts.amazonaws.com If a provider already exists matching the name of your clusters' provider URL, then there is no need to continue with this step.

Create IAM Role for the EKS Service Account

In this step, we create an IAM policy which specifies the permissions our container will need in order to connect to and read from an S3 bucket. Once the policy is created, we require a new role against which the policy will be attached. The following snippet provides a sample policy file which grants permissions to read from a specific S3 bucket.

IAM Policy



{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::my-s3-bucket/*"
      ]
    }
  ]
}

Create a new IAM role. Attach the previously described IAM policy. Edit the trust policy and add the following statement;

IAM Role for Service Account



{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<aws account>:oidc-provider/oidc.eks.<region>.amazonaws.com/id/<oidc provider ID>"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.<region>.amazonaws.com/id/<oidc provider ID>:sub": "system:serviceaccount:<cluster namespace>:<service account name>"
                }
            }
        }
    ]
}

OIDC provider ID = OIDC provider URL for your cluster
Region = AWS region where your cluster is deployed
Cluster namespace = Namespace where your EKS service account lives
Service account name = Associate the Role with a service account in our EKS cluster

Detailed instructions for creation of both IAM Policy and Role are available at EKS Documentation

Associate IAM Role with K8s Service Account

Add the following annotation to your k8s service account definition to associate the newly created IAM Role



apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<aws account>:role/eks-s3-role
    eks.amazonaws.com/sts-regional-endpoints: "true"
  name: my-serviceaccount
  namespace: my-cluster-namespace

The service account is also annotated to use the AWS Security Token Service Regional endpoint rather than the global endpoint to reduce latency, build in redundancy and increase session token validity.

The service account is associated with pods running in your namespace through the _serviceAccountName _directive in your deployment definition.



spec:
      serviceAccountName: my-serviceaccount
      containers:
      ....

The next time EKS attempts to schedule a pod with this definition, it will trigger a mutating webhook, which is responsible for writing the following environment variables to our pod

AWS_WEB_IDENTITY_TOKEN_FILE
AWS_ROLE_ARN

We can verify the environment variables exist by running the command:
kubectl exec -n <namespace> <pod-name>-- env | grep AWS

Testing S3 Access

Once we have verified the presence of the AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN environment variables, our pod is ready to connect to and read from S3. The following Typescript snippet illustrates a simple example which fetches the names of all the S3 buckets in our account. It uses the AWS SDK for JavaScript V3



import {

  ListBucketsCommand,

  ListBucketsCommandOutput,

  S3Client,

} from '@aws-sdk/client-s3';

export const s3API = (): Client=> {

  return {

    listBuckets: async () => {

      let bucketNames: string[] = [];

      const s3Client = new S3Client(

          {region: "eu-west-1"}

      );

      let data: ListBucketsCommandOutput;

      try {

        data = await s3Client.send(new ListBucketsCommand({}));

        data.Buckets.forEach((bucket) => {

          bucketNames.push(bucket.Name);

        });

      } catch (err) {

        console.log('Error', err);

      }

      return bucketNames;

    },

Summary

By using IAM Roles with k8s native service accounts, we obviate the need to provide extended permissions to the EKS node IAM Role. This allows us to follow the principle of least privilege. We can scope IAM permissions for each service account, ensuring containers only have access to those privileges needed to complete its task.

References:

Creating an IAM OIDC provider https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html
EKS Workshop IAM Roles for Service Accounts https://www.eksworkshop.com/beginner/110_irsa/oidc-provider/
Create IAM Role for Service Account https://docs.aws.amazon.com/eks/latest/userguide/create-service-account-iam-policy-and-role.html
Associate IAM role to Service Account https://docs.aws.amazon.com/eks/latest/userguide/specify-service-account-role.html
AWS SDK for JavaScript v3 https://github.com/aws/aws-sdk-js-v3

Hosting a static WordPress Site on AWS S3

Colin Duggan — Thu, 27 Jan 2022 10:45:26 +0000

AWS S3 is a good choice for hosting static WordPress sites. It's secure and inexpensive, and frees up time which would otherwise be spent ensuring plugins, themes, and security procedures were appropriately configured.

The article describes one approach for generating the static site. The local development environment requires Docker Compose, and the only requisite plugin is the Simply Static WP Plugin, which exports the static site to a zip file before being uploading to S3.

Prerequisites

Docker Engine
Docker Compose
S3 bucket configured to host a static website

Preparing the WordPress Site

A quick start guide for running WordPress with Docker Compose can be found at docs.docker.com. I've included the reference docker-compose.yml file included in that startup guide in the snippet below. It can be used to build a local WordPress instance by running the command docker-compose up -d. The resulting containerized version of WordPress will be available at http://localhost:8000.



version: "3.9"

services:

  db:

    image: mysql:5.7

    volumes:

      - db_data:/var/lib/mysql

    restart: always

    environment:

      MYSQL_ROOT_PASSWORD: somewordpress

      MYSQL_DATABASE: wordpress

      MYSQL_USER: wordpress

      MYSQL_PASSWORD: wordpress

wordpress:

    depends_on:

      - db

    image: wordpress:latest

    extra_hosts:

      - localhost:172.23.0.3

      - localhost:172.23.0.1

    volumes:

      - wordpress_data:/var/www/html

    ports:

      - "8000:80"

    restart: always

    environment:

      WORDPRESS_DB_HOST: db

      WORDPRESS_DB_USER: wordpress

      WORDPRESS_DB_PASSWORD: wordpress

      WORDPRESS_DB_NAME: wordpress

volumes:

  db_data: {}

  wordpress_data: {}

Export Static Site

The static site generation part of this work can begin once the WordPress site has been fully configured. Configuration includes selection of the site's theme. Installation of plugins which support the site's functionality. Addition of site specific content, as well as any other customizations required to produce the final result.

1 Navigate to _Plugins\Add New_ and search for _Simply Stati_c.

2 Once installed, navigate to Simply Static\Settings and ensure the following options are configured for destination URLs and delivery method

3 Update the location for Temporary Files by navigating to Simply Static\Settings\Advanced. (This path will write to our wordpress_data volume)

4 Generate static files by navigating to Simply Static\Generate and clicking the Generate Static Files button

5 Once step 4 is complete, click the link to download the static content. Inspect the contents of the .zip file. It should include the following items

- index.html
- wp-content/
- wp-includes/

Uploading to AWS S3

In this step, output from running the Simply Static Plugin, is uploaded to S3.

At the point, the S3 bucket should be configured for static website hosting. This can be enabled through all the usual AWS paths (REST API, SDK, CLI, CloudFormation). We will be sticking with the console.

Click on the S3 bucket. Navigate to the Properties tab. Scroll down to the section titled Static website hosting. Enable and select Host a static website as the hosting type. Update the Index document field with the index.html landing page. Save changes.

Now upload the contents of the Zip file (index.html, wp-content, wp-includes) using the Upload option on the S3 bucket landing page.

The shiny new static WordPress site will now be accessible using the public URL for the site (available under the Properties tab for the S3 bucket under the section Static website hosting).

Secure S3 Bucket with Origin Access Identity (OAI)

The S3 bucket can be secured behind a CloudFront distribution to give greater control over access to the S3 bucket. The S3 bucket should be marked private. An Origin Access Identity (OAI) will be created and associated with a CloudFront distribution. The S3 bucket permissions are then updated to allow the CloudFront distribution to use the OAI to access the static site. The following snippet illustrates the bucket policy which will allow the CloudFront distribution access. Full details can be found on docs.aws.amazon.com.



{

    "Version": "2012-10-17",

    "Id": "PolicyForCloudFrontPrivateContent",

    "Statement": [

        {

            "Effect": "Allow",

            "Principal": {

                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity <distribution ID>"

            },

            "Action": "s3:GetObject",

            "Resource": "arn:aws:s3:::<s3-bucket>/*"

        }

    ]

}

Future Enhancements

The previous guide acts as an introduction to getting a static WordPress site up and running in AWS. In a follow-up article, I refine this current setup and include automation for building and deploying to S3. If you have any comments or suggestions on how this process could be refined, please leave them in the comments section below.