DEV Community: Cyril Rohr

How to verify that VPC traffic to S3 is going through your S3 gateway?

Cyril Rohr — Fri, 02 Feb 2024 09:42:02 +0000

Gateway endpoints for Amazon S3 are a must-have whenever your EC2 instances send and receive traffic from S3, because they allow the traffic to stay within the AWS network, hence better security, bandwidth, throughput, and costs. They can easily be created, and added to your VPC route tables.

But how do you verify that traffic is indeed going through the S3 gateway, and not crossing the outer internet?

Using traceroute, you can probe the routes and see whether you are directly hitting the S3 servers (i.e. no intermediate gateway). In this example, the instance is running from a VPC located in us-east-1:

$ traceroute -n -T -p 443 s3.us-east-1.amazonaws.com
traceroute to s3.us-east-1.amazonaws.com (52.216.215.72), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  52.216.215.72  0.890 ms  0.916 ms  0.892 ms

$ traceroute -n -T -p 443 s3.amazonaws.com
traceroute to s3.amazonaws.com (52.217.139.232), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  52.217.139.232  0.268 ms  0.275 ms  0.252 ms

Both outputs produce the expected result, i.e. no intermediary gateway. This is what would happen if you were accessing a bucket located in the us-east-1 region.

Let's see what happens if we try to access an S3 endpoint located in another zone:

$ traceroute -n -T -p 443 s3.eu-west-1.amazonaws.com
traceroute to s3.eu-west-1.amazonaws.com (52.218.25.211), 30 hops max, 60 byte packets
 1  * * *
 2  240.4.88.37  0.275 ms 240.0.52.64  0.265 ms 240.4.88.39  0.215 ms
 3  240.4.88.49  0.205 ms 240.4.88.53  0.231 ms 240.4.88.51  0.206 ms
 4  100.100.8.118  1.369 ms 100.100.6.96  0.648 ms 240.0.52.57  0.233 ms
 5  240.0.228.5  0.326 ms * *
 6  240.0.32.16  0.371 ms 240.0.48.30  0.362 ms *
 7  * 240.0.228.31  0.251 ms *
 8  * * *
 9  * * 240.0.32.27  0.392 ms
10  * * *
11  * 242.0.154.49  1.321 ms *
12  * * 52.93.28.131  1.491 ms
13  * * 100.100.6.108  1.286 ms
14  100.92.212.7  67.909 ms 52.218.25.211  67.356 ms  67.929 ms

As you can see, the route is completely different, and as expected does not hit straight to the S3 endpoint.

TL;DR: make sure your route tables are correct, and only point to S3 buckets located in the same region.

Gateway endpoints for Amazon S3

Introducing RunsOn: 10x cheaper GitHub Action runners

Cyril Rohr — Tue, 23 Jan 2024 12:37:39 +0000

Let's face it, sometimes you need faster execution of your GitHub Action workflows.

I've worked for a few companies where the full test suite runtime was greater than 20min, and this is not good for developer feedback.

The usual fix here is to deploy faster self-hosted runners, but then this means you must ensure that those runners stay online, regularly patched, and sufficiently used to justify their cost.

Wouldn't it be nice if you could spawn self-hosted runners on demand, with a large choice of CPU/RAM configurations, and for cheap? This is what RunsOn provides, with up to 10x cheaper runners, and the widest choice of runner types on the market.

RunsOn can be installed in your own AWS account, which means there is no middleman between your workflows and your runner. It also supports both x64 and arm64 architecture, and is a one-line change from your current workflow files.

- runs-on: ubuntu-latest
+ runs-on: runs-on,runner=16cpu-linux,image=ubuntu22-full-x64

Whenever a new workflow starts, a runner is automatically provisioned, and will be terminated as soon as the workflow ends. You only pay the per-minute cost of the runner, and you can easily track your costs in your AWS Cost Explorer page.

Some quick pricing comparison:

runner	cpu	$/min (spot)	$/min (github)	GitHub vs RunsOn
`1cpu-linux`	1	0.0008
`2cpu-linux`	2	0.0011	0.008	7x more expensive
`4cpu-linux`	4	0.0022	0.016	7x more expensive
`8cpu-linux`	8	0.0035	0.032	9x more expensive
`16cpu-linux`	16	0.0068	0.064	9x more expensive
`32cpu-linux`	32	0.0132	0.128	10x more expensive
`48cpu-linux`	48	0.0170
`64cpu-linux`	64	0.0196

The cost is fully open, can be installed in one click thanks to a cloudformation template, and a license only costs $250 once. Find out more at https://runs-on.com

Automated preview deployments from GitHub to AWS in 10 minutes

Cyril Rohr — Mon, 12 Jun 2023 15:44:01 +0000

Preview deployments are a way to spin temporary environments from branches or pull requests. They're used to allow teammates and product managers to visually review your work before changes are merged, without relying on code only.

3 ways to get preview deployments

To use preview deployments in your development workflow, you have 3 solutions:

your hosting provider has preview deployments sorted out (e.g. Heroku / Vercel / Render / Scalingo). Easy to setup, but you get vendor lock-in, and some providers are costly.
you find a 3rd-party SaaS to provide the service for you. Setup can be complicated if they use custom definition files, it's costly, and your code is now at risk in another provider's infrastructure.
you build or use a solution that doesn't rely on 3rd-parties. This could be a GitHub Action, Jenkins pipeline, semi-automated script etc.

Building a custom solution is costly in terms of development and maintenance, so today I want to talk about PullPreview, a GitHub Action which makes preview deployments very easy to setup.

PullPreview

PullPreview is an open-source GitHub Action which has the following features:

works with any app that can be booted with Docker Compose (i.e. all).
can be triggered by applying a label on a pull request.
deploys code directly from GitHub onto AWS (Lightsail).
servers are started in your own AWS account.
SSH access to the servers.
you can use your own custom domain if you want.
environments are persistent across deployments.
fully integrated into GitHub UI, no need to go through another app.

Let's get started

Add AWS credentials to your repo

PullPreview requires AWS credentials to deploy servers, so let's create a new user.

For some obscure reason AWS doesn't provide a default policy for Lightsail, so you will need to create a new Policy for this user, with the following content:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "lightsail:*",
            "Resource": "*"
        }
    ]
}

Then attach the newly created policy and create the user:

Final step is to retrieve an access key and secret access key for that new user, and declare those keys into your repository settings on GitHub, as new Repository Secrets.

Label, workflow file, and example Docker Compose app

The PullPreview action will be triggered only for pull requests that have a specific label applied. By default the label name is pullpreview, so let's create it in your GitHub repository. Use any color you like.

Now, create a new branch, and add the following files:

(1) the pullpreview workflow file:

# .github/workflows/pullpreview.yml
name: PullPreview
on:
  pull_request:
    types: [labeled, unlabeled, synchronize, closed, reopened]

jobs:
  deploy:
    permissions:
      contents: read # to fetch code (actions/checkout)
      deployments: write # to delete deployments
      pull-requests: write # to remove labels
      statuses: write # to create commit status
    name: deploy
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
    - uses: actions/checkout@v2
    - uses: pullpreview/action@v5
      # see https://github.com/pullpreview/action/wiki/Inputs
      with:
        admins: your-github-username
        compose_files: docker-compose.pullpreview.yml
        ports: 80,443
        default_port: 443
      env:
        AWS_ACCESS_KEY_ID: "${{ secrets.AWS_ACCESS_KEY_ID }}"
        AWS_SECRET_ACCESS_KEY: "${{ secrets.AWS_SECRET_ACCESS_KEY }}"
        AWS_REGION: "us-east-1"

(2) an example docker-compose file, with SSL support:

# docker-compose.pullpreview.yml
services:
  # easily set up SSL termination
  proxy:
    image: caddy:2
    restart: unless-stopped
    command: "caddy reverse-proxy --from '${PULLPREVIEW_URL}' --to web:4567"
    depends_on:
      - web
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/data"

  # simple ruby web server for demo purposes, which connects to a postgres DB
  web:
    restart: unless-stopped
    depends_on:
      - db
    build:
      context: .
      dockerfile_inline: |
        FROM ruby:3.2
        RUN gem install sinatra pg puma
        CMD ruby -rsinatra -rpg -rpuma \
          -e'get("/"){ "Hey PullPreview user, here is the postgres version: #{PG.connect(ENV["PGURI"]).exec("SELECT VERSION()").getvalue(0,0)}" }'
        EXPOSE 4567
    environment:
      PGURI: "postgres://postgres:p4ssw0rd@db/postgres"
      APP_ENV: production

  db:
    restart: unless-stopped
    image: postgres:13
    environment:
      POSTGRES_PASSWORD: p4ssw0rd

Let the magic begin

Now commit the previous files, push your new branch, and open a pull request with the pullpreview label applied. GitHub will start executing the PullPreview job, and a new preview environment will be up and running a few minutes later!

Going further

Next steps would be to replace the example docker-compose.ci.yml file with your own Docker Compose file, and see if the environment gets successfully deployed as well! You might need a few attempts to get to the point where your environment is fully setup, but remember that you can always SSH into the temporary server if the need arises!

Conclusion

We've only scratched the surface of what PullPreview can do. The fact that it runs entirely within the GitHub ecosystem and in your own AWS account allows for a wide variety of use cases:

access restriction from specific IP ranges
custom domains
various instance types
support for private docker registries
support for multiple docker compose files
support for multiple preview environments per pull request
support for seed data
etc.

You should definitely have a look at the documentation.

Also note that the PullPreview action is free to use for personal use, but comes with a commercial license (300€/year) for businesses. This ensures that the action is maintained and new features added.