How to verify that VPC traffic to S3 is going through your S3 gateway?

Cyril Rohr — Fri, 02 Feb 2024 09:42:02 +0000

Gateway endpoints for Amazon S3 are a must-have whenever your EC2 instances send and receive traffic from S3, because they allow the traffic to stay within the AWS network, hence better security, bandwidth, throughput, and costs. They can easily be created, and added to your VPC route tables.

But how do you verify that traffic is indeed going through the S3 gateway, and not crossing the outer internet?

Using traceroute, you can probe the routes and see whether you are directly hitting the S3 servers (i.e. no intermediate gateway). In this example, the instance is running from a VPC located in us-east-1:

$ traceroute -n -T -p 443 s3.us-east-1.amazonaws.com
traceroute to s3.us-east-1.amazonaws.com (52.216.215.72), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  52.216.215.72  0.890 ms  0.916 ms  0.892 ms

$ traceroute -n -T -p 443 s3.amazonaws.com
traceroute to s3.amazonaws.com (52.217.139.232), 30 hops max, 60 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  52.217.139.232  0.268 ms  0.275 ms  0.252 ms

Both outputs produce the expected result, i.e. no intermediary gateway. This is what would happen if you were accessing a bucket located in the us-east-1 region.

Let's see what happens if we try to access an S3 endpoint located in another zone:

$ traceroute -n -T -p 443 s3.eu-west-1.amazonaws.com
traceroute to s3.eu-west-1.amazonaws.com (52.218.25.211), 30 hops max, 60 byte packets
 1  * * *
 2  240.4.88.37  0.275 ms 240.0.52.64  0.265 ms 240.4.88.39  0.215 ms
 3  240.4.88.49  0.205 ms 240.4.88.53  0.231 ms 240.4.88.51  0.206 ms
 4  100.100.8.118  1.369 ms 100.100.6.96  0.648 ms 240.0.52.57  0.233 ms
 5  240.0.228.5  0.326 ms * *
 6  240.0.32.16  0.371 ms 240.0.48.30  0.362 ms *
 7  * 240.0.228.31  0.251 ms *
 8  * * *
 9  * * 240.0.32.27  0.392 ms
10  * * *
11  * 242.0.154.49  1.321 ms *
12  * * 52.93.28.131  1.491 ms
13  * * 100.100.6.108  1.286 ms
14  100.92.212.7  67.909 ms 52.218.25.211  67.356 ms  67.929 ms

As you can see, the route is completely different, and as expected does not hit straight to the S3 endpoint.

TL;DR: make sure your route tables are correct, and only point to S3 buckets located in the same region.

Gateway endpoints for Amazon S3

Introducing RunsOn: 10x cheaper GitHub Action runners

Cyril Rohr — Tue, 23 Jan 2024 12:37:39 +0000

Let's face it, sometimes you need faster execution of your GitHub Action workflows.

I've worked for a few companies where the full test suite runtime was greater than 20min, and this is not good for developer feedback.

The usual fix here is to deploy faster self-hosted runners, but then this means you must ensure that those runners stay online, regularly patched, and sufficiently used to justify their cost.

Wouldn't it be nice if you could spawn self-hosted runners on demand, with a large choice of CPU/RAM configurations, and for cheap? This is what RunsOn provides, with up to 10x cheaper runners, and the widest choice of runner types on the market.

RunsOn can be installed in your own AWS account, which means there is no middleman between your workflows and your runner. It also supports both x64 and arm64 architecture, and is a one-line change from your current workflow files.

- runs-on: ubuntu-latest
+ runs-on: runs-on,runner=16cpu-linux,image=ubuntu22-full-x64

Whenever a new workflow starts, a runner is automatically provisioned, and will be terminated as soon as the workflow ends. You only pay the per-minute cost of the runner, and you can easily track your costs in your AWS Cost Explorer page.

Some quick pricing comparison:

runner	cpu	$/min (spot)	$/min (github)	GitHub vs RunsOn
`1cpu-linux`	1	0.0008
`2cpu-linux`	2	0.0011	0.008	7x more expensive
`4cpu-linux`	4	0.0022	0.016	7x more expensive
`8cpu-linux`	8	0.0035	0.032	9x more expensive
`16cpu-linux`	16	0.0068	0.064	9x more expensive
`32cpu-linux`	32	0.0132	0.128	10x more expensive
`48cpu-linux`	48	0.0170
`64cpu-linux`	64	0.0196

The cost is fully open, can be installed in one click thanks to a cloudformation template, and a license only costs $250 once. Find out more at https://runs-on.com

DEV Community: RunsOn

How to verify that VPC traffic to S3 is going through your S3 gateway?

Introducing RunsOn: 10x cheaper GitHub Action runners