DEV Community: Mark Nunnikhoven

This One Mistake Will Stop a DevSecOps Shift Left Strategy Dead in Its Tracks

Mark Nunnikhoven — Wed, 10 Nov 2021 18:30:00 +0000

DevSecOps is the latest in a long line of buzzwords. The core makes sense: work on security earlier. But why isn’t this everywhere? Here’s the biggest mistakes teams are making trying to “do” DevSecOps.

Learn more in the video 👆 or read through the transcript 👇.

Transcript

I see security teams making the same mistake over and over again when it comes to “shifting left.” It’s frustrating from afar and infuriating when you have to deal with it day-to-day.

Let’s dig in to the disaster that is DevSecOps…

[00:15]

Imagine for a minute, you’re in your kitchen preparing dinner. You’re a reasonably good home cook. More often than not, what you put on the table is enjoyed by those you’re sharing with it.

Sure, every once and a while you miss. But that’s the rare case, so when it does happen everyone smiles, you laugh, and then place an order for take out. Mistakes happen.

Not too bad, right?

[00:29]

Now, let’s say while you’re getting ready to sit down for a wonderful home cooked meal, you neighbour invites themselves in. They immediate start hammering you with questions like, “How sharp is that knife?”, “Do you know who grew that broccoli?”, “Are there too many ovens in this neighbourhood?”

Taken aback, you politely ask, “Um, are you a professional chef? Do you have a lot of experience cooking?”

They reply, “Oh no, I don’t even have a kitchen in my place. I just order food every once and a while.”

That’s basically the scenario I see play out in organizations around the world.

The development teams and builders are working to solve business problems and address customer needs.

Then the security team shows up out of no where and starts asking seemingly irrelevant questions and demanding that priorities change in the name “reducing risk” and “improving the overall security posture” without understanding what you’re working on or how you work.

[01:37]

This is why even the name DevSecOps frustrates me to no end. The DevOps philosophy already assumes that you want to build a resilient, reliable system. There’s no need to jam another acronym in there.

Teams know that security is important, they just need the information and support to make smart decisions at the right time.

So is this whole “shift left” thing doomed?

No.

Not if you do it well.

[02:06]

If you’re on the security team, the first thing you need to understand is that you probably don’t understand how the builders are working.

You can fix that.

Spend some time with them. Ask lots of questions to better understand their workflow and concerns.

Most important of all, make sure that the information from security tools that shift left provide information with the proper context and enough data for teams to make an informed decision.

[02:34]

Just because it’s a security priority, doesn’t mean it’s a business priority.

For developers and builders, understand that security controls can provide real value to you. The whole goal of these controls is to make sure things work as intended.

Network security tools look for malicious activity and malformed traffic. You don’t want that anywhere near your app.

Threat detection on your servers and containers is looking for errant processes and other indicators of compromise. This makes sure that your resources are only working for you instead of doing things like mining cryptocurrency for cybercriminals.

Posture management—ugh, horrible name—looks at the cloud services you’re using to make sure that you have configured them in a way that matches your risk appetite.

Vulnerability scanners look at your tech stack trying to find known issue before so they don’t bite you in the you-know-what.

[03:26]

Everything on this list and most of the other security controls out there can dramatic HELP you meet your goals.

With that understanding, you need to make sure that you have access to the outputs of these tools. You need to know that they are in place and doing their job, so that you can focus on other parts of yours.

By now, you’ve figured out that the number one mistake I see security teams making when they “shift left” is IGNORING the developers and builders.

For some reason, security teams assume that to “shift left” means doing their isolated security work earlier in the development process. That’s an archaic way of thinking.

[04:05]

To truly shift left, you need to leverage the capability of security tools and processes to help developers and builders identify risks with their systems earlier in THEIR processes.

This data will help the teams make informed decisions about what actions should be taken to meet the business goals.

Shifting security left can help reduce the risks to the business while improving the quality of the systems your build.

Who wouldn’t want that?

Stop Your Password From Opening The Door To Hackers

Mark Nunnikhoven — Fri, 22 Oct 2021 16:52:53 +0000

It's cybersecurity awareness month and we all should be doing out part to #BeCyberSmart.

The one thing I see people struggling with the most is using passwords and I get it.

A lot of what we've been subjected too about passwords is wrong and actually makes things less secure. Making matters worse, security folks—myself included!—aren't known as being the most communicative.

So, I set out to demystify passwords. In the video above 👆, I walk through how passwords are attacked, the UX around them, what makes a truly strong password, and finally I lay out a practical path for dealing with the mishmash of systems out there.

Here in this post, I'll give you the highlights...

Strength

A strong password is a long password...or more probably, a passphrase.

Length is the single most important factor in determining the strength of a password.

The second most important factor is the variety of characters you pick from (so, not just a-z). That's the reason for those crazy password rules we're all so familiar with.

Start thinking pass*phrase*, not password.

Old Rules

Those old rules I mentioned 👆? The whole "at least one capital letter, a number, a symbol, and be at least 8 characters long" thing?

Those rules actually lead to weaker passwords.

Thankfully the most commonly used guidelines were updated in 2017 but a lot of systems are still behind the times. That means we still have to deal with them. 😔

Password Manager

In addition to dealing with those older systems and rules, we also need different passwords for every site and app we use.

Why? Because it reduces your risk if one of those sites is hacked or has a breach.

One of the first things cybercriminals do when they get new credential sets is test them against popular sites.

But keeping track of all of those passwords is a pain. The solution is to use a password manager.

Which one doesn't matter much. Just make sure it runs on all of your preferred devices and has a nice user experience.

That's going to keep your passwords safe and sound...and generate long, gibberish passwords for any new logins.

Taking things a step further, the manager will actually log you in to those sites and apps when needed.

One Password To Rule Them All

To keep all of those passwords in the manager safe and secure, you'll need a password (couldn't avoid them completely 🤣).

Thankfully, almost all password managers are up to date on the rules and we can use a passphrase here.

This passphrase is only going to be used with your manager and you should only change it when you think someone might have figured it out or about every year or so.

Remember, this is the only password you're going to be typing in yourself. Make it a good one!

Here are some simple guidelines to follow to create a really strong and easy to remember passphrase:

use a random word generator to select at least 4 (more if you can) truly random words
throw in a symbol or number (or both) just because

Boom. Easy to remember, super strong password.

Something like: polite2vacuumcensusmonkey!narrowfrozen

polite 2 vacuum census monkey ! narrow frozen

Not only is that a fun passphrase (which I swear was randomly generated) but it's easy to remember and crazy strong.

Stay safe out there and #BeCyberSmart!

Transferring Files in Amazon S3

Mark Nunnikhoven — Tue, 11 Aug 2020 18:10:31 +0000

Fellow AWS Hero, Matt Bonig, recently asked a very interesting question on Twitter as a poll;

Matthew Bonig

@mattbonig

Happy Monday. AWS pop quiz time:

When using
`aws s3 cp s3://somebucket/somefile s3://otherbucket/somefile`

'somefile' is transferred through the host machine

14:27 PM - 10 Aug 2020

Three Possible Answers

It's an interesting question because depending on your perspective and experience, each of the three possible poll answers make sense.

If you don't know about the S3 bucket to bucket copy feature (which, while introduced in 2008 isn't crystal clear in the docs), passing data through the system that called the command makes sense. That's how most file transfers work.

If you've been working with Amazon S3 regularly, you've no doubt seen the ultrafast transfer speeds even over bad connections. The only way that makes sense is if the data is only moving within the S3 infrastructure.

There are always exceptions to everything which is why, "It depends" holds up as a valid answer. Anyone that's been working with tech for any length of time intuitively understands this...after many, many frustrating stories & experiences.

What Actually Happens

Let's breakdown this command and figure out what's going on.

aws s3 cp s3://SOURCE_BUCKET/KEY s3://DESTINATION_BUCKET/

aws calls the AWS CLI program.

s3 filters the commands to the S3 service. a/k/a, "We're using S3!"

cp is the action to call within the specified AWS service.

Now, cp is a little different than most of the AWS CLI commands. Most commands are directly parallels to the AWS API for the service in question.

In this case, there's a lot of syntactic sugar applied here with the goal of making aws s3 cp work as similar as possible to Linux's cp command.

The rest of the command provides a mix of options to indicate the source and destination of the file copy. For our example;

s3://SOURCE_BUCKET/KEY is the source file. A key is the S3 term for what we commonly think of as the directory structure + filename. In this case, the file is in an existing S3 bucket.

s3://DESTINATION_BUCKET/ is the destination. Here, we've indicated another S3 bucket and because the path ends in /, we are telling cp that we want the same filename (or key) in the destination bucket.

API Calls

Behinds the scenes, the S3 API command the cp calls depends on what you've asked it to do.

Here are the most common possibilities;

If you're copying a file from the local system into S3, it calls the PutObject action or—if it's a really large file—the CreateMultipartUpload action
If you're copying a file from a bucket to another bucket, it calls the CopyObject action
If you're copying a file from a bucket to the local system, it calls the GetObject action

Our Transfer

In the case of our command;

aws s3 cp s3://SOURCE_BUCKET/KEY s3://DESTINATION_BUCKET/

The CLI translates that to the CopyObject action which means the data never leaves AWS. The contents of our file (or key) are copying via the S3 backend from the source bucket to the destination bucket.

We can verify that by looking at the outbound traffic on your local system. Here's a screenshot of the original file upload.

I've run the command;

aws s3 cp LOCAL_FILENAME s3://BUCKETA/

This results in an outbound transfer running at ~17 MB/s to AWS.

You can see that not only as reported by the AWS CLI but also by my outbound firewall. That outbound firewall is reporting ~14 MB/s, but the difference is just a matter how each tool updates.

The reported speed also lines up with the amount of time the command was running based on the file size (just over a minute for a 1.1 GB file).

The key point here is that they are reporting very similar numbers.

After that upload completes, to copy the file one bucket to another, I run the command;

aws s3 cp s3://BUCKETA/KEY s3://BUCKETB/

Here are the transfer results from my local system for this command;

The AWS CLI is reporting ~180 MB/s while my local network traffic is only 30 KB/s to AWS. The file transfer takes about 10 seconds to complete.

This proves that the CopyObject API is being used to copy the file through the S3 backend.

What's Next?

The bucket-to-bucket copy feature is a massive time and bandwidth saver when you're working with files in AWS. This works not only between buckets in the same account and region but using different accounts and regions (when the appropriate permissions are in place).

The results of this little experiment also highlight a key rule of working with data in Amazon S3;

Keep data inside of Amazon S3 for as long as possible

You don't want to have to wait on data to be downloaded outside of AWS, nor do you want to have to pay for it.

The AWS CLI itself is a really interesting open source project that has a lot of very cool code behind the scenes to make it work. You can check that out on GitHub.

Applying the Well-Architected Framework, Small Edition

Mark Nunnikhoven — Sat, 27 Jun 2020 15:08:01 +0000

Do you ever tackle a problem and know that you’ve just spent way too much time on it? But you also know that it was worth it? This post (which is also long!) sums up my recent experience with exactly that type of problem.

tl:dr

AWS Lambda has a storage limit for /tmp of 512MB
AWS Lambda functions needs to be in a VPC to connect to an Amazon EFS filesystem
AWS Lambda functions within in a VPC require a NAT gateway to access the internet
Amazon EC2 instances can use cloud-init to run a custom script on boot
A solution needs to address all five pillars of the AWS Well-Architected Framework in order to be the “best”

Read on to find out the whole story…

The Problem

I love learning and want to keep tabs on several key areas. Ironically, tabs themselves are massive issue for me.

Every morning, I get a couple of tailored emails from a service called Mailbrew. Each of these emails contains the latest results from Twitter queries, subreddits, and key websites (via RSS).

The problem is that I want to track a lot of websites and Mailbrew only supports adding sites one-by-one.

This leads to a problem statement of…

Combine N feeds into one super feed

Constraints

Ideally these super feeds would be published on my website. That site is build in Hugo and deployed to Amazon S3 with CloudFlare in front. This setup is ideal for me.

Following the AWS Well-Architected Framework; it’s highly performant, low cost, has minimal operational burden, a strong security posture, and is very reliable. It’s a win across all five pillars.

Adding the feeds to this setup shouldn’t compromise any of these attributes.

I think it’s important to point out that there a quite a few services out there that combine feeds for your. RSSUnify and RSSMix come to mind, but there are many, many others…

The nice thing about Hugo is that it uses text files to build out your site, including custom RSS feeds. The solution should write these feed items as unique posts (a/k/a text files) in my Hugo content directory structure.

🐍 Python To The Rescue

A quick little python script (available here) and I’ve got a tool that takes a list of feeds and writes them into unique Hugo posts.

Problem? Solved.

Hmmm…I forgot these feeds need to be kept up to date and my current build pipeline (a GitHub Action) doesn’t support running on a schedule.

Besides, trying to run that code in the action is going to require another event to hook into or it’ll get stuck in an update loop as the new feed items are committed to the repo.

New problem statement…

Run a python script on-demand and a set schedule

This feels like a good problem for a serverless solution.

AWS Lambda

I immediately thought of AWS Lambda. I run a crazy amount of little operational tasks just like this using AWS Lambda functions triggered by a scheduled Amazon CloudWatch Event. It’s a strong, simple pattern.

It turns out that getting a Lambda function to access a git repo isn’t very straight forward. I’ll save you the sob story but here’s how I got it running using a python 3.8 runtime;

add the git binaries as a Lambda Layer, I used git-lambda-layer
use the GitPython module

That allows a simple code setup like this:

repo = git.Repo.clone_from(REPO_URL_WITH_AUTH)
...
# do some other work, like collect RSS feeds
...
for fn in repo.untracked_file:
  print("File {} hasn't been added to the repo yet".format(fn))

# You can also use git commands directly-ish
repo.git.add('.') 
repo.git.commit('Updated from python')

This makes it easy enough to work with a repo. With a little bit of hand wavy magic, I wired a Lambda function up to a scheduled CloudWatch Event and I was done.

…until I remembered—and by, “remembered”, I mean the function threw an exception—about the Lambda /tmp storage limit of 512MB. The repo for my website is around 800MB and growing.

Amazon EFS

Thankfully, AWS just released a new integration between Amazon EFS and AWS Lambda. I followed along the relatively simple process to get this set up.

I hit two big hiccups.

The first, for a Lambda function to connect to an EFS file system, both need to be “in” the same VPC. This is easy enough to do if you have a VPC setup and even if you don’t. We’re going to come back to this one in a second.

The second issue was that I initially set the path for the EFS access point to /. There wasn’t a warning (that I saw) in the official documents but an off-handed remark in a fantastic post by Peter Sbarski highlighted this problem.

That was a simple fix (I went with /data) but the VPC issue brought up a bigger challenge.

The simplest of VPCs that will solve this problem is one or two subnets with an internet gateway configured. This structure is free and only incurs charges for inbound/outbound data transfer.

Except that my Lambda function needs internet access and that requires one more piece.

That piece is a NAT gateway. No big deal, it’s a one click deploy and a same routing change. The new problem is cost.

The need for a NAT gateway makes completely sense. Lambda runs adjacent to your network structure. Routing those functions into your VPC requires an explicit structure. From a security perspective, we don’t want an implicit connection between our private network (VPC) and other random bits of AWS.

Well-Architected Principles

This is where things really start to go off the of the rails. As mentioned above, the Well-Architected Framework is built on five pillars;

The AWS Lambda + Amazon EFS route continues to perform well in all of the pillars except for one; cost optimization.

Why? Well I use accounts and VPCs as strong security barriers. So the VPC that this Lambda function and EFS filesystem are running in is only for this solution. The NAT gateway would only be used during the build of my site.

The cost of a NAT gateway per month? $32.94 + bandwidth consumed.

That’s not an unreasonable amount of money until you put that in the proper context of the project. The site costs less than $0.10 per month to host. If we add in the AWS Lambda function + EFS filesystem, that skyrockets to $0.50 per month 😉.

That NAT gateway is looking very unreasonable now

Alternatives

Easy alternatives to AWS Lambda for computation are AWS Fargate and good old Amazon EC2. As much as everyone would probably lean towards containers and I’ve heard people say it’s the next logical choice…

// Detect dark theme var iframe = document.getElementById('tweet-1273575669390327809-115'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1273575669390327809&theme=dark" }

…I went old school and started to explore what a solution in EC2 would look like.

For an Amazon EC2 instance to access the internet, it only needs to be in a public subnet of a VPC with an internet gateway. No NAT gateway needed. This removes the $32.94 each month but does put us into a more expensive compute range.

But can we automate this easily? Is this going to be a reliable solution? What about the security aspects?

Amazon EC2 Solution

The 🔑 key? Remembering the user-data feature of EC2 and that all AWS-managed AMIs support cloud-init.

This provides us with 16KB of space to configure an instance on the fly to accomplish out task. That should be plenty...if you haven't figured it out from my profile pic, I'm approaching the greybeard powers phase of my career 🧙‍♂️.

A quick bit of troubleshooting (lots of instances firing up and down), and I ended up with this bash script (yup, bash) to solve the problem;

#! /bin/bash
sleep 15
sudo yum -y install git
sudo yum -y install python3
sudo pip3 install boto3
sudo pip3 install dateparser
sudo pip3 install feedparser

cat > /home/ec2-user/get_secret.py <<- PY_FILE
# Standard libraries
import base64
import json
import os
import sys

# 3rd party libraries
import boto3
from botocore.exceptions import ClientError

def get_secret(secret_name, region_name):
    secret = None
    session = boto3.session.Session()
    client = session.client(service_name='secretsmanager', region_name=region_name)
    try:
        get_secret_value_response = client.get_secret_value(SecretId=secret_name)
    except ClientError as e:
        print(e)
    else:
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
        else:
            decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])

    return json.loads(secret)

if __name__ == '__main__':
    github_token = get_secret(secret_name="GITHUB_TOKEN", region_name="us-east-1")['GITHUB_TOKEN']
    print(github_token)
PY_FILE

git clone https://\`python3 get_secret.py\`:x-oauth-basic@github.com/USERNAME/REPO /home/ec2-user/website

python3 RUN_MY_FEED_UPDATE_SCRIPT

cd /home/ec2-user/website

# Build my website
./home/ec2-user/website/bin/hugo -b https://markn.ca

# Update the repo
git add .
git config --system user.email MY_EMAIL
git config --system user.name "MY_NAME"
git commit -m "Updated website via AWS"
git push

# Sync to S3
aws s3 sync /home/ec2-user/website/public s3://markn.ca --acl public-read

# Handy URL to clear the cache
curl -X GET "https://CACHE_PURGING_URL"

# Clean up by terminating the EC2 instance this is running on
aws ec2 terminate-instances --instance-ids `wget -q -O - http://169.254.169.254/latest/meta-data/instance-id` --region us-east-1

This entire run takes on average 4 minutes. Even at the higher on-demand cost (vs. spot), each run costs $0.000346667 on a t3.nano in us-east-1.

For the month, that’s $0.25376244 (for 732 scheduled runs).

We’re well 😉 over the AWS Lambda compute price ($0.03/mth) but still below the Lambda + EFS cost ($0.43/mth) and certainly well below the total cost including a NAT gateway. It’s weird, but this is how cloud works.

Each of these runs is triggered by a CloudWatch Event that calls an AWS Lambda function to create the EC2 instance. That’s one extra step compared to the pure Lambda solution but it’s still reasonable.

Reliability

In practice, this solution has been working well. After 200+ runs, I have experienced zero failures. That’s a solid start. Additionally, the cost of failure is low. If this process fails to run, the site isn’t updated.

Looking at the overall blast radius, there are really only two issues that need to be considered;

If the sync to S3 fails and leaves the site in an incomplete state
If the instance fails to terminate

The likelihood of a sync failure is very low but if it does fail the damage would only be to one asset on the site. The AWS CLI command copies files over one-by-one if they are newer. If one fails, the command stops. This means that only one asset (page, image, etc.) would be in a damaged state.

As long as that’s not the main .css file for the site, we should be ok. Even then, clean HTML markup leaves the site still readable.

The second issue could have more of an impact.

The hourly cost of the t3.nano instance is $0.0052/hr. Every time this function runs, another instance is created. This means I could have a few dozen of these instances running in a failure event…running up a bill that would quickly top $100/month if left unchecked.

In order to mitigate this risk, I added another command to the bash script; shutdown. Also ensuring that the API parameter of —instance-initiated-shutdown-behavior set to terminate is set on instance creation. This means the instance calls the EC2 API to terminate itself and shuts itself down to terminate…just in case.

Adding a billing alert rounds out the mitigations to significantly reduce the risk.

Security

The security aspects of this solution concerned me. AWS Lambda presents a much smaller set of responsibilities for the user. In fact, an instance is the most responsibility taken on by the builder within the Shared Responsibility Model. That’s the opposite way we want to be moving.

Given that this instance isn’t handling inbound requests, the security group is completely locked down. It only allows outbound traffic, nothing inbound.

Additionally, using an IAM Role, I’ve only provided the necessary permissions to accomplish the task at hand. This is called the principle of least privilege. It can be a pain to setup sometimes but does wonders to reduce the risk of any solution.

You may have noticed in the user-data script above that we’re actually writing a python script to the instances on boot. This script allows the instance to access AWS Secrets Manager to get a secret and print its value to stdout.

I’m using that to store the GitHub Personal Access Token required to clone and update the private repo that holds my website. This reduces the risk to my GitHub credentials which are the most important piece of data in this entire solution.

This means that the instance needs to the following permissions;

secretsmanager:GetSecretValue
secretsmanager:DescribeSecret
s3:ListBucket
s3:*Object
ec2:TerminateInstances

The permissions for secretsmanager are locked to the specific ARN of the secret for the GitHub token. The s3 permissions are restricted to read/write my website bucket.

ec2:TerminateInstances was a bit trickier as we don’t know the instance ID ahead of time. You could dynamically assign the permission but that adds needless complexity. Instead, this is a great use case for resource tags as a condition for the permission. If the instance isn’t tagged properly (in this case I use a “Task” key with a value set to a random, constant value), this role can’t terminate it.

Similarly, the AWS Lambda function has standard execution rights and;

iam:PassRole
ec2:CreateTags
ec2:RunInstances

Cybersecurity is simply making sure that what you build does what you intend…and only what is intended.

If we run through the possibilities for this solution, there isn’t anything that an attacker could do without already having other rights and access within our account.

We’ve minimized the risk to a more than acceptable level, even though we’re using EC2. It appears that this solution can only do what is intended.

What Did I Learn?

The Well-Architected Framework isn’t just useful for big projects or for a point-in-time review. The principles promoted by the framework apply continuously to any project.

I thought I had a simple, slam dunk solution using a serverless design. In this case, a pricing challenge required me to change the approach. Sometimes it’s a performance, sometimes it’s security, sometimes it’s another aspect.

Regardless, you need to be evaluating your solution across all five pillars to make sure you’re striking the right balance.

There’s something about the instance approach that still worries me a bit. I don’t have the same peace of mind that I do when I deploy Lambda but the data is showing this as reliable and meeting all of my needs.

This setup also leaves room for expansion. Adding addition tasks to the user-data script is straightforward and would not dramatically shift any of the concerns around the five pillars if done well. The risk here is expanding this into a custom CI/CD pipeline which is something to avoid.

“I built my own…”, generally means you’ve taken a wrong turn somewhere along the way. Be concerned when you find yourself echoing those sentiments.

This is also a reminder that there are a ton of features and functionality within the big three (AWS, Azure, Google Cloud) cloud platforms and that can be a challenge to stay on top of.

If I didn’t remember the cloud-init/user-data combo, I’m not sure I would’ve evaluated EC2 as a possible solution.

One more reason to keep on learning and sharing!

Btw, checkout the results of this work at;

And if I'm missing a link with a feed that you think should be there, please let me know!

Total Cost

If you’re interested in the total cost breakdown for the solution at the end of all this, here is it;

Per month
===========
24 * 30.5 = 732 scheduled runs
+ N manual runs
===========
750 runs per month

EC2 instance, t3.nano at $0.0052/hour
+ running for 4m on average
===========
(0.0052/60) * 4 = $0.000346667/run

AWS Lambda function, 128MB at $0.0000002083/100ms
+ running for 3500ms on average
============
$0.00000729/run

Per run cost
============
$0.000346667/run EC2
$0.00000729/run Lambda
============
$0.000353957/run

Monthly cost
=============
$0.26546775/mth to run the build 750 times
$0.00 for VPC with 2 public subnets and Internet Gateway
$0.00 for 2x IAM roles and 2x policies
$0.40 for 1x secret in AWS Secrets Manager
$0.00073 for 732 CloudWatch Events (billed eventually)
$0.00 for 750 GB inbound transfer to EC2 (from GitHub)
$0.09 for 2 GB outbound trasnfer (to GitHub)
=============
$0.75620775/mth

This means it'll take 3.5 years before we've spent the same as one month of NAT Gateway support.

* Everything is listed in us-east-1 pricing