Securely Accessing Private AWS Resources from GitHub Actions with TailScale

Fady — Thu, 14 Dec 2023 09:49:49 +0000

The Problem

One of the common problems of CI/CD automation is how to handle private resources without exposing these resources directly to the public or relying on complex VPN setups may not be the ideal solution.

Typically, you go with something like OpenVPN, but OpenVPN can be complicated, with something like OpenVPN,you have to set up your own server, and configure, maintain, etc. On the other hand, it takes less than 2 minutes to set up TailScale, Another thing, if you live in a country where OpenVPN is blocked, you don't have many options.

Here is the diagram that we are going to be implanted in this blog, the blog post will guide you with step-by-step through its implementation. The initial steps involve setting up a GitHub Actions workflow that installs the TailScale client and authorizes it into a TailScale network. Subsequently, the blog post will cover the creation and configuration of the TailScale router, ensuring proper traffic routing and security group settings on AWS.

Connecting GitHub Actions to the Tailscale Network

Before starting this guide, you'll need to have a Tailscale network set up and configured with at least one existing device, If you do not know how, follow this guide and of course you're going to need an AWS account as well.

Now the next thing that I'm going to do is go out to GitHub and create a new repository,You can find the code for the end result in this repository.

You will then need to create an OAuth client to use with the Github action via this link, copy the Client ID and Client Secret to a safe place for now.

Go to the GitHub repo you created earlier and go to Settings > in the Security section of the sidebar, select Secrets and Variables, then click Actions, Create a GitHub secret with the name TS_OAUTH_CLIENT_ID and assign your OAuth client ID as the secret value and name TS_OAUTH_SECRET and assign your OAuth client secret as the secret value.

You will then need to create a server role account using ACL tags in Tailscale, basically ACL tags allow you to assign an identity to a device that is separate from human users, and use that identity as part of an ACL to restrict access. This should be used when adding servers to your Tailscale network, to learn more about ACL tags, click here

Open your favorite editor and create a .github directory and underneath .github we'll create a Workflows directory, this is the special directory that GitHub Actions is going to look for workflow files in, create a new workflow, I'm going to call it tailscale.yaml.

And inside of this file, we're going to go ahead and set up our basic workflow structure.

on:
  workflow_dispatch:
  push:

jobs:
  deploy-via-tailscale:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Tailscale
      uses: tailscale/github-action@v2
      with:
        oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
        oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
        tags: tag:ci

Let me break it down for you.

the workflow dispatch event. Allows us to trigger workflows manually, which is great for debugging.
The push event means that whenever we push a change, it goes ahead and triggers a build.
Define the job and name it deploy-via-tailscale. It will run on the latest release of Ubuntu.
connect to your tailnet by using the Tailscale GitHub Action

Go ahead and push the code to Github, you should see an ephemeral node. The node can access nodes in your Tailnet, subject to the access rules that apply to the specified ACL tag(s).

Set Up Amazon VPC and Tailscale Router

What we're going to do is we're going to create a VPC, and then we're going to have this public subnet, and we're going to put an EC2 instance on that public subnet, and we're going to have that EC2 instance connect out to our Tailscale network as well, and then we're going to set that EC2 instance up as a router.

Let's go to the AWS Management Console. First, we'll create an elastic IP and EC2 instance on the public subnet, which will act as a subnet router.

After the instance has passed the checks we need to download and install Tailscale onto your subnet router machine.

Step 1: Install the Tailscale client

ssh into the instance and follow the instructions

curl -fsSL https://tailscale.com/install.sh | sh

Step 2: Enable IP forwarding and Advertise subnet routes

echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
echo 'net.ipv6.conf.all.forwarding = 1' | sudo tee -a /etc/sysctl.d/99-tailscale.conf
sudo sysctl -p /etc/sysctl.d/99-tailscale.conf

sudo tailscale up --advertise-routes==10.0.16.0/20 --accept-dns=false

Replace the subnets in the example above with the correct ones for your network, it will give you a link that you will need to go to in order to authenticate.

The EC2 instance should appear in the Tailscale console.

Step 3: Enable subnet routes from the tailscale console

Locate the device that advertised subnet routes, from the ellipsis menu at the bottom of the table, select Edit Route Settings. This opens the Edit Route Settings panel.

Everything should be working to test connectivity, we will add test connectivity to the Github action, commit and push the code.

on:
  workflow_dispatch:
  push:

jobs:
  deploy-via-tailscale:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Tailscale
      uses: tailscale/github-action@v2
      with:
        oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
        oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
        tags: tag:ci
    - name: test-connectivity
      run: |
        ping -c 10 100.73.32.96

Replace the IP address with the IP address of the subnet router.

Create Private Amazon RDS Instance and Test Connectivity

After reviewing and verifying, we will provision an Amazon RDS instance in our VPC and test connectivity from our GitHub Actions workflow.

First, we will create a security group that will allow connection from the subnet router, I'm going to call it the database security group.

In a production environment, it is bad practice to allow all connections from a security group, Only allow access from trusted sources for specific ports and protocols, following the principle of least privilege.

I will not cover how to create an RDS database here. I assume you have already created an RDS database, please refer to the AWS documentation if you don't know how to create one.

If you want to test connectivity between your machine and the private Amazon RDS instance type the following command. This action allows you to connect to the MySQL DB instance using the MySQL client, Replace endpoint with the DB instance endpoint (DNS name) and replace admin with the master username you used. Enter the master password that you used when prompted for a password.

mysql -h endpoint -P 3306 -u admin -p

If everything is configured correctly, you should be able to see this massage.

We will return to the Github action workflow to test connectivity, we will create three secrets for the database endpoint, password, and username.

Update the workflow with this to test connectivity. We can now connect to RDS from any node in the Tailnet, using the same DNS name used inside AWS.

on:
  workflow_dispatch:
  push:

jobs:
  deploy-via-tailscale:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Tailscale
      uses: tailscale/github-action@v2
      with:
        oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
        oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
        tags: tag:ci
    - name: Test-Connectivity-to-AWS-RDS
      run: |
        mysql -h ${{ secrets.ENDPOINT }} -u ${{ secrets.USER }} -p"${{ secrets.PASSWORD }}" -e "show databases;"

Conclusion

In this blog, we've demonstrated how to create a secure and seamless connection between GitHub Actions workflows and private AWS resources. This approach not only improves the overall security of your CI/CD pipeline, but also ensures that sensitive data and interactions remain within your network, TailScale is easy to use, and has robust encryption mechanisms.

This is not sponsored or endorsed by TailScale, I use the TailScale at work and in my home lab, I am just a happy customer.

One more thing, you can host Tailscale Control Server yourself if you want, which is a plus.

Protect your Web Apps Against DDoS Attacks by Using AWS Shield

Fady — Mon, 24 Apr 2023 09:52:09 +0000

AWS offers really great tools to protect you from threats and ensure security in the cloud, one of these tools is AWS shield which is essential tools to safeguard your web apps or websites from Distributed Denial of Service (DDoS) attacks.

what is DDoS attacks?

DDoS attacks have become increasingly common in today's digital world, with the number of attacks are rising, DDoS stands for “Distributed Denial-of-Service” attack, DDoS attacks are a type of Cyberattacks in which the attacker (Often, it's not just one person who is responsible for this attack; sometimes, the attacker is a nation) floods a server with internet traffic by botnets (A botnet is a network of computers infected by malware that are under the control of the attacker), rendering connected online services and sites inaccessible to legitimate users. These attacks are sophisticated and designed to overwhelm the network with unwanted traffic. As a result, network performance is either degraded or critical infrastructure experiences a service outage.

AWS Shield

AWS Shield is a managed DDoS protection service offered by AWS, It works in partnership with ELB, Amazon CloudFront, and Amazon Route 53 to protects you from DDoS attacks of many types of DDoS Attacks.
AWS Shield protects against three different types

Network volumetric attacks

These are designed to simply overwhelm the system being attacked.
So to direct as much raw network data at a target as possible

Network protocol attacks or TCP SYN floods

There are various types of protocol attack, but one common one is to generate a huge number of connections from a spoofed IP address (computers infected by malware) and then just leave these connections open and never terminating them, the server will be impacted by the huge volume of fake connections.

Application layer attacks

Imagine you have a part of your web app which allows searches, think of something like Facebook or Twitter Search which lets you search for new friends or people to follow, Attackers can use web app searches to launch DDoS attacks by sending a large number of requests to a web application that has a search feature. The attacker sends requests that are designed to consume as much server resources as possible. And it can be done hundreds, thousands, or millions of times per second. Some parts of the application are cheap to request, but expensive to deliver the result.

So those are the types of things which Shield protects against.

AWS Shield comes in two forms, Shield Standard and Shield Advanced. While both offer protection against DDoS attacks, there are significant differences between the two.

Shield Standard

Shield Standard is a free service that comes with all AWS accounts, you benefit from its protection automatically without you having to do anything.

AWS Shield Standard doesn’t provide much in the way of proactive capability or any form of explicit configurable protection. It’s just there working away in the background

Shield Standard is designed for applications that require basic protection against DDoS attacks but have a low risk tolerance.

Shield Advanced

Shield Advanced, which comes with additional costs (it costs $3,000 US dollars per month,per organization) and benefits, is a commercial extra product that offers additional detection and mitigation against large and sophisticated DDoS attacks, near real-time visibility into attacks, and integration with AWS WAF, a web application firewall.
Shield Advanced provides a 24/7 to AWS Shield Response Team (SRT), advanced attack mitigation techniques, and protection against attacks at the application layer.
It also includes features such as traffic logging and metrics, which allow customers to monitor their traffic and identify potential attacks.

Shield Advanced is designed for applications that require a higher level of protection against DDoS attacks and have a higher risk tolerance.

If you have feedback about this blog post, submit comments in the Comments section below.

DEV Community: Fady