DEV Community: JT

Making a Bastion Host

JT — Tue, 30 Mar 2021 12:05:33 +0000

If we follow good security practices and deploy our database in a private subnet, direct database access is closed off from the wider internet. This is great for security, but can be frustrating when we just need to run some quick SQL commands from our local machines, connect a server that's not in the same network, or download some data without creating an API for this singular purpose. Luckily, we can get around this without exposing our database to the public internet with a "bastion host".

What is a Bastion Host?

In medieval architecture, a bastion is a protrusion from a fortification that provides the most strategic vantage point possible without introducing any weak points to the structure. So if we relate this to our problem, a bastion host in computer science will function similarly - providing a point of access into a "fortified" private network without adding to the surface area of cyber attacks of your infrastructure. In practical terms, we will be using AWS infrastructure throughout this article, but the principles apply to any system of any scale we may work with. In our case, let's say we have a PostgreSQL - configured RDS instance living in a private subnet of our VPC. We're proud of our foresight and sleep soundly knowing we've done more than most to ensure the security of our application. A few sprints later, our project manager asks us to collect some data on the past few months of data for a big client meeting and, since this impromptu meeting was unforeseen, there is not much time to collect it. Since we've deployed our database in a private subnet, we can't connect to it to run our SQL queries and there isn't enough time to update our application code to retrieve this data either, so we will need to create our bastion host to proxy our connection.

How to Create a Bastion Host

The architecture we will be creating

Deploying the Server

We can deploy a simple EC2 instance (T2 Micro running Amazon Linux or Ubuntu will work fine) using the dashboard or command line making sure to launch it in the same VPC as our database, but in a public subnet. This will be our bastion host:

aws ec2 run-instances --image-id ami-xxxxxxxx --count 1 --instance-type t2.micro --key-name MyKeyPair --security-group-ids sg-903004f8 --subnet-id subnet-6e7f829e

Note: replace the ids and credentials with your own information.

Configuring the Firewall

After the server is launched (or while creating it in the dashboard), adjust the security groups for both the bastion host and the database to allow your machine to establish an ssh connection to your bastion host on port 22 and tunnel your psql connection to your database from your bastion.

The following section will describe how to configure our firewalls using the AWS CLI to ensure this information stays up to date with AWS's frequently changing landscape.

We will be allowing only connections from our IP in this example, so let's first get that information.

curl https://checkip.amazonaws.com

This will return our IP address (e.g. 203.0.113.57).

Next let's update our EC2 bastion host to accept incoming connections on port 22.

aws ec2 authorize-security-group-ingress --group-name ec2-bastion-sg --protocol tcp --port 22 --cidr 203.0.113.0/24

Last we'll update the database security group to allow connections on port 5432 (postgres) from our bastion host.

aws ec2 authorize-security-group-ingress --group-name rds-postgre-sg --protocol tcp --port 5432 ---source-group ec2-bastion-sg

*Note: we should replace group-name with the security group ids of our bastion host and database instance respectively, and cidr with our IP address.

Connecting to a Bastion Host

After the instance is launched and the firewalls are configured and tested (we should be able to ssh into our bastion host), we can create our connection.

Opening an SSH Tunnel

ssh -i "~/bastion_key.pem" -NL 8886:postgress.cpypigm0kth7.us-east-1.rds.amazonaws.com:5432 ec2-user@132.5.10.11 -v

8886 is the port on our local machine we will be using to open the connection; we can choose any available port we want here. The flag -N will set up the tunnel without creating a session with the bastion server, -L will forward the port, and -v will print the output logs to the console.

Once the output shows the connection is established successfully, we should check the port to make sure our machine is listening to the port correctly.

sudo lsof -iTCP -sTCP:LISTEN -n -P | grep -i --color 8886

Optionally, we can add a configuration to our ssh config file to keep the connection alive to avoid annoying session timeouts.

cat <<EOT >> ~/.ssh/config
Host *
    ServerAliveInterval 30
    ServerAliveCountMax 2
EOT

Connecting to the Database

Now we should be able to connect to our database! If using postgres, the command will look something like this:

psql -h localhost -p 8886 -U <username> -W

That's it. Now we can use this connection to download our information in time for the important client meeting as well as reuse this solution for any future problems we may face.

Conclusion

In summary, by creating an EC2 instance in our database VPC in a public subnet, configuring the firewalls, and establishing an ssh tunnel, we can interact with our database in a private subnet without introducing too much extra infrastructure or reducing the security of our application.

If you would like alternative solutions for specific vendors, databases, or local machines, please leave a comment and I'll add it to the post!

How to Track Indoor Air Pollution with a Raspberry Pi

JT — Fri, 19 Feb 2021 06:14:59 +0000

You ever wonder what's actually in the air you breathe every day? I've lived in Beijing for a few years now and I'm always running a couple of air purifiers in my home, but how well do they actually work? Sure some have built in sensors, but how accurate are they? Well with a little Python, a Raspberry Pi, and some moxie, we can find our own answers.

This will focus primarily on measuring PM2.5 and PM10 in the air and converting the values to an Air Quality Index (AQI). For measuring other harmful chemicals like Nitrogen Dioxide (NO2) or Carbon Monoxide (CO), see the "Adding Sensors" section at the end of the article

Prerequisites

Raspberry Pi Configured, Headless (I used a 3b, but anything with a USB port should do just fine, though built in wifi is recommended for portability) (~$35)
SDS011 PM Sensor (~$15)
Adafruit IO Account (free)

Setting up Adafruit IO

Create an account on Adafruit IO. This is a great site to collect data streams and display them on custom dashboards. After you've created an account, let's create a couple of feeds. We'll need three: a pm2.5, a pm10, and a log feed (I named mine "beijing-twofive", "beijing-ten", and "logs" respectively). After that, you can either create a dashboard now or later and play around with how you want to display the data. This is how I have mine set up.

Configure your Raspberry Pi

Configure your Raspberry Pi with the installers on the product website. Once configured with either Raspbian or a Linux distribution of your choice (this project should be compatible with most distributions, but it's only been tested on Debian, Raspbian, and MacOS), install the following dependencies:

Python3 (you can skip the virtual env setup for our purposes)
Git

Download the Code and Install Dependencies

SSH into your Raspberry Pi (or login if GUI is installed and open your terminal) and download the repo in your home directory:

git clone git@github.com:HoukasaurusRex/RaspberryPi-AQIPi.git

Then install the python dependencies:

pip3 install -r requirements.txt

Configure ENV with Adafruit Keys and Feed Names

Retrieve your AdafruitIO username and key from the dashboard and add them to you env.

echo 'AIO_USERNAME="Hackerman"
AIO_KEY="aio_xXxXx"
CITY="beijing"
AIO_LOGS="logs"' > .env

Both CITY and AIO_LOGS are feed names created in the AdafruitIO dashboard.

Run the Code

Now you can run the code. I like to use screen to save my terminal process to be accessible later, but you can just run it in your main shell as well.

screen -S aqipi

cd ~/RaspberryPi-AQIPi
chmod +x run.sh
sh run.sh

There won't be any output in your terminal, but you should be able to go to your AdafruitIO feeds and start seeing results!

Sensor Placement

Standard advice for locating your sensor is that it should be outside and four metres above ground level. That’s good advice for general environmental monitoring; however, we’re not necessarily interested in general environmental monitoring – we’re interested in knowing what we’re breathing in.

Choose a location where you spend most of your time or where you might be particularly interested in the general air quality (e.g. in the kitchen or garage) and place your sensor in a safe place where it won't be affected by excessive moisture or humidity.

Understanding the Code

def read_data():
  pm_twofive_data = []
  pm_ten_data = []
  readings = 0

  # This will take 11 data samples and use the built in `statistics` module to upload a median value
  # to filter out excessive data spikes in the readings
  while readings < 11:
    data = []
    for index in range(0, 10):
      datum = ser.read()
      data.append(datum)

    # Convert the readings from bytes to ints and append to the array of data samples
    pm_twofive = int.from_bytes(b''.join(data[2:4]), byteorder='little') / 10
    pm_twofive_data.append(pm_twofive)
    pm_ten = int.from_bytes(b''.join(data[4:6]), byteorder='little') / 10
    pm_ten_data.append(pm_ten)
    readings += 1

    # Take a little break ☕️
    sleep(1)

  # Calculate the AQI from ppm^2 using the US EPA API table detailed in the section below
  # then upload to your AdafruitIO feeds using the Adafruit_IO SDK
  pm_twofive_aqi = calc_aqi('pm_twofive', median(pm_twofive_data))
  send_data('twofive', pm_twofive_aqi)
  pm_ten_aqi = calc_aqi('pm_ten', median(pm_ten_data))
  send_data('ten', pm_ten_aqi)
  return [pm_twofive_aqi, pm_ten_aqi]

Converting PPM^2 to AQI

This is converting to the US EPA AQI, in order to use a different standard, you might need to tweak the formula to fit your country's AQI model.

Exponential Backoff

This repo implements an exponential backoff policy to retry connections after an exponentially longer period to account for common network errors or connection issues with your sensor.

def exponential_backoff(n):
  return (2 ** n) + (random.randint(0, 1000) / 1000)

Results

First testing against the air quality outside, make sure it seems to match up with IQ Air's Air Quality Index.

After a few weeks of running the sensor, it seems the sensors on my air purifiers would often underreport pm2.5 values by as much as 50% and were seldom correlated with each other. The Raspberry Pi on the other hand, filtering out for data spikes, seems to respond very reasonably to real world phenomena such as rising with the outdoor AQI and falling linearly when the purifiers are all left on high for a few hours. I have also learned how much barometric pressure differences will also increase the penetration of outdoor pollution to my home. Interestingly, poor kitchen ventilation also causes quite noticeable spikes in indoor air pollution!

I've been using this sensor a few months now and have since felt much more empowered with managing the quality of the air I breathe each day.

Adding Sensors

This project can be easily extended by adding additional sensors, such as an Ozone (O3), Carbon Monoxide (CO), Nitrogen Dioxide (NO2), or any other harmful air pollutants that might be more relevant to your area. If you do, please let me know and I'd like to compare your findings and update the AQIPi repository to extend the project.

Acknowledgements

Big thanks to Andrew Gregory from raspberrypi.org on his work providing the inspiration for this project.

You can view this article and more like it as well as sign up to my newsletter to get the latest posts on https://jt.houk.space