The video version of this blog is here. I used box-01/02/03 instead of server-01/02/03 in the video, but the rest is exactly the same.
A lot of people who are interested in breaking into DevOps (or whatever title seems most appropriate for "learning about managing the things that actually run the software") often have the same question:
"What should I do?"
And they often get the same answer:
"Just build something"
But then they want to know:
"What should I build?"
And this is where it gets tricky...
While building something that you actually want to use yourself is often one of the most motivating ways to learn while doing, for a lot of beginners, they're not even sure what the basic building blocks of "stuff that runs the software" are.
What to build or how to build
In very basic terms, those building blocks (especially as related to SaaS products) are easy enough to read about:
- a database
- a backend server
- a webserver sending a javascript frontend bundle to a browser
A lot of the web apps we use are basically web frontend wrappers around databases running CRUD, and that's okay!
Though even knowing this, a lot of people can get stuck on the stage of "what kind of web frontend wrapper around a database should I build?"
A lot of the tutorials walk learners through running everything locally, which is indeed a great way to get started on the software side of things.
But what about the operating of that software on VMs somewhere? And yes, even containers run inside VMs, although a lot of platforms like ECS Fargate or Google Cloud Run abstract that away...
Forget the apps, lets deal with some VMs!
It used to be that if you wanted to deploy an application on a VM and get experience doing that, you would have needed to sign up to a cloud provider like AWS, Digital Ocean, or Hetzner. This took time and oftentimes involved a financial commitment (even just entering your credit card details, which kept some people from going this route).
I've written before about Iximiuz Labs, which is the perfect place to play around with VMs for free*!
*- requires a GitHub user
In the rest of this post, I'll walk through deploying and configuring a "3-Tier Web App" ™️ in one of the playgrounds, completely from scratch.
...and here's what it will look like.
We will:
- Set up a postgres database
- Set up a backend application to interact with the database
- Set up a frontend application served via nginx to load in the browser
We won't have to deal with the tougher aspects of DevOps, like DNS, firewalls, subnets, monitoring, or CI/CD pipelines...we want to keep this as simple as possible (though maybe we'll look at some of those in later blog posts).
But don't worry...just in case you're worried it'll be too easy, we will need to care about IP addresses and ports, so there's still a little bit of networking going on here.
I cheated a little bit...the code for both the backend and frontend is already written (it's VERY simple), but that makes it easier for other people, especially those who have no idea what to build, to follow along themselves!
My Kingdom for a Database!
Let's fire up the Flexbox playground and start installing stuff!
You'll just need to add another machine and make sure the Rootfs (the root file system) for each of them is Ubuntu 24.04.
Ubuntu 22.04 would also still work, but we might as well go for the latest OS version.
I've called my machines server-01, server-02, and server-03, but you can call them whatever you want. Once you hit the "Start Playground" button, the playground will boot up, and you should see the following screen:
Let's go ahead and install the database on server-03:
sudo apt update && sudo apt install -y postgresql postgresql-client-16
and now let's start the database
sudo systemctl start postgresql
sudo -u postgres psql -x
This is a neat trick to get into the database without needing the password, since it automatically lets a user named "postgres" in if you're on the same server.
Now you're in the database and can start configuring the application user
CREATE DATABASE drizzle;
\c drizzle;
CREATE USER laborant WITH PASSWORD 'laborant';
(in the above, because these are isolated ephemeral environments, it doesn't really matter what you set as your password, so you might as well make it easy to remember...don't do this in production, obviously)
and now we can grant the privileges for the new user to interact with the database:
GRANT ALL PRIVILEGES ON DATABASE drizzle TO laborant;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO laborant;
GRANT USAGE, CREATE ON SCHEMA public TO laborant;
now the user is ready to be used by the backend service.
Backend user service...get that data!
Now that we've got a database, we can use that for our backend application. On server-02, let's clone the code:
git clone https://github.com/lpmi-13/speed-run-backend
Because this backend service is written in typescript, we have to install node and npm to build and run it. I like fnm for this, but you could use nvm if you want to.
Once that's installed and ready (don't forget to run source ~/.bashrc, since you can't restart the shell inside the VM), go ahead and run the following:
npm i
Inspecting the code in src/index.ts, you can see that it looks for an environment variable called DATABASE_URL, and if that's not set, it tries a default connection.
import express from 'express';
import { drizzle } from 'drizzle-orm/node-postgres';
import { migrate } from 'drizzle-orm/node-postgres/migrator';
import { seed } from 'drizzle-seed';
import { Pool } from 'pg';
import dotenv from 'dotenv';
import path from 'path';
import { usersTable } from './db/schema';
dotenv.config();
const app = express();
app.use(express.json());
const connectionString =
process.env.DATABASE_URL ||
'postgres://postgres:mypassword@localhost:5432/drizzle';
const pool = new Pool({ connectionString });
We have a user and password (the one we created in the last section in our database), so let's create a .env file and set up our DATABASE_URL.
...there's a problem though...what's the hostname going to be? You can't use localhost, because the database isn't running on the same VM.
That's easy enough to fix, let's jump back on server-03 and find out what the IP address is (hint: they will all be 172.X.X.X, because that's the virtual subnet they get run in).
You can run either ip addr or good ole ifconfig and find the address for the eth0 interface that starts with 172..., that's the IP address of server-03.
Now with that information, you can fill in the connection string:
echo 'DATABASE_URL=postgres://laborant:laborant@172.16.0.4:5432/drizzle' > .env
We're specifying the
drizzledatabase, since that's the one the application is configured to try and connect to, but if you're feeling adventurous, go ahead and try with your own custom-defined database.
Once that's all configured, let's build and run the application!
npm run build
node dist/index.js
OH NO! Our first big problem!
laborant@server-02:speed-run-backend$ node dist/index.js
Connecting to PostgreSQL database...
Running migrations if needed...
Failed to start server: Error: connect ECONNREFUSED 172.16.0.4:5432
at /home/laborant/speed-run-backend/node_modules/pg-pool/index.js:45:11
at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
at async PgDialect.migrate (/home/laborant/speed-run-backend/node_modules/drizzle-orm/pg-core/dialect.cjs:56:5)
at async migrate (/home/laborant/speed-run-backend/node_modules/drizzle-orm/node-postgres/migrator.cjs:27:3)
at async startServer (/home/laborant/speed-run-backend/dist/index.js:38:9) {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '172.16.0.4',
port: 5432
}
We can't connect to the database. Let's debug that 🕵️...
With a quick netcat command, we can see that the port is not accepting connections
laborant@server-02:speed-run-backend$ nc -vz 172.16.0.4 5432
nc: connect to 172.16.0.4 port 5432 (tcp) failed: Connection refused
And the reason is simple enough. We didn't update the postgres config to accept incoming connections from outside its own VM 💡.
Let's update that now.
Back on server-03, we want to edit /etc/postgresql/16/main/pg_hba.conf to change the following line:
host all all 127.0.0.1/32 scram-sha-256
into this
host all all 0.0.0.0/0 scram-sha-256
That basically means "instead of just listening for connections from inside the VM, listen to connections trying from outside the VM". See? Easy networking fix!
...but we're not done yet. We still need to tweak one more bit of config. Let's also edit /etc/postgresql/16/main/postgresql.conf to change the following line:
#listen_addresses = 'localhost' # what IP address(es) to listen on;
into this
listen_addresses = '*' # what IP address(es) to listen on;
make sure to remove the initial
#, which turns the above line into a comment, as well as update thelisten_addresses💡
The above is a similar update to tell the database that instead of just listening for connections from inside the VM on localhost, we want to listen for connections from outside.
To apply these configuration changes, we need a full restart of the database, so we can run:
sudo systemctl restart postgresql
And now let's try our netcat test from server-02 again, and we should see:
laborant@server-02:speed-run-backend$ nc -vz 172.16.0.4 5432
Connection to 172.16.0.4 5432 port [tcp/postgresql] succeeded!
Great, now we're ready to try to run the application again.
we could run this as a background process, but we're not going to keep this playground around very long, so we're not doing anything fancy like setting up a systemd service or anything.
laborant@server-02:speed-run-backend$ node dist/index.js
Connecting to PostgreSQL database...
Running migrations if needed...
Migrations completed successfully
Database seeded with 10 users
Server running on port 4000
🎉 our server is running and connected to the database!
If you'd like to confirm it's running fine, let's send a request to its endpoint...FROM ANOTHER VM!!!! (so exciting!)
Frontend user service...show that data!
Just like we planned in the previous step, let's head over to server-01 and connect to the backend server and query the list of users:
curl 172.16.0.3:4000/users | jq
we're just using
jqbecause it comes pre-installed on these VMs, and it makes the output easier to read.
Awesome, we can see the users data coming through from the backend server
laborant@server-01:~$ curl 172.16.0.3:4000/users | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 815 100 815 0 0 85411 0 --:--:-- --:--:-- --:--:-- 90555
[
{
"id": 1,
"name": "Mouhamed",
"age": -118655047,
"email": "motivational_shakeia@outlook.com"
},
{
"id": 2,
"name": "Gisele",
"age": 1415217029,
"email": "bronchial_myrtice@orange.fr"
},
{
"id": 3,
"name": "Garvin",
"age": -1302366560,
"email": "logical_mariacristina@live.com"
},
{
"id": 4,
"name": "Klay",
"age": 1243263009,
"email": "blameless_annalyn@web.de"
},
...
ignore the ludicrous ages, this post isn't gonna cover a data-cleaning pipeline...
So we have a working backend server getting data from a database...but how do we get that to load in the browser for a user?
You might think that you could just clone the frontend code and run the dev server with npm start, but you're gonna have all kinds of problems with binding to 0.0.0.0/0 (remember that means "listens for outside connections") as well as host headers and general frontend-y types of issues.
...but don't worry! We're gonna use Nginx to do what Nginx does (serve bundled frontend code to users and redirect requests to the backend server).
So let's first clone the frontend code:
git clone https://github.com/lpmi-13/speed-run-frontend
Now we can do the same trick of installing fmn/nvm then building the frontend assets.
...previous step of installing fnm or nvm...
npm i
npm run build
The assets (here, we mean html/js/css) are now located in a build/ directory, and we just need to put those in a place where Nginx knows how to find them.
Let's also install Nginx, since we haven't done that yet:
sudo apt update && sudo apt install -y nginx
sudo service nginx start
You'll know that Nginx is up and running since it listens on localhost port 80 by default, so you should be able to run
laborant@server-01:speed-run-frontend$ curl localhost
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
And see the delightful Nginx html coming back.
Now we need to copy over the files in the build/ directory to an Nginx location.
sudo rm -rf /var/www/html/*
sudo cp -r build/* /var/www/html
You'll know it's copied correctly because you can now re-run the curl command and you'll see the react html coming back
laborant@server-01:speed-run-frontend$ curl localhost
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="/favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="Web site created using create-react-app"/><link rel="apple-touch-icon" href="/logo192.png"/><link rel="manifest" href="/manifest.json"/><title>React App</title><script defer="defer" src="/static/js/main.d1e5e332.js"></script><link href="/static/css/main.e6c13ad2.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
We also need to update the Nginx config a bit so that it sends the requests made by the browser to the correct backend (the request will come into Nginx first running on server-01, which will pass it on to the backend running on server-02)
Now we can cheat again (since I have a working nginx config in the frontend code repo we cloned earlier), and copy it into the right place:
sudo cp nginx/nginx.conf /etc/nginx/nginx.conf
You can take a look at it yourself if you want, it's very basic:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 1024;
}
http {
server {
listen 80;
root /var/www/html;
# Serves the frontend html/js on page load
location / {
try_files $uri $uri/ /index.html;
}
# proxy the requests for the backend to the other server
# the domain for server-02 is resolvable by the hostname
location /users {
proxy_pass http://server-02:4000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
}
It basically says "listen for connections on port 80, and send the html/javascript for the root domain, and also forward any requests with /users at the end to the other server on port 4000."
Once that's all copied, go ahead and reload the config with sudo service nginx reload (I forgot this step in the demo and had to quickly debug it).
So we've got the index.html in the correct place, we just need to expose it to the world.
This is one of the best parts of Iximiuz Labs...you can expose a port directly and get a nice public URL.
In the upper right of the UI, there's a little box icon with an arrow coming out of it...click that, and type in 80 for the port (the default nginx port).
and BOOM! We've got users showing up in the browser
Wrapping up
Actually deploying and connecting real things (even if they're temporary and disappear when you close the playground) is the heart of DevOps.
Ideally, there would also be a bit of automation in there, but in this speedrun, the whole idea was to do something super super simple and connect it across independent VMs.
It would be a great addition to add in some database replication or caching or load balancing or...
...but that's probably gonna be a different speed run.
Top comments (0)