DEV Community: Adam Berg

Using Github Actions to Deploy a React App and Express API Using Rsync in 15 seconds

Adam Berg — Sat, 04 Dec 2021 21:41:28 +0000

Motivation

I've been looking to revamp the deployment process for several projects that I'm working on and start building towards my preferred method of deployment.

My biggest requirements are simplicity and speed. I have used Docker, Kubernetes, Docker Swarm, and various other methods of deployment in the past. I recognize these tools have their advantages, but have found that for small to medium sized projects they are more effort than they are worth to maintain.

At the end of the day, all I need to do is build the code and copy the built files to the server. Before starting the project I told myself to get it under a minute, but I'm happy to report that Github Actions starts up much faster than Travis CI and brought this down to 15 seconds to deploy a React frontend and express.js backend.

I've provided full instructions for how to recreate this entire project, but if you're just interested in the workflow part skip ahead to the My Workflow section.

Creating a Simple App to Demonstrate

Before I can demonstrate the workflow, we need to have something to deploy. Below are instructions for how the simple app is structured. Most of you are probably used to the templates provided by Create React App, but here I provide some opinionated alternatives for how to structure the app. The same principles should be possible to transfer over to any existing setup.

Creating a Basic React App

mkdir github-actions-tutorial
cd github-actions-tutorial
yarn init
yarn add react react-dom
yarn add --dev @types/react @types/react-dom
mkdir -p client/src

Create index.tsx

// client/src/index.tsx
import React from "react";
import ReactDom from "react-dom";
import { App } from "./App";

ReactDom.render(<App />, document.getElementById("root"));

Create App.tsx

// client/src/App.tsx
import React, { useEffect, useState } from "react";

export const App: React.FC = () => {
  return (
    <>
      <div>Hello Github Actions!</div>
    </>
  );
};

Building React App with esbuild

Now that we have a simple React app we are going to output a minified production build using esbuild.

Install esbuild

yarn add --dev esbuild

Add client:build script to package.json

// package.json
{
  "name": "github-actions-tutorial",
  "version": "1.0.0",
  "main": "index.js",
  "repository": "git@github.com:adamjberg/github-actions-tutorial.git",
  "author": "Adam Berg <adam@xyzdigital.com>",
  "license": "MIT",
  "scripts": {
    "client:build": "esbuild client/src/index.tsx --bundle --minify --outfile=built/app.js",
  },
  "dependencies": {
    "react": "^17.0.2",
    "react-dom": "^17.0.2"
  },
  "devDependencies": {
    "@types/react": "^17.0.37",
    "@types/react-dom": "^17.0.11",
    "esbuild": "^0.14.1"
  }
}

You can test this is working correctly by running yarn client:build and you should see a built/app.js file in the folder tree with the minified output.

You are probably used to have a yarn start script as well, but for the purposes of this tutorial we're going to skip it and test this all out directly in "production".

Create `public/index.html`

<html>

<head>
  <script src="/js/app.js" defer async></script>
</head>

<body>
  <div id="root"></div>
</body>

</html>

This will be the file that is served by our nginx static file server when clients hit the http://github-actions-tutorial.devtails.xyz URL.

Prepping a Server

I'm going to assume the reader has some knowledge about how to register a domain and create a server on some hosting platform. I already have a domain devtails.xyz with Namecheap and I have created a droplet with Digital Ocean.

In the example below, I have mapped github-actions-tutorial.devtails.xyz to my Digital Ocean IP: 143.198.32.125

As long as you have the ability to ssh into your server, the following instructions should suffice regardless of your hosting platform.

SSH into server

ssh root@143.198.32.125

Create github-actions-tutorial User

To prevent our Github Action from getting root access to our server, we will create a sub-user called github-actions-tutorial

useradd -s /bin/bash -d /home/github-actions-tutorial -m github-actions-tutorial

Install nginx

apt-get install nginx

Create Virtual Host File

# /etc/nginx/sites-available
server {
  listen 80;
  server_name github-actions-tutorial.devtails.xyz;

  location / {
    root /home/github-actions-tutorial/static;
  }
}

This tells nginx to route requests to the github-actions-tutorial.devtails.xyz subdomain to the static folder under our github-actions-tutorial user.

Create `static` folder on `github-actions-tutorial` user

su github-actions-tutorial
mkdir static

This allows us to avoid having our Github Action ssh into the server just to create this folder. This folder will house the js/app.js and index.html. The virtual host file set up previously tells nginx to serve files from the static folder.

Creating a Basic Express REST API

Install express

yarn add express
yarn add @types/express

Create `server/src/server.tsx`

// server/src/server.tsx
import express from "express";

const app = express();

app.get("/api/message", (_, res) => {
  return res.json({
    data: "Hello from the server!",
  });
});

app.listen(8080);

This creates a basic REST API with a single /api/message route that we will use to demonstrate that it is running correctly.

Add server:build script to package.json

We will re-use the esbuild package to build a bundle for our server code as well. For more details on this approach please see this post.

"server:build": "esbuild server/src/server.ts --bundle --minify --outfile=built/server.js --platform=node"

Add this right below the client:build script. You can then run it to confirm working as expected with yarn server:build. It should output a bundled file to built/server.js.

Add build script that runs both client and server builds

"build": "yarn client:build && yarn server:build"

Prepare the Server to Run the API

There are a few one time configurations that need to be applied in order to prepare our server for deployment.

Switch to github-actions-tutorial user

su github-actions-tutorial

Install NVM

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash

Install Node

nvm install 16

Install pm2

npm i -g pm2

Update Virtual Host File to Route to API

Again ssh into the root user and update /etc/nginx/sites-available/github-actions-tutorial.devtails.xyz file

# /etc/nginx/sites-available/github-actions-tutorial.devtails.xyz
upstream github-actions-tutorial-api {
  server localhost:8080;
}

server {
  listen 80;
  server_name github-actions-tutorial.devtails.xyz;

  location /api {
    proxy_pass http://localhost:8080;
  }

  location / {
    root /home/github-actions-tutorial/static;
  }
}

This tells nginx to route any URLs that start with /api to the express app that we added.

Bootstrapping the pm2 process

Before the final step - run: ssh github-actions-tutorial "pm2 reload all" can run, you must first manually start your server with pm2.

After running the Github Action for the first time, it should have copied the built server.js file to ~/api/server.js. You can then start this process with pm2 start api/server.js.

Now that it is running, the pm2 reload all command will reload this server process so it can pick up the changes in your server code.

My Workflow

Phew, with all that set up out of the way, we can now look at what our Deploy workflow does.

Below I'll break it down section by section

Define workflow name and triggers

name: Deploy

on:
  push:
    branches: [ main ]

This creates a workflow called "Deploy" that will be run whenever a push is made to the main branch.

Define build-and-deploy job

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

This creates a job called build-and-deploy that will run the latest ubuntu distribution.

env:
  SSH_KEY: ${{secrets.SSH_KEY}}

This adds a Github Secret to the environment. We will use this in a later step to allow us to rsync to our specified server.

steps:
  - uses: actions/checkout@v2

This checks out the code for the current commit.

- name: Use Node.js 16
  uses: actions/setup-node@v2
  with:
    node-version: 16
    cache: 'yarn'

This installs node 16 and specifies that the Workflow should cache files for yarn. This cache ensures that if no packages are added or removed, yarn install won't have to do anything. This saves a significant amount of time.

- run: yarn install
- run: yarn build

These lines run the install and build which ultimately outputs all the files that we would like to deploy.

- run: mkdir ~/.ssh
- run: 'echo "$SSH_KEY" >> ~/.ssh/github-action'
- run: chmod 400 ~/.ssh/github-action
- run: echo -e "Host static\n\tUser github-actions-tutorial\n\tHostname 143.198.32.125\n\tIdentityFile ~/.ssh/github-action\n\tStrictHostKeyChecking No" >> ~/.ssh/config

This is the most complicated section. What's happening here is that we are adding the SSH_KEY secret to the ~/.ssh/github-action file. The final line creates a ~/.ssh/config file that looks like the following:

Host static
  User github-actions-tutorial
  IdentityFile ~/.ssh/github-action
  StrictHostKeyChecking No

With that set up, the rsync commands look quite simple:

- run: rsync -e ssh public static:~/static
- run: rsync -e ssh built/app.js static:~/static/js/app.js
- run: rsync -e ssh built/server.js static:~/api/server.js

The -e ssh specifies to use rsync over ssh. We copy over all files from the public folder. Then we copy over the built/app.js to ~/static/js/app.js. Finally we copy built/server.js to ~/api/server.js.

- run: ssh github-actions-tutorial "pm2 reload all"

This final line uses pm2 (which we installed earlier) to reload the server process.

Conclusion

While I could get an even faster deployment just by running this on my local machine, having this run as a Github Action provides a big benefit for my open source projects. In order to deploy a contributor's changes, I can simply merge their pull request in to the main branch without having to give direct server access to anyone else.

There's plenty more that could be tidied up or improved, but in the spirit of a hackathon, I'm calling this "done" for now. I now have a baseline of how long I should expect an app to be built and deployed using Github Actions.

Submission Category:

DIY Deployments

Yaml File or Link to Code

name: Deploy

on:
  push:
    branches: [ main ]

jobs:
  build-and-deploy:

    runs-on: ubuntu-latest

    env:
      SSH_KEY: ${{secrets.SSH_KEY}}

    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js 16
      uses: actions/setup-node@v2
      with:
        node-version: 16
        cache: 'yarn'
    - run: yarn install
    - run: yarn build
    - run: mkdir ~/.ssh
    - run: 'echo "$SSH_KEY" >> ~/.ssh/github-action'
    - run: chmod 400 ~/.ssh/github-action
    - run: echo -e "Host github-actions-tutorial\n\tUser github-actions-tutorial\n\tHostname 143.198.32.125\n\tIdentityFile ~/.ssh/github-action\n\tStrictHostKeyChecking No" >> ~/.ssh/config
    - run: rsync -e ssh public github-actions-tutorial:~/static
    - run: rsync -e ssh built/app.js github-actions-tutorial:~/static/js/app.js
    - run: rsync -e ssh built/server.js github-actions-tutorial:~/api/server.js
    - run: ssh github-actions-tutorial "pm2 reload all"

adamjberg / github-actions-tutorial

View on GitHub

Additional Resources / Info

engram is an Open Source project where I first prototyped this style of deploy. It currently takes 3-4 minutes to deploy, which is why I'll be switching over to a workflow closer to the one provided here.

Building My First Command Line Interface (CLI) with Rust

Adam Berg — Fri, 26 Nov 2021 00:07:11 +0000

After telling myself over and over that today is the day I start learning rust. I finally successfully built a (very small) cli for engram.

This post will cover some of the things I learned along the way. I mostly from a TypeScript/Node background and will make comparisons between the two where applicable.

Inventing Some Requirements

I have found that having a tangible end goal increases my odds of project completion by nearly 100%. In this case, the goal is to create a command line program that simply POSTs requests to my personal notes application engram.

This is not just a learning or for fun project as I recently realized that the command line is a great place for the input of quick notes. When I’m working, I sometimes find myself investigating something in the terminal and realize that I have a thought that I’d like to follow up on later, or a I’d like to store a command I just ran so I can remember what I did later on.

Getting Started

Install rust with the command below found from their Getting started page

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

cargo new project-name creates a new folder called project-name with the bare minimum configuration for a rust project

Cargo.toml

This is essentially the rust equivalent of a package.json file. Lists some metadata about the project and eventually allows you to specify any dependencies to be managed by the cargo package manager.

[package]
name = "rust-new-project"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at [https://doc.rust-lang.org/cargo/reference/manifest.html](https://doc.rust-lang.org/cargo/reference/manifest.html)

[dependencies]

main.rs

Who doesn’t love a good Hello, world! application?

fn main() {
  println!("Hello, world!");
}

The Final Output

I find it helpful to see the whole picture at first. I will then break down each line of code and introduce the rust concepts that are used.

Cargo.toml

# Cargo.toml
[package]
name = "eg"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at [https://doc.rust-lang.org/cargo/reference/manifest.html](https://doc.rust-lang.org/cargo/reference/manifest.html)

[dependencies]
reqwest = { version = "0.11", features = ["json", "cookies"] }
tokio = { version = "1", features = ["full"] }

main.rs

# main.rs
use std::io;

use std::collections::HashMap;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
  let mut note = String::new();
  io::stdin().read_line(&mut note)?;

  let mut note_post_map = HashMap::new();
  note_post_map.insert("body", note);

  let client = reqwest::Client::new();
  let resp = client.post("https://engram.xyzdigital.com/api/notes")
    .json(&note_post_map)
    .send()
    .await?;

  if resp.status() != 200 {
    println!("failed to submit note");
    return Ok(());
  }

  Ok(())
}

Breaking it Down Line by Line

Get input from stdin

In order to submit a new note, I need to be able to accept input from the command line. This can be achieved with the std::io package.

use std::io;

...

// Initializes a new string
// the mut specifies that it is mutable (e.g. can be modified)
let mut note = String::new();

// Uses the stdin package to read a line from the input
// The & denotes that we are passing a reference to the note variable
// mut is required to specify that the reference can be mutated
// The ? at the end propagates the error to the outer function
// see [https://doc.rust-lang.org/book/ch09-02-recoverable-errors-with-result.html#a-shortcut-for-propagating-errors-the--operator](https://doc.rust-lang.org/book/ch09-02-recoverable-errors-with-result.html#a-shortcut-for-propagating-errors-the--operator) for more details on rust errors
io::stdin().read_line(&mut note)?;

Once you have added the above to the body of the main function, you will see a compiler error:

the ? operator can only be used in a function that returns Result or Option (or another type that implements std::ops::FromResidual)
cannot use the ? operator in a function that returns ()

One option would be to simply remove the “?”. This instead throws the warning: unused std::result::Result that must be used . This is because the read_line function may fail and ignoring this error may have negative consequences for your application.

In order to fix the error without a warning we must update the return type of the main function to return the special rust Result type.

fn main() -> Result<(), Box<dyn std::error::Error>> {
  ...
  Ok(());
}

The “()” type is called “unit”. And can be thought of like void in other languages.

The Box is a pointer type for heap allocation. As I currently understand it, the Error that may be thrown has a dynamic size (e.g. the message passed along with the error could be variable length). Therefore the Box specifies that the Error will be some dynamically allocated memory. Without this you would see the error below:

the size for values of type (dyn std::error::Error + ‘static) cannot be known at compilation time
doesn’t have a size known at compile-time

Finally, we add an Ok(()) at the end of the main function. This fulfils the Result type with the () unit type.

Write a POST Request in rust

1. Install reqwest library

A quick search pointed me to the reqwest library for handling sending HTTP requests. Installing the library is accomplished by adding the following to the Cargo.toml file.

# Cargo.toml
...
[dependencies]
reqwest = { version = "0.11", features = ["json", "cookies"] }

2. Create HashMap with body property

My server expects a json object that looks like { body: "note contents" } and so we use the std::collections::HashMap library to make what is essentially a JavaScript Object.

use std::collections::HashMap;

...

let mut note_post_map = HashMap::new();
note_post_map.insert("body", note);

3. POST JSON data with reqwest

let client = reqwest::Client::new();
let res = client.post("http://engram.xyzdigital.com/api/notes")
    .json(&note_post_map)
    .send()
    .await**?**;

The await keyword is new here. This works similar to async/await in JavaScript. The function will not continue until the asynchronous result from the HTTP request has returned.

4. Add async keyword to outer main function

In order to support the use of await above, we need to add the async keyword to the main function

async fn main() -> Result<(), Box<dyn std::error::Error>> {
  ...
}

Unfortunately, this gives the following error

main function is not allowed to be async

This is where the tokio library comes in to play. We add it to the Cargo.toml under [dependencies] .

...
[dependencies]
tokio = { version = "1", features = ["full"] }

Now we can add [tokio::main] to our main function. From this page, it appears an async runtime is required to actually execute async code. Tokio is one of the popular libraries that provides this async runtime.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
  ...
}

5. Checking the Result

if resp.status() != 200 {
  println!("failed to submit note");
  return Ok(());
}

Finally some code that should look pretty understandable. The response object has a status method to extract the status code of the response. For now, I simple log out that it failed an exit the program.

Wrap Up

I have started and stopped working with rust multiple times. Each time I stopped I noticed it was because I didn’t have a strong understanding of the syntax, which frustrated me. Writing this article has forced me to think a bit deeper about how it all works (I learned most of the terminology along the way). Hopefully something here clicks for you as well.

I plan to continue writing and learning more rust and will add new posts with relevant learnings along the way. Mostly so when I look back in 3 years I can laugh at how much I didn’t really understand 😂.

DEV Community: Adam Berg

Using Github Actions to Deploy a React App and Express API Using Rsync in 15 seconds

Motivation

Creating a Simple App to Demonstrate

Creating a Basic React App

Create index.tsx

Create App.tsx

Building React App with esbuild

Install esbuild

Add client:build script to package.json

Create public/index.html

Prepping a Server

SSH into server

Create github-actions-tutorial User

Install nginx

Create Virtual Host File

Create static folder on github-actions-tutorial user

Creating a Basic Express REST API

Install express

Create server/src/server.tsx

Add server:build script to package.json

Add build script that runs both client and server builds

Prepare the Server to Run the API

Switch to github-actions-tutorial user

Install NVM

Install Node

Install pm2

Update Virtual Host File to Route to API

Bootstrapping the pm2 process

My Workflow

Define workflow name and triggers

Define build-and-deploy job

Conclusion

Submission Category:

Yaml File or Link to Code

adamjberg / github-actions-tutorial

Additional Resources / Info

Building My First Command Line Interface (CLI) with Rust

Inventing Some Requirements

Getting Started

Cargo.toml

main.rs

The Final Output

Cargo.toml

main.rs

Breaking it Down Line by Line

Get input from stdin

Write a POST Request in rust

Wrap Up

Create `public/index.html`

Create `static` folder on `github-actions-tutorial` user

Create `server/src/server.tsx`