DEV Community: Erik Vullings

Publishing a Web site using node, pm2 and nginx

Erik Vullings — Thu, 26 Nov 2020 12:04:14 +0000

During development of a website, I often have the need to share the intermediate results with my clients using basic authentication. On one of my servers, I perform the following steps to get it up and running.

Publish the site

Assuming you have already created the website, build and deploy it to a folder that can be served by node.js. In my case, I often use parcel to build the client side, using something along the following lines as part of the package.json. As I don't want to hard-code the backend server's location in my client code, it is written in an .env file, e.g.

cat packages/gui/.env
SERVER="http://xxx.xxx.xxx.xxx/bob"

And in my GUI code, I reference process.env.SERVER, which Parcel will replace with the value found in the .env file.

If you use TypeScript, you also need to add a declaration file for this to work, e.g. env.d.ts

declare const process: { env: { SERVER: string } };

To actually do the build process, I have something like this in my package.json (and rimraf and parcel-bundler are part of my devDependencies), so I can use npm run build:domain to deploy the site:

  "scripts": {
    "start": "parcel index.html",
    "clean": "rimraf ./public ./.cache ./dist",
    "build": "parcel build index.html --out-dir ../server/public --public-url ./",
    "build:domain": "npm run clean && npm run build"
  },

The most important script is the build script: it tells parcel to bundle all my stuff, put it in the out-dir folder, and use the public-url to make all file references relative to the final location where it will be running. If you do not include the latter, all file references will be hard-coded e.g. /app.123ved.css instead of app.123ved.css. See also the description here.

Deploying the service

Since my backend is on Node, I normally use a simple pm2 service to run it, e.g.

sudo pm2 start PATH/TO/SERVER.js --name APP_NAME
sudo pm2 ls # to see if it is running

For more information about pm2, see here.

Other useful commands are sudo pm2 stop APP_NAME_OR_APP_ID and sudo pm2 start APP_NAME_OR_APP_ID.

As the server is used for many sites, the application needs to run on a dedicated port, preferably one behind a firewall, so it is not reachable from the outside. We use Nginx for that.

Configuring Nginx

Assuming you have Nginx up-and-running, add a new entry for your server in /etc/nginx/sites-available and create a symbolic link to the /etc/nginx/sites-enabled folder, e.g.

sudo vi bob
ln -s bob ../sites-enabled

Where the content of the file is something like the following:

upstream demoserver {
   # port 55555 not reachable from Internet (firewall) 
   server 127.0.0.1:55555;
}

server {
    location = /bob {
        return 302 /bob/;
    }

    location /bob/ {
        auth_basic "Demonstration area";
        auth_basic_user_file /etc/apache2/.htpasswd;

        allow 127.0.0.1;

        # NOTE: The final / is important, as it will strip the prefix 'bob' from all URLs
        proxy_pass http://demoserver/;
    }
}

That means that our service is running on port 55555, and will be reachable via HOST_IP/bob. Assuming, of course, that you have setup a password file. Read on...

Creating a Password File

Source

To create username-password pairs, use a password file creation utility, for example, apache2-utils or httpd-tools.

Create a password file and a first user. Run the htpasswd utility with the -c flag (to create a new file), the file pathname as the first argument, and the username as the second argument:

$ sudo htpasswd -c /etc/apache2/.htpasswd user1

Press Enter and type the password for user1 at the prompts.

Create additional user-password pairs. Omit the -c flag because the file already exists:

$ sudo htpasswd /etc/apache2/.htpasswd user2

You can confirm that the file contains paired usernames and encrypted passwords:

$ cat /etc/apache2/.htpasswd
user1:$apr1$/woC1jnP$KAh0SsVn5qeSMjTtn0E9Q0
user2:$apr1$QdR8fNLT$vbCEEzDj7LyqCMyNpSoBh/
user3:$apr1$Mr5A0e.U$0j39Hp5FfxRkneklXaMrr/

Restarting Nginx

Test your configuration first using

sudo service nginx configtest

And if that returns OK, restart Nginx using

sudo service nginx restart

Getting environment variables to your web application

Erik Vullings — Mon, 02 Dec 2019 16:21:44 +0000

Getting environment variables to your web application

It is often useful during development to inform your build process that it is working in development mode or production mode. And there are multiple ways to do this, e.g. you can often specify them on the command line, use an environment variable, or put them all together in a .env file.

When you are developing a node.js application, getting access to these environment variables in your code is also easy, for example by using process.env.SOME_VAR, optionally combined with dotenv to read those .env files before startup.

However, when you need those environment variables also in your web application, e.g. to specify the URL for the production or development service, it is less well documented if you don't know where to look. As it turns out, fortunately, it is not too difficult either:

Parcel

Parcel is a very easy to use build tool, which transpiles and minifies your JavaScript or TypeScript code, CSS files, etc. without any configuration. But as it happens, Parcel also exposes your environment variables to your web application's code. So you can use process.env.SOME_VAR directly in your JavaScript code. In case you are working with TypeScript, you need to add a declaration for it too, e.g.

declare const process: {
  env: {
    SOME_VAR: string;
  };
};

Webpack

Webpack is also a very popular choice to build your application, but requires much more configuration too. So in order to pass environment variables to your web application code, you need to use a default Webpack plugin, the DefinePlugin (see also here). For example, your webpack config file could look as follows (when you use TypeScript). Note that you use JSON.stringify to pass the value of your variable.

const webpack = require('webpack');
const HtmlWebpackPlugin = require('html-webpack-plugin');
const path = require('path');

// Import environment variables into your build configuration.
module.exports = (env = {}) => {
  const plugins = [
    new HtmlWebpackPlugin({
      meta: {
        description: 'MY_DESCRIPTION',
      },
      title: 'MY_TITLE',
      favicon: './src/assets/favicon.ico',
    }),
    new webpack.HotModuleReplacementPlugin(),
  ];
  // In non-production environment, set the SERVICE_URL
  if (!env.production) {
    plugins.push(
      new webpack.DefinePlugin({
        SERVICE_URL: JSON.stringify('http://localhost:3210'),
      })
    );
  }

  return {
    entry: './src/index.ts',
    mode: 'development',
    devtool: 'inline-source-map',
    devServer: {
      contentBase: './dist',
      hot: true,
    },
    plugins,
    module: {
      rules: [
        {
          test: /\.tsx?$/,
          use: 'awesome-typescript-loader',
          exclude: /node_modules/,
        },
        {
          test: /\.css$/,
          use: ['style-loader', 'css-loader'],
        },
        {
          test: /\.(png|svg|jpg|gif)$/,
          use: ['file-loader'],
        },
        {
          test: /\.(woff|woff2|eot|ttf|otf)$/,
          use: ['file-loader'],
        },
      ],
    },
    resolve: {
      extensions: ['.ts', '.js'],
    },
    output: {
      filename: 'bundle.js',
      path: path.resolve(__dirname, 'dist'),
    },
  };
};

In order to use the previously declared SERVICE_URL in your web app, you still need to declare it (e.g. in declarations.d.ts)

declare var SERVICE_URL: string;

As Webpack replaces your SERVICE_URL everywhere, complete dead code blocks (if (false) { ... }) can be removed during the minification, so that is rather need.

Other build environments

For React or Vue applications, their build tools do this automatically for you: environment variables that follow a specific naming syntax, e.g. start with VUE_, are exposed into your web app.

REST-EASY-LOKI

I follow a similar approach for a small REST service that I built around the in-memory database, lokijs, rest-easy-loki. All environment variables that start with LOKI_ are exposed at SERVER/api/env dynamically.

Client-server communication setup via REST and WebSockets

Erik Vullings — Fri, 14 Jun 2019 15:34:55 +0000

TLDR: In production, or when your client-server app is running in a Docker container, the server port is typically configured outside of your control. So how can the client still discover the server's port dynamically in order to access REST resources or setup a WebSocket connection? The approach taken here is too serve the client as a static resource via the server, use document.location.href to infer the location of the server, and fallback to the hard-coded development settings if that fails.

When developing client-server web applications, I normally use a node.js-based server and a Single-Page Application client. Typically, the server provides some REST endpoints, for example to fetch some lists, and also allows for real-time WebSocket communication.

This works pretty easy during development: the server offers a REST and WebSocket interface on a configured port, let's say 1234, and the client connects to it http://localhost:1234/[REST-RESOURCE] or io(http://localhost:1234). As the client may be served by a development server (like webpack-dev-server or parcel) to have hot reloading, you only need to enable CORS on the server to allow the client to connect to it. On the server, if you use nest.js, it may be as simple as

import * as bodyParser from 'body-parser';
import { NestFactory } from '@nestjs/core';
import { NestExpressApplication } from '@nestjs/platform-express';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create<NestExpressApplication>(AppModule, { cors: true });
  app.use(bodyParser.json({ limit: '10mb' }));
  app.use(bodyParser.urlencoded({ limit: '10mb' }));

  const port = 1234;
  await app.listen(port, () => {
    console.log(`Server is listening on port ${port}.`);
  });
}
bootstrap();

This works all well during development, but in production or in a Docker container, we most likely need to specify the port of the server too, so instead of hard-coding the port, we get it from the environment by replacing const port = 1234; with:

  const port = process.env.MY_SERVER_PORT || 1234;

Since now you no longer control the port, how do you tell your client where your server is located? One solution is the following: when building the application for production, the generated output is moved to the server's public folder used for serving static content, i.e.

  app.use(express.static(path.join(process.cwd(), 'public')));

This enables the client to discover the server using window.location or document.location.href. As the client is not aware whether it is running in production mode or in development mode, it assumes it is running in production mode, and if the connection fails, it switches seamlessly to development mode. So for socket.io the client first tries production mode, and if this fails, i.e. it receives a connect_error, it tries development mode:

let socket: SocketIOClient.Socket;

const setupSocket = (productionMode = true) => {
  socket = productionMode ? io() : io('http://localhost:1234');
  socket.on('connect_error', () => {
    socket.close();
    if (productionMode) {
      socket = setupSocket(false);
    } else {
      console.error('event', data);
    }
  });
};

For the REST services, a similar approach is taken: first try document.location.href to reach the server, and when that fails, try the hard-coded development location. Something like

  public async loadList(): Promise<T[] | undefined> {
    try {
      const result = await m
      .request<T[]>({
        method: 'GET',
        url: this.baseUrl,
        withCredentials,
      });
      return result;
    } catch {
      if (this.developmentMode) {
        throw Error('Help');
      }
      // The initial value was based on production mode, so when you fail, switch
      this.baseUrl = this.createDevModeUrl();
      return this.loadList();
    }
  }

If you are dealing with a reverse proxy like nginx, traefik or redbird, you can even get a bit fancier using the following snippet, grabbing everything before the first hash tag.

const getRootUrl = () => {
  // Regex matching everything until the first hash symbol, so should also be able to deal with route rewriting...
  const regex = /https?:\/\/.*(?=\/#)/i;
  const route = document.location.href;
  const m = route.match(regex);
  return (m && m.length === 1) ? m[0].toString() : '';
};

In an actual application, I normally store the development or production mode in the application state or store, so I only fail once per connection.

This solution, I profess, is not very elegant, and requires a bit of code to set it up properly. Also, in development, you most likely get one or two warnings due to failed communication with the server, as it assumes it is running in production mode (still, better than to have these errors while running in production). Therefore, I am happy to hear your suggestions or improvements.

Meeting-o-meter

Erik Vullings — Fri, 24 May 2019 18:34:45 +0000

Did you ever attend a meeting that went on and on. And everybody had to say something to make it last even longer. Well, if this a recurring event in your life, maybe the meeting-o-meter offers a solution for you. By specifying the number of people, and their hourly rate, you can measure the cost of each second that is (potentially) wasted. At the least, it gives everyone an insight in the costs of attending such a meeting. Its open source, so please, give it a spin and fork it to improve it.

How to use it

Switch to the settings (top right menu) to change the hourly rates.
Add the bottom of the screen, specify the number of people
Press start to start measuring the costs

Building

Originally developed in Visual Studio 2012, it still works in VS2015. But as it was originally developed in 2013, I have included the libraries that I've used too, as you may not able to find them again.

About

This is an old project that I developed in 2013, and it recently came up in a conversation. So I looked up the code and am sharing it here. If you want to continue my work, please fork the project, as I am currently using TypeScript for most of my projects, so I can share them on the web. For example, you could:

Port it to a web environment (I recommend using the mithril framework).
Add a page to show an agenda
Add a page to enter the results (so you can do a cost/benefit analysis
Send/save the results