DEV Community: Chris Muir

Kubernetizing a basic Slack app on Oracle cloud (OCI)

Chris Muir — Mon, 21 Jun 2021 03:00:53 +0000

In my previous two blog posts I spoke about how to create a basic Slack app using NodeJS and Slack's Bolt JavaScript framework, and then Dockerizing it.

In this blog post I want to capture my notes about deploying the application to a Kubernetes cluster using Oracle's cloud offerings a.k.a. Oracle Cloud Infrastructure (OCI). As per the previous blog posts, there's nothing staggeringly new in this blog post, I'm simply capturing my notes on how to do this so I don't have to remember all the details for next time.

In this post we're going to use OCI to setup a Kubernetes cluster, configure an OCI user account to be remotely managed from my Mac, and then use Docker and kubectl on my Mac to deploy the application to the Kubernetes cluster on OCI.

Assumptions

For this blog post I'm making a few assumptions:

(a) We have access to an OCI tenancy
(b) We have privileges in OCI to create a Kubernetes Cluster and deploy to the Container Registry

For (b) if our user account is an OCI administrator we have all the privileges required, otherwise we need to follow these OCI documents to grant the appropriate privileges:

Setting up a Kubernetes cluster on OCI

In OCI there is a number of ways to setup cloud resources like a Kubernetes cluster, such as using the OCI web console, Terraform, a command line client (a.k.a. OCI CLI) and more. We'll take a shortcut (read: cheat) and use the Kubernetes wizard in the OCI console.

First we'll create a compartment to segment the Kubernetes work from other work in our OCI tenancy.

(1) With an OCI tenancy created and a login to the OCI console, via the hamburger menu select Identity & Security then Compartments from the sub menu under the Identity heading.

(2) Create a compartment "k8s-demo-compartment" under the root compartment

(3) Via the hamburger menu select Developer Services then Kubernetes Clusters (OKE) under the Containers & Artifacts heading.

(4) On the left hand side of the screen, under Compartment select the compartment "k8s-demo-compartment" that you created earlier.

(5) Select the Create Cluster button.

(6) In the result Create Cluster dialog select the Quick Create option. This option is designed to create all the necessary Kubernetes artifacts including a virtual cloud network (VCN), various gateways, the Kubernetes cluster working nodes and so on. Select Launch Workflow

(7) In the resulting Quick Create Cluster set the following:

Change the cluster Name to "k8s-demo-cluster"
Ensure the Kubernetes API Endpoint is set to Public Endpoint
Ensure the Kubernetes Worker Nodes is set to Private Workers
Select the VM.Standard.E3.Flex Shape
Limit the VM to one OCPU and 16GB memory
Leave the Number of nodes at 3.

(8) Click Next

The resulting screen will show us the progress in creating the various artifacts needed for the Kubernetes cluster.

(9) Once all work is completed click Close

The OCI console will then place us in the Cluster Details screen. Note the status of the cluster will be Creating. Wait until it reads Active.

(10) Once done select the Node Pools option in the left hand-side menu, and in the resulting page select pool1 and then Nodes in the left hand-side menu.

The resulting screen will show us the progress in creating the nodes needed for the Kubernetes cluster. Once set to 'Ready' the Kubernetes cluster is fully setup ready for the next steps.

Where to from here?

In the previous blog we deployed the Slack application to our local Docker instance. Our goal in this blog hereafter is to deploy the application to OCI to run inside the Kubernetes cluster we just created. But there's a few steps we must do before we can get to this.

First we need to configure an OCI user and an authentication token to allow connections from the various tools we're going to use on our local machine, a Mac in my case, to deploy our application to OCI.
Once we've setup an OCI user with the appropriate settings, we'll then use Docker to deploy our local image to an OCI Container Registry (OCIR).
Finally we'll use a local installation of the Kubernetes kubectl command line tool to hook up the Kubernetes cluster with the OCIR deployed image.

Let's run through those steps now:

Configure our OCI user with an auth token to deploy to OCI

As mentioned previously, our next piece of work is to setup our OCI user with an authentication token. This will allow us to connect to OCI from various tools we're going to use on our local machine to deploy our application to OCI.

(11) In the OCI Console in the top right of the screen, select the user Profile button then select the User Settings option.
(12) In the left hand-side menu select Auth Tokens.
(13) Select the Generate Token button, and in the resulting dialog give the Description "Token for Kubernetes deployment". Select the Generate Token button.
(14) In the resulting screen select the Show button, and record the hidden token. Think of this as a password we're going to use momentarily to access OCI from your local machine.

Create an OCI Container Registry

Next we'll setup an OCI Container Registry ready in a moment to receive/push our image from our local machine:

(15) In the OCI Console select the hamburger menu, then Developer Services followed by Container Registry under the Containers & Artifact heading.
(16) In the resulting screen on the left hand side, select the "k8s-demo-compartment" Compartment.
(17) Select the Create Repository button.
(18) In the resulting Create Repository dialog enter a Repository Name of "scratchslackapp" (all in lowercase) and set the Access to Public.

Push our local application Docker image to the OCI Container Registry

In order to push our local application Docker image to the OCI Container Registry we need to login to OCI via Docker on our local machine. To do this we will issue the following command:

docker login <tenancy-region-key>.ocir.io --username <tenancy-id>/<user-name>

(19) To determine the tenancy-region-code, in the OCI Console click on the user Profile button again in the top right hand of the screen then select the Tenancy option. In the resulting screen we will have the value Home Region which is the region your tenancy sits. For example "US East (Ashburn)".

(20) Once we known the region, in the table in the following documentation page lookup the corresponding region key needed for the login statement above in lowercase. For example for "US East (Ashburn)" the region key is "IAD". Convert this to lowercase.
(21) Still on the same screen, also note the Object Storage Namespace value. This is the tenancy ID value. For customer purchased tenancies this will be something we've agreed with Oracle beforehand (e.g. 'acme' or some other meaningful tenancy identifier for our org). If we're using the OCI free tier this will be an Oracle generated identifier, such as "kdhkv4bxlaap".
(22) Also record the tenancy Name. This is the tenancy name and will be required in (much) later steps in this blog.
(23) Finally the user name is simply the user name we use to login into OCI, such as "joe.doe@oracle.com".

In returning to the docker login command using the examples above, it would look like:

(24) docker login iad.ocir.io --username kdhkv4bxlaap/joe.doe@acme.com

(25) At this point we'll be prompted for a password. Provide the authentication token we were provided earlier.

Next within our local Docker repository we want to tag the image we wish to deploy to the OCIR container registry with the name of OCIR container repository name. We do this by issuing:

docker tag <local-image-name>:latest <tenancy-region-key>.ocir.io/<tenancy-id>/<oci-repository-name>:latest

Using the examples used in the blog so far, this would be:

(26) docker tag scratchslackapp:latest iad.ocir.io/kdhkv4bxlaap/scratchslackapp:latest

From here we push the image to OCI:

(27) docker push iad.ocir.io/kdhkv4bxlaap/scratchslackapp:latest

(28) To see the results, return to the OCI Console, via the hamburger menu select Developer Services followed by Container Registry under the Containers & Artifact heading.
(29) Ensure the "k8s-demo-compartment" is still selected on the left-hand side of the screen, then in the middle select and expand the "scratchslackapp" repository you created earlier. We should now find a single image which represents what we just uploaded.

Setup kubectl ready to configure Kubernetes with the image

At this point our image is now in OCI, but hasn't been associated with the Kubernetes cluster.

In order to work on Kubernetes, Kubernetes provides a command line tool called kubectl (a.k.a. kubernetes control). This can be downloaded from the Kubernetes tools website. Once installed it must then be configured to talk to our OCI Kubernetes cluster. The configuration typically goes in the local machine user's home directory in a .kube/config file.

OCI makes it easy to configure this by providing a handy dialog under the Kubernetes cluster to show us how exactly to set this up.

(30) Via the OCI console hamburger menu again open the Kubernetes Clusters (OKE) option again, then select our cluster.
(31) Press the Access Your Cluster button.
(32) In the resulting dialog it will explain how to setup the .kube/config file by downloading and installing & configure the OCI CLI tool, and then running a number of steps. Follow those steps!

(33) We can then verify kubectl is working properly by listing the running nodes in your OCI cluster: kubectl get nodes

Use kubectl to configure Kubernetes with the image

Okay, we're getting close!

In order to associate the Kubernetes cluster with our image, we first need to setup a Kubernetes 'secret' using the following command:

kubectl create secret docker-registry <secret-name-to-create> --docker-server=<tenancy-region-key>.ocir.io --docker-username='<tenancy-namespace>/<oci-username>' --docker-password='<auth-token>' --docker-email='<email-address>

Most of the above values are now self explanatory, but the requires some more explanation. If our tenancy's users are federated with Oracle Identify Cloud Service such as the OCI free tier uses, we use the format oracleidentitycloudservice/, for example:

(34 v1) kubectl create secret docker-registry my-kube-secret --docker-server=iad.ocir.io --docker-username='my-tenancy-name/oracleidentitycloudservice/joe.doe@acme.com' --docker-password='our auth token' --docker-email='joe.doe@acme.com

If alternatively our tenancy is a customer owned tenancy without Oracle Identity Cloud Service, the command takes the form:

(34 v2) kubectl create secret docker-registry my-kube-secret --docker-server=iad.ocir.io --docker-username='my-tenancy-name/joe.doe@acme.com' --docker-password='our auth token' --docker-email='joe.doe@acme.com

(35) The secret can be verified by issuing: kubectl get secret my-kube-secret --output.yaml

With the secret in place we're then ready to instruct Kubernetes to use the image. This can be done via the kubectl file, but ends up being repetitive and prone to manual errors.

An alternative approach to create a Kubernetes YAML file that can be used multiple times.

(36) In your favourite text editor, return to our application's source code and create a new file kube.yaml

(37) Add the following source code:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: scratchslackapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: scratchslackapp
  template:
    metadata:
      labels:
        app: scratchslackapp
        version: v1
    spec:
      containers:
      - name: scratchslackapp
        image: iad.ocir.io/kdhkv4bxlaap/scratchslackapp:latest
        imagePullPolicy: Always
        ports:
        - containerPort: 5000
          protocol: TCP
      imagePullSecrets:
        - name: my-kube-secret
---
apiVersion: v1
kind: Service
metadata:
  name: scratchslackapp-lb
  labels:
    app: scratchslackapp
spec:
  selector:
      app: scratchslackapp
  type: LoadBalancer
  ports:
  - port: 8080
    targetPort: 5000
    name: http

This file contains two main parts, that above the 3 dashes (---) and that below the 3 dashes.

Above - defines the application to deploy to the Kubernetes pod. Note how the image maps to our image we just deployed to the OCI Container Registry, the imagePullSecrets uses my-kube-secret we just created, and the containerPort maps to the Docker EXPOSE port of 5000 we configured in the previous blog.

Below - in order for the application to be exposed to the internet, this creates an OCI load balancer, mapping port 8080 externally to the targetPort 5000 internally for our application.

(38) With this file saved, return to our command line, cd to the same directory as the yaml file and execute:

kubectl create -f kube.yaml

This command fires Kubernetes to deploy our application to the Kubernetes pod, and spins up a load balancer for the cluster so external calls from Slack can be routed to our running Slack code. The command will return when the pod is ready. However the load balancer may take longer to start so be patient!

(39) To check if the load balancer is ready issue the following command:

kubectl get services

We will see an output similar to the following:

NAME                 TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)          AGE
kubernetes           ClusterIP      10.11.0.1       <none>           443/TCP          27h
scratchslackapp-lb   LoadBalancer   10.11.157.133   132.221.40.151   8080:31467/TCP   5m16s

If the external-ip for the load balancer says '' we need to wait longer.

While we're waiting other commands we can issue:

kubetcl get all
kubectl get services
kubectl describe services scratchslackapp
kubectl get pods
kubectl get pod <pod-id-from-previous-command>
kubectl get deployments scratchslackapp
kubectl describe deployments scratchslackapp

Once the load balancer is running, the very last thing we need to do is return to Slack, visiting the api.slack.com, open our application, select the Event Subscriptions page and update the Request URL to the http (not https as we haven't configured it in this example) address of the external IP address from above, and port 8080 we configured in the kube.yaml file.

For example:

http://132.221.40.151:8080/slack/events

(40 optional) There after if we're happy with our testing from Slack, you can delete the pod deployment by executing:

kubectl delete -f kube.yaml

Done, in 40 steps! (phew)

Dockerizing a basic Slack app

Chris Muir — Wed, 16 Jun 2021 05:42:25 +0000

In a previous blog I wrote about creating a basic Slack app using NodeJS, Slack's Bolt JavaScript framework, and ngrok. In this blog I will extend this to include deploying the application to Docker.

Like the previous blog, this blog is just capturing my notes on how to do this so I don't have to remember all the steps.

The main inspiration for the Dockerfile in this post comes from Kathleen Juell's post How to Build a Node.js Application with Docker. All credit goes to Kathleen for summarizing these steps so neatly.

Adding the Dockerfile

On the assumption we already have Docker installed on our local machine, within the application we create a Docker configuration file literally called the Dockerfile.

(1) In the previous application's root directory, in our favourite text editor, create a new file Dockerfile.
(2) Copy in the following code:



FROM node:14.7.0
USER node
RUN mkdir -p /home/node/app/node_modules && chown -R node:node /home/node/app
WORKDIR /home/node/ScratchSlackApp
COPY package*.json ./
RUN npm install
COPY --chown=node:node . .
EXPOSE 5000
CMD [ "node", "app.js" ]

What this does:

FROM node:14.7.0 - defines the version of Node to run in the Docker container. The tag 14.7.0 represents the current NodeJS LTS version at the time of writing. Other supported version can be found via the Docker Official Node Images page.
USER node - allows us to create a new user besides root to run the application
RUN mkdir && chown - creates the application directory structure and gives the node user the appropriate ownership on the files
WORKDIR - sets the default directory for the container to start executing the code from. This is linked to the CMD call explained below
COPY package*.json ./ - copies the package.json file into the container
RUN npm install - within in the container downloads the necessary dependencies/libraries defined in the package.json file
COPY --chown-node:node . . - the two full stops say to copy the code from the base directory of the source, to the base directory of the container. The chown statement makes the owner the node user we defined earlier in USER.
EXPOSE 5000 - defines which port the application running in the container will be listening on
CMD - defines how the application will be started in the container

Of special note from the Node Bolt application example in my previous blog, remember that the application runs on port 5000. This is why I've set EXPOSE to 5000 in the Dockerfile.

(4) When building the container we want it to download the node_modules fresh. To avoid the existing node_modules being copied over, we create a .dockerignore file and add the following entries:



node_modules

.dockerignore

Build the Docker image

With the Dockerfile in place we can then build the first Docker application image using the following command from the application's source base directory:

(5) docker build -t scratchslackapp .

The -t flag represents the image name to build in Docker. + The image name must be in lowercase.
The final full-stop implies the image is built from the current directory

Docker images can be listed by executing docker images after the build.

Create and run a Docker container

With the image in place, we can now create a running container based on the image:

(6) docker run --name scratchslackapp -p 5000:5000 -d scratchslackapp

The --name flag defines the container name. I've made this the same as the image, but it can be different.
The -p flag defines the host post mapped to the container port we defined earlier in the Dockerfile. I've kept this to 5000 in all cases to keep things simple.
The -d flag runs the container in the background

Some useful additional Docker commands to know:

docker ps - lists the running Docker containers, which includes a unique generated container-id per instance, used in the following commands.
docker stop <container-id - stops a container
docker rm <container-name> - deletes a container
docker logs --follow <container-id> - tails the container's STDOUT & STDERR output to the screen.

If the code needs to be updated, stop and remove the previous container, then build and run via the docker commands described above.

Accessing the Docker container via Slack

From the above, assuming the Docker container is now running, from the previous blog post, remember if we've restarted ngrok, that the ngrok port must align with the Docker EXPOSE port, and, if we're using the free version of ngrok that rotates the URL, we must also update the URL in the Slack manifest file. Look to the previous blog post for how that was done.

Creating a basic Slack app

Chris Muir — Tue, 15 Jun 2021 05:53:54 +0000

Slack has done a good job of documenting how to create a basic Slack app to get you over the configuration hump. And there are some handy simple examples on Glitch which save you from having to create your own server to host the backend code. The following blog post captures my notes on how to do the same on my local Mac using NodeJS, Slack's Bolt JavaScript framework, and ngrok to expose the application to the internet.

There's nothing revolutionary in the following blog post, this is just a documentation exercise so I don't have to keep remembering every step!

Setup ngrok

Mac install instructions:
(1) Download ngrok (https://ngrok.com/download)
(2) Unzip the ngrok zip file, and drag and drop the resulting ngrok file to our user's Mac Application folder
(3) Setup a symbolic link so ngrok can be accessed across our system:
cd /usr/local/bin
ln -s /Applications/ngrok ngrok
(4) Start ngrok:
ngrok http 5000

Note we're forwarding to port 5000 in this example. Change this to whatever we want, but record the value. This will be important when we create the backend NodeJS code for the application shortly.

From the ngrok's output note the https:// forwarding address. Record this, you will need this shortly. The following is an example output:

Session Status                online                                                                
Session Expires               1 hour, 59 minutes                                                    
Version                       2.3.40                                                                
Region                        United States (us)                                                    
Web Interface                 http://127.0.0.1:4040                                                 
Forwarding                    http://165a936dd19b.ngrok.io -> http://localhost:5000                 
Forwarding                    https://165a936dd19b.ngrok.io -> http://localhost:5000

Configure the app at api.slack.com

Slack's api.slack.com website is where we setup and maintain the configuration of our application such as the application name, privileges, and the URLs for Slack to call our backend NodeJS application that we will create shortly.

In the following example we will use Slack's new beta manifest file format to setup the basic configuration.

(5) Create or reuse our own Slack account and Slack workspace
(6) Login to api.slack.com using our Slack account
(7) Select Your Apps button top right, then the Create an App button centre of the screen
(8) Select From an app manifest
(9) Step 1 of 3: Select your workspace
(10) Step 2 of 3: Copy in the following manifest file:

_metadata:
  major_version: 1
  minor_version: 1
display_information:
  name: ScratchSlackApp
features:
  app_home:
    home_tab_enabled: false
    messages_tab_enabled: true
    messages_tab_read_only_enabled: false
  bot_user:
    display_name: ScratchSlackApp
    always_online: false
oauth_config:
  scopes:
    bot:
      - chat:write
      - im:write
      - im:history
settings:
  event_subscriptions:
    request_url: https://165a936dd19b.ngrok.io/slack/events
    bot_events:
      - message.im
  org_deploy_enabled: false
  socket_mode_enabled: false

Update the request_url with the ngrok https:// forwarding address from earlier.

(11) Continue with step 3 of 3: Select Create

(12) The website will now take us to the settings for the application. Ensure the Basic Information settings page is selected, then within this page under the Install your app heading, select the Install to Workspace button.
(13) In the proceeding permissions page, select the Allow button. This step will make the application available to our user in the Slack client under the Apps section. At this stage the application has no backend logic so the app will do nothing, we will create this next.
(14) Returning to the Basic Information setting page, navigate down to the App Credentials section and Show the Signing Secret (commonly known as the 'Slack Signing Secret'). Record this value, we will need this value in a moment.
(15) Select the OAuth & Permissions features page, and note the Bot User Oauth Token. Also record this value, you will need this in a moment.

ngrok note: if you stop and restart ngrok, assuming you are using the free version which does not provide a static URL, the ngrok URL will change each restart. To update the above request_url in the Slack app, visit the Event Subscriptions page and update the Request URL. However note if you manually update this value (rather than defining it through the manifest file when you first create the app), Slack will attempt to check that the URL is live and running. As such you need to create and run the Slack application first as defined in the next section before you can update the URL.

Create the NodeJS backend application

On completing the above steps, we are now ready to create the backend NodeJS application that will listen and respond to events sent by the user via the Slack client, relayed through Slack, as configured in the above settings.

In other words if the user types a message into our Slack app in the Slack client, the message will be relayed from the Slack client to Slack's own servers, whereupon Slack will look up the request_url defined above, and send the message to the defined URL where our Slack code is running and can respond.

(16) On our local Mac:
cd ~/Desktop
mkdir ScratchSlackApp
cd ScratchSlackApp
npm init -y

(17) In the same directory open the generated package.json file in our favourite text editor and overwrite the complete file with the following contents:

{
  "name": "scratchslackapp",
  "version": "1.0.0",
  "description": "",
  "main": "app.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@slack/bolt": "^3.4.0",
    "dotenv": "^10.0.0"
  }
}

(18) Still within our favourite text editor, create a new file app.js in the same directory and copy in the following code:

const { App, ExpressReceiver } = require('@slack/bolt');

require('dotenv').config();

const app = new App({
  token: process.env.SLACK_BOT_TOKEN,
  signingSecret: process.env.SLACK_SIGNING_SECRET
});

app.message(async ({ body, message, context, say }) => {
  try {
    if (body.event.type === 'message' && message.text !== undefined) {
      say('Hi!');
    }
  } catch (error) {
    say(error.message);
  }
});

(async () => {
  await app.start(process.env.PORT);
  console.log('⚡️ Bolt app is running on port ' + process.env.PORT + '!');
})();

The basic code here:

New App() sets up a Slack app using the Slack Bolt JavaScript framework. This includes all the magic for setting up the required HTTP listeners, that will receive HTTP/HTTPS requests sent from Slack to our code via the request_url we configured in the manifest file earlier.
The app.message() handler receives 'text' messages from Slack which we can then respond to.
The final app.start() call in the async block starts the Bolt server, including the HTTP listeners.

(19) Note the 3 process.env calls to PORT, SLACK_BOT_TOKEN & SLACK_SIGNING_SECRET. These are the 3 values we asked you to record earlier. To set these in our favourite text editor create an additional file .env in the application's base directory, and copy in the values we recorded:

PORT=5000
SLACK_BOT_TOKEN=xoxb-123472270484-1234804366771-cEXFXx0jgAA9bnAAaS16fvgc
SLACK_SIGNING_SECRET=263b7d12a7ccaea3f838f3ef123062ef

The above values are examples only.

(20) Finally on the command line we need to install the libraries/dependencies defined in the package.json file then run our application:

npm install
node app.js

fast-csv for CSV files

Chris Muir — Fri, 05 Mar 2021 01:52:41 +0000

I recently had to undertake pre-processing on a CSV file with NodeJS+Typescript before ingesting it into a system.

The CSV file in question presents a number of challenges:

The CSV file is large @ ~125k rows
Includes a header row but individual headers need to be renamed
There are redundant columns to remove
There may be additional columns that we also don't know about that need to be dropped
The columns need reordering
Blank lines must be skipped

Via a quick Google I found fast-csv.

An initial & superficial look at fast-csv highlights a few qualities making it attractive enough to explore further:

It is still actively being developed (at the time of this post) giving some assurance around bug fixes
Uses the MIT friendly open source license
Has no runtime dependencies minimizing any down stream license issues

In looking at the feature set, fast-csv is comprised of 'parse' and 'format' routines for ingesting and transforming CSV files. It also supports streams for fast processing of large files. The following describes how I made use of fast-csv features to meet the above requirements.

To start with here's the initial CSV file we will ingest:

beta,alpha,redundant,charlie,delta

betaRow1,alphaRow1,redundantRow1,charlieRow1,deltaRow1
betaRow2,alphaRow2,redundantRow2,charlieRow2,deltaRow2
betaRow3,alphaRow3,redundantRow3,charlieRow3,deltaRow3

Our goal is to rename and reorder the columns, drop the blank line, drop the 'redundant' column, and our program should be able to also drop the 'delta' column which it wont know about at all. The final output should look like:

NewAlpha,NewBeta,NewCharlie
alphaRow1,betaRow1,charlieRow1
alphaRow2,betaRow2,charlieRow2
alphaRow3,betaRow3,charlieRow3

The following code shows the solution:

import * as fs from 'fs';
import * as csv from 'fast-csv';

const inputFile = __dirname + '/../sample-data/input.csv';
const outputFile = __dirname + '/../sample-data/output.csv';

(async function () {

  const writeStream = fs.createWriteStream(outputFile);

  const parse = csv.parse(
    { 
      ignoreEmpty: true,
      discardUnmappedColumns: true,
      headers: ['beta','alpha','redundant','charlie'],
    });

  const transform = csv.format({ headers: true })
    .transform((row) => (
      {
        NewAlpha: row.alpha, // reordered
        NewBeta: row.beta,
        NewCharlie: row.charlie,
        // redundant is dropped
        // delta is not loaded by parse() above
      }
    ));

  const stream = fs.createReadStream(inputFile)
    .pipe(parse)
    .pipe(transform)
    .pipe(writeStream);
})();

In explaining the solution:

parse() options

ignoreEmpty takes care of skipping the blank line(s)
discardUnmappedColumns will drop any columns we don't specify in the following headers option, taking care of dropping the 'delta' column
headers maps the columns we are loading. Note how I've used discardUnmappedColumns to drop 'delta' but I'm still loading 'redundant'. The 'redundant' column is dropped in the format() options described next

format() options

headers directs the output to include the header row
The transform() row post-processor allows us to reorder the columns, rename the columns, and also drop the 'redundant' column

With a larger CSV file in hand, testing shows the above routine can process ~125k rows with 126 columns, from a file of approx 135MB in size, in ~19 seconds on my MBP 3.2Ghz i7.

fast-csv indeed.