DEV Community: Emmanuel Odongo

Reading Secrets With the 1Password CLI

Emmanuel Odongo — Wed, 06 Sep 2023 20:15:17 +0000

I use 1Password as my password manager but didn't really see much need for the CLI that they provide until fairly recently. I'll go over a couple of use cases where the CLI has integrated really well into my workflow.

Keeping Dot Files Password-Free

I've been using aerc¹ for a few weeks. When you add an email account to aerc, it saves the password in a configuration file (~/.config/aerc/accounts.conf for me), an example of which can be seen below:

[Fastmail]
source   = imaps://user%40fastmail.com:agvsbg8gd29ybgqh@imap.fastmail.com
outgoing = smtps://user%40fastmail.com:agvsbg8gd29ybgqh@imap.fastmail.com

Having the password stored in plain text² is less than ideal, even if it is on a device you own. Helpfully, aerc provides a way to specify an arbitrary command that can be executed to retrieve the password. To use the 1Password CLI, the accounts configuration file can be modified as follows:

[Fastmail]
source            = imaps://user%40fastmail.com@imap.fastmail.com
source-cred-cmd   = op read op://MyVault/Fastmail/aerc-password
outgoing          = smtps://user%40fastmail.com@imap.fastmail.com
outgoing-cred-cmd = op read op://MyVault/Fastmail/aerc-password

The command we want executed is op read, and we pass it the URL³ of the secret to access. The next time aerc is launched, a TouchID prompt, or a prompt to Allow Access, will be presented as shown below:

Autofilling One-Time Passwords

As a publisher of npm packages, it's a good idea to enable 2FA on your npm account. This makes a leaked token with write-access less of a risk since no writes (such as publishing a new version of a package) can be performed without a valid OTP.

When publishing an npm package using npm publish, a prompt is shown in the terminal asking the user to type in the OTP. However, there is also an --otp flag we can make use of to provide the OTP upfront:

npm publish --otp $(op item get NPM --otp)

This time we use the op item get command⁴, passing it the name of the item and the --otp flag. Upon execution, a TouchID prompt or an Allow Access prompt is presented, removing the need to manually type or paste the OTP. As an added convenience, the above command can be bound to a shell alias.

Footnotes

aerc is a terminal-based email client. ↩︎
This helpful URL scheme shows that the password is agvsbg8gd29ybgqh. ↩︎
The URL takes the form:
```
op://<vault>/<item>[/<section>]/<field>
```
↩︎
Note that if we tried using the op read command:
```
op read 'op://MyVault/NPM/Security/one-time password'
```
instead of the current OTP being returned, we would get the reference URL used to generate the OTP:
```
otpauth://totp/croccifixio?secret=AGVSBG8GD29YBGQHIGDVB2QGBMLNAHQ1&issuer=npm
```
↩︎

Testing Async Generators

Emmanuel Odongo — Mon, 03 Apr 2023 20:03:16 +0000

Event Emitters and Async Generators

I recently came across a situation where I needed to stream realtime updates from server to client. After some research, I opted not to go with the defacto solution of web sockets, and instead went with the equally well-supported approach of Server Sent Events (SSE).

SSE is a one-directional communication channel with an impressively simple browser API:

// establish connection
const eventSource = new EventSource(url);

// listen and handle events
eventSource.addEventListener(eventName, eventHandler);

// close connection
eventSource.close();

If the connection is interrupted without explicitly being closed by the client, the browser will automatically attempt to reestablish the connection.

On the server side, I used the Fastify SSE Plugin which supports using an event emitter to handle the firing of events.

Here's a simplified version of a GET /rates endpoint used to subscribe to receive exchange rates:

import fastify from "fastify";
import { FastifySSEPlugin } from "fastify-sse-v2";
import { EventEmitter, on } from "events";

const eventEmitter = new EventEmitter();
const server = fastify();
server.register(FastifySSEPlugin);

server.get("/rates", (_request, reply) => {
  reply.sse(
    (async function* () {
      for await (const [payload] of on(eventEmitter, "ratesUpdated")) {
        yield {
          data: JSON.stringify(payload),
          event: "update",
        };
      }
    })()
  );
});

The async generator – async function* () – is what allows us to listen to events fired by the event emitter.

It's a good idea to use an abort controller to clean up when the connection drops. Here's what the code now looks like:

import fastify from "fastify";
import { FastifySSEPlugin } from "fastify-sse-v2";
import { EventEmitter, on } from "events";

const eventEmitter = new EventEmitter();
const server = fastify();
server.register(FastifySSEPlugin);

server.get("/rates", (request, reply) => {
  const abortController = new AbortController();

  request.socket.on("close", () => {
    abortController.abort();
  });

  reply.sse(
    (async function* () {
      for await (const [payload] of on(eventEmitter, "ratesUpdated", { signal: abortController.signal })) {
        yield {
          data: JSON.stringify(payload),
          event: "update",
        };
      }
    })()
  );
});

We can extract the async generator into a reusable and testable unit:

import { EventEmitter, on } from "events";
import { EventMessage } from "fastify-sse-v2";

interface Params {
  abortController: AbortController;
  eventEmitter: EventEmitter;
  eventName: string;
}

function makeEventListenerGenerator({
  abortController,
  eventEmitter,
  eventName,
}: Params) {
  return async function* (): AsyncGenerator<EventMessage> {
    for await (const [data] of on(
      eventEmitter,
      eventName,
      { signal: abortController.signal }
    )) {
      yield { 
        data: JSON.stringify(data), 
        event: "update",
      };
    }
  };
}

This function can then be used in the GET /rates handler as follows:

reply.sse(
  makeEventListenerGenerator({
    abortController,
    eventEmitter,
    eventName: "ratesUpdated",
  })()
);

Writing the Test

Before we can test our makeEventListenerGenerator function, it's important to understand that it returns an async generator function. Calling this function returns an async iterator: an object that can generate a sequence of values asynchronously.

The on function, which we imported from node's events module, is roughly equivalent to the browser's addEventListener method. We can subscribe to events that are fired by the event emitter using the on function.

Firing events is done using the event emitter's emit method.

Here's the whole flow of publishing and consuming events:

import { EventEmitter, on } from "events";

const eventEmitter = new EventEmitter();
const iterator = on(eventEmitter, "ping");

eventEmitter.emit("ping", { key: "value" });

await iterator.next(); // => { value: [{ key: "value" }], done: false }

Armed with this knowledge, we can now unit test the makeEventListenerGenerator function:

import { EventEmitter } from "events";
import { describe, expect, test } from "vitest";

import { makeEventListenerGenerator } from "./makeEventListenerGenerator";

describe("makeEventListenerGenerator", () => {
  test("iterates over emitted events", () => {
    const abortController = new AbortController();
    const eventEmitter = new EventEmitter();
    const eventName = "ratesUpdated";
    const eventPayload = [{ from: "USD", to: "EUR", rate: 0.94 }];

    const eventIterator = makeEventListenerGenerator({
      abortController,
      eventEmitter,
      eventName,
    })();

    (async () => {
      expect(await eventIterator.next()).toHaveProperty("value", {
        data: JSON.stringify(eventPayload),
        event: "update",
      });
    })();

    eventEmitter.emit(eventName, eventPayload);
  });
});

With that, our unit test is complete and we can give ourselves a pat on the back. But before I close off, there is one final key point that I feel needs to be covered.

Typically, unit tests take the form: arrange -> act -> assert. If we read the test we just wrote from top to bottom, it seems like we are doing arrange -> act -> assert -> act. What gives?

The last part of our test that runs is not the eventEmitter.emit(...) line, but rather our assertion: expect(...).toHaveProperty(...). This is because, as soon as the await keyword is encountered, the evaluation of the expression to its right – eventIterator.next() – will be pushed onto the Microtask Queue. The main thread continues executing to the end, and only then can the result of the evaluated expression be processed.

The 2 code snippets below should help clarify this:

console.log("top");
(() => {
  console.log("middle");
})();
console.log("bottom");

// logs "top", "middle", "bottom"

console.log("top");
(async () => {
  console.log(await "middle");
})();
console.log("bottom");

// logs "top", "bottom", "middle"

Great care needs to be taken, not to be caught unawares by this behaviour. The following test passes even though the assertions are clearly wrong:

import { EventEmitter } from "events";
import { describe, expect, test } from "vitest";

import { makeEventListenerGenerator } from "./makeEventListenerGenerator";

describe("makeEventListenerGenerator", () => {
  test("iterates over emitted events", () => {
    const abortController = new AbortController();
    const eventEmitter = new EventEmitter();
    const eventName = "ratesUpdated";
    const eventPayload = [{ from: "USD", to: "EUR", rate: 0.94 }];

    const eventIterator = makeEventListenerGenerator({
      abortController,
      eventEmitter,
      eventName,
    })();

    eventEmitter.emit(eventName, eventPayload);

    (async () => {
      expect(await eventIterator.next()).toHaveProperty("value", "false positive");
      expect(false).toBe(true);
    })();
  });
});

Pushing to Multiple Git Repos

Emmanuel Odongo — Thu, 30 Mar 2023 21:36:27 +0000

The problem

Assume that you had a repo that was hosted on github, and you decided that for some reason you would like to have a copy of your repo on gitlab as well. Perhaps the obvious solution would be to add another remote to your git config.

git remote add gitlab git@gitlab.com/username/my-repo.git

Pushing to both repos would then be achieved as follows:

git push origin main
git push gitlab main

Remembering to run both these commands every time you wanted to push your changes seems like a tall ask. The good new is that you can get the desired behaviour with just a single command that's likely already part of your muscle-memory:

git push origin main

The solution

A good place to start implementing our solution to this problem would be to check for existing remotes.

git remote -v

The above command lists out our fetch and push remotes, which may look something like this:

origin  git@github.com:username/my-repo.git (fetch)
origin  git@github.com:username/my-repo.git (push)

For the superstitious amongst us, you can optionally clear and re-add the origin remote.

git remote remove origin
git remote add origin git@github.com:username/my-repo.git

We now have one push and one pull URL. The solution to our problem lies in setting a second push URL as shown:

git remote set-url --add --push origin git@gitlab.com:username/my-repo.git

To wrap up, we then set the upstream branch of our choosing (main in this case).

git fetch origin main
git branch --set-upstream-to origin/main

From now on, whenever we run git push origin main, git will push our changes to both remote repositories (github and gitlab). Fetching or pulling changes from origin will always refer to just the one repo (github).

Bonus points

As a final touch, we can give both of our repo hosts a unique name in case we ever need to explicitly push or fetch from a particular one.

git remote add github git@github.com:username/my-repo.git
git remote add gitlab git@gitlab.com:username/my-repo.git

Once this is done, listing our remotes with git remote -v gives the following output:

github  git@github.com:username/my-repo.git (fetch)
github  git@github.com:username/my-repo.git (push)
gitlab  git@gitlab.com:username/my-repo.git (fetch)
gitlab  git@gitlab.com:username/my-repo.git (push)
origin  git@github.com:username/my-repo.git (fetch)
origin  git@github.com:username/my-repo.git (push)
origin  git@gitlab.com:username/my-repo.git (push)

Using Key Pairs with JWTs

Emmanuel Odongo — Sun, 26 Mar 2023 20:20:58 +0000

One common way of handling authentication and authorisation in web-based systems is to have a client send their login credentials to the backend, which generates and returns a signed JWT linked to an identity. The client can then access or modify protected resources by attaching the JWT to the request. Before handling the request, the backend verifies the JWT's authenticity.

Signing and Verifying JWTs

JWTs can be signed and verified using a secret. In this case, the same secret is used for signing and verifying. This is a reasonable approach in a monolithic architecture, since only one program has access to the secret.

// GENERATING A JWT USING A SECRET
import { randomUUID } from "crypto";
import * as jwt from "jsonwebtoken";

const SECRET = "123";

const user = {
  id: randomUUID(),
};

const claimSet = {
  aud: "Audience",
  iss: "Issuer",
  jti: randomUUID(),
  sub: user.id,
};

const token = jwt.sign(
  claimSet,
  SECRET,
  {
    algorithm: "HS256",
    expiresIn: "20 minutes",
  }
);

console.log(token); // => eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJBdWRpZW5jZSIsImlzcyI6Iklzc3VlciIsImp0aSI6ImY1NGEzOGVmLTQ4NzctNGJmYy05N2RmLWFkYzFiNjQxNzU5YiIsInN1YiI6IjRlNzQ5ZTAwLTE1NWItNGNlNi1iYWQyLWExOTE5MWM0MmQ2NyIsImlhdCI6MTY3OTc3OTUwOSwiZXhwIjoxNjc5NzgwNzA5fQ.X94g8OkecnaOYLMuVFmy_hcjJ7nvBMhDEvrUpTvvxQE


// VERIFYING A JWT USING A SECRET
import { verify } from "jsonwebtoken";

const SECRET = "123";

verify(token, SECRET);

An alternative way of signing and verifying JWTs is by using key pairs. This involves signing the JWT using a private key and subsequently verifying it using the corresponding public key.

In a service-oriented architecture, the borders between services are generally drawn in a way that separates concerns. That separation should go hand in hand with the principle of least privilege. From the point of view of a service, it should have the least permissions needed for it to perform its duties.

More concretely, only the service responsible for generating JWTs should have access to the private key. This means that other services are unable to generate valid JWTs; all they can do is use the public key to verify a JWT they have received.

// GENERATING A JWT USING A PRIVATE KEY
import { randomUUID } from "crypto";
import { readFileSync } from "fs";
import { sign } from "jsonwebtoken";

const PRIVATE_KEY = readFileSync("./privateKey.pem");

const user = {
  id: randomUUID(),
};

const claimSet = {
  aud: "Audience",
  iss: "Issuer",
  jti: randomUUID(),
  sub: user.id,
};

const token = sign(
  claimSet,
  PRIVATE_KEY,
  {
    algorithm: "ES512",
    expiresIn: "20 minutes",
  }
);

console.log(token); // => eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJBdWRpZW5jZSIsImlzcyI6Iklzc3VlciIsImp0aSI6ImE0NzVhYTU5LTIwMGQtNDlkOS1iODVmLTJkZmExM2Q3NTMyMSIsInN1YiI6ImE2YWFkNWY0LTE3NjctNDUwYy04MWNjLTIyMmI3OWI1NzNiYSIsImlhdCI6MTY3OTc4MDI3NiwiZXhwIjoxNjc5NzgxNDc2fQ.AIuJlLZCvpSWLh_ez6pBVX4lcrVbOiUc2NuwCNiw5ms4ELAZRvQFT5-UlKC-PBWXWzWpHh7eO-WWmfOgRnObk_vpAYAo5Wu8Wu-YaL2lBLvaQp2oG5YnXJ9S1kCKGF9i0UloUeYCK6-bdhRvh-rrOqpCOPepWEiQDiWgEzAdPOl75pY4


// VERIFYING A JWT USING A PUBLIC KEY
import { readFileSync } from "fs";
import { verify } from "jsonwebtoken";

const PUBLIC_KEY = readFileSync("./publicKey.pem");

verify(token, PUBLIC_KEY);

Generating Key Pairs

A key pair can be generated from the terminal using openssl.

Before we start generating a key pair, we need to know which curve openssl should use. The ES512 algorithm used in the previous code snippet corresponds to the secp512r1 curve^[1].

We can generate the private key by running the following command:

openssl ecparam -name secp521r1 -genkey -out privateKey.pem

The private key is then used to generate the public key^[2] using the command below:

openssl ec -in privateKey.pem -pubout -out publicKey.pem

Footnotes

[1]: If you wanted to use a different algorithm, say ES256, but provided the key generated above, jsonwebtoken would throw a helpful error message specifying which curve it expects.

import { randomUUID } from "crypto";
import { readFileSync } from "fs";
import { sign } from "jsonwebtoken";

const PRIVATE_KEY = readFileSync("./privateKey.pem");

const user = {
  id: randomUUID(),
};

const claimSet = {
  aud: "Audience",
  iss: "Issuer",
  jti: randomUUID(),
  sub: user.id,
};

const token = sign(
  claimSet,
  PRIVATE_KEY,
  {
    algorithm: "ES256",
    expiresIn: "20 minutes",
  }
); // => throws `"alg" parameter "ES256" requires curve "prime256v1".`

Generating a new key pair with the expected curve (prime256v1 instead of secp521r1) should resolve the error.

[2]: The node crypto module can generate a public key based off of the private key. So if the service issuing tokens needed to verify them too, we would only need to configure the private key:

import { createPublicKey } from "crypto";
import { readFileSync } from "fs";
import { verify } from "jsonwebtoken";

const PRIVATE_KEY = readFileSync("./privateKey.pem");

const PUBLIC_KEY = createPublicKey(PRIVATE_KEY)
  .export({ format: "pem", type: "spki" });

verify(token, PUBLIC_KEY);