DEV Community: Sanjar Afaq

Keyboard input in Node.js

Sanjar Afaq — Sat, 06 Jan 2024 08:19:52 +0000

This article talks about working with keyboard input in Node.js. This code will run in the terminal.

We'll be discussing the default way in Node to do this. Practically, people prefer using a library like inquirer to create interactive CLIs.

The readline module can be used for reading input into programs.
A promise version readline/promises is it's marked experimental. Works by default in v20.

The APIs of the module allow for very granular control if needed - like position of cursor, row/column position, reading a line vs per character reads.

I'll demonstrate two commonly needed interfaces:

Read a sentence (until user presses enter). Like C++'s cin.
Read single character

1. Read sentence (until 'Enter' is pressed)

Since we get back something once, I'm using async-await.

Logic (snippet)

const readline = require("readline/promises");

/**
 * Take sentence input, until you press 'Enter'
 * Like C++ cin
 *
 * @param {String} message
 * @returns {String}
 */
const prompt = async (message) => {
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  const answer = await rl.question(message);

  rl.close(); // stop listening
  return answer;
};

A simple program

// copy code from above here
async function simpleSum() {
  const a = await prompt("Enter first number: ");
  const b = await prompt("Enter second number: ");
  console.log("Sum is", Number(a) + Number(b));
}
simpleSum();

A more interesting program

async function getGitHubName() {
  const username = await prompt("GitHub username: ");
  const resp = await fetch(`https://api.github.com/users/${username}`);
// yes, `fetch` is available by default in Node v20
  if (!resp.ok) {
    console.log(
      "Error occurred",
      ",",
      "Code:",
      resp.status,
      ",",
      "Message:" + (await resp.text())
    );
  } else {
    const data = await resp.json();
    const name = data.name;
    console.log("User found. Name:", name);
  }
}

getGitHubName();

2. Run code for single character

More like run some code on keypress. This continuously listens for keypresses.
Since each keypress runs some code, I'm using callback.

Example: this is what Metro (React native dev process) runs like - when 'r' is pressed it reloads.

Logic (snippet)

const readline = require("readline/promises");
/**
 * Continues listening for keypresses, and run code for each keypress
 */
const listenKeyPresses = (callback = (key, data) => console.log({ key, data })) => {

  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  rl.input.on("keypress", callback);
  return rl;
};

listenKeyPresses.example = () => {
  listenKeyPresses((key, data) => {
    const isLetter =
      key.toLowerCase().charCodeAt() >= "a".charCodeAt() &&
      key.toLowerCase().charCodeAt() <= "z".charCodeAt();

    console.log(`\b${key} is a ${isLetter ? "letter" : "Non-letter"}`);
    console.log(data);
  });
};
listenKeyPresses.example() // run the example

This is very handy when testing, trying something new. Instead of creating a small frontend using HTML, CSS or JS, or setting up Postman. Just use this, setup key press and corresponding code run, and test quickly. Both input + output in the same terminal!

Detect file run as main or module

Sanjar Afaq — Mon, 01 Jan 2024 07:52:03 +0000

There's a simple way in Python to know if current file is being run directly or as an import, using the __main__ variable. Any such way in Node.js?

Yes, there is.

CommonJS

if (require.main === module) {
  console.log('This file is being run directly.');
} else {
  console.log('This file is being imported as a module.');
}

ESM

// mainModule.mjs

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log('This ESM module is being run directly.');
} else {
  console.log('This ESM module is being imported as a module.');
}

What's the use of this?

I write Node.js scripts for personal use, and it's good to have usage examples that console.log something. If a script is being run directly, it'll print the usage examples, otherwise it'll stay silent.

Node version enforcement and convenience

Sanjar Afaq — Tue, 26 Dec 2023 19:48:29 +0000

Node versions

Created Wed Dec 27, 2023 at 12:38 AM

Many projects have bad documentation or ways around what Node.js version they use, and how to load it.
But there's a simple explicit way to do both enforcement and convenience at once.

It involves making changes to 4 files - .nvmrc, .npmrc, package.json and terminal hook (.zshrc for example).

Enforcement - the app will raise errors for wrong versions during npm install. .npmrc and package.json do this, since any app is started by an npm script. This raises an error during npm install itself for wrong Node version, which is a good thing. See StackOverflow

/.npmrc

   engine-strict=true

/package.json. See engines property on npmjs

    "engines": {
        "node": "<=16"
    },
    // supports range like so `>=0.10.3 <15`
    // optionally, even `npm` version can be specified

Convenience
1. Manual load - Add .nvmrc with the node version (by running node -v > .nvmrc), then manually do nvm use inside the directory. Don't have to remember "x" in nvm use x, good.
/.nvmrc and (for safety/other environments) /.node-version file
```
  v16.20.2
```
1. Auto load - remembering and doing nvm use for each new terminal is lame. So, add .nvmrc to the project as usual, and also add a function that loads the node version. Fully automatic. See Deeper shell integration
~/.zshrc
```
  # for ZSH: Add to .zshrc https://github.com/nvm-sh/nvm?tab=readme-ov-file#zsh
  # for bash: Add to .bashrc https://github.com/nvm-sh/nvm?tab=readme-ov-file#bash
  # optionally can do
```

Notes and gotchas:

Set global default: nvm has a bad UI. To set global default, one has to run nvm use alias default _version_
Use specific node version (temporarily - for terminal session) - nvm use _version_
On mac, make sure to run brew uninstall node. Homebrew node messes up with nvm global default - it just takes over the default.

PS

Time to take asdf seriously?

Hot reload setup for Obsidian plugin development

Sanjar Afaq — Tue, 14 Nov 2023 15:27:07 +0000

Obsidian is a markdown based note taking tool. It has a rich plugin ecosystem. Most plugins are free and open-source.

This article talks about general development setup of an Obsidian plugin. It's not focused on creating a new plugin from scratch, but for making changes to existing plugins - so you can fix bugs directly, make your life easier and help everyone at the same time.

Vault setup

By vault I mean an Obsidian notebook (folder).

Step 1: Create a new Obsidian vault
Step 2: Install and enable this hot-reload plugin as usual in this new vault.

Note: you can install/develop a plugin in an existing vault too, assuming you have a safe backup of the vault.

Setting up the plugin code

Example plugin - https://github.com/JYC333/obsidian-attachment-name-formatting
Step 1: Clone the plugin repo inside .obsidian/plugins
Step 2: Open the cloned folder in an editor.
Step 3: Change the "name", "id" and "description" in the manifest to some recognizable name. I usually do this in a new git branch.
Step 4: Install dependencies
Step 5: Start the plugin development mode, and make sure that the built files are directly kept in the .obsidian/plugins/cloned-plugin-name folder. You need to look make sure atleast main.js, manifest.json exist.

Install the plugin (from folder)

In Obsidian (inside the vault), go to Settings > Community Plugins > Scroll down until you reach 'Installed Plugins' section.

Press the refresh icon (beside folder) of the section. The "name" (as changed in manifest.json) should appear here. Enable the plugin. If it doesn't appear here, click the folder icon and see if the cloned folder is present or not.

Developing/observing

Open the Obsidian developer tools panel. Top app panel > View > Toggle Developer tools.
Start editing the app code, and changes will show up in Obsidian. Make sure the development server is running.
Most development modes (React/Vue etc) will automatically re-build the code (produce main.js) on change of app code.

CRA to Vite migration (JS, not TS), for beginners

Sanjar Afaq — Sun, 24 Sep 2023 11:16:58 +0000

GitHub link to article, vite-react-js-template

Situation (ignorable)

You have a React app, created using create-react-app (CRA) project that you wish to migrate to Vite.

Why migrate (ignorable)

CRA is really slow, especially the npm install step.

This usually results in failure during free deployments (like render.com) - due to memory usage and timeouts.

Moving to Vite.js solves this - npm install for projects is very fast, development experience is good and free deployments don't fail.

Preparation

You have your CRA project. Ok.

We need to add Vite config files to this project. For this, create a new (blank) Vite project -

Run npm create vite in the terminal.
In the prompt - add project name ("hello-world"), select framework as React and variant as JavaScript + SWC.

Steps

Rename component .js files to .jsx. Util files don't need to be renamed. Just to be clear: a component file means any file having JSX code. See changes (view GitHub commit diff)
Move index.html and change shell files - move index.html from public to src and change other "shell" files like App.js, index.js, App.css etc. See changes
Add vite.config.js, package.json, package-lock.json - just copy this from the fresh vite project into your CRA project. Replace conflicting files. See changes
Handle environment variables (optional - if your project uses them) - code process.env.PORT becomes import.meta.env.PORT. Make similar changes at other places. See changes Handle environment variables (optional - if your project uses them)
Run the app - npm run dev should work now.
Cleanup - delete extra files if any.
Deployment build folder (optional) - Vite's build folder is named dist (as opposed to CRA's build). Change corresponding backend code if needed. See changes

For more info, see the GitHub trail (commits) starting from https://github.com/exemplar-codes/posts-express-api-app/commit/52a0fe58befad2bda8eacf06383fa6f247fe04ef

EOT

Unawaited async middleware errors in Express v4

Sanjar Afaq — Sun, 27 Aug 2023 19:39:06 +0000

Context

I am working on a plain JS Express.js app. It has middlewares, a few routers.

Situation - strange error

I was getting an error from Mongoose package. The error would crash the app, in spite of having an error sink - which would never get triggered (unexpected).

I am trying to use the .save() mongoose function, an async function, without await. But there's a reason I do this - I wish to send the response before the call succeeds.

Exact problem (comparison):

// error
app.get('/xyz', async (req, res, next) => {

  try {
    // some code here
    product.save(); // this _will_ throw an error (expected, ok)
  } 
  catch (e) {
    next(e);
  }

  console.log('Everything is fine.'); // runs (unexpected)

});

app.use((err, req, res, next) => {})

// error middleware doesn't run (app crashes)

app.get('/xyz', async (req, res, next) => {

  try {
    // some code here
    await product.save(); // this _will_ throw an error (expected, ok)
  } 
  catch (e) {
    next(e);
  }

  console.log('Everything is fine.'); // doesn't run now, as expected

});

app.use((err, req, res, next) => {})

// error middleware runs (app doesn't crash)

Fix (here)

I needed to add await.

Explanation

Since I did not wait for product.save() (which is async - so error is thrown after some time), to express, the error never occurred.

So, it never ran the code inside catch.

And therefore, the error middleware never ran.

But the product.save() does throw an error, and it does bubble up at the runtime level (Node.js process instead of within Express), crashing the app.

Hack - catch error at a lower level (runtime instead of Express)

The following is a native way to catch errors. Add the following as the first middleware:

app.use((req, res, next) => {
  process.on("unhandledRejection", function (reason, p) {
    console.log("hack, Unhandled Rejection at: Promise ", p, " reason: ", reason);
    // application specific logging, throwing an error, or other logic here
  });
})

Source: https://stackoverflow.com/a/30650609/11392807

Here:

Crash is prevented - since this will catch the unhandled error, prevent bubbling to the runtime level, so the crash will be prevented.
The error middleware is still not called.

Why would you I use this? If the app is old, there may be a lot of such "non-awaited" middlewares.

Actual fixes

Any of these:

Use await if possible.
For non awaited parts, use .then and call next(e) from .catch. Example: If the user doesn't care about the result immediately.

Conclusion

If async middlewares (Express.js) contain code that doesn't use await.
Then execution of such code starts happening outside the middleware chain.
This leads to problems like failed triggering of error middlewares, app crashes, and inability to send a response.

Use positives for conditionals - if, ternary true case, boolean variables

Sanjar Afaq — Thu, 24 Aug 2023 11:31:36 +0000

Prefer positive conditionals/checks

In conditionals - like if, ternary, boolean variables, prefer storing/checking positive conditions over equivalent negative ones.

what is positive depends on the goal of the feature.

// example 1
const hasNoEnergy = !getStatus(); // problem: negative stored in boolean

if (hasNoEnergy) {
  injectFuel();
} else {
  changeOrbit();
}

// example 2
const hasEnergy = getStatus();

if (!hasEnergy) { // problem: negative checked in if
  injectFuel();
} else {
  changeOrbit();
}

// these are complicated, need to be read deliberately

// simple, better
const hasEnergy = getStatus();

if (hasEnergy) {
  changeOrbit();
} else {
  injectFuel();
}

How Express.js detects if a middleware is an error or request one

Sanjar Afaq — Tue, 22 Aug 2023 22:19:47 +0000

Situation

Suppose we have this code:

const express = require('express');
const app = express();

// usual code - app.METHOD, app.use, routers etc

// below: wont' be treated as error middleware
app.use((req, res, next) => {}); 


// below: will be treated as an error middleware
app.use((err, req, res, next) => {});

Question

How does Express.js know which one is an error middleware, given both app.use take a callback (function).

Explanation

Well it's quite simple. JS allows us to calculate the number of params in a function (MDN). Syntax example:

const mySumFunction = (a, b) => { return a + b; };

console.log(mySumFunction.length); // 2

This was surprising for me since I thought callback (or functions in general) are black-boxes, we could just call them, and that was it. But no.

Evidence

Coming back to the topic, Express internally does this .length to decide if the middleware is a "request" or an "error" one.

See (Express.js repo at GitHub): https://github.com/expressjs/express/blob/3531987844e533742f1159b0c3f1e07fad2e4597/lib/router/layer.js#L89

Conditional middlewares in Express.js

Sanjar Afaq — Thu, 17 Aug 2023 21:32:21 +0000

Github link to article

Situation

There is a backend app with many middlewares, including two named X and Y. The requirement is that only X runs or Y runs based on a condition.

Solution

How do we handle this?
There are a few cases here, with minor variations.

Keep it linear - we add the middlewares one after the other. In the body of these middlewares we return early if the condition is not matching.

If both are custom middlewares

function mwX(req, res, next) {
    if(condition)
        return next();
    // do X mw logic
}

function mwY(req, res, next) {
    if(condition)
        return next();
    // do Y mw logic
}

app.use([mwX, myY]);

- If any of them is library generated. Or if you don't want to litter conditional code into middlewares.

```js
// of course, this can be done using array too
function xConditional(req, res, next) {
    if(condition)
        return X;
    return ((req, res, next) => { return next(); });
}

function yConditional(req, res, next) {
    if(condition)
        return Y;
    return ((req, res, next) => { return next(); });
}


app.use([xConditional, yConditional])
```

Ternary - works only if the condition is static (i.e. not dependent on req or res), we can use the ternary operator. app.use(condition ? X : Y);. X and Y could be custom or library generated doesn't matter. Code:
```
app.use(staticCondition ? X : Y);
```

Note:

Prefer differentiation based on route, method and router before doing this. They are in-built control mechanisms which are intuitive and easy to use.
Can be generalized to more than 2 middlewares too.

Direct wrapper (special case) - if the middlewares are exclusively conditional. Code:

function wrapperMiddleware() {
  if(condition) // based on req or res logic or otherwise
    return X;
  else
    return Y;
}

app.use(wrapperMiddleware());

Chained middlewares in Express.js

Sanjar Afaq — Thu, 17 Aug 2023 21:22:57 +0000

GitHub link to the article

Situation

Suppose I need to run multiple middlewares for the exact same method, route, router or maybe with .use() itself.

Let's try with the .use() one, should be simple, I just write them one below the other.

But what if the number of middlewares are not known, or I'm lazy. Can I use a for loop?
No. Since all .use(callback) are callbacks, we never run them until we get a request - so it's not possible with a for loop.

Solution - how

Since looping is not possible.
Express supports comma separated syntax or an array of middlewares passed to .use (and other functions).
The syntax is very flexible - everything is linearized.

Solution - syntax

Here's the syntax:

app.use(myMiddleware); // already familiar


// 1. Comma-separated works fine too (all will be run in order)
app.use(myMiddleware1, myMiddleware2, myMiddleware3);

// 2. Array also works 
app.use([myMiddleware1, myMiddleware2, myMiddleware3]);

// 3. can use both - it still works
app.use(myMiddleware1, [myMiddleware2, myMiddleware3], myMiddleware4);

// 4. btw, nested arrays work too, Express just flattens them
app.use(myMiddleware1, [myMiddleware2, [myMiddleware3]], myMiddleware4);

Works with .router(), method('', ) too.

Doubt (ignorable)

Q: Could it work with this notation of comma, or array if we can write a wrapper with a loop?
A: It would work, but not properly or for call cases:

For middleware that don't respond.
1. It would seem to work. Since each middleware in the loop would call next(), but the first iteration would end the wrapper (which is a middleware itself). So the others in the loop would run (but not in the queue - they will run independently). Doesn't work
If they respond - won't work, see why:
- If the first one is a non-responding middleware, point 1 happens (out of order exec) - not acceptable.
- If we ignore out of order/independent execution. Let's see what happens, the responding middleware runs... fine, no. When the next middleware (be it responding or non-responding), we would get headers set error - since it would reach the 404.

I know, we can update res.locals and catch that in 404, but the out of exec order is till not solved.

Conclusion

Construct of arrays/comma separated middleware is a REQUIRED feature.

It makes it easy to develop Express apps.