DEV Community: Liran Tal

Getting started with Neural Networks in JavaScript

Liran Tal — Fri, 07 Mar 2025 00:00:00 +0000

Much of the tooling and educational content around machine learning is focused on Python and its ecosystem. Rightly so, to an extent, given that data scientists and machine learning engineers have been immersed with Python related tooling for a long time. However, as someone coming from the JavaScript ecosystem, I wanted to get started with machine learning using JavaScript.

About 12 years ago, while at HPE Software, I underwent through a machine learning course which was super fun. We learned the basics and coded some exercises like KNN algorithm from scratch. It was a nice experience and at the time it wasn’t surprising that we got exposed to this as engineers at HPE Software. Back then, the buzzword was “Big Data” and HPE had dipped its toes into this space with products like Vertica and others.

Fast forward to 2025 (well, technically, November 2022) and artificial intelligence, machine learning, and neural networks are everywhere.

Can we bring this magic into JavaScript? A little library called Brain.js makes it possible.

In this post, I’ll just put the fundamentals that helped me get started so you can grab the basics too and get started playing with this.

Key Concepts in Neural Networks

Hardly is this going to be an expert or good enough coverage of key concepts in machine learning, so I highly recommend you perhaps start here but then go through other resources to get a deeper understanding.

In a nutshell, the following are a few key concepts you’ll need to know in particular because they are part of the building blocks of the Brain.js library so you will be interacting with these concepts directly.

Neurons and Hidden Layers

A neuron is one of those fundamental building blocks of neural networks. The simplest and most reductive way to think of a neuron is that it represents a value. That’s really oversimplifying it and technically inaccurate too. To expand a bit more about it, think - how would that neuron get a value? If it gets a value, would it do something with it? This helps unveil the very basic idea of a neuron. A Neuron is a sort of computational unit that takes in some input, applies a mathematical function to it, which produces an output.

If a neuron is the very basic building block, a node if so to say, then a layer is a collection of neurons. Many such layers then make up for what is the neural network. In a classic representation of a neural network you can divide it to three parts - the input layer, the hidden layers, and the output layer. Visually it would look something like this:

Input Layer: 0 0 0.9 0 0.4 0 0.2 0 0
    | \ | | | | | | /
Hidden Layer: 0.1 0.3 0.5 0.2 0.1
    | / | | | | | \
Output Layer 0.1 0.2 0.3 0.4 0.5

The Hidden Layer part is where a lot of the “magic” happens. In simple terms you can think of it in a way that the more hidden layers you have, the more complex the neural network can be. The more “intelligence” or “room for understanding and learning” you can give to the neural network. Sometimes, if you try to train and test a neural network with not enough neurons or hidden layers, the network will simply be unable to learn. It’s like trying to program a calculator that adds numbers from 1 to 1 million but you only provide it with an integer size of 8 bits. It technically can’t do it.

Forward and Backward Propagation

Ok so, neurons hold values, layers hold neurons and the role of the neural network is to learn. How does it learn? This is where forward and backward propagation come in.

Think of a person at a bowling night. They throw the ball and it needs to hit the pins. Perhaps the first throw is going to be off the mark and hit the side rails. What does the person do? They adjust their aim accordingly, perhaps pivot their hand a bit to the right, and throw again. This time they hit, but only 3 pins. They adjust again, maybe this time they also adjust the force of the throw. Did it help or did it make it worse?

    0
    |
   / \

    O
   / \
  / \
 / \
/ \
| |
| |
| |
| |
| |
| | | | |
| | | | |
| | | | |
| | | | |

This is the learning process that we humans go through. A neural network follows this process through a method that it calls forward propagation and backward propagation. The forward propagation is the process of taking the input, passing it through the neural net, firing the neurons with some input, and getting an output. What was the output? did it match the expected output? If not, the neural network needs to adjust its weights. This is where the backward propagation comes in - it is the process of taking the output, comparing it to the expected output, and adjusting the weights of the neurons in the network.

Training a Neural Network

The learning process of forward and backward propagation that we described above is repeated many times until the neural network is able to produce the expected output. This process is called training a neural network. The neural network is trained on a dataset that has input and output pairs. The neural network is trained to produce the output given the input. The more the neural network is trained, the better it gets at producing the expected output.

Getting started with Brain.js for Neural Networks in JavaScript

If you’re on a Linux machine you’ll likely need to install the following dependencies because the Brain.js library uses native bindings:

sudo apt-get update
sudo apt-get install -y libgl1-mesa-dev
sudo apt-get install -y libxi-dev libx11-dev libxext-dev

For more detailed install instructions you can refer to the Brain.js GitHub repository.

Next, make sure you have allowed npm installs to run scripts. This is because the Brain.js library uses a script to compile the native bindings. You can do this by running:

npm config set ignore-scripts false

Then you can continue with installing the Brain.js library:

npm install brain.js

Creating a Neural Network with Brain.js

Many of the examples for Brain.js and neural networks show a simple example to predict the output of a XOR operation. Let’s try something different.

Let’s train a neural network to learn to go beyond XOR with a simple, yet valuable, example using Brain.js: Predicting if a number is even or odd.

What would be your input for the neural network? We’ll represent numbers through their binary representation. So, for example, the number 3 would be represented as 0011 and the number 4 would be represented as 0100. So our input could be represented as follows:

const trainingData = [
  { input: [0, 0, 0, 0], output: [1] }, // 0 (even)
  { input: [0, 0, 0, 1], output: [0] }, // 1 (odd)
  { input: [0, 0, 1, 0], output: [1] }, // 2 (even)
  { input: [0, 0, 1, 1], output: [0] }, // 3 (odd)
  { input: [0, 1, 0, 0], output: [1] }, // 4 (even)
  { input: [0, 1, 0, 1], output: [0] }, // 5 (odd)
  { input: [0, 1, 1, 0], output: [1] }, // 6 (even)
  { input: [0, 1, 1, 1], output: [0] }, // 7 (odd)
  { input: [1, 0, 0, 0], output: [1] }, // 8 (even)
  { input: [1, 0, 0, 1], output: [0] }, // 9 (odd)
  { input: [1, 0, 1, 0], output: [1] }, // 10 (even)
  { input: [1, 0, 1, 1], output: [0] }, // 11 (odd)
  { input: [1, 1, 0, 0], output: [1] }, // 12 (even)
  { input: [1, 1, 0, 1], output: [0] }, // 13 (odd)
  { input: [1, 1, 1, 0], output: [1] }, // 14 (even)
  { input: [1, 1, 1, 1], output: [0] }, // 15 (odd)
];

As you can see we’re showing the neural net “what good looks like”, so we provide it with the input of a binary representation of a number, and what I expect it to provide back as an output: 1 for even, 0 for odd.

The neural net, however, isn’t going to reply back with 1 or 0 though, it’s going to reply back with a number between 0 and 1 that represents the probability of the number being even or odd. We can then round this number to get the final result, or more accurately, we’d check if the threshold is closer to 0 or 1, to base our decision on.

How would we test the neural net? We will provide it with some number, say 5 (well, its binary representation 0101), and see what the neural net predicts. However, you might wonder, what’s the point of this? what’s the magic? we already told the neural network that 5 is odd. Yes?

So, I suggest maybe we remove some of the training data and see how the neural net performs. Consider the following:


// 1. Prepare training data
const trainingData = [
  { input: [0, 0, 0, 0], output: [1] }, // 0 (even)
  // { input: [0, 0, 0, 1], output: [0] }, // 1 (odd)
  // { input: [0, 0, 1, 0], output: [1] }, // 2 (even)
  { input: [0, 0, 1, 1], output: [0] }, // 3 (odd)
  { input: [0, 1, 0, 0], output: [1] }, // 4 (even)
  // { input: [0, 1, 0, 1], output: [0] }, // 5 (odd)
  // { input: [0, 1, 1, 0], output: [1] }, // 6 (even)
  // { input: [0, 1, 1, 1], output: [0] }, // 7 (odd)
  // { input: [1, 0, 0, 0], output: [1] }, // 8 (even)
  // { input: [1, 0, 0, 1], output: [0] }, // 9 (odd)
  { input: [1, 0, 1, 0], output: [1] }, // 10 (even)
  { input: [1, 0, 1, 1], output: [0] }, // 11 (odd)
  // { input: [1, 1, 0, 0], output: [1] }, // 12 (even)
// { input: [1, 1, 0, 1], output: [0] }, // 13 (odd)
// { input: [1, 1, 1, 0], output: [1] }, // 14 (even)
// { input: [1, 1, 1, 1], output: [0] }, // 15 (odd)
];

Next, let’s not forget to import the Brain.js dependency:

const brain = require('brain.js');

Ok, so now let’s go on. Next step is to create the neural network:

// 2. Create the neural network
const net = new brain.NeuralNetwork({
  // 8 neurons in the layer
  hiddenLayers: [8],
})

Note: we’ll get back to the hidden layer configuration here but I specifically didn’t leave the default. As an exercise, you should play with the hidden layer and try to expand on providing more neurons and more layers. How does the neural network perform when you set hidden layers to [8,8,8,8,8,8,8,8,8,8,8] for example?

The above neural network is initialized using defaults, including an activation function and other parameters. Now we’re ready to train:

// 3. Train the network
net.train(trainingData);

This is a CPU-bound synchronous process that happens in the Node.js main thread. You can also read the training data result from the returned object to get some statistics about the training process.

Finally, let’s test the network:

// 4. Test the network
function testNumber(number) {
  const binary = number.toString(2).padStart(4, '0').split('').map(Number);
  const result = net.run(binary);
  const isEven = result[0] > 0.5;
  const correctNess = isEven === (number % 2 === 0);
  console.log(`${correctNess ? '✅' : '❌'} Number: ${number} (Binary: ${binary}) - Even: ${isEven} (Accuracy: ${result[0]})`);
}

testNumber(1);
testNumber(2);
testNumber(3);
testNumber(5);
testNumber(15);
testNumber(4);
testNumber(7);
testNumber(10);
testNumber(13);

In the testNumber() function we convert the number to the binary representation of it as is aligned with the same format of the training data. Then run the neural network, and then check if the result is above 0.5 to determine if the number is even or odd. We then compare this to the actual number to see if the neural network was correct.

Just like that, you’ve created a neural network that can predict if a number is even or odd. You can expand on this by providing more training data, or by changing the hidden layers and neurons in the neural network.

If you want to continue from this point on with a full reproducible example, see my git repository at https://github.com/lirantal/neural-network-predicts-even-or-odd.

How I use GenAI to Speed Up Demo Apps in My DevRel Role

Liran Tal — Sun, 02 Mar 2025 00:00:00 +0000

Often the questions posed to software developers is whether AI will replace them. If you’re a developer advocate or in the field of Developer Relations (DevRel) then you’ve most probably thought about this too - will AI replace you? make you expandable?

The key, really, is to fundamentally leverage GenAI to speed up your work and make you more productive. This is true whether you’re an engineer or a developer advocate. As the saying goes:

“AI will not replace you. Developers who use AI will replace developers who don’t.”

Indeed, this applies too for DevRel. Let me show you a practical example from my day-to-day.

Using GenAI to Make Developer Advocates More Productive

I’m a Developer Advocate at a security company (snyk.io) and I often need to create demo applications to showcase security vulnerabilities. These are deliberately vulnerable apps that we build in Java, JavaScript, Golang and other languages and frameworks for education purposes.

As you can imagine, a lot of work is spent on building these demo apps. Here’s a nutshell of what goes into it:

Prep and research: what sort of vulnerabilities? what’s the use-case?
Storyline: what is a realistic example that developers can relate to?
Proof-of-concept: building the core of the exploitation and making sure it works so that it creates the basis for the demo app
Building the demo app: this is where the actual app is built and where the exploitation is showcased
Make it fun: hopefully, make the app look actually fun and engaging, using proper frontend and not some boring UI
Make it reproducible: make sure that the app can be easily deployed and run by anyone in my DevRel team, SEs, or by the community
Document it: properly document the how-to for the exploits
Spread the word: write a blog post, create a video, and make sure that the app is well-documented

What if I told you that out of that list, which is a lot but there’s even more that goes into it, I really only need to focus on a couple of to-do bullets and the rest can be automated by LLMs and GenAI?

Follow me for a practical, real-world example where I employed the above strategy to build a demo app that showcases a security vulnerability in image analysis and prompt injection in a Node.js app.

Step 1: Ideate the Use-Case and Fun Demo

This is my starting point. I often either see something in the news, community, social feed or other interactions that sparks an idea. In other times, I just sit down and think what would make a fun use-case to demo. You can use AI to brainstorm this too but I often find it more authentic when the idea comes naturally (or from subconsciously browsing the web and prior engagements).

So, I ideate what would be the use-case and fun demo to showcase a type of security vulnerability (in this case it’s an LLM hacking showcase)

Step 2: Proof-of-Concept

Once I locked down an “idea”, I iterate on the proof-of-concept which is the core of the program and I want to make sure that I’ve got a working example. This is where I spend most of my time because it requires effort in research, prompt injections, and various methods to find the “path” to a successful working demo.

That successful moment when everything just works! The vulnerability is exploited, the prompt injection is successful and I can see the results in the console. Yay! 🙌

Step 3: Building the Demo App

Ok so I’ve got a story, an idea, an actual proof-of-concept of the underlying insecure code and insecure code flow that would lead to a security vulnerability and break the app. Now I need to build the actual demo app.

I bet many developer advocates and engineers spend a ton of time on, for honestly in my opinion, not a good reason. Here are all the stuff you can get stuck with:

Which framework should I use? React, or Vue? Should I try the new Svelte?
I need to dig into the docs and see how to set up a new project
How do I scaffold a new project? should I use a ready-made template on GitHub? use the framework’s own CLI?
Which UI component library to choose from?
How do I get all of the different libraries put together and configured: Shadcn, TypeScript, the metaframework, etc

And much more messy details that go into building an application. But what if I can just settle on the high-level requirements and let GenAI do the rest?

This is where the new generation of application scaffolding and Generative AI comes in! I’m not talking about Cursor or Zed. I’m not talking about prompting IDEs or ChatGPT. I’m talking about a new breed of blended IDEs for low-code development that is fine-tuned at the intersection of AI and software development.

A prime example of that is Bolt which is a Generative AI tool from the folks who built StackBlitz.

I prompt Bolt, asking it to scaffold a new Node.js app with Nuxt.js and it does all the heavy lifting for me. It sets up the project, installs the dependencies, and even gives me a working example that I can then build upon.

I can specify the pages I want, the routing, which components are needed (for example an upload file component), and even the styling. I don’t need to worry about API endpoints, or how to setup the new Nuxt project. It’s all done for me.

Conclusion

I’ve used the demo app that Bolt scaffolded as a template to get started with and just baked on it my proof-of-concept code. You can take a look here if you want to learn more: https://github.com/lirantal/event-ticket-admission-with-ai

What is an LLMs.txt File?

Liran Tal — Fri, 28 Feb 2025 00:00:00 +0000

Large Language Models have made it into mainstream fields of technologies, beyond code generation, beyond documentation and quite significantly into many sorts of human-computer interactions. How do we give these LLMs more true context so that they do not hallucinate? so that these models, whether GPT-4o, Claude 3.7 Sonnet, or any other, can be more reliable, trustworthy vessel of information? Meet the llms.txt file format.

What is the LLMs.txt file?

The llms.txt file is a newly proposed standard that is intended to provide large language models with relevant context and metadata in the form of a simple text file (it may be formatted as plain-text markdown ascii).

What are the use-cases for LLMs.txt file?

Originally, the llms.txt file was intended to be used in the context of allowing AI-based agents processes to more easily scrape data off of websites so that these self-learning and autonomous agents do not need to deal with HTML parsing, loading JavaScript and any other web scraping struggles. Instead, websites can provide a simple llms.txt file that contains the relevant context for each page, and LLMs can easily and quickly digest them without requiring further compute for parsing.

LLMs.txt for Websites

Due to the directory structure of websites, you can generate and plant llms.txt files in the root directory of your website but also they can be placed in documentation subdomain to allow GenAI code assistants to better embed and create the context for code snippets and suggested code examples. Some examples of these websites include:

The Turbo build tool: https://turbo.build/llms.txt
Anthropic’s documentation: https://docs.anthropic.com/llms.txt
Dotenv’s all around favorite Node.js environment variable management tool: https://dotenvx.com/llms.txt
CrewAI agentic framework docs: https://docs.crewai.com/llms.txt

We’re already seeing emerging llms context related tools such as llmstxt python project that compresses files into a single, LLM-friendly text file designed to get codebases ready for analysis by Large Language Models.

What’s next for LLMs.txt?

Given that contextual information is at the core of LLM integrations and agentic frameworks, are we going to see llms.txt in different shapes and forms, making it to more than just websites?

I personally think so. Some ideas that come to mind are to put llms.txt files in the following hubs as a starting point:

GitHub repositories
DockerHub images
The npm registry

LLMs.txt Directory

With the newly proposed llms.txt file standard, new directories have been emerging that index the llmstxt file format and allow to search and discover websites that have embraced this new file format. Some of which are:

LLMs.txt Hub: https://llmstxthub.com/
LLMStxt Site: https://llmstxt.site/

Next up

LLMs are more ubiquitous than ever, but if you don’t want to risk privacy or spend, learn how to run a local LLM for inference with an offline-first approach.

The CJS module system, globals and other hardships with maintainable code in Node.js

Liran Tal — Wed, 19 Feb 2025 00:00:00 +0000

In the next set of examples we will review some common scenarios of tight coupling in Node.js applications and the challenges they present.

These are pretty common code pattern I’ve seen in many Node.js “seed” applications - source code repositories that are meant to provide you with a starting point for your application. I’ve also seen many Node.js tutorials and blog posts that follow this code pattern for database access in their examples.

In this article, we will review the following prime examples of tight coupling in a Node.js codebase:

Using global variables: Global variables are a common source of tight coupling in Node.js. When a variable is declared as global, it is accessible to all parts of the application. This can make it difficult to track down the source of a problem and can make it difficult to test the application.
Hardcoding dependencies: Hardcoding dependencies means specifying the dependencies of a class or function explicitly. This can make it difficult to change the dependencies of the class or function without modifying the code.
Using singleton patterns: Singleton patterns create a single instance of a class that is accessible to all parts of the application. This can make it difficult to test the class and can make it difficult to change the implementation of the class without affecting other parts of the application.

Scenario 1: hard-wired database dependency in a repository

In the following example, we have a UserRepo class that is responsible for fetching users from a database. The UserRepo class is tightly coupled to the database connection:

// FILE: user.repository.js
// ========================
const db = require('./db')

export class UserRepo {
  constructor () {};

  getUsers (): Promise<User[]> {
    return db.query('SELECT * FROM users');
  };
};

A classic follow-up usage of this repository is when a service or a controller requires the user.repository.js file and uses it to fetch users from the database. Here’s an example of a Fastify route definition that does so:

// FILE: user.route.js
// ========================
import { UserRepo } from './user.repository.js'

fastify.get('/users', async (request, reply) => {
  const userRepo = new UserRepo();
  const users = await userRepo.getUsers();
  return reply.status(200).json({ users });
});

What is the problem with the above code?

Requiring, or using, the UserRepo class impoliticly requires the database connection that is hard-wired in the UserRepo class. This means that if we want to change the database part of the repository, we will have to change the UserRepo class as well, or find ways around it.

Next, you are faced with the following test code example for the Fastify route:

// FILE: test.js
// ========================
const test = require("node:test");
const assert = require("node:assert");

test("users route returns a list of users ", async (t) => {

    const fastify = require("fastify")();
    const route = require("./user.route.js");

    fastify.register(route);

    const response = await fastify.inject({
        method: "GET",
        url: "/users",
    });

    assert.strictEqual(response.statusCode, 200);
});

For this test to run successfully it needs a database due to the hard-wired database dependency in the UserRepo class. However, it’s common to avoid requiring the real the database connection as-is bootstrapped with the application for different reasons:

the database connection is configured to use your local testing database, where-as you want to configure another database for executing tests.
you want to skip the database connection altogether, and use a mock database connection instead.

What is the common solution to work around the hard-wired database dependency?

Using packages like sinon, proxyquire or rewire to change the db dependency in Node.js internal module cache that exchanges the db.js file with a mock of the database.
Spinning up a real database for the test using Docker containers or other means.

Scenario 2: Global modules in Node.js

The CommonJS module system in Node.js acts as a singleton. This means that when you require a module, the module is cached and the same instance of the module is returned every time you require it.

This is a common pattern in Node.js applications, where a module is required once and then used throughout the application. For example, a database connection module is required once and then used throughout the application to execute queries against the database.

However, this pattern proves again to be problematic and is a source of tight coupling in Node.js applications. Let’s untangle the problem with an example that revolves around configuration.

The CJS module system is an example of a module system that can lead to tight coupling. In the cjs module system, modules are loaded as global objects. This means that any module that imports another module will have direct access to the exported objects of the imported module. This can make it difficult to test the modules and can make it difficult to change the implementation of the modules without affecting other modules.

In this example Node.js application, we have a route that returns the weather for a city. The route uses a service to make an API call, and the service relies on accessing a configuration module to receive the URL for the API call.

Here’s the Fastify route definition which makes use of the getWeather call:

// FILE: route.js
// ========================
const { getWeather } = require("./service");

async function routes(fastify, options) {
  fastify.get("/", async (request, reply) => {
    const city = request.query.city;
    const weather = await getWeather({ city });
    return reply.send({ weather });
  });
}

module.exports = routes;

The getWeather function is defined in the service.js file which accesses the configuration module to receive the URL for the API call:

// FILE: service.js
// ========================
const config = require("./config");
const weatherApiUrl = config.WEATHER_API_URL;

module.exports.getWeather = async ({ city }) => {
  // in reality this is an API call to a weather service
  // but for the sake of tests we will just print a console log
  // that shows the API call that is being made
  console.log(`making an API call to ${weatherApiUrl} for ${city}`);

  const weatherDatabase = {
    Seattle: "Rainy",
    Singapore: "Sunny",
  };

  const weather = weatherDatabase[city] || "Unknown";

  return weather;
};

Lastly, the configuration module is defined in the config.js file which is a highly common pattern I’ve seen in many Node.js applications - a configuration module that creates a configuration from static json files, environment variables or other means and then export an object that is created once and then used throughout the application.

The config.js file:

// FILE: config.js
// ========================
const config = {
  WEATHER_API_URL: "https://api.example.com/weather",
  PORT: 4000,
};

module.exports = config;

Now, consider the following test code for the Fastify route:

// FILE: test.js
// ========================
const test = require("node:test");
const assert = require("node:assert");

test("the weather API should return the current weather but don't call the real API because its expensive", async (t) => {
  const config = require("./config");
  config.WEATHER_API_URL = "https://api.example.com/FAKE-WEATHER-API";

  const fastify = require("fastify")({ logger: false });
  const route = require("./route");

  fastify.register(route);

  const response = await fastify.inject({
    method: "GET",
    url: "/?city=Seattle",
  });

  // use the assert module to verify the server's response
  assert.strictEqual(response.statusCode, 200);
  assert.strictEqual(response.payload, '{"weather":"Rainy"}');

  fastify.close();
});

In this test, we don’t want to make an actual API call to the weather service, because it’s expensive. Instead, we want to mock the API call and return a fake weather response.

However, if you run the test, you’ll see that it still logs the API call to the real weather service with a log entry in the test suite as follows:

Making an API call to https://api.example.com/weather for Seattle

What is actually happening is not entirely config.js fault.

The service.js file, is a module too, and is also globally cached within the Node.js internal module cache. Taking a quick look again at the service.js file, we can see that the weatherApiUrl constant is defined in the global code of the file:

const config = require("./config");
const weatherApiUrl = config.WEATHER_API_URL;

module.exports.getWeather = async ({ city }) => {}

As you can see, we’ve accessed the configuration and defined the API URL constant within the global module code of the service.js file which means that now the weatherApiUrl is cached too.

The “module” terminology in Node.js is quite confusing here because of the singleton pattern that the CommonJS module system adheres to which creates an effect of “globals” in the application.

What to do next to avoid tight coupling in Node.js applications

It’s easy to give in and write quick and functional code that works, but it’s much harder to write code that is maintainable and easy to change. Your future self will thank you for writing code that is more cohesive, testable, and over well well designed.

Strive to write code that is loosely coupled. This means that the code should be easy to change without affecting other parts of the application. This is a common principle in software engineering and is often referred to as the Open/Closed Principle. The Open/Closed Principle states that software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.

Those two words that have been lingering in your head for the length of this article - dependency injection - are the key to writing maintainable code in Node.js applications, but that is a whole other topic for another article.

What would you say are your top most struggles when it comes to securing your Node.js apps today?

Liran Tal — Thu, 13 Feb 2025 09:18:35 +0000

Hi Node.js Devs 👋

Trying to get a handle of how can I best help unblock server-side developers in their appsec workflows...

Did we get the whole 3rd-party dependency vulnerabilities figured out?
What sort of help do you need?
What tool or resource can help unblock you?
What are you spending time on to secure your apps? (like is it secrets, env vars, authentication, thinking about your API security? something else?)

How to Read and Parse PDFs with PDF.js and Create PDFs with PDF Lib in Node.js

Liran Tal — Sun, 02 Feb 2025 00:00:00 +0000

You probably caught up on the title that we are going to mention two different npm packages to handle manipulation of PDF files in Node.js. That’s because the more popular option and the one that is more widely used and maintained - pdf-lib is unfortunately unable to read data off of PDF files and only allows creating new PDFs or modify existing ones by adding pages, images, creating forms and such.

So we’re going to split this write-up into those two parts and the respective npm packages that will be used for each task:

pdf-lib - for creating and modifying PDFs. This npm package receives regularly more than 500,000 downloads a week and is maintained by a single developer who updated it lastly around 3 years ago in July 2021.
pdfjs-dist - this one is the popular PDF.js library that is maintained by the Mozilla team and is used by many other projects. It is more widely downloaded, at about 2 million weekly downloads and is updated more frequently, including provenance which is a great security feature to be utilized by library maintainers.

How to Create PDFs with PDF Lib in Node.js

PDF creation with PDF Lib is very granular. You can create a new PDF document of a specific width and height, add pages, draw text, images and create or fill-in form elements.

A simple PDF creation program in Node.js is as follows:

import { PDFDocument, StandardFonts, rgb } from 'pdf-lib'
import fs from 'node:fs/promises'

// Create a new PDFDocument
const pdfDoc = await PDFDocument.create()

// Embed the Times Roman font
const timesRomanFont = await pdfDoc.embedFont(StandardFonts.TimesRoman)

// Add a blank page to the document
const page = pdfDoc.addPage()

// Get the width and height of the page
const { width, height } = page.getSize()

// Draw a string of text toward the top of the page
const fontSize = 30
page.drawText('Creating PDFs in JavaScript is awesome!', {
  x: 50,
  y: height - 4 * fontSize,
  size: fontSize,
  font: timesRomanFont,
  color: rgb(0, 0.53, 0.71),
})

// Serialize the PDFDocument to bytes (a Uint8Array)
const pdfBytes = await pdfDoc.save()

// write to file
await fs.writeFile('./uploads/mal.pdf', pdfBytes)

Very simple, intuitive API. You can refer to the GitHub repository of the library to see more examples and use cases.

How to Read and Parse PDFs with PDF.js in Node.js

Reading and parsing text off of PDF files is a bit more involved than it seems at first.

The reason is that you can’t just read the entire contents of a PDF as one giant text strings because there’s granularity involved - pages. And so indeed, the PDF.js library is built around the concept of pages and you have to iterate over each page in the PDF file to extract the text from it.

Here’s how you extract all the text from all pages in a PDF file:

import fs from "node:fs/promises";
import path from "node:path";
import * as pdfjsLib from "pdfjs-dist/legacy/build/pdf.mjs";

const filePath = "uploads/document.pdf";

async function extractTextFromPDF(pdfPath) {

  // Read the PDF file into a buffer and then
  // parse it with PDF.js
  const pdfData = await fs.readFile(pdfPath);
  const pdfDataArray = new Uint8Array(pdfData);
  const pdfDocument = await pdfjsLib.getDocument({
    data: pdfDataArray,
    standardFontDataUrl: path.join(
      import.meta.dirname,
      "node_modules/pdfjs-dist/standard_fonts/"
    ),
  }).promise;

  let extractedText = "";

  // Iterate through all the pages in the PDF file
  // and extract the text from each page, then assign it
  // to an accumulator variable  
  for (let pageNum = 1; pageNum <= pdfDocument.numPages; pageNum++) {
    const page = await pdfDocument.getPage(pageNum);
    const textContent = await page.getTextContent();
    const pageText = textContent.items.map((item) => item.str).join(" ");
    extractedText += pageText + "\n";
  }

  return extractedText;
}

async function main() {
  const text = await extractTextFromPDF(filePath);
  console.log(text);
}

main().catch(console.error);

How to fix “warning standardFontDataUrl” error with PDF.js in Node.js

Lastly, you might encounter a console warning when using Mozilla’s PDF.js library in Node.js like I did, which relates to fonts.

You’ll notice that when we passed the PDF document data to the pdfjsLib.getDocument function, I also specified a standardFontDataUrl option. This is because the PDF.js library needs to be specified some directory to locate fonts related to the PDF document.

If you omit this option you’ll get a console warning when you run the Node.js PDF parsing program such as:

Warning: fetchStandardFontData: failed to fetch file

To solve that, I specified the location of the fonts that are shipped in the pdfjs-dist package:

standardFontDataUrl: path.join(
  import.meta.dirname,
  "node_modules/pdfjs-dist/standard_fonts/"
),

That’s it. Happy hacking!

p.s. to riff off of my hacking blessings to you - here’s a crazy idea: change the color of the text added in the PDF from black or whatever you chose to white on white background, making it invisible, throw in some classic prompt injection instructions and pass that over to an LLM powered service and see what happens :-)

TypeScript in 2025 with ESM and CJS npm publishing is still a mess

Liran Tal — Tue, 14 Jan 2025 00:00:00 +0000

How does the JavaScript ecosystem tooling looks like in 2025 for TypeScript developers and publishing to the registry?

Well, first off, there are now several JavaScript ecosystem points in the toolchain that are on the verge of hopefully unlocking a better developer experience for TypeScript developers and for a modern JavaScript ecosystem as a whole. This extends and includes gaps related to dual publishing of ESM and CJS modules to a registry and so on.

So what’s in store right now with regards to modules and publishing toolchain in 2025?

JSR.io - A new JavaScript registry that aims to replace npmjs.com. This isn’t just a new storage place to push your packages to. JSR aims to be the natural choice to publish modern JavaScript packages and that means that code is written with TypeScript and packages are distributed via the modern ECMAScript modules as is the web standard for.
Node.js v22 and v23 introduced native support for CommonJS modules to require ESM modules. This was previously available via the experimental command-line flag --experimental-require-module but with v23 and a backport to v22 of Node.js, this is now supported out of the box. To recall why this is a big deal in terms of relieving the pain of dual publishing, it’s because prior to this, Node.js would refuse to load ESM modules from a CJS module and would require you to manage package.json exports, declare the type field, and so on. That’s what lead to the dual publishing of ESM and CJS packages in the first place.

TypeScript and ESM Package Toolchain in 2025

With all of the above context and new features in Node.js, and possibly the rise of the new JSR.io registry, not everyone are on the latest edge of the Node.js runtime and there’s some catching up to do.

For this reason, I’m still likely going to manage dual publishing of ESM and CJS packages for a while. This is where the package build toolchain comes in and I want to share what I’ve come up with as a good balance for supporting TypeScript and ECMAScript modules that get transpiled and bundled to CommonJS modules (and to ESM) with a dual module system publishing strategy.

tsup

So tsup’s purpose is to be a bundler for Node.js and browsers that is simple to use. It might claim to be zero-config but I ended up having to manage a tsup.config.ts regardless to customize and explicitly declare the behavior I’m expecting from the toolchain.

The role for tsup is that I run it as part of the npm run build process in which it transpiles TypeScript to JavaScript (and can bundle it into a single file if you choose to) and allows me to output both CJS and ESM modules, taking care of the dist/ directory and all of that. I still however had to manage the package.json exports field and the type field to declare the module system.

Here’s the related package.json snippet that I use tsup with:

{
  "types": "dist/main.d.ts",
  "type": "module",
  "bin": "./dist/bin/cli.cjs",
  "exports": {
    ".": {
      "import": {
        "types": "./dist/main.d.ts",
        "default": "./dist/main.mjs"
      },
      "require": {
        "types": "./dist/main.d.cts",
        "default": "./dist/main.cjs"
      },
      "default": "./dist/main.mjs"
    },
    "./dist/*": {
      "types": "./dist/*.d.ts",
      "import": "./dist/*.mjs",
      "require": "./dist/*.cjs"
    }
  },
  "scripts": {
    "build": "tsc && tsup"
  }
}

And then my tsup.config.ts has some specific declarations in it:

My entry points. You’ll notice there are two, due to the CLI executable that gets shipped with the package.
The extension for the output files being .mjs and .cjs for ESM and CJS respectively so that older Node.js versions can still require the CJS module based on the filename convention.

import { defineConfig } from 'tsup'

export default defineConfig([
  {
    entryPoints: ['src/main.ts', 'src/bin/cli.ts'],
    format: ['cjs', 'esm'],
    dts: true,
    minify: false,
    outDir: 'dist/',
    clean: true,
    sourcemap: false,
    bundle: true,
    splitting: false,
    outExtension (ctx) {
      return {
        dts: '.d.ts',
        js: ctx.format === 'cjs' ? '.cjs' : '.mjs',
      }
    },
    treeshake: false,
    target: 'es2022',
    platform: 'node',
    tsconfig: './tsconfig.json',
    cjsInterop: true,
    keepNames: true,
    skipNodeModulesBundle: false,
  },
])

tshy

An alternative to tsup is tshy. tshy is much higher-level and opinionated than tsup — it builds CJS and ESM (using tsc, writes to your package.json, and reads config from package.json & tsconfig only and is not not intended as a CLI tool or bundler.

I’ve had friends such as Eric Allam from Trigger.dev who recommended tshy but I’ve settled on tsup for now as the balance of control and simplicity is what I’m looking for.

TypeScript executors

You’re likely going to need what’s referred to as TypeScript executors which are tools that allow you to run TypeScript files ad-hoc, without having to transpile them to JavaScript first as a build step and only then run them. You can think of them as a replacement for node, since node doesn’t support TypeScript files out of the box (yet! there’s work in progress there too :-))

ts-node - I’ve been using ts-node for running the tests. I have it set up so that I execute node and provide it with the --loader tsnode/esm flag to be able to load TypeScript files.

Here is an example of ts-node in my scripts section of package.json:

{
  "scripts": {
        "test": "c8 node --loader ts-node/esm --test __tests__ /**",
        "test:watch": "c8 node --loader ts-node/esm --test --watch __tests__ /**"
  }
}

Another option you can consider is tsx.

Template for TypeScript and Dual Publishing in 2025

So instead of having to re-configure a project from scratch every time I want to build a new project I’ve managed to get a template going that I can re-use easily.

It’s a scaffold project that I’m publishing to the npm registry and it’s called create-node-lib. You can easily scaffold a new project with:

$ npx create-node-lib my-new-lib

This will scaffold a new project with tsup, TypeScript, ts-node, coverage, a bunch of useful GitHub Actions and and all the necessary configurations for dual publishing ESM and CJS modules. You can find an example of an already scaffolded project that I use as a testing ground called baboop which is actually a fun little CLI to pop-up desktop notifications.

Home Assistant YouTube DNS Blocking with AdGuard and Lovelace Buttons Setup

Liran Tal — Tue, 07 Jan 2025 00:00:00 +0000

In a prior article I’ve written how to block client devices in your LAN from accessing YouTube on Home Assistant but that setup was somewhat clunky because of the switch interface on the Lovelace UI. Since then, I’ve been wanting to move to a more action friendly interface via the Lovelace UI with the help of purpose-specific buttons that can be clicked to toggle the DNS blocking on and off.

The reason for using buttons instead of a switch is that a switch inherently needs to derive a state to be able to properly send the “on” or “off” state to the AdGuard Home integration. This is because the switch is a binary state and it’s not clear whether the switch is on or off when the Lovelace UI loads. If you aren’t planning to overly complicate the state management via GET requests to the AdGuard integration then we can keep things easier with buttons.

Setting up Home Assistant YouTube DNS blocking with Lovelace Buttons

So first thing we need to do is define the command line API calls with curl via the following configuration.yaml definition:

command_line:
  - switch:
      name: Strict Entertainment Media
      unique_id: strict_entertainment_media
      command_on: 'curl -X POST -m 10000 -H "content-Type:application/json" -s -u "username:password" http://a0d7b954-adguard:3000/control/clients/update -d "{\"name\": \"HouseholdPublic\",\"data\":{\"name\":\"HouseholdPublic\",\"ids\": [\"192.168.68.52\", \"192.168.68.50\", \"192.168.68.61\", \"192.168.68.51\",\"192.168.68.58\"],\"tags\": [],\"upstreams\": [],\"filtering_enabled\": true,\"parental_enabled\": true,\"safebrowsing_enabled\": true,\"safesearch_enabled\": true,\"use_global_blocked_services\": false,\"use_global_settings\": true, \"blocked_services\": [\"youtube\"]}}"'
      command_off: 'curl -X POST -m 10000 -H "content-Type:application/json" -s -u "username:password" http://a0d7b954-adguard:3000/control/clients/update -d "{\"name\": \"HouseholdPublic\",\"data\":{\"name\":\"HouseholdPublic\",\"ids\": [\"192.168.68.52\", \"192.168.68.50\",\"192.168.68.61\", \"192.168.68.51\",\"192.168.68.58\"],\"tags\": [],\"upstreams\": [],\"filtering_enabled\": true,\"parental_enabled\": true,\"safebrowsing_enabled\": true,\"safesearch_enabled\": true,\"use_global_blocked_services\": true,\"use_global_settings\": true, \"blocked_services\": []}}"'

In this setup, I still maintain separate commands for on and off state via a so-called switch but we’re going next to create a script for each of these commands that will be used by the Lovelace UI buttons.

Open up your scripts.yaml file and add the following:

block_youtube:
  alias: Block YouTube
  sequence:
    - service: switch.turn_on
      target:
        entity_id: switch.strict_entertainment_media

unblock_youtube:
  alias: Unblock YouTube
  sequence:
    - service: switch.turn_off
      target:
        entity_id: switch.strict_entertainment_media

These are now service scripts which can be called by the Lovelace UI buttons. Next up, we’re going to create the Lovelace UI buttons.

Creating Lovelace UI buttons for YouTube DNS blocking

We are going to end up with the following buttons layout:

And here is how the visual editor looks like to get that setup:

The Lovelace UI configuration for the buttons is as follows:

type: horizontal-stack
cards:
  - show_name: true
    show_icon: true
    type: button
    tap_action:
      action: call-service
      service: script.block_youtube
      target: {}
    icon: mdi:youtube
    name: Block YouTube
  - show_name: true
    show_icon: true
    type: button
    tap_action:
      action: call-service
      service: script.unblock_youtube
    name: Unblock YouTube
    icon: mdi:youtube

Customizing Astro Starlight Sidebar for Gated Content with Authentication

Liran Tal — Mon, 06 Jan 2025 00:00:00 +0000

The Astro framework powers this personal blog, my Node.js Secure Coding website, and now my newly launched Bun Security website. I’ve been using Astro for a while now and I’m a big fan of the framework. It’s fast, it’s simple, and it’s a joy to work with.

For the Bun Security course, because this is an online self-paced educational content, I settled on using Starlight as the documentation framework. Starlight is a static site generator that’s built on top of Astro, and it comes with pre-built components, layout and design that provides a UI for documentation websites and that fits well with course content too.

Specifically for the Bun Security course, I wanted to created a mix of open content about news and security topics for the Bun server-side JavaScript runtime, but also to have gated content which makes up the course material. This is where things get a bit more interesting and require some customization to get this working.

So to set us off for the rest of this Astro Starlight customization journey, we’re going to leverage the following parts:

Astro as the foundational framework that powers the website
Starlight as the documentation framework that provides the UI and layout to put content in
Clerk as the authentication provider that will be used to decide on access to the gated content

Next up, the customizable part of Starlight are going to be the sidebar items which we don’t want to just publicly show and have pre-generated but rather generate the content via server-side rendering (so we can decide on access to the content based on the user’s authentication status), and the ability to customize Starlight’s sidebar to get granular control of the sidebar items (allows us to decide whether to show the chapter name and the sub-chapters or not).

Let’s begin.

Astro’s Starlight configuration

The following is my stock configuration for the Starlight integration part of Astro:

    starlight({
      prerender: false,
      title: "Bun Security Essentials | Course",
      sidebar: [
        {
          label: "Getting started",
          autogenerate: { directory: "course/getting-started" },
        },
        {
          label: "Bun Security Introduction",
          autogenerate: { directory: "course/introduction" },
        },
        //
        // ... more chapters go here...
        //
        {
          label: "👉 ",
          link: "/",
          badge: { text: 'Buy the Course', variant: 'note' },
        },
      ],
      social: {
        github: "https://github.com/lirantal",
      },
      disable404Route: true,
      customCss: ["./src/assets/styles/starlight.css"],
      favicon: "/favicon.ico",
      components: {
        SiteTitle: "./src/components/ui/starlight/SiteTitle.astro",
        Sidebar: "./src/components/ui/starlight/Sidebar.astro",
      },
    }),

In the above, I’ll call out the two important parts:

The prerender option has to be set to false so that we can control the sidebar items and generate them dynamically and also generate the actual site’s content dynamically based on the user’s authentication status.
The Sidebar component override which allows to customize the sidebar items and generate them dynamically.

Customizing the Starlight Sidebar

The Sidebar component is where we can decide which content to render for the sidebar items and we’re going to use the Clerk authentication SDK to do so based on the user’s authentication status.

---
import type { Props } from "@astrojs/starlight/props";
import Default from "@astrojs/starlight/components/Sidebar.astro";

const user = await Astro.locals.currentUser();

const strippedSidebar = Astro.props.sidebar.filter((entry) => {

  const allowedEntries = [
    'Getting started',
  ]

  if ((entry.type === "group" || entry.type ==='link') && allowedEntries.includes(entry.label)) {
    return true;
  }

  if (user) {
    return true;
  } else {
    entry.label = `[Locked]`;
    entry.entries = [];
    entry.badge = {
      text: "Get Full Course",
      variant: "tip",
    };
    return true;
  }
});

const strippedProps: Props = { ...Astro.props, sidebar: strippedSidebar };
---

<Default {...strippedProps} />

The user variable information is available due to the Clerk integration and the middleware that gets used as part of the Clerk setup into an Astro project.

Next up, we create our own instance of strippedSidebar which is made up of our own logic (authentication based) on how to render the items for the sidebar. In this case, we’re only allowing the “Getting started” chapter to be publicly available, and the rest of the chapters are gated behind the authentication.

How does it look?

Here is a screenshot of the Starlight sidebar with the customization applied so you can get a visual. This is from my Bun Security website:

How to Setup Google Cloud Project and Store Images in Google Cloud

Liran Tal — Sat, 21 Dec 2024 00:00:00 +0000

In this write-up I will describe how to setup a Google Cloud project (on GCP) and use it to store images in Google Cloud Storage. If you’re looking into storing uploaded images via an S3-like alternative with Google through a step-by-step tutorial and this is a write-up for you.

Init a new Google Cloud project with the `glcoud` CLI

1. Install the gcloud CLI
1. Initiate a new project:

gcloud projects create credster --name="Credster project" --labels=type=credster

1. Set the active project from now on to the rest of the usage for gcloud:

gcloud config set project PROJECT_ID

1. Link the project to a billing account so you can open up APIs. This has to be done from the UI as follows:
2. 4.1. Choose the project to be active:
3. 4.2. Click on Billing from the available boxes:
4. 4.3. Then choose Link a billing account:
5. 4.4. Then set and choose the existing one already (if not, then create one):

Store images in Google Cloud Storage

In this case we need to achieve 3 things:

1. Create a bucket to hold all image files
1. Create a credentials API key in JSON format that we can load on the backend server to access and perform operations on the storage bucket
1. Update the bucket permissions to allow public access

Create images bucket

Create a bucket to hold uploaded images:

gcloud storage buckets create gs://cert_images --project credster

With an output of:

Creating gs://cert_images/...

Create service account for cloud storage

1. Go to API & Services → Credentials and click on Create Credentials and choose Service account:

1. Give it a name (backend-storage-access)
1. Assign it the role Storage admin:

1. Save it with Done
1. Now edit the service account and go to KEYS tab and create a new key and choose JSON key:

and then

it now downloaded a JSON file to your computer and gave the following dialog to draw attention on keeping it secure:

Update bucket permissions to view-all

Add allUsers to members of the bucket resource and give them role Storage object viewer.

gsutil iam ch allUsers:objectViewer gs://$BUCKET_NAME

Update bucket permissions to allow CORS

Read about CORS here and why it is needed

Create a local temporary JSON file to define allowed CORS for the bucket called cors-bucket-policy.json

[
    {
      "origin": ["*"],
      "method": ["GET", "PUT"],
      "responseHeader": ["Content-Type"],
      "maxAgeSeconds": 3600
    }
]

We can then update it properly to something like this when we go to production:

[
    {
      "origin": ["<https://your-example-website.appspot.com>"],
      "method": ["GET", "PUT"],
      "responseHeader": ["Content-Type"],
      "maxAgeSeconds": 3600
    }
]

gcloud storage buckets update gs://$BUCKET_NAME --cors-file=$CORS_CONFIG_FILE

View the CORS configuration for a bucket and confirm:

gcloud storage buckets describe gs://$BUCKET_NAME --format="default(cors)"

If needed to remove CORS completely:

gcloud storage buckets update gs://$BUCKET_NAME --clear-cors

Google Cloud Static Website Hosting

More information about Google Cloud static website hosting, with Firebase and otherwise can be found in the following resources:

Thinking Fast and Slow in Application Security

Liran Tal — Thu, 12 Dec 2024 00:00:00 +0000

Imagine if we applied behavioral economics principles to application security methodologies and practices, what would be able to unlock? System1 and System2, All Systems Go.

No doubt, if you read Daniel Kahneman’s book “Thinking, Fast and Slow” you’ve likely been similarly fascinated by his insights into the two systems of thinking. And no doubt. He is, after all, a Nobel prize winning psychologist who revolutionized the field of economics. Together with Amos Tversky, they created immense value and contribution into the understanding the nuances and oddities of human decision making.

Inspired by this, I set out to see how could we apply the same human behavior principles to the domain of application security.

System1 and System2 in AppSec

System1 and System2 are two modes of thinking that we humans apply to various tasks during decision making. System1 is known for its speed and intuition while System2 is more deliberate and analytical.

How do we apply this foundational thinking fast and slow behavioral systems to the application security domain to derive better results?

System1: Fast Thinking in AppSec

To put System1 into the application security context, I’ll suggest some areas where this applies:

Running software composition analysis (SCA) tools: They are fast, deterministic and can be easily automated by security teams and developers alike. Developers in fact pay little overhead to running these tools - whether they get automated pull requests to upgrade vulnerable dependencies, or they spin off a snyk test command on the CLI to check for vulnerabilities in their project, this process is fast. The scanning process is fast, and the decision making is fast too - upgrade the dependency or not.
Generating SBOMs: Like SCA scanning, this is also a minimal effort win for developers and security teams. Generating an SBOM is an automated process too and have been largely commodotized.

System2: Slow Thinking in AppSec

Where System1 maps to automated tools, almost boolean decision making, System2 requires more work. Let me put some practical appsec exercises into the context:

Threat modeling: mostly a manual process that gets stakeholders in a room to discuss security threats and risks to a system. Can it be automated? Sure, but its effectiveness comes from the human interaction that uncovers hidden layers, assumptions and biases. You often threat model early into the design process of a feature or a project, so at that point, you also have little data to work with.
Secure code review: One of the most foundational and yet illusive concepts in secure coding is context. Context is key to understanding if a risk we identified is applicable or not. Does the data flow from the user impact the security of the system? Depends. It depends on the context of how the system uses the data, and which system uses the data. A secure code review can be automated to an extent but is more often than not a valuable exercises when expert and deep understanding of the system involved is present.
Analyzing SAST security findings: Somewhat similar to secure code review, when a developer reviews a list of security findings from a static analysis tool (SAST), they are burdened with figuring out call path flow, data flows, and context of the finding. Much of the information is also not easily available. For example, a developer who reviews the code may not have enough understanding of what type of data is being processed. Perhaps they have a bias to think the data is always text, a string, a literal, but in reality, an attacker may end up sending a payload that gets interpolated to an array by a middleware or a library. Analyzing SAST security findings requires understanding of source-to-sink and some security expertise to be easily actionable by developers in terms of applying mitigation and security controls effectively.

Fixing AppSec with Systematic Thinking

Kahneman research demonstrated that System1 thinking is in charge 95% of the time where as rational thinking through System2 is only 5% of the time. That makes sense, because System2 thinking takes effort, and is relatively slow. System2 requires more energy and time to process and apply logical thinking.

From the prism of developers, they are inherently less likely to focus on cross-cutting concerns like security due to the foundational business aspects (focus on features, value, delivery). This is where System1 thinking comes into play, especially for security tasks that developers would want automated so they don’t need to divert their train of thought to System2 thinking in order to address security issues brought up by the security team.

What impact would we unlock if we could move traditional System2 tasks into System1 for developers?

Anchoring Bias in AppSec

Anchoring bias is a cognitive bias that describes the common human tendency to rely too heavily on the first piece of information offered when making decisions. So imagine you walk into a store and you see a $1,000 Synology NAS (yes, I’ve been on the market for that :D) and then you see a $450 NAS. Your initial anchor is the $1,000 NAS, so you’re more likely to think the $450 NAS is a good deal.

So we established that during decision making we anchor initial piece of information as the relative point to which we make subsequent judgments.

Let me show you a real-world example of this in everyday life of a developer. Imagine a developer is installing a new package in their project. Their workflow is as follows:

npm install lodash

added 1 package, and audited 57 packages in 15s

found 17 vulnerabilities (1 low, 16 high)
  run `npm audit fix` to fix them, or `npm audit` for details

How did the 17 vulnerabilities that they were told about just now impacted them? They shipped to production. Nothing happened. They looked into some of them, found that they are as high as CVSS 9.8 but they are irrelevant to them because they are detected in development dependencies. Their anchor is set to “I’ve seen this before, it’s not a big deal”. Their first impression has been skewed to a point where they are less likely to seriously consider vulnerabilities as a threat to their system.

This is, in fact, a common problem in developer security and has been coined the term “vulnerability fatigue”. Developers are provided with too much information and findings of security vulnerabilities that get reported to them, most of which were justified to be deemed as irrelevant.

Loss Aversion in AppSec

To put simply, you’re more likely to feel awful about losing a $100 than you would feel good about randomly finding $100 on the back pocket of your jacket. This is loss aversion in action.

Or let me try to maybe give it a developer spin for an analogy - imagine a developer has now spent 10 hours fixing a ridiculously hard bug in their code. They’ve been debugging, reading logs, and working their brains out to fix this thing relentlessly. Suddenly, and right on point with a developer analogy, they hit the wrong git command and poof all of the staging area changes are gone. They’ve lost 10 hours of deep work and a proper solution to the bug. In that context, developers are more likely to feel awful about losing 10 hours of work than they would feel good about finding an npm dependency that saved them 10 hours of work.

How do we apply loss aversion to application security? To an extend, this depends on the sort of spin you want to give it. Do you want to use loss aversion to deter developers from making security mistakes? Or do you want to use it to encourage developers to fix security issues? Some ideas come to mind:

Rely on loss aversion to encourage developers to unveil the invisibility cloak of security issues by showing them the impact of a security vulnerability. You can do this by practicing a hands-on vulnerable and exploitation demonstration that creates the “aha” moment for developers.
Another option that is practiced more regularly is to set a policy where the CI pipeline breaks when a security vulnerability of a certain severity is detected. This is a form of loss aversion since developers are more likely to fix the issue to begin with by applying secure code review, secure coding practices, or even better - running static security analysis in their IDE, to already catch the issue before it gets to the CI pipeline and frustrates them.

Have more dramatic ideas on how to apply loss aversion to application security? I had some red-team thoughts but maybe these are a bit too abusive for developers and better kept in the drawer ;-)

Availability Bias in AppSec

If our brain can recall something easily we’re more likely to overestimate its significance.

How do we instill the availability bias to raise awareness for application security? Here are some suggestions:

Regularly share security incidents, data breaches and security news with the team.
Put developers into the “blue team” shoes by running security exercises like capture the flag (CTF) events but instead of attacking, they’re defending.

Developers will be more likely to remember security incidents that they were actively involved in the process or defending against during one of those fun exercises, and as a consequence will have a higher awareness of security issues day in and day out.

Confirmation Bias in AppSec

What would confirmation bias look like fo developers?

Developers are possibly more likely to confirm their own beliefs about how the organization handles security. For example, if a developer believes that security of the application (or the business) is the responsibility of the security team, they are more likely to confirm this belief by not taking security at all into account in their daily work. That’s someone else’s job, right?
What if developers were swayed by application security own’s catch 22? For example, if a developer believes that the application is secure because they’ve never seen a security incident, they never had a data breach, or they were never confronted with a security issue, they are more likely to confirm this belief by, once again, not practicing security in their day to day development practices.

Planning Fallacy in AppSec

Ever heard a developer say “this one is going to be a quick fix” - famous last words, right? Only to discover that there’s just so much more involved, your peer is out of office today, and some system credentials you needed are unavailable to you so the task ends up taking 3 days to fix.

Applying planning fallacy to application security can be reasoned with tasks relating to handling invalid user input. Let’s put that into a practical example - a developer implements a GET HTTP request which receives user input in the form of query parameters. Here’s a code example to illustrate:

app.get('/search', (req, res) => {
  const query = req.query.q;
  // do something with query
});

Developers fall into the trap of planning fallacy in two ways:

They either overestimate the simplicity of this task and don’t even consider handling invalid user input, validating or sanitizing the input as needed.
Or maybe they do take input validation into account but they underestimate the complexity of the task and end up with a half-baked solution. For example, consider the following code snippet:

app.get('/search', (req, res) => {
  const query = req.query.q;

  if (typeof query == 'string') {
    const sanitizedQuery = query.trim();

  }

  // do something with sanitizedQuery
});

But now what if the attacker sends in an array as the query parameter? Such as /q[]=1&q[]2. The developer didn’t account for that because they haven’t even thought of considering the possibility of an array being passed in as the query parameter. This is a popular attack vector known as HTTP parameter pollution.

Where do we go from here?

I would say that being aware of these cognitive biases and how developers may have their own biases is a good start for application security teams and security champions in the organization because it can clarify and provide deeper understanding of why developers may not be as security conscious as they should be. And that’s not something to hold against them. It’s not that developers don’t care about security, there are deeper forces at play that impact their decision making.

Not sure when we discover about System3 thinking, but for the time being, goodluck with System1 and System2 in application security! ;-)

Component auto import in Astro framework

Liran Tal — Fri, 06 Sep 2024 00:00:00 +0000

The Astro frontend framework is such a delight to work with but I was missing a feature with regards to islands and framework components that allowed the page data, such as blog posts, to automatically import the components that are used in the markdown file. This is how I solved it.

About MDX and Astro

Astro is a frontend framework that allows you to build websites, like a personal blog, using a mix of static and dynamic content. It’s a great way to build websites that are fast, optimized for SEO due to the server-side rendering and server-generated pages support, and easy to maintain.

One of the features of Astro is that it allows you to use MDX files to create pages. MDX is a way to write JSX in markdown files. This allows you to extend the Markdown syntax and use framework components in your markdown files to create rich content. MDX is well-supported in Astro using the @astro/mdx package and the MDX syntax is built on-top of Markdown so you don’t really need to learn a new syntax and can maintain the frontmatter and markdown syntax that you are already familiar with.

Her’es an example of MDX files:

---
title: "Hello, world!"
date: "2024-06-06"
---

import SomeComponent from './SomeComponent.jsx';

# Hello, world!

<SomeComponent prop="propValue" />

This is a blog post written in MDX, <Button name="Click me" />.

The rendered page will have the SomeComponent and Button components rendered in the page content but will not show the import SomeComponent ... statement. This is because the @astro/mdx package processes the MDX file and injects the component imports into the page. Of course, don’t forget saving this file with an .mdx extension: blog-hello-world.mdx.

Auto import components in Astro

If you search “astro framework component auto import” long enough you’ll land on the community contributed Astro package called astro-auto-import by the splendid Chris Swithinbank.

In essence, the astro-auto-import package reads the component file from the path that you specify in the Astro integration configuration file and then generates the appropriate ESM module tree for it that gets injected into the markdown imports and looks something like this:

return {
        type: 'mdxjsEsm',
        value: '',
        data: {
            estree: {
                body: [],
                ...parseJs(js, { ecmaVersion: 'latest', sourceType: 'module' }),
                type: 'Program',
                sourceType: 'module',
            },
        },
    };

Benefits of using astro-auto-import:

You can import React, Vue, and other framework components.
You don’t need to explicitly and manually import the components in the markdown file.
You can maintain both .md and .mdx files side by side in your Astro blog.

Configure the Astro Auto Import integration by adding the following to your astro.config.mjs file:

// .. other imports
import AutoImport from 'astro-auto-import';

export default defineConfig({

    // ... other config options
    integrations: [
        // The Auto Import configuration here:
        AutoImport({
            imports: ['./src/components/widgets/BlogCallToAction.vue'],
        }),
        // Make sure that the mdx() integration is *after* the Auto Import integration
        // *AND IMPORTANT* to make sure that the mdx integration doesn't repeat any prior markdown configured
        // plugins like remarkToc or remarkReadingTime
        mdx(),
    ],

    markdown: {
        remarkPlugins: [remarkToc, remarkReadingTime],
        extendDefaultPlugins: true,
    },
});

The following are especially important to note:

The imports array in the AutoImport configuration should contain the path to the components that you want to auto-import in the markdown files.
The mdx() integration should be placed after the AutoImport integration in the integrations array.
The mdx() integration should not repeat any prior markdown configured plugins like remarkToc or remarkReadingTime.

If you skip the last one and you have some markdown plugins stated in the mdx() integration, you will get an error like this:

[Vue warn]: Component <Anonymous> is missing template or render function.
[Vue warn]: Component <Anonymous> is missing template or render function.
 error Could not render `[object Module]`. No matching import has been found for `[object Module]`.
  Hint:
    Please make sure the component is properly imported.
  Error reference:

or some other Vite error similar to the above which states that it can’t find the component that you are trying to render in the MDX file and the whole page won’t load. Astro will crash.

Once everything is installed and setup, just use the component in your MDX files without having to declare the import statement:


# Hello, world!

<SomeComponent prop="propValue" />

This is a blog post written in MDX, <Button name="Click me" />.

No more manual imports, kind of like using globals in Markdown files 😅 but hey, at least those Markdown files are portable and consistent and won’t have a weird import looking statement in the beginning of the file when you migrate or import them to a new system next time!