DEV Community: NodeSecure

NodeSecure hidden capability: mama

Thomas.G — Sat, 10 Jan 2026 07:12:29 +0000

Hello 👋

I'm starting a new short-article series focused on highlighting lesser-known parts of the NodeSecure project. The goal is to help new contributors by giving them a clearer view of the back-end building blocks that power the project.

Chapter 1: mama

Mama stand for ManifestManager. This package was designed to manage and load an npm manifest (a package.json file, for simplicity).

Under the hood, it uses @nodesecure/npm-types to provide precise, up-to-date types (including runtime-related fields). We will dive into that package in another article.

import { ManifestManager } from "@nodesecure/mama";

// synchronous version: ManifestManager.fromPackageJSONSync
const mama = await ManifestManager.fromPackageJSON(
  process.cwd()
);
console.log(mama.document);

This package provides many utilities used across back-end components in the Scanner monorepo.

Here are a few of them:

Integrity

You can easily extract a hash by using the readonly getter integrity.

console.log(mama.integrity);

Scanner uses this to assert that the package.json in the tarball matches the one uploaded to the registry (known as manifest confusion).

Here are the properties we hash:

{
  name,
  version,
  dependencies,
  license: license ?? "NONE",
  scripts
}

When a mismatch is detected, the tool reports it as a global warning, as shown in the CLI UI:

module type

Inspired by the recent node-modules-inspector tool built by Antfu, we re-implemented the same module type detection:

// "dts" | "faux" | "dual" | "esm" | "cjs"
console.log(mama.moduleType);

entry files

mama can recursively extract entry files using the Node.js exports field (or legacy fields like main). The API is lazy and returns an IterableIterator<string>:

console.log([...mama.getEntryFiles()]);

This API is used in the tarball package in combination with JS-X-Ray’s EntryFilesAnalyser.

async scanFiles(): Promise<ScannedFilesResult> {
  const location = this.manifest.location;
  const [
    composition,
    spdx
  ] = await Promise.all([
    getTarballComposition(location),
    conformance.extractLicenses(location)
  ]);

  const code = await new SourceCodeScanner(this.manifest).iterate({
    manifest: [...this.manifest.getEntryFiles()]
      .flatMap(filterJavaScriptFiles()),
    javascript: composition.files
      .flatMap(filterJavaScriptFiles())
  });

  return {
    conformance: spdx,
    composition,
    code
  };
}

author

Parse the NPM author field if present and then return a Contact interface.

interface Contact {
  email?: string;
  url?: string;
  name: string;
}

For example, John Doe <john.doe@gmail.com> produces the following object:

{
  "name": "John Doe",
  "email": "john.doe@gmail.com"
}

Others

The module also provides additional utilities around reading and managing manifests, such as:

Parsing package specs (including scope/org, package name, and semver range)
Detecting local lockfiles

The end

The full module documentation is available here.

Thanks you for reading

Securizing your GitHub org

Thomas.G — Sun, 19 Feb 2023 15:48:01 +0000

Hello 👋

I started open source a bit naively (like everyone I guess 😊).

But the more I progress and the more important/popular some of my projects become 😎. That's great, but at some point you have to deal with a lot of things related to security (like Vulnerability disclosure).

You start to hear and see a lot of scary stories around you 😱. Not to mention all the acronyms where you don't understand anything at first 😵 (VMT, CVE, SAST, SCA, CNA ...).

As I was working on an open source security project, I put pressure on myself to be ready. Also as a member of the Node.js Security WG I thought it was an interesting topic and that I was probably not the only one who was worried about not being up to the task 😖.

So I rolled up my sleeves and tackled the problem 💪. Here is my feedback/journey on how I improved the security of my NodeSecure GitHub organization.

👀 We use Node.js and JavaScript (but most recommendations are valid for other ecosystems).

Security Policy and Vulnerability Disclosure

Adding a root SECURITY.md file explaining how developers and security researchers should report vulnerability is important. You don't want a security threat to be turned into a public issue (This gives you time to analyze and possibly fix before disclosure).

⚠️ If you are a developer, never report a security threat using a public GitHub issue. This is a serious mistake. This could even put your business/team at risk.

I don't want to bullshit you, so let me share with you the OpenSSF guide that helped me set up my first reporting strategy: Guide to implementing a coordinated vulnerability disclosure process for open source projects.

I started from scratch by reading this guide and taking inspiration from their templates 🐤. As a small open source team we don't especially have DNS or mail servers (not even a defined Vulnerability Management Team A.K.A VMT).

I was a bit puzzled to put my personal email as I'm not alone 😟.

I quickly learned that Github added a new feature to report/create private security issue 😍. You can enable it in the Security tab (I think it's also now possible to enable it on every repositories at once).

And this is what it finally looks like:

Use OpenSSF scorecard

The OSSF scorecard initiative is really good to assess your project against security best practices. I am not the first to write about this.

You can easily setup the GitHub action workflow by following those instructions.

Once configured, you will have a set of alerts available in the Security tab.

This will give you an overview of the different subjects to improve (workflows, dependencies etc). Each of these alerts contains a full description of the actions to be taken to fix the problem.

I have personally used these recommendations to dig and train myself. The next chapters will help you improve your score.

📢 By the way NodeSecure CLI has a first-class support of the scorecard.

🔓 Enable branch protection

I am a bad student 😳. Almost all of my projects had no branch protection on the main / master branch 🙈.

To set up the protection, go to Settings > Branches and edit your main branch.

GitHub has quite a few options on the subject 😵. If you don't know what to choose in terms of options, don't check anything (it's ok to begin ✔️).

If you want to be more restrictive, be careful because it could block you (some options are only viable in projects with many contributors/reviewers).

As far as I am concerned I often choose:

Require a pull request before merging
Require conversation resolution before merging
Require status checks to pass before merging

🐲 Workflows Hardening

I fell down when I saw all that it was necessary to know to secure workflows with GitHub actions 😲.

You must pay attention to the permissions granted to your jobs / GITHUB_TOKEN
Ensure your GitHub Actions are pinned to a SHA
Hardening the runner (see the StepSecurity HardenRunner)
Probably a lot of other stuff I haven't had time to see yet 😆

Fortunately there is a great free online tool that help you by doing all the hard work (it will open a pull-request and automatically fix issues).

// Detect dark theme var iframe = document.getElementById('tweet-1617557370728767488-427'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1617557370728767488&theme=dark" }

The tool was created by StepSecurity. I had the opportunity to talk with the CEO and they listen to the maintainers which is really cool.
Thanks to them ❤️!

Configure Dependabot

It is recommended to use Dependabot for updating your dependencies and GitHub actions (yes, it also supports updating workflows in a secure way 😍).

You only need to add a .github/dependabot.yml config file. Personally I recommend a weekly interval (with a lot of projects daily is a bit horrible).

version: 2
updates:
  - package-ecosystem: github-actions
    directory: /
    schedule:
      interval: weekly

  - package-ecosystem: npm
    directory: /
    schedule:
      interval: weekly

The StepSecurity tool we have seen in the previous chapter is also capable of doing it 🚀.

Also, think to enable Dependabot alerts in the Security tab. This will allow the bot to open pull-request to fix known vulnerabilities by looking at your dependencies (referenced in package.json or others).

🔬 Adding CodeQL scanning

To enhance security even more you can add a SAST tool like CodeQL. Like scorecard it will report security scanning alert but for your codebase.

Here an example:

A great way to make sure that newly added code does not contain vulnerabilities that were obvious to detect.

👀 Note that once again StepSecurity can set up the workflow for you.

📜 Enable Security advisories (and others)

Github Security tab as a lot of cool features that help you maintain the security of your project. If you have followed all my previous chapters, most of them should be enabled now.

Make sure to also enable Secret scanning alerts.

For an organization many of these parameters can be forced on all repositories. Go to Settings > Code security and analysis. You will have the options to enable/disable all.

💡 OpenSSF Best Pratices program

Previously known as CII-Best-Practices, this program indicates that the project uses a set of security-focused best development practices for open source software.

So I registered my first project on the website. It was a good surprise because it allowed me to question the quality of my documentation and tests 😬.

Seeing the different levels and questions really helps you think about what you're missing (and possibly learn about the concepts you don't know about yet.. Like SBOM).

I am still working on completing the first step/badge for the CLI project which now has a score of 8.7 out of 10 🎉 on the OpenSSF scorecard.

🎯 Conclusion

That's it for this article. I've covered what I've done/learned in the last couple of months. Here are some really cool additional links 💃:

If you work with NPM, I invite you to read our latest article about package managers:

📦 Everything you need to know: package managers

Antoine Coulon for NodeSecure ・ Nov 18 '22

#npm #node #opensource

Obviously, I probably still have a lot to learn. But I hope this will help other maintainers/developers ❤️.

🙏 Thanks for reading me 🙏

JS-X-Ray 6.0

Thomas.G — Mon, 16 Jan 2023 15:48:18 +0000

Hello 👋

It's been a while since the last article on JS-X-Ray 😲!

JS-X-Ray 3.0

Thomas.G for NodeSecure ・ Feb 28 '21

#node #javascript #security #opensource

In this article I will present you the latest major version 👀. I didn't do an article on version 4 and 5 because they didn't introduce new features (only breaking changes on the API).

📢 What is JS-X-Ray ?

If you are new in town, JS-X-Ray is an open source JavaScript SAST (Static Application Security Testing). The tool analyzes your JavaScript sources for patterns that may affect the security and quality of your project 😎.

Among the notable features:

Retrieving dependencies (CJS & ESM support) and detecting suspicious import/require.
Detecting unsafe RegEx.
Detecting obfuscated source (and provide hints on the tool used).

As well as a lot of other detections.

Major release 4 and 5

These versions introduced changes on warnings (and we improved how we manage them in the codebase). We added new descriptors for each of them:

i18n (for translation in CI or CLI).
experimental
severity (Information, Warning, Critical)

Those information are visible in the NodeSecure CLI interface:

Major release 6

🐬 Ok, let's dive into this major release to discover the surprises 🎉 it has in store for us.

🚀 Introducing VariableTracer

Almost a year of work on this new mechanism / class that brings a whole new dimension to JS-X-Ray.

const tracer = new VariableTracer()
  .enableDefaultTracing()
  .trace("crypto.createHash", {
    followConsecutiveAssignment: true, moduleName: "crypto"
  });

tracer.walk(node);

This class is able to follow all declarations, assignments and patterns (and those even through very obscure patterns).

const aA = Function.prototype.call;
const bB = require;

const crypto = aA.call(bB, bB, "crypto");
const cr = crypto.createHash;
cr("md5"); // weak-crypto warning is throw here

This allows us to implement Probes in a much simpler way (which makes maintenance and testing much easier).

Here an example with the isWeakCrypto probe:

function validateNode(node, { tracer }) {
  const id = getCallExpressionIdentifier(node);
  if (id === null || !tracer.importedModules.has("crypto")) {
    return [false];
  }

  const data = tracer.getDataFromIdentifier(id);

  return [
    data !== null &&
    data.identifierOrMemberExpr === "crypto.createHash"
  ];
}

By default the Tracer follows all ways of requiring dependencies with CJS and also usage of eval or Function.

🚧 Removing unsafe-assign warning

This warning was required at the beginning of the project because it was difficult for me to correctly identify some malicious patterns.

However, with the introduction of the new Tracer, which is very complete and precise, this warning no longer makes sense has it only generates unnecessary noise and false positives.

📜 Better ESM source parsing

We previously had a lot of parsing-error warnings because the NodeSecure scanner failed to detect if the file was using either CJS or ESM.

That new version will automatically retry with ESM enabled if it fails with CJS.

📉 Reducing false positives

To continue the momentum of the previous sections. This version drops a lot of warnings and significantly improves others.

Reducing false positives for encoded-literal warning by introducing new way of detecting safe values.
Improve short-identifiers by also storing ClassDeclaration, MethodDefinition and Function parameters.

We are also introducing a new suspicious-file warning when a file contain more than 10 encoded-literal warnings to avoid having file with hundreds or thousands of warnings.

Of the 500 most popular NPM packages, we previously had 24k warnings with version 5. The latest version brings that number down to approximatively 5k warnings.

🔬 Improving coverage

A lot of work has been done to add unit tests on all the probes of the project. We are near 100% of coverage 💪.

Thanks to the amazing work of our contributors:

👀 What's next ?

Here what I'm working for the next major release:

Adding support of TypeScript sources (probably by allowing a customization of the parser).
A new API that allows to dynamically extend the SAST with new custom probes (and custom warnings).
Introducing new built-in detections and warnings (unsafe URL etc).

I will continue to work to reduce the number of false positives and keep improving obfuscated codes detection.

Please think to drop a star on github ❤️!

NodeSecure / js-x-ray

JavaScript & Node.js open-source SAST scanner. A static analyser for detecting most common malicious patterns 🔬.

JavaScript AST analysis. This package has been created to export the NodeSecure AST analysis to enable better code evolution and allow better access to developers and researchers.

The goal is to quickly identify dangerous code and patterns for developers and security researchers. Interpreting the results of this tool will still require you to have basic knowledge of secure coding.

Goals

The objective of the project is to detect potentially suspicious JavaScript code. The target is code that is added or injected for malicious purposes.

Most of the time hackers will try to hide the behaviour of their code as much as possible to avoid being spotted or easily understood. The work of the library is to understand and analyze these patterns that will allow us to detect malicious code.

Feature Highlight

Retrieve required dependencies and files for Node.js
Detect unsafe regular expressions
Get warnings when the AST analysis detects a…

View on GitHub

That's it for today! Thanks for reading me 😉

📦 Everything you need to know: package managers

Antoine Coulon — Fri, 18 Nov 2022 09:51:21 +0000

Welcome everyone! This article is the first one of the Everything you need to know, a Software Engineering series.

In this series, I will try to give you a solid basic understanding about Software Engineering concepts I consider important.

All modern computer systems include tools that automate the process of installing, uninstalling and updating software.

This responsibility is that of a package manager and several can intervene within the same computer system.

Operating system

The majority of Unix-based operating systems embed a package manager as standard, providing a multitude of different packages very simply.

If you have ever used a Linux distribution such as Ubuntu or Debian, you've probably used a package manager before. If I say apt-get update does that ring a bell?

This command tells APT to update all versions of packages installed. APT (Advanced Packaging Tool) is a package manager embedded very widely as standard on Linux operating systems. To install a package, you can for example enter the command apt-get install <package>.

Programming language

Most programming languages can embed their own with package managers, either natively or provided within their respective ecosystem.

Take for example npm, the default package manager for Node.js. We can also mention pip for Python, NuGet for C#, Composer for PHP, etc. Similar to APT, npm makes it easy to install packages using the npm install <package> command.

For this article, I decided to take npm as an example.
npm is indeed a very good support to highlight the advantages but also the disadvantages that a package manager can have.
The advantages and disadvantages listed in the following part are valid for all package managers.

npm is installed alongside Node.js. To reproduce these examples, [you only need to install Node.js here].

In four parts, we will see what are the main reasons for such an expansion of package managers to all layers of a computer system.

1. Ease of use and maintenance of packages

The main interest of a package manager is obviously to simplify the installation of dependencies external to our application. Before the rise of npm in January 2010, the dependencies of a JavaScript application were mostly installed manually. By "manual installation" I mean:

downloading a zip archive from a remote server
unzipping the archive in the project
manual referencing of the installed version, and this with each update of a dependency.

With a package manager like npm, we therefore benefit from:

Simplified installation of a package npm install <package>
The simplified update of a package npm update <package>
The simplified removal of a package npm uninstall <package>

The packages are installed in a node_modules folder adjacent to the application and which is entirely managed by npm. All packages located in the node_modules folder can be directly imported from the application.

In general, each programming language natively embeds its own module resolution management mechanism.

1.1. Install

In order for a package to be installed, we first need a name which is in most cases used as a unique identifier. Naming conventions can differ from one ecosystem to another.

$ npm install rxjs

With this command, the package manager will search within the registry for a package that has the name rxjs. When the version is not specified, the package manager will usually install the latest available version.

1.2. Use

// ECMAScript Modules (ESM)
import { of } from "rxjs";
// CommonJS
const { of } = require("rxjs");

The module systems integrated into the programming languages make it possible to import a library installed locally and sometimes remotely (like Go or Deno for example). In this case with Node.js, the package must be installed locally in a node_modules folder. With Node.js, the module resolution algorithm allows the dependency to be in a node_modules folder either adjacent to the source code or in a parent folder (which sometimes leads to an unexpected behavior).

2. Managing the consistency of installed packages

Now, let's dive into a little more detail on one very important aspect that a package manager must manage: state consistency between installed packages. So far, installing a package looks like a trivial task, which is just to automate downloading a package of a certain version and making it available in a conventional folder that the application has access to.

However this management of consistency between packages turns out to be relatively difficult and the way of modeling the dependency tree varies according to the ecosystems. Most of the time, we talk about a dependency tree, but we can also talk about a dependency graph, in particular a directed graph.

If you are not familiar with the concept of directed graphs, I invite you to read the series of articles I wrote about it on dev.to with examples in JavaScript.

The implementations of these data structures can be drastically different depending on the ecosystem of a package manager, but also between package managers of the same ecosystem (npm, yarn, pnpm for Node.js for example).

How to ensure that all developers share the same dependencies and therefore the same versions of each underlying library?

Still in the context of npm, let's take for example a very simple list of dependencies, expressed as an object in the package.json file:

package.json

{ 
  "dependencies": {
    "myDependencyA": "<0.1.0"
  }
}

This object describes a dependency of our project on the myDependencyA library downloadable from the npm registry. Semantic Versioning here constrains the version of the library to be installed (here lower than 0.1.0).

Semantic version management (commonly known as SemVer) is the application of a very precise specification to characterize the version of software. For more information on this subject, I invite you to take a look at the official specification https://semver.org/lang/fr/

In our case, by remaining on the classic <major>.<minor>.<patch> scheme, we express the possibility of installing all the versions of myDependencyA from "0.0.1" to "0.0.9". This therefore means that any version of the dependency that respects the range is considered valid. On the other hand, this also means that if a developer A installs the dependency at 2 p.m. and a developer B installs the dependency at 5 p.m., they may both not have the same dependency tree if ever a new version of myDependencyA is released in the meantime.

The npm dependency resolution algorithm will by default favor the installation of the most recent dependency that respects the semantic management described in the package.json. By specifying npm install myDependencyA, the most recent version of myDependencyA will be installed respecting the constraint "<1.0.0" (version strictly lower than "1.0.0").

The major problem with this approach is the lack of stability and reproducibility of the dependency tree from one computer to another, for example between developers or even on the machine used in production. Imagine that version 0.0.9 of myDependencyA has just been released with a bug and your production machine is about to do an npm install on Friday at 5:59 PM…

The very simple example is often referred as version drift. This is why a single description file (in this case package.json) cannot be enough to guarantee an identical and reproducible representation of a dependency tree.

Other reasons include:

using a different version of the package manager whose dependency installation algorithm may change.
publishing a new version of an indirect dependency (the dependencies of the dependencies we list in the package.json here), which would result in the new version therefore being uploaded and updated.
the use of a different registry which for the same version of a dependency exposes two different libraries at a time T.

Lockfiles to the rescue

To ensure the reproducibility of a dependency tree, we therefore need more information that would ideally describe the current state of our dependency tree. This is exactly what lockfiles do. These are files created and updated when the dependencies of a project are modified.

A lockfile is generally written in JSON or YAML format to simplify the readability and understanding of the dependency tree by a human. A lockfile makes it possible to describe the dependency tree in a very precise way and therefore to make it deterministic and reproducible from one environment to another. So it's important to commit this file to Git and make sure everyone is sharing the same lockfile.

package-lock.json

{
  "name": "myProject",
  "version": "1.0.0",
  "dependencies": {
    "myDependencyA": {
      "version": "0.0.5",
      "resolved": "https://registry.npmjs.org/myDependencyA/-/myDependencyA-0.0.5.tgz",
      "integrity": "sha512-DeAdb33F+"
      "dependencies": {
        "B": {
          "version": "0.0.1",
          "resolved": "https://registry.npmjs.org/B/-/B-0.0.1.tgz",
          "integrity": "sha512-DeAdb33F+"
          "dependencies": {
            // dependencies of B
          }
        }
      }
    }
  }
}

For npm, the basic lockfile is called package-lock.json. In the snippet above, we can precisely see several important information:

The version of myDependencyA is fixed at "0.0.5" so even if a new version is released, npm will install "0.0.5" no matter what.
Each indirect dependency describes its set of dependencies with versions that also describe their own versioning constraints.
In addition to the version, the contents of the dependencies can be checked with the comparison of hashes which can vary according to the registers used.

A lockfile therefore tries to accurately describes the dependency tree, which allows it to remain consistent and reproducible over time at each installation.

⚠️ But...

Lockfiles don't solve all inconsistency problems! Package managers implementations of the dependency graph can sometimes lead to inconsistencies. For a long time, npm's implementation introduced Phantom Dependencies and also NPM doppelgangers which are very well explained on the Rush.js documentation website (advanced topics that are out of the scope of this blog post).

3. Provision of distributed and transparent databases via open-source

Distributed registries

A package manager is a client that acts as a gateway to a distributed database (often called a registry). This allows in particular to share an infinite number of open-source libraries around the world. It is also possible to define company-wide private registries in a secured network, within which libraries would be accessible.

Verdaccio allows to setup a private proxy registry for Node.js

The availability of registries has greatly changed the way software is developed by facilitating access to millions of libraries.

Transparent access to resources

The other benefit of open-source package managers is that they most often expose platforms or tools that allow browsing through published packages. Accessing source code and documentation has been trivialized and made very transparent. It is therefore possible for each developer to have an overview or even to fully investigate the code base of a published library.

4. Security and integrity

Using open-source registries with millions of publicly exposed libraries is pretty convenient, but what about security?

It is true that open-source registries represent ideal targets for hackers: all you have to do is take control of a widely used library (downloaded millions of times a week) and inject malicious code into it, and no one will realize!

In this part, we will see the solutions implemented by package managers and registries to deal with these attacks and limit the risks.

Integrity safety for each installed package

Given that a package can be installed from any registry, it is important to implement verification mechanisms at the level of the content of the downloaded package, to ensure that no malicious code has been injected during the download, regardless of its origin.

For this, integrity metadata is associated with each installed package. For example with npm, an integrity property is associated with each package in the lockfile. This property contains a cryptographic hash which is used to accurately represent the resource the user expects to receive. This allows any program to verify that the content of the resource matches what was downloaded. For example for @babel/core, this is how integrity is represented in package-lock.json:

"@babel/core": {
   "version": "7.16.10",
   "resolved": "https://registry.npmjs.org/@babel/core/-/core-7.16.10.tgz",  
   "integrity": "sha512 pbiIdZbCiMx/MM6toR+OfXarYix3uz0oVsnNtfdAGTcCTu3w/JGF8JhirevXLBJUu0WguSZI12qpKnx7EeMyLA=="
}

Let's take a closer look at how integrity can drastically reduce the risk of injecting malicious code by hashing source code.

As a reminder:

We call hash function, a particular function which, from a datum supplied as input, calculates a digital fingerprint used to quickly identify the initial datum, in the same way as a signature to identify a person. Wikipedia

Let's take for example a simple case:

// my-library
function someJavaScriptCode() {
  addUser();
}

Let's imagine that this JavaScript code represents a resource that a user might want to download. Using the SHA1 hash function, we get the hash 7677152af4ef8ca57fcb50bf4f71f42c28c772be.
If ever malicious code is injected, the library's fingerprint will by definition change because the input (source code here) to the hash function will have changed:

// my-library
function someJavaScriptCode() {
  processMaliciousCode(); // this is injected, the user is not  expecting that
  addUser();
}

After injecting the malicious code, still using the same SHA1 hash function, we obtain 28d32d30caddaaaafbde0debfcd8b3300862cc24 as the digital fingerprint.
So we get as results:

Original code = 7677152af4ef8ca57fcb50bf4f71f42c28c772be
Malicious code = 28d32d30caddaaaafbde0debfcd8b3300862cc24

All package managers implement strict specifications on this approach to integrity. For example, npm respects the W3C's "Subresource Integrity or SRI" specification, which describes the mechanisms to be implemented to reduce the risk of malicious code injection.
You can jump directly here to the specification document if you want to dig deeper.

Security constraints at the author level

To strengthen security at the level of open-source packages, more and more constraints are emerging on the side of project authors and maintainers. Recently, GitHub, which owns npm, announced that it is forcing two-factor authentication (2FA) for contributors to the 100 most popular packages. The main idea around these actions is to secure resources upstream by limiting write access to open-source packages and identifying people more precisely.

It's important to also mention that there are tools that can be used to perform automatically scans and audits continuously.

Built-in tools

In order to automate the detection of vulnerabilities, many package managers natively integrate tools allowing to scan the installed libraries. Typically, these package managers communicate with databases that list all known and referenced vulnerabilities. For example, GitHub Advisory Database is an open-source database that references thousands of vulnerabilities across multiple ecosystems (Go, Rust, Maven, NuGet, etc) e.g. npm audit command uses this database.

Third-party tools

NodeSecure

At NodeSecure we are building free open source tools to secure the Node.js & JavaScript ecosystem. Our biggest area of expertise is in package and code analysis.

Here are some example of the available tools:

@nodesecure/cli, a CLI that allow you to deeply analyze the dependency tree of a given package or local Node.js project
@nodesecure/js-x-ray, a SAST scanner (A static analyser for detecting most common malicious patterns)
@nodesecure/vulnera, a Software Component Analysis (SCA) tool
@nodesecure/ci, a tool allowing to run SAST, SCA and many more analysis in CI/CDs or in a local environment

Snyk

Snyk is the most popular all-around solution for securing applications or cloud-based infrastructures. Snyk offers a free-tier with SAST and SCA analysis.

To ensure continuous detection of vulnerabilities, it is recommended to run scans each time packages are installed/modified.

Conclusion

There you go, you now know what issues are addressed and solved by package managers!

Package managers are complex tools that aim to make life easier for us as developers, but can quickly become problematic if misused.

It is therefore important to understand the issues they deal with and the solutions provided in order to be able to put into perspective several package managers of the same ecosystem. In the end, it's a tool like any other and it must mobilize thinking in the same way as when libraries/frameworks/programming languages are used.

Don't also forget to take into account security issues and use automated tools which can drastically reduce the attack surface!

NodeSecure Vuln-era

Thomas.G — Thu, 21 Jul 2022 08:15:29 +0000

😍 Logo and cover by our beloved medhi bouchard ❤️

Hello 👋,

Back for a little article about the rebranding of one of the NodeSecure tools: Vulnera (previously vuln, the vuln-era has begun!).

An opportunity for me to also write about this wonderful project that was born with the redesign of the back-end less than a year ago ⌚. If you don't remember I wrote an article:

Announcing new NodeSecure back-end

Thomas.G for NodeSecure ・ Sep 11 '21

#node #javascript #security

Don't wait and dive in 🌊 with me to discover this tool 💃.

What is Vulnera ? 👀

Vulnera is a package that allows you to programmatically fetch your Node.js project vulnerabilities from multiple sources or strategies:

NPM Audit (Github Advisory Database)
Sonatype OSS Index
deprecated Node.js Security WG Database
Snyk

📢 Feel free to push new sources (we have a guide on how to add/contribute one).

The code was originally designed for vulnerability management within the Scanner. Yet, its API is evolving with the objective of making it a full-fledged project.

import * as vulnera from "@nodesecure/vulnera";

const def = await vulnera.setStrategy(
  vulnera.strategies.NPM_AUDIT
);

const vulnerabilities = await def.getVulnerabilities(process.cwd(), {
  useStandardFormat: true
});
console.log(vulnerabilities);

Standard vulnerability format 👯

We have created a standard format to reconcile the different sources.

export interface StandardVulnerability {
  /** Unique identifier for the vulnerability **/
  id?: string;
  /** Vulnerability origin, either Snyk, NPM or NodeSWG **/
  origin: Origin;
  /** Package associated with the vulnerability **/
  package: string;
  /** Vulnerability title **/
  title: string;
  /** Vulnerability description **/
  description?: string;
  /** Vulnerability link references on origin's website **/
  url?: string;
  /** Vulnerability severity levels given the strategy **/
  severity?: Severity;
  /** Common Vulnerabilities and Exposures dictionary */
  cves?: string[];
  /** Common Vulnerability Scoring System (CVSS) **/
  cvssVector?: string;
  /** CVSS Score **/
  cvssScore?: number;
  /** The range of vulnerable versions */
  vulnerableRanges: string[];
  /** The set of versions that are vulnerable **/
  vulnerableVersions: string[];
  /** The set of versions that are patched **/
  patchedVersions?: string;
  /** Overview of available patches **/
  patches?: Patch[];
}

You can always use the original formats of each source of course 😊. We have implemented and exposed TypeScript interfaces for each of them.

Usage in Scanner 🔬

On the scanner we have all the necessary information because we go through the dependency tree 🎄. At the end of the process, we recover all vulnerabilities by iterating spec by spec within the hydratePayloadDependencies strategy method.

const {
  hydratePayloadDependencies,
  strategy
} = await vulnera.setStrategy(
  userStrategyName // SNYK for example
);
await hydratePayloadDependencies(dependencies, {
  useStandardFormat: true,
  path: location
});

payload.vulnerabilityStrategy = strategy;

The following diagram explains the overall behavior and interactions between the Scanner and Vulnera.

If you want to learn more about the Payload you can check the TypeScript interface here.

What's next ? 🚀

Some sources are more difficult to exploit than others (for NPM we use Arborist which simplifies our lives).

const { vulnerabilities } = (await arborist.audit()).toJSON();

However, we have to think and create mechanics to exploit sources like Sonatype 😨. This is required for API like getVulnerabilities().

Among the major subjects and ideas we are working on:

Create a private database to benchmark the sources between them (see #29).
Merging multiple sources in one (see #25).
Fetch vulnerabilities of a given remote package (with support for private registry like verdaccio). At the moment we only support the analysis of a local manifest or a payload of the scanner.

Credits 🙇

This project owes much to our core collaborator Antoine COULON who invested a lot of energy to improve it 💪.

Fun fact: its first contribution 🐤 on NodeSecure was also on the old version of the code Scanner that managed vulnerabilities.

But I don't forget individual contributions 👏

Mathieu Kahlaoui for adding the getVulnerabilities() API
Oleh Sych for adding Snyk strategy
Medhi for his work on the logo

NodeSecure / vulnera

Programmatically fetch security vulnerabilities with one or many strategies (NPM Audit, Sonatype, Snyk, Node.js DB).

The vuln-era has begun! Programmatically fetch security vulnerabilities with one or many strategies. Originally designed to run and analyze Scanner dependencies it now also runs independently from an npm Manifest.

Requirements

Node.js v22 or higher

Getting Started

This package is available in the Node Package Repository and can be easily installed with npm or yarn.

$ npm i @nodesecure/vulnera
# or
$ yarn add @nodesecure/vulnera

Usage example

import * as vulnera from "@nodesecure/vulnera";

await vulnera.setStrategy(
  vulnera.strategies.GITHUB_ADVISORY
);

const definition = await vulnera.getStrategy();
console.log(definition.strategy);

const vulnerabilities = await definition.getVulnerabilities(process.cwd(), {
  useFormat: "Standard"
});
console.log(vulnerabilities);

Available strategy

The default strategy is NONE which mean no strategy at all (we execute…

View on GitHub

Thanks 🙏 for reading me and see you soon for another article!

NodeSecure CLI v2.0.0

Thomas.G — Wed, 29 Jun 2022 09:48:03 +0000

Hello 👋,

I am writing this article with excitement and after several months of work. With the core team we are thrilled to announce that we are publishing a new version of the UI.🚀.

As you are reading these lines I am probably under the sun ☀️ of Tel Aviv for the NodeTLV conference where I will give a talk about NodeSecure and some other tools.

What an incredible journey 😍. Four years ago I was working on my tool alone 😥... But now more than a dozen developers are contributing to the project and I can only thank all of you for your precious support 🙏.

If you are new, then let me introduce you to the project

🐤 Getting started with NodeSecure

NodeSecure is an organization gathering a lot of individual projects that will allow you to improve the security and quality of your projects 💪. With our tools you can visually discover the dependencies you use on a daily basis and learn more about them 📚.

Our most notable project is:

NodeSecure / cli

JavaScript security CLI that allow you to deeply analyze the dependency tree of a given package or local Node.js project.

🐢 Node-Secure CLI 🚀

a Node.js CLI to deeply analyze the dependency tree of a given NPM package or Node.js local app

📜 Features

Run a static scan on every JavaScript files and sort out warnings (unsafe-regex, unsafe-import etc) and the complete list of required expr and statements (files, node.js module, etc.).
Return complete composition for each packages (extensions, files, tarball size, etc).
Packages metadata from the npm registry API (number of releases, last publish date, maintainers etc).
Search for licenses files in the tarball and return the SPDX expression conformance of each detected licenses.
Link vulnerabilities from the multiple sources like GitHub Advisory, Sonatype or Snyk using Vulnera.
Add flags (emojis) to each packages versions to identify well known patterns and potential security threats easily.
First-class support of open source security initiatives like OpenSSF Scorecard.
Generate security report (PDF).

🚧 Requirements

Node.js v22 or higher

💃 Getting

…

View on GitHub

How can you use it? It's easy, you just have to install globally the CLI with npm:

$ npm i @nodesecure/cli -g

# Analyze a remote package on the NPM Registry.
# Note: also work with a private registry like gitlab or verdaccio
$ nsecure auto fastify

# Analyze a local manifest (or local project).
# -> omit the package name to run it at the cwd.
$ cd /myproject
$ nsecure auto

We have many other projects and many opportunities for you to contribute. Feel free to join us on Discord to discuss.

👀 What's changed in v2.0.0 ?

A lot to be honest 😆. Our initial idea was simply to improve and complete the interface (We went a bit overboard I guess 😅).

One of the things that became problematic was the lack of space in the interface 😨. So we had to completely redesign the UX. I have to thank Medhi Bouchard, who spent dozens of hours designing UI on figma (Without him all this would have been much more difficult to achieve 💪).

Multiple views

This new interface offers several distinct views:

Home (global informations about the project you asked to analyze).
Network (where we are drawing the dependency tree).
Settings (which allows you to customize your experience with the tool)

Note: It is also possible to switch between each view with a keyboard shortcut (which corresponds to the capitalized character).

Home view

The home view is a replacement for the old Global stats button. We have been working to bring more attention to the information.

To summarize the information we find in this view;

Global stats on the project (direct vs indirect, size, downloads)
Licenses and Extensions
Authors
Global warnings (not visible in the screenshot since there is none).
Links to Github and NPM.

We plan to expand this view with even more information and really cool gadgets. We also want to bring more attention and information around the creators and maintainers.

🔧 Settings view

This is the new kid in the town. There is not much to customize yet but that will come with time.

One of the key ideas of NodeSecure is that each developer and maintainer can customize their experience with the tool.

Some of our warnings have a lot of false positives that is real, so you will be able to ignore them if you don't find them relevant.

Eventually the options will allow to make more clear-cut decisions like tagging a maintainer's library (which will be useful during incidents like the one with Faker.js or node-ipc).

🌎 Network view

We have slightly improved the network view and updated the colors for something more pleasant.

In version 1.4.0 of our Vis-network implementation, we have also implemented different theme for parent and child nodes (What you can see in the screenshot below).

Note: We have not abandoned the "Dark" theme. Eventually it will be possible to switch from a light to a dark theme in the settings.

🚀 New left pannel

We wanted to keep the spirit of the old interface where we could retrieve information about a package very quickly. However we want to avoid as much as possible the need to scroll to get the information.

No more popup 💃. All information is now directly accessible in this new panel.

This new design is divided into the following sub-panels:

Overview (Package informations, github stats, etc).
Files and size (with bundlephobia).
Scripts and Dependencies.
Threats and issues in JavaScript source.
Vulnerabilities.
Licenses conformance (SPDX).

There is also much more information than before. For example, I've been wanting to implement vulnerabilities in the interface for two years and it's now done:

Note: I remind you that we support multiple strategy for vulnerabilities like Sonatype or Snyk.

Scripts

This new version allows you to consult the scripts of a package. Really cool combined with the 📦 hasScript flag. Most supply chain attack uses a malicious script ... so it became important for us to be able to consult them in the UI.

Threats in source code

This version implements the latest release of JS-X-Ray which includes new features;

Detecting weak crypto algorithm (md5, sha1 ...).
Warnings now have a level of severity (like vulnerabilities).

There is still a lot of work to be done on the interface, especially to better visualize the faulty code. You will notice that the links to access NPM and Unpkg are now always present in the header.

Licenses conformance

The information is still the same, but the design is a little more enjoyable. We have also added a small tooltip if you want to know more about SPDX.

The title and file name are clickable. The first one will open the license page on the SPDX website and the second one the file itself on unpkg.

Others

We have slightly improved the short descriptions of the flags and they are now clickable (this will open the wiki directly to the relevant flag).

Also in the scripts & dependencies section you will find a show/hide button on the third-party dependencies.

Still the same behavior as in the old version, it will hide in the network all the children of the package.

New documentation/wiki

We have developed a brand new documentation-ui module that allows us to implement a wiki on any of our projects.

In this new version you can open the wiki by clicking on the button with the book icon on the right side of the screen. We now also have documentation on the warnings of our static analyzer JS-X-RAY accessible in the SAST Warnings pannel of the wiki.

👯 Credits

All this work is possible thanks to the different contributors and contributions they made those last few months.

Their simple presence, good mood and spirit were a source of inspiration and motivation for me. Thanks you very much ❤️

Conclusion

As always we move forward and evolve. We continue to work hard to improve security in the JavaScript ecosystem and we look forward to being joined by other developers with the same commitment.

Thanks for reading me and see you soon for another great story!

A technical tale of NodeSecure - Chapter 2

Thomas.G — Mon, 06 Jun 2022 19:24:48 +0000

Hello 👋,

I'm back at writing for a new technical article on NodeSecure. This time I want to focus on the SAST JS-X-Ray 🔬.

I realized very recently that the project on Github was already more than two years old. It's amazing how time flies 😵.

It's been a long time since I wanted to share my experience and feelings about AST analysis. So let's jump in 😉

💃 How it started

When I started the NodeSecure project I had almost no experience 🐤 with AST (Abstract Syntax Tree). My first time was on the SlimIO project to generate codes dynamically with the astring package (and I had also looked at the ESTree specification).

One of my first goals for my tool was to be able to retrieve the dependencies in each JavaScript file contained within an NPM tarball (By this I mean able to retrieve any dependencies imported in CJS or ESM).

I started the subject a bit naively 😏 and very quickly I set myself a challenge to achieve with my AST analyser:

function unhex(r) {
   return Buffer.from(r, "hex").toString();
}

const g = Function("return this")();
const p = g["pro" + "cess"];

const evil = p["mainMod" + "ule"][unhex("72657175697265")];
evil(unhex("68747470")).request

The goal is to be able to output accurate information for the above code. At the time I didn't really know what I was getting into 😂 (But I was passionate about it and I remain excited about it today).

I thank Targos who at the time submitted a lot of code and ideas.

To date the SAST is able to follow this kind of code without any difficulties 😎... But it wasn't always that simple.

🐤 Baby steps

One of the first things I learned was to browse the tree. Even for me today this seems rather obvious, but it wasn't necessarily so at the time 😅.

I discovered the package estree-walker from Rich Harris which was compatible with the EStree spec. Combined with the meriyah package this allows me to convert a JavaScript source into an ESTree compliant AST.

import { readFile } from "node:fs/promises";

import { walk } from "estree-walker";
import * as meriyah from "meriyah";

export async function scanFile(location: string) {
  const strToAnalyze = await readFile(location, "utf-8");

  const { body } = meriyah.parseScript(strToAnalyze, {
    next: true, loc: true, raw: true, module: true
  });

  walk(body, {
    enter(node) {
      // Skip the root of the AST.
      if (Array.isArray(node)) {
        return;
      }

      // DO THE WORK HERE
    }
  });
}

I also quickly became familiar with the tool ASTExplorer which allows you to analyze the tree and properties for a specific code.

As a beginner, you can be quickly scared by the size and complexity of an AST. This tool is super important to better cut out and focus on what is important.

I also had fun re-implementing the ESTree Specification in TypeScript. It helped me a lot to be more confident and comfortable with different concepts that were unknown to me until then.

At the beginning of 2021 I also had the opportunity to do a talk for the French JS community (it's one more opportunity to study).

😫 MemberExpression

JavaScript member expression can be quite complicated to deal with at first. You must be comfortable with recursion and be ready to face a lot of possibilities.

Here is an example of possible code:

const myVar = "test";
foo.bar["hel" + "lo"].test[myVar]();

Computed property, Binary expression, Call expression etc. The order in which the tree is built seemed unintuitive to me at first (and I had a hard time figuring out how to use the object and property properties).

Since i created my own set of AST utilities including getMemberExpressionIdentifier.

🚀 A new package (with its own API)

When NodeSecure was a single project the AST analysis was at most a few hundred lines in two or three JavaScript files. All the logic was coded with if and else conditions directly in the walker 🙈.

To evolve and maintain the project, it became necessary to separate the code and make it a standalone package with its own API 👀.

I wrote an article at the time that I invite you to read. It contains some nice little explanations:

JS-X-Ray 1.0

Thomas.G for NodeSecure ・ Mar 30 '20

#javascript #node #security #ast

The thing to remember here is that you probably shouldn't be afraid to start small and grow into something bigger later. Stay pragmatic.

Easy to write, hard to scale 😭

It's easy to write a little prototype, but it's really hard to make it scale when you have to handle dozens or hundreds of possibilities. It requires a mastery and understanding of the language that is just crazy 😵. This is really what makes creating a SAST a complicated task.

For example, do you know how many possibilities there are to require on Node.js? In CJS alone:

require
process.mainModule.require
require.main.require

I probably forget some 😈 (as a precaution I also trace methods like require.resolve).

But as far as I'm concerned, it's really what I find exciting 😍. I've learned so much in three years. All this also allowed me to approach the language from an angle that I had never experienced or seen 👀.

Probes

On JS-X-Ray I brought the notion of "probe" into the code which will collect information on one or more specific node. The goal is to separate the AST analysis into lots of smaller pieces that are easier to understand, document and test.

Very far from perfection 😞. However, it is much better than before and the team is now helping me to improve all this (by adding documentation and tests).

It was for JS-X-Ray 3.0.0 and at the time i have written the following article (which includes many more details if you are interested).

JS-X-Ray 3.0

Thomas.G for NodeSecure ・ Feb 28 '21

#node #javascript #security #opensource

VariableTracer

This is one of the new killer feature coming to JS-X-Ray soon. A code able to follow the declarations, assignment, destructuration, importating of any identifiers or member expression.

In my experience being able to keep track of assignments has been one of the most complex tasks (and I've struggled with it).

This new implementation/API will offer a new spectrum of tools to develop really cool new features.

const tracer = new VariableTracer().trace("crypto.createHash", {
  followConsecutiveAssignment: true
});

// Use this in the tree walker
tracer.walk(node);

This simple code will allow us, for example, to know each time the method createHash is used. We can use this for information purposes, for example to warn on the usage of a deprecated hash algorithm like md5.

Here an example:

const myModule = require("crypto");

const myMethodName = "createHash";
const callMe = myModule[myMethodName];
callMe("md5");

The goal is not necessarily to track or read malicious code. The idea is to handle enough cases because developers use JavaScript in many ways.

We can imagine and implement a lot of new scenarios without worries 😍.

By default we are tracing:

eval and Function
require, require.resolve, require.main, require.mainModule.require
Global variables (global, globalThis, root, GLOBAL, window).

✨ Conclusion

Unfortunately, I could not cover everything as the subject is so vast. One piece of advice I would give to anyone starting out on a similar topic would be to be much more rigorous about documentation and testing. It can be very easy to get lost and not know why we made a choice X or Y.

Thanks for reading this new technical article. See you soon for a new article (something tells me that it will arrive soon 😏).

NodeSecure - What's new in 2022 ?

Thomas.G — Mon, 07 Feb 2022 18:10:40 +0000

Hello 👋,

Back for a different article than usual. This is the opportunity for me to talk about the NodeSecure project and to tell you about what's new since the beginning of the year 💃.

The project has grown significantly and we are now several active contributors on the project 😍. This opens up great opportunities for the organization and our tools as a whole.

Above all, many thanks to all those who participate in this adventure 😘. If you also follow the project and want to contribute and learn, do not hesitate 🙌.

Release 1.0.0 🚀

We have moved and renamed the main project. It became necessary to bring the project into the org to allow everyone to discover our other tools.

Now available on the NodeSecure github under the cli name. The old package has been deprecated and the new release can be downloaded with the name @nodesecure/cli.

Changing the name was necessary. It all started with one tool but now NodeSecure is a family of tools, contributors 👯 etc.

This also marks the beginning of the first major release 🎉.

$ npm install -g @nodesecure/cli

NodeSecure / cli

JavaScript security CLI that allow you to deeply analyze the dependency tree of a given package or local Node.js project.

🐢 Node-Secure CLI 🚀

a Node.js CLI to deeply analyze the dependency tree of a given NPM package or Node.js local app

📜 Features

Run a static scan on every JavaScript files and sort out warnings (unsafe-regex, unsafe-import etc) and the complete list of required expr and statements (files, node.js module, etc.).
Return complete composition for each packages (extensions, files, tarball size, etc).
Packages metadata from the npm registry API (number of releases, last publish date, maintainers etc).
Search for licenses files in the tarball and return the SPDX expression conformance of each detected licenses.
Link vulnerabilities from the multiple sources like GitHub Advisory, Sonatype or Snyk using Vulnera.
Add flags (emojis) to each packages versions to identify well known patterns and potential security threats easily.
First-class support of open source security initiatives like OpenSSF Scorecard.
Generate security report (PDF).

🚧 Requirements

Node.js v22 or higher

💃 Getting

…

View on GitHub

And by the way: this new release include support for Workspaces with the cwd command 😎.

NodeSecure ci 📟

A remarkable work from Antoine who has been actively working on the project for a good month 💪. This will bring a whole new dimension to the NodeSecure project and meet to at least some needs long requested by developers.

He wrote an article to present the tool and explain how to set it up 👀, I recommend you to read it:

🔒 Make your JavaScript project safer by using this workflow

Antoine Coulon for NodeSecure ・ Feb 1 '22

#javascript #node #security #cicd

There is still work to do, don't hesitate to come and contribute to this beautiful project which promises a lot for the future.

NodeSecure / ci

NodeSecure tool enabling secured continuous integration

Secure Continuous Integration

Installation

This package is available in the Node Package Repository and can be easily installed with npm or yarn.

$ npm i @nodesecure/ci
# or
$ yarn add @nodesecure/ci

Getting Started

@nodesecure/ci brings together a set of tools to identify dependencies vulnerabilities and track most common malicious code and patterns.

Before going further, here is an overview of the available features depending on your project configuration:

Static Analysis	Compatibility
JavaScript	✅
TypeScript	❌

Static Analysis is powered by @nodesecure/js-x-ray and @nodesecure/scanner.

For now, TypeScript can't directly be analyzed on the fly. However as you might know, any transpiled TypeScript code is JavaScript code hence can be analyzed Moreover, it is recommended to launch the Static Analysis with a source code state as close as possible to the state of your production code (and before minification). In fact, you want to make sure that you are…

View on GitHub

NodeSecure preview

Working on security accessibility for developers within the JavaScript ecosystem is important to us.

This is why Tony Gorez has taken it upon himself to design the Preview project which will allow to scan online npm packages. We still have some difficulties to put it online but we are working on it.

The goal of the project is to highlight some of the benefits and metrics reported by the NodeSecure tools and why not make more developers sensitive to security subjects.

NodeSecure / preview

Scan your node packages in your browser!

Caution

This project is not maintained anymore. We plan to implement a search mode in the NodeSecure CLI

🕸 Preview

Light NodeSecure in browser

Find your package weaknesses!

⚡️ Features

This project aims to help newcomers to understand the benefits of NodeSecure

👩‍🚀 On demand analysis
🏋️‍♀️ Package size & dependency count
⛳️ Vulnerability flags
🕐 Browser caching
👑 Powered by NodeSecure/scanner

Contributing

First, install dependencies

$ npm i

Run the development server:

$ npm run dev

Open http://localhost:3000 with your browser to see the result.

Run e2e test:

First, install playwright.

$ npx playwright install

Then, run the tests ^^

$ npm run test:e2e

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Tony Gorez
💻 📖 👀 🐛

Gentilhomme
👀 🐛

im_codebreaker
💻

Charles Viterbo
💻 🐛

License

MIT

View on GitHub

NodeSecure authors

In light of the recent events with Marak Squares it is I think quite important to have some insight on the maintainers of the packages we use.

Detect Marak Squires packages with NodeSecure

Thomas.G ・ Jan 10 '22

#javascript #node #security

We must have better tools to warn developers in case of incident like Faker. But also to highlight these maintainers who also need funding.

This could also allow some developers to realize the dependence they have on certain projects and why not encourage them to contribute to help.

That's why we are working on a new package with Vincent Dhennin to optimize and fetch additional metadata for package authors.

NodeSecure / authors

DEPRECATED (replaced by @nodesecure/contact)

NodeSecure authors

Caution

This project (package) has been re-implemented/replaced in Scanner monorepo, here

Requirements

Node.js v18 or higher

Getting Started

This package is available in the Node Package Repository and can be easily installed with npm or yarn.

$ npm i @nodesecure/authors
# or
$ yarn add @nodesecure/authors

Usage example

import { extractAllAuthorsFromLibrary } from "@nodesecure/authors";

const flaggedAuthors = [{ name: "Blake Embrey", email: "hello@blakeembrey.com" }];

const authors = extractAllAuthorsFromLibrary(library, flaggedAuthors);
// Expect authors to be following this schema
// [
//   {
//     name: "Blake Embrey",
//     email: "hello@blakeembrey.com",
//     flagged: true,
//     packages: [
//       {
//         homepage: "https://github.com/blakeembrey/array-flatten",
//         spec: "array-flatten",
//         versions: "3.0.0",
//         isPublishers: false
//       },
//       {
//         homepage: "https://github.com/pillarjs/path-to-regexp#readme",
//         spec: "path-to-regexp",
//         versions: "6.2.0",
//         isPublishers: true
//       }
//   }
// ]

API

TBC

Contributors ✨

Thanks…

View on GitHub

Our goal is to implement these improvements in future releases of Scanner. I'm excited about this because personally I like to get to know the maintainers of the packages I use.

NodeSecure RC

We are working on adding a runtime configuration for our tools (especially the CI project).

import assert from "node:assert/strict";
import * as RC from "@nodesecure/rc";

const writeOpts: RC.writeOptions = {
  payload: { version: "2.0.0" },
  partialUpdate: true
};

const result = (
  await RC.write(void 0, writeOpts)
).unwrap();
assert.strictEqual(result, void 0);

This should improve the experience for many of our tools where we had a CLI with complex settings and commands or pseudo configuration within the project (like report).

That's it for this article. We continue to work and listen to your various feedbacks to improve our tools.

See you soon for another article 😉.

Best Regards,
Thomas

🔒 Make your JavaScript project safer by using this workflow

Antoine Coulon — Tue, 01 Feb 2022 18:41:03 +0000

The security issue

Have you ever been thinking about security in your JavaScript projects? No? Well, you should, because with new thousands of package published on npm everyday, vulnerabilities could come from your own code but also from your direct dependencies (node_modules).

Few months ago, coa npm library was used to steal users' personal data by injecting malicious code.
As a reminder coa was:

Downloaded approximately 9 million times per week

Used by about 5 million GitHub projects

And that's just one story among many others...

If you're using npm to download dependencies, you have probably already encountered this message:

After each npm install, npm runs an audit scan against your updated dependencies. Here, we have 79 vulnerabilities, coming from one or many dependencies. Each one represents a potential threat and should be fixed.

Where do these vulnerabilities come from? Basically, npm maintains a vulnerability Database which is updated on a daily basis. Many other databases exist, here is an exhaustive list about most popular open-source databases for the JavaScript ecosystem:

These ressources are great, but we are lazy developers focused on productivity and we want to automate that, so we don't have to manually check all databases at 8 am every day before processing new features.

The security solution

First things first, I want to warn you about the fact that there is no silver bullet for security concerns.

If security were all that mattered, computers would never be turned on, let alone hooked into a network with literally millions of potential intruders. Dan Farmer, pioneer in the development of vulnerability scanners for Unix operating systems and computer networks.

Nevertheless, you can drastically reduce the amount of vulnerabilities by using tools that can be easily integrated with your projects.
However most of the time these tools are not open-source hence not for free use.

NodeSecure Continuous Integration

NodeSecure is an open source organization that aims to create free JavaScript security tools. Our biggest area of expertise is in npm package and code analysis.

To see more, read these NodeSecure series, written by Thomas @fraxken, founder of the GitHub organization.

What is @nodesecure/ci

@nodesecure/ci brings together a set of tools to identify dependencies vulnerabilities and track most common malicious code and patterns using Static Code Analysis and Vulnerabilities Analysis

If your project (custom configuration is available) passes all security checks, the process exit with no error code otherwise, it fails.

Here is a preview:

How to use

- GitHub Action

If you use GitHub Actions, you have a very straightforward way to add the official NodeSecure ci-action action to your workflow:

workflow.yaml

steps:
      - uses: actions/checkout@v2
      - uses: NodeSecure/ci-action@v1

Now your source code and its dependencies will be automatically analyzed, ironically without even adding new dependencies to your projects. That's also a perfect fit if your tech lead doesn't want you to add new dependencies (node_modules already heavier than the universe).

- Node.js Script

Install the @nodesecure/ci package and start using the entry script node_modules/.bin/nsci

As well as for the GitHub Action, you can provide a custom configuration through CLI arguments.

First, reference the binary script in the package.json

{
   "scripts": {
       "nsci": "nsci"
   }
}

Then start it providing different arguments (all can be used at once, by the way):

$ npm run nsci -- --directory=/Users/user1/myproject
$ npm run nsci -- --strategy=npm
$ npm run nsci -- --vulnerability=all
$ npm run nsci -- --warnings=error
$ npm run nsci -- --reporters=console

- Module API

@nodesecure/ci exposes its pipeline runner as an API to allow use in any other combined workflow.

import { runPipeline } from "@nodesecure/ci";

const optionsExample = {
    directory: process.cwd(),
    strategy: "node",
    vulnerabilities: "all",
    warnings: "error",
    reporters: ["console"]
}

await runPipeline(optionsExample);
// => the process can either exit with error code (1) 
// or no error code (0), depending on the pipeline status.

That's it, now you have no more excuses not to practice DevSecOps =)

Any feedback on @nodesecure/ci is welcome, the library is just getting started.

Feel free to reach me on GitHub @antoine-coulon

Thanks for reading.

NodeSecure v0.9.0

Thomas.G — Tue, 07 Dec 2021 17:15:51 +0000

Hello 👋,

After more than ten long months of work we are finally there 😵! Version 0.9.0 has been released on npm 🚀.

This is a version that required a lot of effort. Thank you to everyone who contributed and made this possible 🙏.

So what are the features of this new release v0.9.0? This is what we will discover in this article 👀.

For newcomers you can learn more about NodeSecure here or by reading the series.

V0.9.0 💪

This new version uses the new back-end and especially version 3 of the scanner.

ESM instead of CJS

This is a choice we explained in a previous article. This version has been completely rewritten in ESM.

We also made the choice to abandon Jest which causes too many problems 😟. We now use tape.

Better CLI

All commands are now separated by file and the bin/index.js file has been cleaned of all unnecessary code.

We are also working on adding UT for each command (which should avoid regressions and allow better contributions).

New front-end network management

This release heavily improves the front-end code with the addition of a package dedicated to vis-network management.

NodeSecure / vis-network

NodeSecure vis.js network front-end module

Vis-network

NodeSecure Vis.js network front-end module.

Getting Started

This package is available in the Node Package Repository and can be easily installed with npm or yarn.

$ npm i @nodesecure/vis-network
# or
$ yarn add @nodesecure/vis-network

Usage example

// Import Third-party Dependencies
import { NodeSecureDataSet, NodeSecureNetwork } from "@nodesecure/vis-network";

document.addEventListener("DOMContentLoaded", async () => {
  const secureDataSet = new NodeSecureDataSet();
  await secureDataSet.init();

  new NodeSecureNetwork(secureDataSet);
});

API

Scripts

The project scripts are used for those who want to test the code.

npm start to start an httpserver from ./dist
npm run build to build the ./example with esbuild.

Note: The start command run the build command before launching the http server.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Gentilhomme
💻 📖

…

View on GitHub

This should also allow us to migrate more easily to D3.js in 2022 🚀.

Better resolver support

The new version of the scanner has support for github: and git: spec.

The scanner is now able to analyze the following dependencies:

"dependencies": {
  "zen-observable": "^0.8.15",
  "nanoid": "github:ai/nanoid",
  "js-x-ray": "git://github.com/NodeSecure/js-x-ray.git",
  "nanodelay": "git+ssh://git@github.com:ai/nanodelay.git",
  "nanoevents": "git+https://github.com/ai/nanoevents.git"
}

Better payload structure

The structure of JSON has been improved to be more consistent (especially on the management of versions by dependency).

The latest version of the scanner also corrects many inconsistencies in the management of authors and maintainers.

"author": {
  "name": "GENTILHOMME Thomas",
  "email": "gentilhomme.thomas@gmail.com"
},
"publishers": [
  {
    "name": "fraxken",
    "email": "gentilhomme.thomas@gmail.com",
    "version": "2.2.0",
    "at": "2021-11-11T18:18:06.891Z"
  }
],
"maintainers": [
  {
    "name": "kawacrepe",
    "email": "vincent.dhennin@viacesi.fr"
  },
  {
    "name": "fraxken",
    "email": "gentilhomme.thomas@gmail.com"
  },
  {
    "name": "tonygo",
    "email": "gorez.tony@gmail.com"
  }
]

Brand new vulnerabilities management

We have already presented it, but now we use our own package that allows to recover vulnerabilities using several strategies (Security WG, NPM Audit etc..).

NodeSecure / vulnera

Programmatically fetch security vulnerabilities with one or many strategies (NPM Audit, Sonatype, Snyk, Node.js DB).

Requirements

Node.js v22 or higher

Getting Started

This package is available in the Node Package Repository and can be easily installed with npm or yarn.

$ npm i @nodesecure/vulnera
# or
$ yarn add @nodesecure/vulnera

Usage example

import * as vulnera from "@nodesecure/vulnera";

await vulnera.setStrategy(
  vulnera.strategies.GITHUB_ADVISORY
);

const definition = await vulnera.getStrategy();
console.log(definition.strategy);

const vulnerabilities = await definition.getVulnerabilities(process.cwd(), {
  useFormat: "Standard"
});
console.log(vulnerabilities);

Available strategy

The default strategy is NONE which mean no strategy at all (we execute…

View on GitHub

This is just the beginning and I think it will soon be a fully featured project. Among the new features there is a new standard format dedicated for NodeSecure:

export interface StandardVulnerability {
    id?: string;
    origin: Origin;
    package: string;
    title: string;
    description?: string;
    url?: string;
    severity?: Severity;
    cves: string[];
    cvssVector?: string;
    cvssScore?: number;
    vulnerableRanges: string[];
    vulnerableVersions: string[];
    patchedVersions?: string;
    patches?: Patch[];
}

Trojan source detection with JS-X-Ray 4.2.0

The new backend implements the version 4 of JS-X-Ray. In this latest release we added a warning for Trojan source.

Documentation and tests

A lot of effort has been put into adding documentation and unit testing to all of the projects.

There is still a long way to go to make this even more accessible and you are welcome to help us.

What's next ?

We are now working as a group on different topics. We have many ongoing projects/subjects:

Specification of a configuration file for our projects.
Better analysis and identification of authors and maintainers. See NodeSecure/authors.
Creating new tools to be executed in CI.
Working on the next Web UI (TypeScript + Catalyst).

Conclusion 🙏

We should be able to produce more frequent releases until the new UI comes.

Thanks again to the core contributors of the project without whom we would not have arrived here today!

See you soon for the release v0.10.0 💃.

A technical tale of NodeSecure - Chapter 1

Thomas.G — Mon, 22 Nov 2021 13:20:10 +0000

Hello 👋

I have been working on the NodeSecure project for almost three years now 😵. I have personally come a long way... At the beginning I didn't know much about the field in which I started 🐤.

That's why I thought that writing articles about "some" of the technical difficulties and the tools I used could be valuable 🚀.

I will try to make articles that focus on one aspect 🎯. Let's get started 💃.

🔍 Fetching the dependency tree

One of the first challenges I had to solve was how to get the dependency tree and all the information attached to the packages.

My first instinct was to work with the public API of the npm registry.
This sounds like a very good idea, but you will soon run into a set of problems (cache, private registry etc..).

What I wanted to do has already been implemented in the package named pacote.

Note: Arborist did not exist yet. I will come back to this in a future article. The first versions of NodeSecure did not support the analysis of a local project anyway.

Pacote

As its README suggests, Pacote is a library that allows you to retrieve various data for a given package. To be more precise:

A package manifest (A manifest is similar to a package.json file. However, it has a few pieces of extra metadata, and sometimes lacks metadata that is inessential to package installation.)
A packument (A packument is the top-level package document that lists the set of manifests for available versions for a package.)
A tarball (The archive containing the package itself with the published files)

These terms are really important and are explained in the pacote README.

Note: There is a package with the type definitions @npm/types.

In the NodeSecure/scanner these methods are used at different stages of the analysis. When we browse the dependency tree for example we use the manifest() method with the range version (or specifier) of the package.

await pacote.manifest(gitURL ?? packageName, {
  ...NPM_TOKEN,
  registry: getLocalRegistryURL(),
  cache: `${os.homedir()}/.npm`
});

The library allows you to manage a whole set of things quite quickly without too much difficulty 💪.

Note that in the above code there is a notion of Git URL 👀.

🔬 Dependency resolution

You are probably used to see SemVer versions or ranges within your package.json. Quite similar to this:

"dependencies": {
    "@nodesecure/flags": "^2.2.0",
    "@nodesecure/fs-walk": "^1.0.0",
    "@nodesecure/i18n": "^1.2.0",
    "@nodesecure/js-x-ray": "^4.1.2",
    "@nodesecure/npm-registry-sdk": "^1.3.0"
}

But there are many other ways to install/link a dependency within a package.json 😲:

One of the advantages of pacote is that it handles most of these resolutions for you 😎. I discovered all this while working on the subject (because I had never dealt with those types of resolutions).

If you want to be able to spot them here is a regular expression:

if (/^([a-zA-Z]+:|git\+|\.\\)/.test(version)) {
  // Version with custom resolution
}

This also explains why in NodeSecure we have a "hasCustomResolver" flag allowing quick identification of packages using resolutions to dependencies that diverge from the usual.

Pacote also exposes a resolve() method:

import pacote from "pacote";

const tarURL = await pacote.resolve("@slimio/is@^1.0.0");

It resolve a specifier like foo@latest or github:user/project all the way to a tarball url, tarball file, or git repo with commit hash.

📦 Download and extract tarball

One of the steps is to retrieve the package on the local system to be able to analyze it and retrieve a set of information.

const spec = ref.flags.includes("isGit") ?
  ref.gitUrl : `${name}@${version}`;

await pacote.extract(spec, dest, {
  ...NPM_TOKEN,
  registry: getLocalRegistryURL(),
  cache: `${os.homedir()}/.npm`
});

The package will be extracted into a temporary directory generated when the scanner is launched.

Note: see fs.mkdtemp

Once the extraction is finished, we will retrieve the information we need:

Files, extensions, size on disk etc..
Execute NodeSecure/JS-X-Ray on each JavaScript files.
Fetch licenses and retrieve their SPDX conformance.

We will dig deeper into the steps of static code analysis in a future article.

😈 It can't be that simple

In all this there are things quite complex to manage:

Same packages but with different "range" of versions 🎭.
Ensure the integrity of the links (relations) between packages.

The first one is hard because most of the time we are dealing with SemVer range and not with the EXACT version of the package. There is quite a bit of connection here with how npm handles conflict during installation (also how npm algorithms pick the right manifest).

I think I probably still lack some vision and experience on the subject. The current code is probably quite heavy too.

Today the cwd API of the Scanner use Arborist. For the from API i would like to avoid having to deal with a packument.

For the second one it is mainly a problem with the behaviour of the walker that will browse asynchronously the tree. We must therefore avoid that a package already analyzed is taken into account again. The problem with this is that we will be missing relationship links between some packages in the tree.

The current scanner solves the problem by going through all the dependencies one last time to create the missing link.

for (const [packageName, descriptor] of payload.dependencies) {
  for (const verStr of descriptor.versions) {
    const verDescriptor = descriptor[verStr];

    const fullName = `${packageName}@${verStr}`;
    const usedDeps = exclude.get(fullName) ?? new Set();
    if (usedDeps.size === 0) {
      continue;
    }

    const usedBy = Object.create(null);
    const deps = [...usedDeps].map((name) => name.split(" "));
    for (const [name, version] of deps) {
      usedBy[name] = version;
    }
    Object.assign(verDescriptor.usedBy, usedBy);
  }
}

✨ Conclusion

That's it for this article where we have explored a little bit the difficulties around going through the dependency tree.

If you like the concept don't hesitate to like and share.

🙏 Thanks for reading and see you soon for a new article.

Announcing new NodeSecure back-end

Thomas.G — Sat, 11 Sep 2021 17:40:34 +0000

Hello 👋

In the last article of the series I announced the future of NodeSecure. Well, we have just finished rebuilding our back-end 😲 (or at least a first version of it).

So what are the particularities of this new back-end? This is what we will discover in this article 👀.

But first let me make an introduction for the newcomers.

What is NodeSecure ❓

NodeSecure is an open source organization that aims to create free JavaScript security tools. Our biggest area of expertise is in npm package and code analysis.

Our most notable projects are:

Nsecure
JS-X-Ray - SAST Scanner
Report - HTML & PDF security report

The main project is a CLI that will fetch and deeply analyze the dependency tree of a given npm package (Or a local project with a package.json) and output a .json file that will contain all metadata and flags about each package.

The CLI is able to open the JSON and draw a Network of all dependencies (UI and emojis flags will help you to identify potential issues and security threats).

More information on our Governance page.

New back-end 🚀

Moving everything to the NodeSecure github org 🏠

All packages have been moved to the github organization. You will notice that we have a nice new logo ✨ (created by Tony).

This should make it simple to implement a new set of tools and collaborate more effectively. The integration of new maintainers should also be greatly simplified.

Moving to Node.js 16 and ESM

One of the major choices was to use ESM instead of CJS. Many maintainers like Sindresorhus made the choice to switch to ESM which prevented us from updating some of our packages 😭.

There are still a lot of things that are not stable, but we are convinced that it is the right choice for the future of our tools 💪.

Knowing that we still have time before completely finalizing the version 1 we also made the choice to have a limited support to the next LTS of Node.js.

New segmentation and packages 📦

We have segmented the back-end into a multitude of packages. That makes them reusable in other tools.

It will also greatly improve the quality of documentation and testing 💎.

name	description
scanner	⚡️ A package API to run a static analysis of your module's dependencies.
vuln	NPM Audit, Snyk and Node.js Security WG vulnerability strategies built for NodeSecure.
flags	NodeSecure security flags 🚩 (configuration and documentation)
i18n	NodeSecure Internationalization
npm-registry-sdk	Node.js SDK to fetch data from the npm API.

And there is still a lot more to discover (fs-walk, sec-literal , npm-tarball-license-parser etc).

Scanner API 🔬

Even though we now have a dedicated package the API has not changed.

import * as scanner from "@nodesecure/scanner";
import fs from "fs/promises";

// CONSTANTS
const kPackagesToAnalyze = ["mocha", "cacache", "is-wsl"];

const payloads = await Promise.all(
  kPackagesToAnalyze.map((name) => scanner.from(name))
);

const promises = [];
for (let i = 0; i < kPackagesToAnalyze.length; i++) {
  const data = JSON.stringify(payloads[i], null, 2);

  promises.push(fs.writeFile(`${kPackagesToAnalyze[i]}.json`, data));
}
await Promise.allSettled(promises);

The PDF & HTML report project has been updated to use this new back-end.

Team and contributors 👯

We are integrating Vincent Dhennin as a new maintainer. His help and contributions have been important and I can only thank him for this investment.

We are now three (including Tony Gorez and me).

I would like to thank the other contributors who participated a lot:

What's next ?

To be clear, the objective is to prepare a version 0.9.0 of NodeSecure implementing the new back-end (already in progress).

This will allow us to continually improve and update the back-end features. It will also now be easier to work on the evolution of the CLI.

We still don't have a roadmap or vision for the new interface. We will start working on it by October or November I think.

🙏 Thanks for reading and see you soon for an article on the next version of the CLI 😍.