DEV Community

Devil's Dev
Devil's Dev

Posted on Originally published at devilsdev.github.io

RAG Pipeline Utils v3.7.1: Security Fixes From a Code-Scanning Review, and a New npm Page

#node #ai #rag #javascript

v3.7.1 is a patch release with no breaking changes. It fixes a slow pattern in QueryPlanner that a long query could use to tie up a server, and two ways a crafted __proto__ key could write to Object.prototype in the CLI. It also gives the npm page a new README. Upgrading is recommended if you pass user queries to QueryPlanner.

npm install @devilsdev/rag-pipeline-utils@^3.7.1
Enter fullscreen mode Exit fullscreen mode

How these were found

Main had 52 open code-scanning alerts from CodeQL and OpenSSF Scorecard, most of them old. We went through every one of them against the code:

  • timed each slow-pattern alert;
  • reproduced each prototype-pollution alert;
  • checked whether the flagged code is actually shipped in the published package.

The result:

  • 19 are fixed in this release: the code issues below, plus Docker image pins and tighter workflow permissions.
  • 27 were false alarms, and each was dismissed on GitHub with its reason. Examples: modules that aren't in the published package, test fixtures with attack strings, and a random session label that was never a secret.
  • The 6 still open are repository settings (code review, branch protection, fuzzing) or an example that has no lockfile on purpose.

QueryPlanner and long runs of spaces

CodeQL flagged QueryPlanner's ; pattern as slow on long input. Timing the whole plan() call found a worse one it hadn't flagged:

  • The pattern for "…, and how …" clauses was cubic in the length of a run of spaces: 4,000 spaces took 21 seconds, and 20,000 didn't finish within a minute.
  • The "vs" and comparison patterns were quadratic.

QueryPlanner is exported from the main entry and from /edge, and it's given user queries, so one long query could hold a server's CPU.

The planner now reads each run of whitespace as one space before matching, and the clause boundaries can split a run only one way. On every query in the test suite and the edge cases we added, it splits exactly as before, and 200,000 characters of any shape we tried plan in under 100 ms. The one visible change: a sub-query has single spaces where the query had runs of whitespace.

__proto__ in the CLI

  • rag-pipeline plugin install. A plugin whose package.json declared "stage": "__proto__" passed the stage check, and registering it wrote to Object.prototype in the CLI process. Only the stage names the installer knows are accepted now.
  • rag-pipeline config set. The key path was checked, but a __proto__ key inside the JSON value was merged into Object.prototype. Such keys are skipped now, and the merge only goes into the config's own objects.

Both fixes come with regression tests that fail on 3.7.0.

A new README on npm

The npm page now speaks to developers meeting the library for the first time. It opens with the problem the library solves, an answer that says thirty vacation days when the source says twenty, and shows checkAnswer catching exactly that sentence. Then a pipeline in a dozen lines, and what else is in the box. Both examples run in CI, so they stay correct.

Also

  • The eval Markdown report escapes backslashes, so an error message containing \| no longer breaks a table cell.
  • Docker base images are pinned by digest, and Dependabot keeps those digests current.
  • The Security and Supply Chain workflows are read-only by default.

The full list is in the CHANGELOG.

Originally published on the rag-pipeline-utils blog.

Top comments (0)