Secure Node.js File Uploads in Minutes with Pompelmi

Malware scanning, deep ZIP inspection, MIME/size guards, and optional YARA — with adapters for Express, Fastify, Koa, and examples for Next.js.

Repo: https://github.com/pompelmi/pompelmi
Docs / Demo: https://pompelmi.github.io/pompelmi/

Dropping a simple file upload endpoint into production can backfire fast:

• Users send ZIP bombs or polyglot files.
• Your app stores PHP shells, obfuscated JS, or a weaponized document.
• Backups and CDN spread the mess.

Pompelmi is a TypeScript toolkit that catches most of that before the file touches disk. It gives you:

• ✅ MIME & size allow‑lists (content‑based, not just extension)
• 🧰 Deep ZIP inspection (depth, file count, nested archives)
• 🧪 Optional YARA rules integration
• 🧯 Safe defaults + framework adapters
• 🧩 Works with in‑memory buffers, streams, or your existing upload stack

This post shows you how to wire it in 5–10 minutes.

---

Quick Start (Engine‑only, any framework)

Install the core engine with your package manager:

# pick one
npm i @pompelmi/engine
# or
yarn add @pompelmi/engine
# or
pnpm add @pompelmi/engine

Create a scanner instance with strict defaults:

// scanner.ts
import { createScanner } from "@pompelmi/engine";

export const scanner = createScanner({
  maxSizeBytes: 10 * 1024 * 1024, // 10 MB
  mimeAllowList: [
    "image/png",
    "image/jpeg",
    "application/pdf",
  ],
  // Block risky double extensions like .php.jpg, enforce sane names
  filenamePolicy: {
    allowUnicode: true,
    denyDoubleExtensions: true,
    maxLength: 120,
  },
  zip: {
    deepScan: true,
    maxDepth: 3,
    maxFiles: 128,
    maxExpandedBytes: 50 * 1024 * 1024, // 50 MB after unzip
    denyEncrypted: true,
  },
  yara: {
    enabled: false, // flip to true once rules path is configured
  },
});

Scan a Buffer before you save it:

import { scanner } from "./scanner";

async function handleUpload(name: string, mime: string, buf: Buffer) {
  const result = await scanner.scanBuffer(buf, {
    filename: name,
    mime,
  });

  if (!result.ok) {
    // result.reasons is an array of precise, dev‑friendly messages
    throw new Error(`Upload rejected: ${result.reasons.join(", ")}`);
  }

  // proceed to store the file (local, S3, GCS, etc.)
}

Prefer streams? Use scanner.scanStream(readable, { filename, mime }) to avoid loading large files in memory.

---

Express Middleware (drop‑in)

npm i @pompelmi/engine @pompelmi/express-middleware multer

// app.ts
import express from "express";
import multer from "multer";
import { createScanner } from "@pompelmi/engine";
import { pompelmi } from "@pompelmi/express-middleware";

const app = express();
const upload = multer({ storage: multer.memoryStorage() });

const scanner = createScanner({
  maxSizeBytes: 10 * 1024 * 1024,
  mimeAllowList: ["image/png", "image/jpeg", "application/pdf"],
  zip: { deepScan: true, maxDepth: 3, maxFiles: 128, denyEncrypted: true },
});

app.post(
  "/upload",
  upload.single("file"),
  pompelmi({
    scanner,
    // Optional: customize how to read the file from req
    fileResolver: (req) => ({
      buffer: (req as any).file?.buffer,
      filename: (req as any).file?.originalname,
      mime: (req as any).file?.mimetype,
    }),
    onReject: (res, reasons) => res.status(400).json({ ok: false, reasons }),
  }),
  async (req, res) => {
    // At this point the file is scanned and considered safe under your policy
    res.json({ ok: true });
  }
);

app.listen(3000, () => console.log("Listening on http://localhost:3000"));

Test it quickly:

curl -F file=@./tests/fixtures/clean.pdf http://localhost:3000/upload

---

Fastify Plugin

npm i @pompelmi/engine @pompelmi/fastify-plugin fastify-multipart

import Fastify from "fastify";
import multipart from "fastify-multipart";
import { createScanner } from "@pompelmi/engine";
import { pompelmi } from "@pompelmi/fastify-plugin";

const app = Fastify();
app.register(multipart);

const scanner = createScanner({ /* same config as above */ });

app.register(pompelmi, { scanner });

app.post("/upload", async (req, reply) => {
  const parts = req.parts();
  for await (const part of parts) {
    if (part.type === "file" && part.file) {
      const result = await scanner.scanStream(part.file, {
        filename: part.filename,
        mime: part.mimetype,
      });
      if (!result.ok) return reply.code(400).send(result);
      // store stream or buffer it
    }
  }
  return { ok: true };
});

app.listen({ port: 3000 });

---

Koa Middleware

npm i @pompelmi/engine @pompelmi/koa-middleware koa koa-body

import Koa from "koa";
import koaBody from "koa-body";
import { createScanner } from "@pompelmi/engine";
import { pompelmi } from "@pompelmi/koa-middleware";

const app = new Koa();
const scanner = createScanner({ /* config */ });

app.use(koaBody({ multipart: true }));
app.use(
  pompelmi({
    scanner,
    fileResolver: (ctx) => {
      const f = (ctx.request.files as any)?.file; // depends on your field name
      return f && {
        buffer: f.buffer,
        filename: f.originalFilename,
        mime: f.mimetype,
      };
    },
  })
);

app.use(async (ctx) => (ctx.body = { ok: true }));

app.listen(3000);

---

Next.js (API Route, App Router or Pages)

You don’t need a special adapter — call the engine in your route handler and reject bad files before writing to disk.

App Router (app/api/upload/route.ts)

import { NextRequest, NextResponse } from "next/server";
import { createScanner } from "@pompelmi/engine";

const scanner = createScanner({ /* config */ });

export async function POST(req: NextRequest) {
  const form = await req.formData();
  const file = form.get("file") as File | null;
  if (!file) return NextResponse.json({ ok: false, reason: "No file" }, { status: 400 });

  const arrayBuf = await file.arrayBuffer();
  const result = await scanner.scanBuffer(Buffer.from(arrayBuf), {
    filename: file.name,
    mime: file.type,
  });

  if (!result.ok) {
    return NextResponse.json({ ok: false, reasons: result.reasons }, { status: 400 });
  }

  // store to S3, etc.
  return NextResponse.json({ ok: true });
}

---

Enabling YARA (optional, powerful)

YARA lets you match byte patterns and heuristics to catch known malware families or suspicious code.

npm i @pompelmi/engine
# supply or mount your own rules directory

import { createScanner } from "@pompelmi/engine";

const scanner = createScanner({
  /* ...other config... */
  yara: {
    enabled: true,
    rulesPath: process.env.YARA_RULES_PATH || "/opt/yara/rules", // folder with .yar files
    timeoutMs: 1500, // guard against long‑running rules
  },
});

YARA rule example (very simple):

rule Suspicious_JS_Eval {
  meta:
    author = "you"
    description = "Flags use of eval in JS files"
  strings:
    $eval = "eval("
  condition:
    ext == "js" and $eval
}

Start small: over‑broad rules create false positives. Treat matches as rejections by default, but log and tune in staging first.

---

Deep ZIP Inspection (nested archives, bombs, encrypted zips)

ZIPs are a common attack vector. Pompelmi can:

• Walk nested archives up to a max depth
• Enforce file count and expanded size ceilings
• Deny encrypted archives outright (recommended)
• Reject forbidden MIME types inside the ZIP

const scanner = createScanner({
  zip: {
    deepScan: true,
    maxDepth: 3,
    maxFiles: 128,
    maxExpandedBytes: 50 * 1024 * 1024,
    denyEncrypted: true,
  },
  mimeAllowList: ["image/png", "image/jpeg", "application/pdf"],
});

When scanning a zip buffer, Pompelmi inspects each entry, enforcing your policy for every inner file.

---

Storing to S3 after Scan (safe flow)

1. Receive the file → 2) Scan in memory/stream → 3) Upload to S3

import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";
import { scanner } from "./scanner";

const s3 = new S3Client({ region: "eu-central-1" });

export async function saveToS3(name: string, mime: string, buf: Buffer) {
  const result = await scanner.scanBuffer(buf, { filename: name, mime });
  if (!result.ok) throw new Error(result.reasons.join(", "));

  await s3.send(new PutObjectCommand({
    Bucket: process.env.BUCKET!,
    Key: `uploads/${crypto.randomUUID()}-${name}`,
    Body: buf,
    ContentType: mime,
  }));
}

Prefer pre‑signed uploads? Terminate them on a dedicated ingress, scan there, then copy into the final bucket. Never let unscanned blobs reach your canonical bucket.

---

Security Hardening Checklist

• [ ] Maintain a strict allow‑list of MIME types and (optional) extensions
• [ ] Cap file size, ZIP depth, file count, expanded bytes
• [ ] Reject encrypted ZIPs (unless you truly need them)
• [ ] Normalize and validate filenames; block double extensions
• [ ] Scan before storage (even for presigned flows)
• [ ] Consider YARA for advanced detections
• [ ] Log reasons and sample metadata for tuning/forensics
• [ ] Run your upload worker under least privilege (no shell, no compilers)
• [ ] Periodically re‑scan cold storage with updated rules

---

Observability (what to log)

if (!result.ok) {
  console.warn("upload_rejected", {
    reasons: result.reasons,
    filename: name,
    mime,
    size: buf.length,
  });
}

Don’t log raw file content. Hashes (e.g., SHA‑256) are okay if you need deduplication.

---

Troubleshooting

I get false positives.

• Start with MIME/size + ZIP limits only. Add YARA later.
• Narrow your allow‑list and test with real‑world sample files.

Valid PDFs are blocked.

• Many online tools emit malformed MIME types. Inspect result.details.detectedMime (if available) and adjust your allow‑list.

ZIP bombs time out.

• Lower maxExpandedBytes, maxFiles, maxDepth and set a global timeoutMs if exposed by your version.

My framework stores files on disk first.

• Switch to in‑memory or stream processing (e.g., multer.memoryStorage() or Fastify streams) and scan before any write.

---

FAQ

Is Pompelmi production‑ready?
Yes, but your policy determines safety. Keep the allow‑list tight and test with your real traffic.

Do I need YARA?
Not to start. MIME/size/ZIP guards block a huge class of attacks. Add YARA for high‑risk surfaces.

Can it scan videos or large images?
Yes, up to your maxSizeBytes. Prefer streams for large uploads.

Does it quarantine files?
Pompelmi is a gatekeeper. Quarantine is a downstream storage concern — you can save rejected files to a separate bucket if needed.

---

What’s next?

• ⭐ Star the repo: https://github.com/pompelmi/pompelmi
• Read the docs/demo: https://pompelmi.github.io/pompelmi/
• Open an issue for feature requests or rulesets

---

Full Example Repo

A minimal Express example you can clone and run is in the monorepo. If you want a standalone quickstart, scaffold like this:

npm init -y
npm i express multer @pompelmi/engine @pompelmi/express-middleware typescript ts-node-dev @types/express @types/multer
npx tsc --init --esModuleInterop true --module commonjs --target es2020

src/scanner.ts — config from earlier.
src/app.ts — Express route from earlier.
Run with:

npx ts-node-dev src/app.ts

---

Final Notes

• Keep your allow‑list short; widen only when necessary.
• Treat uploads as untrusted until all checks pass.
• Revisit policies each quarter as your product evolves.

Stay safe and ship fast. 🚀