DEV Community: Evgenii Kalkutin

The Node.js Performance Trap Most Developers Fall Into

Evgenii Kalkutin — Wed, 23 Apr 2025 17:41:33 +0000

The Illusion of Optimization

Many Node.js developers believe they're writing efficient code by using modern array methods. They praise the platform's non-blocking nature while unknowingly creating performance bottlenecks. The reality? Their approaches often backfire, leading to sluggish applications and wasted resources.

Consider this common pattern, an elegant one-liner you wrote:

const results = massiveArray.map(expensiveOperation);

While seemingly elegant, this approach carries significant drawbacks:

Memory Overhead - Creates an entirely new copy of your data in memory
Eager Execution - Processes everything immediately, regardless of need
Blocking Behavior - Ties up the event loop during large operations
Scalability Issues - Risks crashing when handling substantial datasets

In Node.js environments, these problems compound:

Strict default memory limits often trigger out-of-memory crashes
No built-in mechanism for gradual processing or load management
Complete lack of backpressure handling in synchronous operations

A Better Approach: Intelligent Data Processing

The good news is we have better tools at our disposal now. Instead of processing everything up front, modern JavaScript lets us work with data more intelligently through lazy evaluation. This approach only works with items when they're actually needed, which keeps memory usage steady even with massive datasets. It also plays nicely with streaming workflows where data flows continuously.

Generator functions provide a clean way to implement this pattern. Take this example:

function* processData(data) {
  for (const item of data) {
    yield transform(item);
    // Natural break point for event loop
  }
}

From an execution model perspective, generators implement cooperative multitasking natively. By yielding control at each iteration, they preserve the event loop's non-blocking characteristics while enabling deterministic interleaving of operations. This execution model provides several key benefits:

Native Backpressure Implementation: The pull-based iteration model creates natural flow control boundaries where downstream consumption rates dictate upstream production
First-Class Async Integration: The model extends cleanly to asynchronous contexts through async generators, maintaining consistency across sync/async boundaries
Stream Processing Foundation: Generators serve as the primitive building block for Node.js streams, enabling efficient pipeline composition
Memory-Safe Execution: The incremental processing model prevents memory spikes and GC pressure associated with batch operations

For asynchronous workflows, the same principles apply with async generators:

async function* processStream(dataStream) {
  for await (const item of dataStream) {
    yield await transformAsync(item);
  }
}

Evolving Your JavaScript Mindset

Transitioning from traditional array operations to modern iteration protocols requires more than just new syntax - it demands a fundamental shift in how we think about data processing. The table below contrasts these paradigms at a deeper level than just API differences:

🚫 Old Way	✅ Better Way	💡 Benefit
Eager processing	Lazy evaluation	Only works when needed
Copy all data	Process in-place	O(n) → O(1) memory complexity
Blocking	Non-blocking	No app freezes
One big batch	Gradual chunks	Handles huge data
Rigid	Flexible	Works in more situations

The JavaScript ecosystem has evolved - and so must your optimization strategies. While Array methods served their purpose, modern systems demand generator patterns that align with Node.js' non-blocking architecture and memory constraints.

Evgeny Kalkutin - JSNSD/JSNAD Certified specialist

Got your own horror story? Share it in the comments — best one gets a thread breakdown on my Telegram!

Building Universal npm Libraries: Supporting Both CJS and ESM

Evgenii Kalkutin — Tue, 15 Apr 2025 21:05:38 +0000

Why Dual Packages Matter

The JavaScript ecosystem currently operates with two module systems:

CommonJS (CJS) - The traditional Node.js system using require()
ES Modules (ESM) - The modern standard using import/export

This divide creates real challenges. For example, when popular libraries like chalk transitioned to ESM-only in version 5, many existing CommonJS projects faced compatibility issues. While ESM is the future, the reality is that numerous production systems still rely on CJS.

The Solution: Dual-Package Support

By building libraries that support both formats, we can:

Maintain backward compatibility
Support modern JavaScript workflows
Reduce ecosystem fragmentation
Provide a smoother migration path

Here's a straightforward approach to implement dual-package support.

Implementation Guide

Project structure

your-lib/
├── dist/                           # Generated output (added to .gitignore)
├── src/                            # Source files in TypeScript/ES6
│   ├── utils.ts                    # Library functionality
│   └── index.ts                    # Main entry point
├── package.json                    # Dual-package configuration
├── rollup.config.js                # Build setup
├── tsconfig.declarations.json      # TS declarations config 
└── tsconfig.json                   # TS base config

TypeScript Support

We use two tsconfig files for optimal compilation:

Base Config (tsconfig.json)**

{
  "compilerOptions": {
    "strict": true,
    "esModuleInterop": true,
    "target": "ESNext",
    "module": "Preserve",
    "moduleResolution": "bundler",
    "rootDir": "src"
  },
  "include": ["src/**/*.ts"]
}

Key features:

Handles the main transpilation
Outputs modern JavaScript (ESM by default)
Used by Rollup during build

Declarations Config (tsconfig.declarations.json)

{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "declaration": true,
    "emitDeclarationOnly": true,
    "outDir": "dist/types"
  }
}

Key features:

Generates type definitions (.d.ts files)
Runs separately from main build
Ensures clean type output without JS files

Build Configuration (`rollup.config.js`)

import typescript from '@rollup/plugin-typescript';

export default {
  input: 'src/index.ts',
  output: [
    {
      dir: 'dist/esm',
      format: 'esm',
      entryFileNames: '[name].mjs',
    },
    {
      dir: 'dist/cjs',
      format: 'cjs',
      entryFileNames: '[name].cjs',
    },
  ],
  plugins: [
    typescript({
      tsconfig: './tsconfig.json',
    }),
  ],
};

Key features:

Creates separate ESM (.mjs) and CJS (.cjs) builds
Uses TypeScript plugin for compilation
Maintains clean output structure

Configure `package.json`

The package.json file is crucial for dual-package support. Here are the key configuration aspects:

Module System Configuration:

{
  "type": "module",
  "main": "dist/cjs/index.cjs",
  "module": "dist/esm/index.mjs",
  "types": "dist/types/index.d.ts",
  "exports": {
    ".": {
      "require": "./dist/cjs/index.cjs",
      "import": "./dist/esm/index.mjs",
      "types": "./dist/types/index.d.ts"
    }
  }
}

It's critical to properly separate dependencies for build tools (Rollup, TypeScript, etc.)
devDependencies:

{
  "@rollup/plugin-typescript": "^12.1.2",
  "@types/node": "^22.14.1",
  "rollup": "^4.40.0",
  "tslib": "^2.8.1",
  "typescript": "^5.8.3"
}

Use dependencies only for packages your library actually uses at runtime.
Why this separation matters:

Installation Efficiency: npm/yarn won't install devDependencies for end users
Smaller Bundle Size: Prevents unnecessary packages from being included
Clear Dependency Documentation: Shows what's needed for building vs running
Security: Reduces potential attack surface in production

Best Practices:

Only include truly required packages in dependencies
Keep all build/test tools in devDependencies
Specify exact versions (or use ^) for important compatibility
Run npm install --production to test your runtime dependencies

Remember: Well-structured dependencies make your library more reliable and easier to maintain.

For a complete working example, check out this boilerplate project on GitHub:
👉 Dual-Package Library Example
It includes all the configurations discussed, so you can fork it or use it as a reference.

If you found this guide helpful:
⭐ Give it a star on GitHub to support the project!
💬 Leave a comment below with your thoughts or questions.
🔄 Share with others who might benefit from it.

Happy coding, and may your libraries work everywhere! 🚀
With respect,
Evgeny Kalkutin - kalkutin.dev

DEV Community: Evgenii Kalkutin

The Node.js Performance Trap Most Developers Fall Into

The Illusion of Optimization

A Better Approach: Intelligent Data Processing

Evolving Your JavaScript Mindset

Building Universal npm Libraries: Supporting Both CJS and ESM

Why Dual Packages Matter

The Solution: Dual-Package Support

Implementation Guide

Project structure

TypeScript Support

Build Configuration (rollup.config.js)

Configure package.json

Build Configuration (`rollup.config.js`)

Configure `package.json`