DEV Community

💻 Arpad Kish 💻
💻 Arpad Kish 💻

Posted on

Mastering Asynchronous Task Management in TypeScript with q-exec-ts

#node #api

Handling a massive influx of asynchronous tasks in Node.js can be challenging. Whether you are processing a large batch of API requests, dealing with data streams, or managing database operations, unbridled concurrency can quickly overwhelm your system or trigger third-party rate limits. This is where q-exec-ts, a TypeScript library created by @arpad1337, comes into play to elegantly solve this problem.

What is q-exec-ts?

q-exec-ts is a focused, lightweight TypeScript utility built around a QueuedExecutor class that enables developers to smoothly control task concurrency and throttle script executions. Under the hood, it extends the native Node.js events.EventEmitter class, providing an event-driven approach to tracking tasks in a queue.

By leveraging the QueuedExecutor, developers can queue up functions with their respective arguments, enforce a maximum number of concurrent running tasks, and inject a mandatory delay (in milliseconds) between starting each task.

Key Features and Architecture

The library is designed with structural simplicity and strict type safety in mind:

  • Strict Typing: The executor uses a custom PositiveInteger type constraint to strictly enforce positive integer values for setting max concurrency and delay.
  • The Delegate Pattern: Instead of passing one-off callbacks every time an item is pushed, you configure a QueuedExecutorDelegate<T> object. This interface requires a single exec method that automatically processes the typed arguments pushed into the queue buffer.
  • Built-in Throttling: The class constructor accepts a delay parameter. This acts as a throttle, utilizing timeouts behind the scenes to ensure subsequent tasks are not immediately fired before the specified delay window has elapsed.
  • Event-Driven Lifecycle: Because it is an EventEmitter, the queue emits useful lifecycle events. It emits a bufferEmpty event whenever the internal buffer runs dry, and a finished event when the input stream is explicitly closed and all pending executions have completed.

How to Use q-exec-ts

Integrating the queued executor into a codebase is relatively straightforward. First, you define the argument types your tasks will expect and implement the delegate:

import * as events from "events";
import { 
    QueuedExecutor, 
    QueuedExecutorDelegate, 
    QueuedExecutorEvents
} from "./QueuedExecutor"

type ArgsType = [string];

const delegate: QueuedExecutorDelegate<ArgsType> = {
    exec: async (...args: ArgsType): Promise<void> => {
        try {
            // Your async logic here
            await doSomethingAsync(args[0]);
        } catch (e) {
            console.error(e);
        }
    }
};
Enter fullscreen mode Exit fullscreen mode

Next, you instantiate the QueuedExecutor by specifying your desired concurrency limit, throttling delay (in milliseconds), and the delegate you created:

const executor = new QueuedExecutor<ArgsType>(
    maxConcurrency, 
    100, // throttling in ms
    delegate
);
Enter fullscreen mode Exit fullscreen mode

As tasks or data arrive, you seamlessly push them to the executor. The queue stores the parameters in a buffer and automatically runs them as slots become available based on the concurrency threshold:

executor.push(someString);
Enter fullscreen mode Exit fullscreen mode

Finally, when your source data stream is complete, you must notify the executor so it can gracefully wind down and trigger the terminal Finished event:

executor.inputStreamClosed(); // Signals no more items will be pushed
await events.once(executor, QueuedExecutorEvents.Finished);
Enter fullscreen mode Exit fullscreen mode

Conclusion

For developers grappling with API rate limits or system unreliability from concurrent background processes, q-exec-ts offers an elegant, drop-in solution. By combining strict TypeScript boundaries, the robustness of the native Node.js EventEmitter API, and configurable throttling, it handles the heavy lifting of queue coordination for you. The package is completely open-source and released under the permissive MIT License by Arpad K. (copyright 2025), making it a safe choice to embed within personal, commercial, or enterprise-scale projects.

https://github.com/arpad1337/q-exec-ts

Top comments (0)