What is it?
This is a zero-dependency and low-footprint library that allows you to pack a set of asynchronous tasks and execute them in a controlled way.
It allows you to execute those tasks in time-based intervals or in size-based chunks.
The library also allows you to indicate the order of execution of the packeted tasks (strict
or loose
).
It provides many other configurations you can leverage to better control the flow of the pack.
How do I install it?
You can install it by using the following command:
npm install async-task-packer
How can I use it?
Create a packer by calling the createPacker
function and pass it the configuration you want to use.
Then, wrap your async functions with the resulting function to add them to the pack.
Use-case #1 - Delayed logger
const packer = createPacker({
executionMethod: 'interval',
executionType: 'loose',
interval: 1000
});
const logger = (message) => {
const createdAt = new Date();
packer(() => {
sendLogToAnExternalApi(message, createdAt);
});
};
logger('Hello');
logger('World');
Use-case #2 - Analytics chunk-based packer
In this scenario, we want to send a batch of events to an external API.
We want to send the events in chunks of 10, but we also want to send the events after 10 seconds, even if the chunk is not full, to prevent losing important information.
We don't care about event ordering (loose
type) as we provide a createdAt
timestamp in the event payload and we can sort the events on the server side.
const packer = createPacker({
executionMethod: 'chunk',
executionType: 'loose',
chunkSize: 10,
maxChunkLifetime: 10*1000 // Using a debounce mechanism, the packer will execute the pack after 10 seconds, even if it's not full
});
const track = (event, ...args) => {
const createdAt = new Date();
packer(() => {
sendEventToAnExternalApi(event, createdAt, ...args);
});
};
track('user_login', { userId: 123 });
track('item_added_to_cart', { cartUUID: 'abc', itemId: 456 });
track('item_added_to_cart', { cartUUID: 'abc', itemId: 789 });
track('order_completed', { cartUUID: 'abc' });
track('user_logout', { userId: 123 });
(More examples in the GitHub Repo)
API
The createPacker
function accepts an object with the following properties:
-
executionMethod
(required): The execution method to use. It can be eitherchunk
orinterval
.- If you choose
chunk
as the execution method, you can also provide the following properties: -
chunkSize
(required): The size of the chunk to use. It must be a positive integer. -
maxChunkLifetime
(optional): The maximum lifetime of a chunk. It must be a positive integer. Defaults to600000
(10 minutes). -
unref
(optional): A boolean indicating whether the interval should be unrefed or not. Defaults tofalse
. (https://nodejs.org/api/timers.html#timers_timeout_unref) - If you choose
interval
as the execution method, you can also provide the following properties: -
interval
(required): The interval to use. It must be a positive integer. -
debounce
(optional): A boolean indicating whether the interval should be debounced or not. Defaults tofalse
. Setting it totrue
will cause the interval to be reset every time a new task is added to the pack. -
unref
(optional): A boolean indicating whether the interval should be unrefed or not. Defaults tofalse
. (https://nodejs.org/api/timers.html#timers_timeout_unref)
- If you choose
-
executionType
(required): The execution type to use. It can be eitherstrict
orloose
.- If you choose
strict
as the execution type, the packer will execute the tasks in the order they were added to the pack. - If you choose
loose
as the execution type, the packer will execute the tasks in parallel and will not enforce the order they were added to the pack.
- If you choose
To better control the pack flow, you can also pass the following properties:
-
expectResolutions
(optional): A boolean indicating whether the packer should expect the tasks to be resolved or not. Defaults tofalse
.- If set to
false
, the packer will consider the pack as resolved as soon as the tasks are completed, regardless of whether they are resolved or rejected. Setting this option to false still allows you to catch errors by providing anonCatch
method. - If set to
true
, the packer will consider the pack as resolved only if all the tasks are resolved. If one of the tasks is rejected, the packer will consider the pack as rejected and will call the packeronCatch
method if provided or will throw the error if noonCatch
method is provided.
- If set to
-
awaitAllTasks
(optional): A boolean indicating whether the packer should wait for all the tasks to be executed before resolving the pack. Defaults tofalse
.- If set to
false
, the packer will resolve the pack as soon as the tasks are executed. - If set to
true
, the packer will wait for all the tasks to be executed before resolving the pack. Please note that this option will immediately fill the pack with the tasks currently in the queue if conditions are met. It means a new pack could potentially be created and executed even if the previous one is not yet resolved.
- If set to
-
onCatch
: A function to be called when an error is thrown. It will be called with the error as the first argument. If not provided, the error will be thrown instead. -
onPackExecution
: A function to be called when a pack is executed. It will be called with the array of executed tasks (Promises yet to be resolved or rejected) as the first argument.
Considerations about chunk size
As of right now, if a chunk size is provided but not reached, the packer will not execute the tasks in the pack.
This means that if you provide a chunk size of 3 and you add 2 tasks to the packer, the packer will not execute the tasks until you add another task to the packer.
Please, keep this in mind when using the chunk
execution method.
Tests
You can run the tests by using the following command:
npm test
How does it work?
Glossary
task
An async function to execute.
pack
The pack is the current list of async tasks to be / being executed.
queue
A temporary queue used to store every async task provided.
Execution Steps
The library exposes a higher-order function that you can wrap your async functions with to put them in a temporary queue.
The library will then move items from the queue to the pack
based on the provided execution method (size limited for chunk
configuration or time-based for interval
configuration).
The library will execute the tasks in the queue according to the configuration that you pass to it.
Top comments (0)