Problem Statement
We have a one GET endpoint which fetches documents from database, transforms and returns. There were few clients for which we have to fetch large number of documents which used to block the event loop thread of node. When event loop thread is blocked, it creases below problems:
- App becomes unresponsive to new requests.
- Liveliness probe fails and k8s restarts the pod.
- Add unpredictable delay to small and fast requests.
Restarts became very frequent and we can not directly add pagination and deprecate this legacy endpoint. So need to figure out a short term solution which requires less code changes to prevent the restart and improve the responsiveness of application.
Solution
There is a concept called partitioning. It is very simple, break your large synchronous processing into smaller tasks and add event loop yield in between of each task to let event loop work upon other requests in between. Below is the simple pseudocode which unblocks the event loop in between of each task (batch) and let it serve other requests.
// Return a Promise which resolves immediately. But event loop continue processing in next cycle which unblock the event loop and let it work upon other tasks as well.
function yieldToEventLoop(): Promise<void> {
return new Promise<void>((resolve) => setImmediate(resolve));
}
for (let i = 0; i < docs.length; i += smallBatchSize) {
const batch = docs.slice(i, i + smallBatchSize);
// Do your processing of batch here
output.push(...processBatch(batch));
await yieldToEventLoop();
}
Key Take Away
- It does not make processing faster and remove CPU cost.
- It improves responsiveness of the application.
- One heavy request can starve the entire Node process. Chunk + yield can keep the event loop alive and provide temporary relief.
If your server relies heavily on complex calculations, you should think about whether Node.js is really a good fit. Node.js excels for I/O-bound work, but for expensive computation it might not be the best option. Reference
Top comments (0)