Split your app into producer and consumer service. Consider a que between producer and consumer service like Kafka. It will really help to scale and failure of one service won't affect other.
A software engineer interested in solving real problems, developer productivity & learning languages for fun. Primarily working on Node.js, React & databases. Current Interest: Rustlang
Definitely, we can use it with an approach where the input file is attached & data pushed to a PubSub or a messaging queue then we can push the job to a different system altogether.
But that really depends on the specific use case though.
Split your app into producer and consumer service. Consider a que between producer and consumer service like Kafka. It will really help to scale and failure of one service won't affect other.
Definitely, we can use it with an approach where the input file is attached & data pushed to a PubSub or a messaging queue then we can push the job to a different system altogether.
But that really depends on the specific use case though.
I do achieved the same result using Pub/Sub. Specifically backed by Redis and Bull Queue -> check this out github.com/OptimalBits/bull