Context
We had a platform running with Symfony where we needed to execute a lot of heavy background jobs daily.
Initially we used a bundle which handles our jobs and their execution.
With the growing user base, it had started to let us down.
We were getting more and more jobs that are stuck, hung, etc…
Solution
To solve this, i developed a new bundle that offers a batch processing system with the Docker engine.
All jobs are executed separately in docker containers.
This way is more viable as Docker handles the execution of the started containers.
Neat thing with this bundle is, you can pretty much execute anything in those containers.
For example, say you need a downloader in your symfony application, rather than implementing it in PHP, you could leverage JS Async and make a quick node.js script with something like bluebird and package it into a Docker image.
Monitoring UI
The bundle also offers a Monitoring user interface which you may choose to activate.
With this interface, you can:
— get an overview of your jobs
— inspect jobs
— stop jobs
— get usage info for each running job
Job details page
Dashboard page
Check out the bundle, see if it's what you needed.
Top comments (0)