I'm a developer-turned-business owner who loves to explore the right tools for the job. I enjoy writing and documenting my journey. I use code as one of the tools to solve real problems.
I've tried a few libraries, extensions, and approaches, including:
ReactPHP components
AmpPHP components
Swoole PHP Extension
Asynchronous jobs on a Redis queue
Asynchronous jobs from the database
Multiple processing servers
They all worked, but only very temporarily. Take the jobs in the Redis queue for example: at a certain point, there were so many queued jobs that I was never able to work through all of them and the queue actually got bigger every day until it ran out of memory on day four.
I'd really like to revisit doing parallel processing in PHP though because it's much easier to implement something in the language the application is built with.
If I do find a nice solution to do everything in PHP, I'll definitely write about it. Finding a solution would be a great end of a very long journey and a great start for a new one!
That's interesting to know, it seem PHP memory limit could be tune if the issue is related.
There are many solutions in PHP which is often difficult for me to make the decisions, has lead my way to Go (from PHP (pure and Laravel) to Swift to Nodejs to Ruby to Crystal to Go), it's a long journey and Go’s minimalistic approach actually… works. I assume it could probably help to avoid out of memory if you could experiment it and my finding that Go consumes less memory and gain better performance in various benchmark found on the web.
I've tried a few libraries, extensions, and approaches, including:
They all worked, but only very temporarily. Take the jobs in the Redis queue for example: at a certain point, there were so many queued jobs that I was never able to work through all of them and the queue actually got bigger every day until it ran out of memory on day four.
I'd really like to revisit doing parallel processing in PHP though because it's much easier to implement something in the language the application is built with.
If I do find a nice solution to do everything in PHP, I'll definitely write about it. Finding a solution would be a great end of a very long journey and a great start for a new one!
That's interesting to know, it seem PHP memory limit could be tune if the issue is related.
There are many solutions in PHP which is often difficult for me to make the decisions, has lead my way to Go (from PHP (pure and Laravel) to Swift to Nodejs to Ruby to Crystal to Go), it's a long journey and Go’s minimalistic approach actually… works. I assume it could probably help to avoid out of memory if you could experiment it and my finding that Go consumes less memory and gain better performance in various benchmark found on the web.
surely Go is faster than PHP as well as any compiled language is often faster than script-based ones.
static typing languages like Go are also often faster than dynamic typing languages as well as the strong type system of Go also helps a lot.
Go is a more strong competitor for Java than PHP and Python IMHO.