Assuming I have a project to create a 1000 wordpress websites using the top 1000 web hosting company in the world for research purpose (ttfb, latency etc), I will create a sample blog and write a post on each blog, then do some lighthouse and load balancing test on each site to compare them against each other.
- Docker (Ubuntu with Xvfb)
The best choice so far seemed AWS lambda or Google Cloud, but they have a pretty nice limitation of 9 to 15 minute per run and I will have to re run the instances again after that time.
While it's impossible, I ran several cost calculator for lambda, and it says it will cost me the following for 15 minutes even though I need it to run for an hour,
$15 for 1000 executions, 1cpu, 1024MB ram, 15 minutes => 900000 ms
I also cannot run them one after another, using same chrome window, nor I can stop them after 15 minutes.
If I create $5 digitalocean/linode instances to do same amount of task, I would need to pay $7 for that one hour, though managing those instances seems like another level of headache.
That leaves me with kubernetes, and docker swarm.
If I buy CPU optimized droplets, even the max 32vCPU package looks like I will need to buy 31 of them at once to get 1 CPU for each chrome instance. Linode has 48 cores package, only 20 nodes enough to hit the 1k limit, though spawning and managing them might come at a cost.
So, my questions is, how do you think you should scale 1k instances that must be running for 1 hour?