DEV Community

Discussion on: “Multithreaded Python: slithering through an I/O bottleneck”

Collapse
 
rhymes profile image
rhymes

Great explanation Victoria!

The GIL is pretty much the microphone in a low-budget conference panel, except where no one gets to shout.

HAHAHAHAHA I can't stop laughing. Love the analogy.

Also a good example of how rich the Python standard library actually is :) I would have probably reached out for httpx without thinking about it but I love you stuck to the stdlib.

For a moment I wondered, as a thought experiment on the topic, if fetching batches of links with HTTP pipelining in threads (each thread pipelines N links) would be much slower than having each thread process one link at a time. Though now that I'm writing this I guess it's not a good idea as you'd need to group links by origin and complicate the threads quite a bit.