Great suggestions Dinny. However I feel we should be gentle on sites and should not send too many requests per second. That is why I didn't feel the need of using these two libraries.
I have written another article where I have used docker cluster to scrape data at a very high speed. Although I was not able to achieve desired results.
Great suggestions Dinny. However I feel we should be gentle on sites and should not send too many requests per second. That is why I didn't feel the need of using these two libraries.
I have written another article where I have used docker cluster to scrape data at a very high speed. Although I was not able to achieve desired results.
pythoncircle.com/post/518/scraping...