It allows you to do so much with such a few lines of codes.
I consider Scrapy as a requests package under big steroïds.
The fact that you can handle parallelization, throttling, data filtering, and data loading in one place is very good. I am specifically fond of the autothrottle feature
However, Scrapy need some extension to work well with proxies and headless browser.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Scrapy is AWSOME !
It allows you to do so much with such a few lines of codes.
I consider Scrapy as a requests package under big steroïds.
The fact that you can handle parallelization, throttling, data filtering, and data loading in one place is very good. I am specifically fond of the autothrottle feature
However, Scrapy need some extension to work well with proxies and headless browser.