DEV Community

loading...

Discussion on: Scraping Facebook groups using Python? Avoid getting blocked with ProxyCrawl

Collapse
yellow1912 profile image
yellow1912

The problem is that most services like Facebook will try to block you if you go over rate limit. In the end you may still have to pay for proxy service like these to ensure you don't get yourself blocked. There are also free proxy out there but in my experience they are unreliable.

Collapse
ashishshetty profile image
AlphaSierra

That's why I said to use selenium. It fools servers to think that an actual user is browsing. Although this will not be feasable if your internet connection is too slow. Here checkout this project of mine where I have used selenium to scrape amazon: github.com/Shetty073/amazon-top-de...

Edit: Also instead of scraping please checkout facebook's API, you might get what you want easily without scraping.

Thread Thread
yellow1912 profile image
yellow1912

I'm not sure. Perhaps my use case is different. I scrap Instagram images for my users (scrap their own accounts). Since there are so many users, so many accounts, I always end up going over rate limit.