DEV Community


Discussion on: How to Create a Web Crawler From Scratch in Python

donaldcr profile image
Info Comment marked as low quality/non-constructive by the community. View code of conduct
Info Comment hidden by post author - thread only visible in this permalink

great article and useful information! Thanks for that. I would also add that it's important to use proxy services when creating and testing your crawler so you would be sure that it works and can stay undetected by the websites you're gonna to crawl.

Some comments have been hidden by the post's author - find out more