DEV Community

Discussion on: Someone could be scraping data from your website - how aware are you?

Collapse
jayjeckel profile image
Jay Jeckel • Edited on

I have to say, I really respect your hustle. Like an arms dealer selling weapons to both sides.

I've never had a problem with people scraping my sites, after all the sites are there to provide information and scraping is just another way people gather that information. I've always told clients, if your business relies on putting information on the web while not wanting competitors to have that information, then you don't really understand the internet and probably shouldn't be running a web based business in the first place.

As for blocking bots, well, at best it's rude and I tend to block and ignore sites that block my ability to use the information they provide.

Collapse
botmenot profile image
BotMeNot Author

Thanks for your feedback!