DEV Community

Ali
Ali

Posted on

1 1 1 1 1

Bypassing Bot Detection Software with Selenium in Python

(Use at your own risk, I'm not condoning anything illegal)

Websites such as Amazon uses bot detection software such as Imperva and FingerprintJS to block automated tools from interacting with their website. And they're rightly doing so to keep web scrapers and hackers away from stealing their precious data.

If they only knew...

Those security controls are fairly easy to bypass if you know what you're doing.

There are a few ways we can bypass those security controls:

  • Rotate proxies
  • Using cookies
  • Rotate User Agents and HTTP Headers
  • Remote Javascript signatures in your webdriver(we'll explore this one soon! 😎)
  • Avoiding patterns when interacting with websites

If you're using Selenium and you're getting blocked by automation software.

There's a little trick we can implement that can save us the hassle of setting up rotating proxies, headers or user agents.

Disable Selenium Automation Flag

But here's another simpler way that might do the trick - disabling Selenium's automation flag.

Selenium by default has automation flags enabled which might block you from accessing websites using anti bot software.

To bypass the block you'll have to disable those automation flags in your Selenium code, let's see how we can do that using a simple one liner.

First, let's call the webdriver.Chromeoptions() function and instantiate it to a variable called options.

This enable's us to add custom options to our webdriver:

options = webdriver.ChromeOptions

Now, let’s add the option disabling the automation flag by adding the following line to our Selenium code:

options.add_argument("--disable-blink-features=AutomationControlled")

This simple one-liner disables the automation flag and tells the website that you're not a bot.

Billboard image

Synthetic monitoring. Built for developers.

Join Vercel, Render, and thousands of other teams that trust Checkly to streamline monitor creation and configuration with Monitoring as Code.

Start Monitoring

Top comments (0)

Heroku

This site is built on Heroku

Join the ranks of developers at Salesforce, Airbase, DEV, and more who deploy their mission critical applications on Heroku. Sign up today and launch your first app!

Get Started

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay