DEV Community

OxyCast

Web Scraping: Another Block In The Wall | Episode 2

In the #2 episode of OxyCast, the host – Augustinas Kalvis (Software Developer at Oxylabs), and his special guest – Martynas Saulius (Python Developer at Oxylabs), explain the blocking process in-depth and what daily web scraping challenges even skilled developers encounter, and how to avoid them.*

In this episode you’ll learn:

  • Why is it essential to ensure web scraper’s reliability and scalability? How to do it?
  • How do websites detect bots?
  • What happens when you get blocked?
  • Which web scraping blocking methods are encountered most?
  • What are the most common ways to mitigate the blocking?

Listen to the #2 OxyCast episode on YouTube, Apple Podcasts, Spotify, and SoundCloud.

*Information provided is based on our clients' use of Oxylabs' innovative solutions and tools for scraping publicly available data.

Oxylabs is a premium proxy service provider that offers tools and resources for public data collection.

© 2022 Oxylabs. All rights reserved.

Episode source