DEV Community: Elen Simonian

Bing Proxies in 2026: How to Scrape Bing SERP Without Getting Banned

Elen Simonian — Fri, 26 Jun 2026 13:22:22 +0000

Scraping Bing at scale sounds straightforward until you hit your first block. Unlike Google, Bing is less discussed in the scraping community, so people underestimate how seriously Microsoft takes automated traffic. The detection logic is different from Google, the rate limits behave differently, and the mistakes that get you blocked are specific to Bing’s infrastructure.
This guide covers why Bing blocks scrapers, how its detection differs from Google, and how to build a Python scraping pipeline with rotating residential proxies that actually holds up under load.

Why Bing Blocks Scrapers

Bing evaluates every incoming request against several signals simultaneously:
• IP reputation and ASN classification (residential vs datacenter vs hosting)
• Request frequency per IP over rolling time windows
• User-agent consistency and header patterns
• Cookie and session behavior across requests
• JavaScript challenge completion (on certain result pages)

The most common failure point is IP type. Datacenter IPs are classified immediately by their ASN - Microsoft’s infrastructure knows which IP ranges belong to cloud providers. Sending SERP requests from an AWS or Azure IP range gets you blocked within a handful of requests, often silently returning empty or altered results rather than an explicit error.
Residential IPs avoid this because they belong to real ISP-assigned addresses on consumer networks. Bing treats them the same way it treats organic traffic. The second most common failure point is request pacing - even residential IPs get flagged if they fire queries at machine speed with no variation.

Bing vs Google: Key Detection Differences

If you have experience scraping Google SERPs, a few things about Bing are worth knowing before you start:
Rate limits are less aggressive early, harder later. Bing tends to allow more requests before the first block, but once an IP is flagged it tends to stay flagged longer than Google’s equivalent cooldown periods.
CAPTCHAs appear less frequently. **Bing more often returns degraded or empty results without triggering a visible CAPTCHA. This means you can silently collect bad data if you are not validating response content.
**User-agent matters more. Bing’s bot detection is more sensitive to user-agent strings than Google’s. Stale or clearly automated user-agents get flagged faster. Rotating realistic user-agents alongside IPs is more important here.
Geo-targeting affects results significantly. Bing returns noticeably different results based on the cc and setlangparameters and the IP’s geographic location. For accurate local SERP data, the proxy location and the query parameters need to match.

Setting Up the Proxy Layer

For Bing SERP scraping, NodeMaven Bing proxies are residential IPs with a 95%+ clean rate across a pool of 30M+ addresses. The key spec for SERP work is the IP Quality Filter, which screens IPs before assignment. Typical unfiltered pools have a large share of IPs with blacklist history - those get caught immediately on Bing.
Relevant NodeMaven specs for this use case:
• 30M+ residential IPs across 190+ countries
• IP Quality Filter: roughly 95% of pool at low fraud scores
• Sticky sessions up to 7 days (useful for multi-page result crawls)
• ZIP-level and city-level geo-targeting in 190+ locations
• Success rate: 99.54% average
• Pricing from $2.20/GB, trial at $3.50 for 750MB

The proxy connection format uses HTTP or SOCKS5 with username/password auth. For rotating residential proxies, each new session string generates a new IP. For sticky sessions, the same session string holds the same IP for the duration you set.

Python Scraping Pipeline: Basic Setup

Install the required packages:
pip install requests beautifulsoup4 fake-useragent

Basic rotating proxy request:
import requests
import time
import random
from bs4 import BeautifulSoup

PROXY_USER = "your_nodemaven_username"
PROXY_PASS = "your_nodemaven_password"
PROXY_HOST = "gate.nodemaven.com"
PROXY_PORT = "8080"

def get_proxy(country="us", session_id=None):
user = PROXY_USER
if country:
user += f"-country-{country}"
if session_id:
user += f"-session-{session_id}-sesstime-60"
proxy_url = f"http://{user}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}

def get_useragent():
agents = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0",
]
return random.choice(agents)

def scrape_bing(keyword, country="us", page=1):
first = (page - 1) * 10
params = {
"q": keyword,
"first": first,
"cc": country.upper(),
"setlang": "en",
}
headers = {
"User-Agent": get_useragent(),
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Upgrade-Insecure-Requests": "1",
}
proxies = get_proxy(country=country)

try:
    response = requests.get(
        "https://www.bing.com/search",
        params=params,
        headers=headers,
        proxies=proxies,
        timeout=15,
    )
    response.raise_for_status()
    return response.text
except requests.exceptions.RequestException as e:
    print(f"Request failed: {e}")
    return None

Parsing Bing SERP Results
Bing’s HTML structure is more stable than Google’s and less frequently updated. Organic results sit inside li elements with the class b_algo:
def parse_results(html):
if not html:
return []

soup = BeautifulSoup(html, "html.parser")
results = []

for item in soup.select("li.b_algo"):
    title_el = item.select_one("h2 a")
    snippet_el = item.select_one(".b_caption p")

    if not title_el:
        continue

    result = {
        "title": title_el.get_text(strip=True),
        "url": title_el.get("href", ""),
        "snippet": snippet_el.get_text(strip=True) if snippet_el else "",
    }
    results.append(result)

return results

def validate_response(html):
if not html:
return False
if len(html) < 5000:
return False
if "b_algo" not in html:
return False
return True
The validate_response function catches the silent failure case: Bing returning a short or empty page instead of a CAPTCHA. Always check that the response contains actual result elements before treating the request as successful.

*Adding Rate Control and Retry Logic
*
Request pacing is as important as IP rotation. Even clean residential IPs get flagged if they fire at machine speed.
import time
import random

def scrape_with_retry(keyword, country="us", page=1, max_retries=3):
for attempt in range(max_retries):
# Randomized delay between requests: 2 to 6 seconds
if attempt > 0:
wait = random.uniform(3, 8)
print(f"Retry {attempt}, waiting {wait:.1f}s")
time.sleep(wait)

    html = scrape_bing(keyword, country=country, page=page)

    if validate_response(html):
        return parse_results(html)

    print(f"Invalid response on attempt {attempt + 1}")

return []

def scrape_keyword_pages(keyword, country="us", pages=3):
all_results = []

for page in range(1, pages + 1):
    print(f"Scraping page {page} for: {keyword}")
    results = scrape_with_retry(keyword, country=country, page=page)
    all_results.extend(results)

    # Delay between pages: 3 to 7 seconds
    if page < pages:
        time.sleep(random.uniform(3, 7))

return all_results

*Geo-Targeting Bing Results by Location
*

Bing’s geo-targeting works through two mechanisms: the cc parameter in the query and the geographic location of the requesting IP. For accurate local results, both need to match.
NodeMaven supports targeting down to city and ZIP level. To get Bing results as they appear to a user in Seattle:
def get_proxy_geo(country="us", city=None, zip_code=None, session_id=None):
user = PROXY_USER + f"-country-{country}"
if city:
user += f"-city-{city.lower().replace(' ', '')}"
if zip_code:
user += f"-zip-{zip_code}"
if session_id:
user += f"-session-{session_id}-sesstime-30"
proxy_url = f"http://{user}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}"
return {"http": proxy_url, "https": proxy_url}

Seattle-specific results

proxies = get_proxy_geo(country="us", city="seattle", session_id="bing-seattle-01")
Pair this with cc=US&setlang=en in the query parameters and the response reflects what a Seattle user would see in Bing, including localized ad placements and map pack results.

*Scraping Bing vs Google: Practical Differences
*

*Running a Full Keyword Batch
*

Putting it together for a multi-keyword rank tracking run:
import json

keywords = [
"best project management software",
"project management tools 2026",
"asana vs monday comparison",
]

all_data = {}

for kw in keywords:
print(f"\nKeyword: {kw}")
results = scrape_keyword_pages(kw, country="us", pages=2)
all_data[kw] = results
print(f" Collected {len(results)} results")

# Delay between keywords: 5 to 12 seconds
time.sleep(random.uniform(5, 12))

with open("bing_serp_results.json", "w") as f:
json.dump(all_data, f, indent=2)

print(f"\nDone. Results saved to bing_serp_results.json")

*What to Watch For
*

Empty result pages. Always run validate_response() before parsing. Bing returns 200 status codes on blocked requests - the response just has no organic results.
Session reuse across keywords. For rank tracking where you want consistent geo, use the same sticky session string per location target. For bulk scraping where you want IP diversity, generate a fresh session ID per request.
Pagination limits. Bing typically returns results up to around page 10 before results degrade significantly. For most rank tracking use cases, pages 1-3 cover what matters.
Bing News and Bing Shopping. Different result types use different CSS selectors. The b_algo class covers standard organic results. News results use div.news-card, shopping results use li.b_shpItem. Scope your parser to the result type you need.

Getting Started
The NodeMaven residential proxies page covers pool specs and pricing. The $3.50 trial gives 750MB to validate the setup against your actual keyword targets before committing to a plan. For Bing SERP work, residential proxies from $2.20/GB with city-level geo-targeting give you everything the pipeline above needs.

How to Test if Your Proxy is Leaking DNS: 2026 Setup Guide

Elen Simonian — Sat, 30 May 2026 14:18:46 +0000

How to Test if Your Proxy is Leaking DNS: 2026 Setup Guide

You configured a proxy. Traffic is routing through it. IP check passes. Job done?

Not necessarily.

DNS leaks are one of the most common ways a proxy setup can expose your real location and ISP while appearing to work correctly. Your IP changes, but your DNS queries still go home.

This guide covers:

What a DNS leak actually is at the network level
How to detect one reliably
How to fix it across different proxy configurations
Programmatic detection methods with code examples

What Is a DNS Leak?

When you type a domain into your browser, a DNS resolver translates it into an IP address.

Normally this happens through your ISP's DNS server. When you use a proxy, the expectation is that DNS resolution also happens inside the proxy network rather than through your local ISP.

A DNS leak occurs when your operating system or application bypasses the proxy and sends DNS queries directly to your ISP's resolver.

The result:

The proxy hides your IP from the destination website
Your ISP still sees every domain you're resolving
Anyone monitoring DNS traffic can reconstruct your browsing activity regardless of what proxy you're using

This matters most for

Automation and scraping jobs where geo-accurate DNS is required for localized responses
Multi-account workflows where a mismatched DNS origin can trigger platform-side identity checks
Privacy-sensitive workflows where DNS history is the actual attack surface

Why DNS Leaks Happen

DNS leaks are usually not a misconfiguration on the proxy provider's side.

They happen because of how OS-level DNS resolution works.

Most operating systems have a DNS resolver that runs independently of application-level proxy settings.

When you configure an HTTP or SOCKS5 proxy in a browser or script, the proxy handles connection routing, but the OS resolver may still handle DNS separately depending on the application and proxy type.

Common causes

HTTP proxies handle DNS on the proxy server by default, but some clients resolve DNS locally first and then send the IP to the proxy
SOCKS4 does not support remote DNS resolution
SOCKS5 supports remote DNS resolution, but only if the client explicitly requests it
WebRTC in browsers can expose local DNS even when a proxy is active
Some automation frameworks use the system DNS resolver instead of routing through the proxy
Split DNS configurations on corporate or managed networks

How to Test for a DNS Leak

There are several reliable ways to verify whether your DNS requests are leaving the proxy tunnel.

Method 1: Manual Test with a DNS Leak Test Tool

The fastest way to check is to use NodeMaven's DNS leak test tool while your proxy is active.

It works by making multiple DNS requests to unique subdomains and checking which DNS server resolved them.

Steps

Configure your proxy in your browser or system settings.
Navigate to the DNS leak test tool with the proxy active.
Run the extended test.
Check the resolver IPs in the results.

Result interpretation

If any resolver belongs to your ISP or home network rather than the proxy provider's infrastructure, you have a DNS leak.

Method 2: dig or nslookup via the Proxy

For terminal-based testing, you can route a DNS query through the proxy using dig with a SOCKS5 proxy:

# Test DNS resolution through a SOCKS5 proxy
# Replace with your NodeMaven proxy credentials and host

dig @8.8.8.8 whoami.akamai.net +short

# Route the query through the proxy using proxychains
proxychains dig whoami.akamai.net +short

# Compare the two outputs
# If they return different IPs, your DNS is routing through the proxy
# If they return the same IP, you have a potential DNS leak

You can also use curl to hit a DNS echo service:

# Check your apparent DNS resolver through the proxy

curl --proxy socks5h://username:password@proxy.nodemaven.com:9999 \
  https://whoami.akamai.net

# Note the 'h' in socks5h - this forces remote DNS resolution
# socks5:// resolves DNS locally
# socks5h:// resolves DNS on the proxy server

Important

The socks5h:// vs socks5:// distinction is one of the most common sources of DNS leaks in automation scripts.

Method 3: Python Script for Programmatic Detection

If you're running automated workflows, you can build DNS leak detection into your setup verification:

import requests
import json

def check_dns_leak(proxy_url: str) -> dict:
    """
    Check for DNS leaks by comparing DNS resolvers visible
    with and without the proxy.
    """

    proxies = {
        "http": proxy_url,
        "https": proxy_url,
    }

    ...