DEV Community: Billy

Finding Exposed APIs and Services: A Developer's Guide with ScanSearch

Billy — Thu, 13 Aug 2026 04:00:13 +0000

As developers, we're constantly building and deploying services. Sometimes, in the rush to get things out the door, or through a simple misconfiguration, an internal API or a development environment might accidentally be exposed to the public internet. This isn't just a theoretical risk; it's a very real vector for security vulnerabilities.

So, how do we proactively check for these kinds of exposures? Manually scanning IP ranges is tedious and often incomplete. This is where tools designed for internet-wide reconnaissance come in handy. In this article, we'll explore how to use a service like ScanSearch to identify potentially exposed services, focusing on finding common API endpoints and documentation.

The Problem: Unintended Exposure

Imagine you've deployed a new microservice. It's meant to be behind an API gateway, accessible only internally. But a firewall rule was misconfigured, or a load balancer setting was overlooked, and suddenly, your dev environment's /api/v1/admin endpoint is directly accessible to anyone on the internet. This isn't an uncommon scenario, and it can lead to serious data breaches or system compromises.

As developers, it's beneficial to have a way to quickly check for these kinds of unintended exposures, not just for our own projects but also to understand the broader landscape of internet-facing services.

Introducing ScanSearch

ScanSearch is an internet-wide search engine for network devices, services, and vulnerabilities. Think of it like Google, but for things connected to the internet. It constantly scans and indexes what's out there, allowing you to query for specific banners, ports, services, or even HTTP responses.

While primarily used by security researchers, developers can leverage ScanSearch to:

Verify external accessibility: Ensure services intended to be internal remain so.
Discover common misconfigurations: Search for default credentials, exposed Git repositories, or .env files.
Identify shadow IT: Find forgotten or unknown internet-facing assets.
Explore API landscapes: See how common API patterns are exposed.

Let's dive into some practical examples.

Scenario 1: Finding Exposed Swagger/OpenAPI Docs

Many API frameworks automatically generate documentation endpoints, often at paths like /swagger, /api-docs, or /redoc. If these endpoints are exposed without authentication, they can provide attackers with a complete blueprint of your API, including sensitive endpoint paths, request/response structures, and even authentication mechanisms.

To search for exposed Swagger UI endpoints, we can use ScanSearch to look for specific HTTP response body content that indicates the presence of Swagger UI. A common string is the title tag or a script reference.

Let's try searching for the HTML title associated with Swagger UI:

http.title: "Swagger UI"

This query will return a list of hosts that have the string "Swagger UI" in their HTTP title. You can further refine this by looking for specific paths or other indicators. For example, to find those specifically on an /api-docs path:

http.title: "Swagger UI" AND http.url: "/api-docs"

This combination is powerful. It allows you to quickly identify services that are not only exposing a web server but specifically exposing their API documentation through a well-known path.

Scenario 2: Discovering Generic API Endpoints

Sometimes, it's not about a specific documentation page, but just generally looking for common API patterns like /api/v1 or /graphql. While these queries might return a lot of noise, they can sometimes highlight unexpected public-facing services.

Let's search for servers that respond to common API path patterns:

http.url: "/api/v1/" OR http.url: "/graphql"

This will show you a range of services that have these paths. Clicking on the results often reveals more details, including HTTP headers and the full response body, which can help you determine if it's a legitimate public API or an accidentally exposed internal one.

Scenario 3: Identifying Development/Staging Environments

Development or staging environments often contain sensitive data, debug information, or unpatched vulnerabilities. If these are accessible from the internet, they become prime targets. We can look for common indicators in HTTP titles or banners.

For example, searching for common keywords in titles:

http.title: "dev environment" OR http.title: "staging server" OR http.title: "test system"

Or even looking for specific default server banners that might indicate a development-focused setup:

http.html: "Welcome to Node.js" AND port: 3000

(Note: The actual content for http.html would be a more specific string from the page, or http.body for the full body content. The example above is illustrative of looking for specific server-side technology indicators combined with a common development port.)

Best Practices for Developers

Assume public by default: When deploying anything, assume it's public until proven otherwise. Double-check firewall rules and load balancer configurations.
Authentication and Authorization: Ensure all API endpoints, even internal ones, are protected by robust authentication and authorization mechanisms.
Regular Audits: Periodically use tools like ScanSearch to scan your own public IP ranges or domains for unintended exposures.
Least Privilege: Configure services with the absolute minimum necessary permissions.
Environment Variables: Never hardcode sensitive credentials or API keys. Use environment variables or secure secret management services.

Conclusion

Proactively searching for exposed APIs and services is a critical step in maintaining the security posture of our applications. ScanSearch provides a powerful lens into the internet's observable surface, allowing developers to identify potential vulnerabilities before malicious actors do. By incorporating such tools into our development and deployment workflows, we can build more secure and resilient systems.

Head over to ScanSearch.net to try out these queries yourself and explore the vast landscape of internet-connected devices and services.

Finding Exposed Services (and Vulnerabilities) on Your Network: A Practical ScanSearch Guide

Billy — Wed, 05 Aug 2026 04:00:13 +0000

Ever wondered what services on your network are truly exposed to the internet? Or perhaps you're trying to track down a specific type of device across your public-facing infrastructure? Manually scanning port by port across a large IP range can be tedious, slow, and often misses the bigger picture.

This article isn't about setting up a traditional port scanner. Instead, we're going to explore how to leverage an internet-wide search engine for network devices and services, ScanSearch, to quickly identify and understand the public-facing footprint of your network. Think of it like Google, but for servers, routers, cameras, and everything else connected to the internet.

We'll cover practical use cases, including:

Identifying unintentionally exposed services.
Searching for specific device types or software versions.
Spotting known vulnerabilities associated with your infrastructure.

Let's dive in.

The Problem: Blind Spots in Your Public-Facing Infrastructure

It's common for development or operations teams to configure services, sometimes forgetting that a default configuration or an overlooked firewall rule might leave something accessible that shouldn't be. This isn't just about malicious actors; it's also about maintaining good security hygiene and understanding your own attack surface.

Traditional internal network scans are crucial, but they don't always give you the external perspective. That's where a tool like ScanSearch comes in – it's constantly indexing the internet's public-facing devices.

Getting Started with ScanSearch

ScanSearch offers a powerful search syntax. The most straightforward way to begin is by searching for your organization's public IP ranges or domain names. For this tutorial, I'll use example.com and a hypothetical IP range, but you should substitute these with your own public IPs or domains.

Use Case 1: Discovering All Services on Your IP Range

Let's say your organization uses the IP range 203.0.113.0/24. To see everything ScanSearch has indexed for this range, you'd simply enter:

ip:203.0.113.0/24

This query will return a list of all devices, services, and associated information (like open ports, banners, and even HTTP response headers) found within that IP range. You might be surprised at what pops up – maybe an old development server you thought was offline, or a service running on a non-standard port.

Use Case 2: Finding Specific Device Types or Software Versions

Perhaps you're concerned about older versions of Apache or Nginx that might still be running. You can combine the IP range with a keyword search for specific banners or technologies. For example, to find all Apache servers on our hypothetical range:

ip:203.0.113.0/24 product:apache

Or, to get more specific and look for an older, potentially vulnerable version (e.g., Apache 2.2):

ip:203.0.113.0/24 product:apache/2.2

This quickly highlights potential upgrade candidates or misconfigurations.

Use Case 3: Identifying Known Vulnerabilities on Your Infrastructure

ScanSearch also indexes publicly known vulnerabilities (CVEs) associated with identified services. This is incredibly powerful for proactively assessing risk.

Let's say you want to see if any of your devices on example.com are running services with known vulnerabilities:

host:example.com has_vulnerability:true

This query would return any hosts under example.com where ScanSearch has identified a service with an associated, known vulnerability. You can then drill down into the results to see the specific CVEs and affected services. This is a critical step in prioritizing patching and mitigation efforts.

Pro-tip: You can also search directly for a specific CVE. If you're tracking a new, critical vulnerability, you could search cve:CVE-2023-XXXX to see if it appears anywhere on your network.

Use Case 4: Searching for Default Credentials or Common Misconfigurations

While ScanSearch doesn't actively exploit systems, its indexing of service banners and HTTP responses can sometimes reveal clues about weak configurations. For instance, you might look for common default administrative interfaces or specific keywords that indicate a lack of proper setup.

ip:203.0.113.0/24 title:"admin panel"

Or, looking for specific HTTP response bodies:

ip:203.0.113.0/24 http.body:"Powered by WordPress"

While not a direct vulnerability, finding an exposed WordPress admin panel on a public IP range you own is certainly something you'd want to investigate further.

Beyond Simple Queries: Combining Filters

The real power of ScanSearch comes from combining these filters. You can search for specific ports, protocols, HTTP headers, and much more. For example, to find all devices on your range running an SSH server (port 22) that also have a known vulnerability:

ip:203.0.113.0/24 port:22 has_vulnerability:true

This quickly narrows down your focus to the most critical issues.

Conclusion

Understanding your external attack surface is a fundamental part of maintaining secure infrastructure. Tools like ScanSearch provide an invaluable perspective by indexing the entire internet, allowing you to quickly query and identify exposed services, potential misconfigurations, and known vulnerabilities associated with your own public-facing assets.

Instead of endless manual scans, integrate ScanSearch into your regular security audits. It's a powerful way to catch those overlooked services and proactively address potential risks before they become a problem. Give it a try with your own public IP ranges and domains – you might be surprised at what you find.

Finding Exposed APIs and Services: A Practical Guide for Developers

Billy — Tue, 28 Jul 2026 04:00:15 +0000

As developers, we often build and deploy services, APIs, and applications that interact with the internet. While we strive for secure configurations, misconfigurations or overlooked details can expose sensitive information or create vulnerabilities. Manually auditing every network device and service for potential exposures can be a monumental task, especially in larger infrastructures.

This article will walk through a practical approach to identify publicly accessible services and potential API endpoints, focusing on how a tool like ScanSearch can assist in this process. We'll look at common scenarios where this kind of reconnaissance is useful and how to interpret the results.

The Problem: Unintended Exposure

Imagine you're developing a new microservice. During development, you might expose a debug endpoint or an internal API without proper authentication, assuming it will only be accessible within your private network. However, a misconfigured firewall, an overlooked security group rule, or even a temporary test deployment left running can inadvertently expose these services to the entire internet.

Similarly, older services or those deployed by other teams might have known vulnerabilities or default credentials that could be exploited if discovered. As developers, understanding what's actually visible from the outside world is a crucial step in building more secure applications.

Enter: ScanSearch for External Visibility

ScanSearch is an internet-wide search engine designed to discover network devices, services, and vulnerabilities. Think of it as a Google for network assets. Instead of crawling websites, it indexes information about what's listening on IP addresses across the globe.

This makes it incredibly useful for developers who want to understand their external attack surface. You can use it to:

Verify external accessibility: Did that API you just deployed accidentally expose port 8080 to the world?
Discover forgotten services: Are there old test servers or services running that you're unaware of?
Identify common vulnerabilities: ScanSearch indexes known vulnerabilities, which can help you prioritize patching efforts.

Let's dive into some practical examples.

Practical Example 1: Finding Exposed API Endpoints by Port

Many APIs run on specific ports. If you suspect an internal API might be exposed, you can search for services listening on that port within a specific IP range or for a particular organization.

Let's say you're looking for an API that should only be internal but might be exposed on port 8080. While I can't provide real-time interactive examples here, the conceptual query on ScanSearch would be straightforward.

You'd typically use a query like port:8080 to find all services listening on port 8080. To narrow it down, you might add an organization name or an IP range. For instance, if your company's public IP range is 192.0.2.0/24, you could construct a more targeted search.

The results would show IP addresses, associated hostnames (if available), and basic information about the service running on that port. This could quickly reveal if your internal API has an unintended public presence.

Practical Example 2: Discovering Services by Banners and Technologies

Services often announce themselves with banners or specific HTTP headers. ScanSearch indexes this information, allowing you to search for particular technologies or patterns.

Suppose you're developing with Nginx and want to ensure no old, unpatched Nginx instances are exposed from your infrastructure. A general query for product:nginx might reveal many instances. To make it more relevant to your specific infrastructure, you'd combine this with an IP range or ASN (Autonomous System Number) associated with your organization.

# Conceptual ScanSearch query for Nginx instances within a specific IP range
product:nginx ip:192.0.2.0/24

If you find an unexpected Nginx instance, you can then investigate its configuration and purpose. Often, these findings lead to discovering forgotten development servers or misconfigured staging environments.

Practical Example 3: Identifying Known Vulnerabilities

One of ScanSearch's powerful features is its ability to index known vulnerabilities. This is invaluable for proactive security. If a new critical vulnerability is announced for a particular service (e.g., a specific version of Apache Struts), you can quickly check if any of your publicly exposed assets might be affected.

You would search for the specific vulnerability identifier (e.g., a CVE number) or a combination of product and version that's known to be vulnerable.

# Conceptual ScanSearch query for a specific CVE
cve:CVE-2023-XXXX

# Or for a vulnerable product version
product:apache-struts version:2.5.0

This can help you prioritize patching efforts and ensure that critical vulnerabilities are addressed promptly, potentially before they are exploited.

Interpreting the Results

When you get results from ScanSearch, it's essential to understand what you're looking at:

IP Address/Hostname: This tells you where the service is located.
Port: The specific port number the service is listening on.
Product/Service: Often, ScanSearch can identify the type of software running (e.g., Apache HTTPD, Nginx, OpenSSH).
Banners/Headers: These provide additional details that the service itself advertises.
Vulnerabilities (CVEs): If ScanSearch has identified known vulnerabilities associated with the detected service/version.

Each result is a data point requiring further investigation. A service showing up doesn't automatically mean it's vulnerable, but it does mean it's publicly accessible and should be intentionally so, with appropriate security controls.

Beyond the Search: Actionable Steps

Discovering exposed services is only the first step. Here's what you should do next:

Verify: Confirm if the service genuinely belongs to your infrastructure and if its public exposure is intended.
Assess Risk: If unintended, evaluate the potential impact of its exposure.
Remediate: If it shouldn't be public, take steps to secure it. This might involve firewall rules, network ACLs, stronger authentication, or simply shutting it down.
Document: Update your documentation to reflect intended exposures and security configurations.
Automate: Consider integrating such checks into your CI/CD pipeline or regular security audits.

Conclusion

Understanding your external network footprint is a critical aspect of modern application development and security. Tools like ScanSearch provide developers with a practical way to gain this visibility, helping to prevent unintended exposures and proactively identify potential vulnerabilities. By integrating such reconnaissance into your development workflow, you can build more robust and secure applications from the ground up.

Remember, what you don't know can hurt you. Regularly checking what the internet sees of your infrastructure is a simple yet powerful security practice.

For more details and to try it out, visit ScanSearch.net.

How to Check SPF, DKIM, and DMARC Records in Python

Billy — Fri, 24 Jul 2026 21:42:16 +0000

If your app sends email — transactional or marketing — three DNS records decide whether it lands in the inbox or the spam folder: SPF, DKIM, and DMARC. Here's how to look them up and sanity-check them in Python, no third-party API required.

Install the one dependency:

pip install dnspython

SPF: who is allowed to send

SPF lives in a TXT record on the domain itself and starts with v=spf1.

import dns.resolver

def get_spf(domain):
    for rec in dns.resolver.resolve(domain, "TXT"):
        txt = b"".join(rec.strings).decode()
        if txt.startswith("v=spf1"):
            return txt
    return None

print(get_spf("github.com"))
# v=spf1 ip4:... include:_spf.google.com ~all

A quick gotcha worth checking: SPF allows at most 10 DNS-querying mechanisms (include, a, mx, ptr, exists, redirect). Go over and receivers return permerror, which quietly breaks authentication:

def spf_lookup_count(spf):
    return sum(spf.count(m) for m in ("include:", "a:", "mx:", "ptr", "exists:", "redirect="))

spf = get_spf("example.com")
if spf and spf_lookup_count(spf) > 10:
    print("⚠️ SPF exceeds the 10-lookup limit")

DMARC: the policy that ties it together

DMARC is a TXT record on the _dmarc. subdomain and starts with v=DMARC1.

def get_dmarc(domain):
    try:
        for rec in dns.resolver.resolve(f"_dmarc.{domain}", "TXT"):
            txt = b"".join(rec.strings).decode()
            if txt.startswith("v=DMARC1"):
                return dict(
                    kv.strip().split("=", 1)
                    for kv in txt.split(";") if "=" in kv
                )
    except dns.resolver.NXDOMAIN:
        return None

print(get_dmarc("github.com"))
# {'v': 'DMARC1', 'p': 'reject', 'rua': 'mailto:...'}

The key field is p: none (monitor only), quarantine (spam folder), or reject (bounce). If a domain sends real mail but has p=none, it's not protected against spoofing yet.

DKIM: the signature key

DKIM is trickier because you need the selector — a label chosen by the sender that lives at SELECTOR._domainkey.DOMAIN. There's no way to enumerate selectors from DNS, so you try common ones:

COMMON_SELECTORS = ["google", "default", "s1", "s2", "k1", "selector1", "selector2", "dkim"]

def find_dkim(domain):
    found = {}
    for sel in COMMON_SELECTORS:
        try:
            for rec in dns.resolver.resolve(f"{sel}._domainkey.{domain}", "TXT"):
                txt = b"".join(rec.strings).decode()
                if "v=DKIM1" in txt or "p=" in txt:
                    found[sel] = txt
        except (dns.resolver.NXDOMAIN, dns.resolver.NoAnswer):
            continue
    return found

print(find_dkim("github.com").keys())

If none of the common selectors resolve, the domain may use a custom one — you'll need to read it from an actual signed message's DKIM-Signature: header (s= tag).

Putting it together

def audit(domain):
    spf = get_spf(domain)
    dmarc = get_dmarc(domain)
    dkim = find_dkim(domain)
    print(f"{domain}")
    print(f"  SPF   : {'✓' if spf else '✗ missing'}")
    print(f"  DMARC : {dmarc.get('p') if dmarc else '✗ missing'}")
    print(f"  DKIM  : {', '.join(dkim) if dkim else '✗ none of the common selectors'}")

audit("your-domain.com")

Run that against your sending domain before you ship a campaign. Missing DKIM or a p=none DMARC are the two most common reasons legitimate mail gets filtered after Gmail and Yahoo tightened bulk-sender rules in 2024.

When you need this at scale

The snippets above are perfect for one domain in a script. If you're validating whole lists, warming a new sending domain, or want the checks plus blacklist and deliverability scoring in one pass, I built these into Bulko's free email tools — same logic, no code to run. But for a quick programmatic check, dnspython is all you need.

Rotating Residential Proxies in Python: requests, Scrapy & Sticky Sessions

Billy — Fri, 24 Jul 2026 21:42:06 +0000

When you scrape at any real volume, the bottleneck is rarely your code — it's the target site's rate limiting and IP bans. Rotating residential proxies solve this by routing each request through a different real-user IP. Here's how to wire them into requests and Scrapy in Python, including the sticky-session trick most tutorials skip.

The proxy URL format

A residential proxy is just an authenticated HTTP/SOCKS endpoint. With a pool gateway, you target a country and control session behavior through the username, not separate endpoints:

http://USERNAME_country-us_session-a1b2c3_lifetime-30:PASSWORD@proxy.gproxy.net:1000

country-us — exit country (ISO code)
session-a1b2c3 — a sticky-session id; reuse it to keep the same IP, change it to rotate
lifetime-30 — how many minutes that session's IP stays fixed

Basic request through a rotating proxy

import requests

USER = "USERNAME"
PWD = "PASSWORD"

def proxy(country="us", session=None, lifetime=30):
    tag = f"_country-{country}"
    if session:
        tag += f"_session-{session}_lifetime-{lifetime}"
    url = f"http://{USER}{tag}:{PWD}@proxy.gproxy.net:1000"
    return {"http": url, "https": url}

# New IP on every call (no session id):
r = requests.get("https://api.ipify.org?format=json", proxies=proxy(), timeout=30)
print(r.json()["ip"])

Run it in a loop and you'll see a different IP each time — the gateway rotates automatically when no session id is present.

Sticky sessions: keep one IP across requests

Some flows (login, multi-step checkouts, paginated results behind a cookie) break if your IP changes mid-session. Pin the IP by passing a stable session id:

import uuid

sess = uuid.uuid4().hex[:8]        # one id for the whole flow
p = proxy(country="de", session=sess, lifetime=30)

s = requests.Session()
s.proxies.update(p)
s.get("https://example.com/login")
s.post("https://example.com/login", data={...})   # same exit IP

When you want a fresh IP, generate a new session id.

Scrapy integration

Scrapy reads the proxy off each request's meta. A tiny middleware rotates a fresh session per request:

# middlewares.py
import uuid

class ResidentialProxyMiddleware:
    USER = "USERNAME"
    PWD = "PASSWORD"

    def process_request(self, request, spider):
        sess = uuid.uuid4().hex[:8]
        cc = request.meta.get("country", "us")
        request.meta["proxy"] = (
            f"http://{self.USER}_country-{cc}_session-{sess}_lifetime-10:"
            f"{self.PWD}@proxy.gproxy.net:1000"
        )

# settings.py
DOWNLOADER_MIDDLEWARES = {
    "myproject.middlewares.ResidentialProxyMiddleware": 350,
}

Set request.meta["country"] = "gb" in a spider to geo-target specific requests.

A few practical notes

Retry on proxy errors. Residential IPs occasionally drop; wrap requests with retries (urllib3 Retry, or Scrapy's RETRY_ENABLED) so a bad exit doesn't kill a job.
Respect the target. Rotating IPs is for spreading legitimate load and geo-testing — not for hammering a site past its terms. Add delays and honor robots.txt.
SOCKS5 works the same way on port 1002 if you need it (socks5://...).

Getting credentials

You need a residential pool that supports country targeting and sticky sessions. I use GProxy — the username-based targeting above is exactly its format, and it's usage-based ($0.90/GB) rather than a fixed monthly seat, which suits burst scraping jobs. Grab a key from the dashboard, drop it into USER/PWD, and the snippets above run as-is.

Happy (responsible) scraping.

Finding Network Device Info Programmatically: A Python Guide with ScanSearch

Billy — Mon, 20 Jul 2026 04:00:16 +0000

Ever needed to quickly check the exposure of a specific type of network device, or verify if a known vulnerability is present across your external-facing infrastructure? Manually probing IPs can be a tedious and time-consuming process. What if you could query the entire internet for devices, services, and even known vulnerabilities, all from a simple API call?

This article will walk you through using Python to programmatically query ScanSearch, an internet-wide search engine for network devices, services, and vulnerabilities. We'll focus on practical examples that demonstrate how to find specific device types, filter results, and extract useful information.

The Problem: Manual Network Reconnaissance is Slow

Imagine you're a security analyst, and a new vulnerability (CVE-2023-XXXX) has just been announced, affecting a specific version of a network-attached storage (NAS) device. Your immediate thought is: Are any of our external-facing devices vulnerable? And what about other organizations I'm responsible for monitoring?

Traditionally, you'd start with IP ranges, port scans, and banner grabbing – a process that's not only slow but also might miss devices outside your known ranges. This is where a tool like ScanSearch comes in handy, providing a fast way to query pre-indexed internet data.

Getting Started with ScanSearch

ScanSearch (https://scansearch.net) offers an API that allows you to perform these queries programmatically. While the specifics of API key acquisition aren't covered here, assume you have an API key ready for the following examples.

We'll use Python's requests library to interact with the ScanSearch API. First, let's set up our basic request structure.

import requests
import json

API_KEY = 'YOUR_SCANSEARCH_API_KEY' # Replace with your actual API key
BASE_URL = 'https://api.scansearch.net/v1/search'

def query_scansearch(query_string, page=1, limit=10):
    headers = {
        'Authorization': f'Bearer {API_KEY}',
        'Content-Type': 'application/json'
    }
    params = {
        'q': query_string,
        'page': page,
        'limit': limit
    }
    try:
        response = requests.get(BASE_URL, headers=headers, params=params)
        response.raise_for_status()  # Raise an exception for HTTP errors
        return response.json()
    except requests.exceptions.HTTPError as err:
        print(f"HTTP error occurred: {err}")
        print(f"Response body: {response.text}")
    except Exception as err:
        print(f"An error occurred: {err}")
    return None

This query_scansearch function takes a query_string, page, and limit and handles the API call, including basic error checking.

Example 1: Finding Specific Device Types

Let's say we want to find all publicly exposed instances of nginx web servers. The query_string for ScanSearch is quite powerful and supports various filters. A simple query for nginx will return services identified as such.

print("\n--- Searching for Nginx Servers ---")
nginx_results = query_scansearch('nginx')

if nginx_results and 'results' in nginx_results:
    print(f"Found {len(nginx_results['results'])} Nginx instances (first {len(nginx_results['results'])} results shown):")
    for i, result in enumerate(nginx_results['results'][:5]): # Print first 5 for brevity
        print(f"  {i+1}. IP: {result.get('ip')}, Port: {result.get('port')}, Banners: {result.get('banners', [])[0] if result.get('banners') else 'N/A'}")
        # You can inspect the full result object for more details
        # print(json.dumps(result, indent=2))
else:
    print("No Nginx results found or an error occurred.")

This code snippet sends a query for nginx and then iterates through the results, printing out the IP, port, and the first banner string found. ScanSearch's data includes a lot more, so feel free to inspect the result object more deeply.

Example 2: Filtering by Port and Service

What if you're interested in SSH services running on non-standard ports, say port 2222? ScanSearch allows combining search terms.

print("\n--- Searching for SSH on Port 2222 ---")
ssh_port_results = query_scansearch('service:ssh port:2222')

if ssh_port_results and 'results' in ssh_port_results:
    print(f"Found {len(ssh_port_results['results'])} SSH services on port 2222:")
    for i, result in enumerate(ssh_port_results['results'][:5]):
        print(f"  {i+1}. IP: {result.get('ip')}, Port: {result.get('port')}, Country: {result.get('location', {}).get('country_name')}")
else:
    print("No SSH on port 2222 results found or an error occurred.")

Here, service:ssh specifically targets services identified as SSH, and port:2222 narrows it down to that specific port. The results also contain geographical information under location.

Example 3: Searching for Vulnerabilities (CVEs)

ScanSearch also indexes known vulnerabilities. This is incredibly powerful for security auditing. Let's look for devices exposed to a hypothetical (or real, if you substitute a current one) CVE.

print("\n--- Searching for Devices with CVE-2021-44228 (Log4Shell) ---")
# Note: ScanSearch indexes vulnerabilities; results will show devices associated with them.
log4shell_results = query_scansearch('cve:CVE-2021-44228')

if log4shell_results and 'results' in log4shell_results:
    print(f"Found {len(log4shell_results['results'])} devices potentially affected by CVE-2021-44228:")
    for i, result in enumerate(log4shell_results['results'][:5]):
        print(f"  {i+1}. IP: {result.get('ip')}, Port: {result.get('port')}, Service: {result.get('service')}")
        # For vulnerability queries, you might want to inspect the 'vulnerabilities' key in the full result.
        # print(json.dumps(result.get('vulnerabilities', []), indent=2))
else:
    print("No CVE-2021-44228 results found or an error occurred.")

This example demonstrates how to query for a specific CVE ID. ScanSearch's data includes an association between identified services/devices and known vulnerabilities, which can be invaluable for threat hunting and patch validation.

Considerations and Best Practices

API Key Security: Never hardcode your API key directly into public repositories. Use environment variables or a secure configuration management system.
Rate Limiting: Be mindful of API rate limits. Implement delays or back-off strategies if you're making a large number of requests.
Result Paging: The page and limit parameters are crucial for fetching all results when your query returns more than the default limit. Implement a loop to fetch subsequent pages.
Data Structure: The exact structure of the results can vary slightly depending on the query. Always inspect the JSON response to understand the available fields.
Query Syntax: Familiarize yourself with ScanSearch's query syntax for more precise searches. This often includes field-specific searches (e.g., product:apache, os:linux).

Conclusion

Programmatically querying an internet-wide search engine like ScanSearch significantly speeds up network reconnaissance, security auditing, and threat intelligence gathering. By leveraging its API with a few lines of Python, developers and security professionals can quickly gain insights into internet-facing infrastructure, identify specific device types, services, and even potential vulnerabilities without resorting to slow, active scanning.

Explore the ScanSearch website (https://scansearch.net) to learn more about its capabilities and detailed query options.

Finding Exposed Services & Vulnerabilities: A Developer's Quick Guide with ScanSearch

Billy — Sun, 12 Jul 2026 04:00:14 +0000

Ever been in a situation where you need to quickly identify what services are exposed on a particular network range, or even across the internet? Maybe you're auditing your own infrastructure, or perhaps performing some ethical hacking. Traditional methods can involve port scanning specific IPs, which is fine for small scopes but quickly becomes impractical for broader reconnaissance.

This article will walk you through a practical approach to quickly discover network devices, their services, and even potential vulnerabilities using a powerful internet-wide search engine called ScanSearch. We'll focus on common developer use cases, showing you how to formulate queries to get actionable insights without running a single nmap command yourself.

The Problem: Blind Spots and Manual Scanning

Imagine you're responsible for securing a fleet of IoT devices. You've configured them, but how do you verify they aren't accidentally exposing a web interface or an SSH port to the public internet? Or perhaps you're developing a new service and want to see what similar services are exposing. Manually scanning vast IP ranges is time-consuming and often requires dedicated infrastructure.

This is where an internet-wide search engine comes in. Instead of actively scanning, it aggregates data from continuous scans, allowing you to query this massive dataset instantly.

Introducing ScanSearch: A Practical Tool

ScanSearch is essentially Google for network devices. It continuously scans the internet and indexes information about network devices, the services they run, and known vulnerabilities. This allows you to search for specific criteria, like devices running a particular version of Apache, or devices with an open RDP port.

Let's dive into some practical examples.

Use Case 1: Discovering Exposed Web Servers

Let's say you want to find all devices on the internet running an Apache web server on port 80. ScanSearch's query language is fairly intuitive.

You'd use a query like this:

service.name:apache AND port:80

This query tells ScanSearch to look for devices where the service name contains "apache" and the port is 80. The results would show you IP addresses, along with other details about the discovered service. This is incredibly useful for understanding the landscape of publicly accessible web servers.

Use Case 2: Identifying Devices with Specific Vulnerabilities

One of the most powerful features for developers and security professionals is the ability to search for devices with known vulnerabilities. ScanSearch indexes information about vulnerabilities associated with discovered services.

Let's say you're concerned about devices running services known to be vulnerable to a specific CVE (Common Vulnerabilities and Exposures) ID, for example, CVE-2021-44228 (Log4Shell).

While ScanSearch doesn't let you query by CVE ID directly in the current iteration, you can achieve similar results by searching for specific service versions or patterns often associated with that vulnerability. For instance, if Log4Shell primarily affected specific versions of Java applications exposing certain services, you could try to narrow it down.

For a more general approach to finding vulnerable services, you might search for a service known to have many common vulnerabilities, like certain versions of an older database or web server.

Let's try to find devices running an older, potentially vulnerable version of nginx.

service.name:nginx AND service.version:"1.14.0"

(Note: Always check the specific version information against public CVE databases like NVD to determine actual vulnerability status.)

This query helps you quickly pinpoint instances that might need immediate patching or further investigation. It's a great first step in a vulnerability management workflow.

Use Case 3: Finding Open RDP Ports for Auditing

Remote Desktop Protocol (RDP) is a common target for attackers, and accidentally exposing it to the internet is a significant security risk. You can quickly check for publicly exposed RDP services.

port:3389 AND protocol:rdp

This simple query will list all devices with an open RDP port (3389). If you manage a large network, running this query periodically can act as a quick check against misconfigurations.

Tips for Effective Querying

Be Specific: The more specific your query, the more relevant your results will be. Use AND and OR operators to combine terms.
Explore Fields: ScanSearch offers various fields you can query, such as ip, port, service.name, service.version, protocol, etc. Experiment with these to narrow down your searches.
Regular Checks: For ongoing security, integrate ScanSearch queries into your regular audit routine. It's a fantastic way to passively monitor your public-facing assets.

Beyond the Basics

While we've only scratched the surface, ScanSearch's capabilities extend to discovering a wide array of network services, like databases (MongoDB, MySQL), industrial control systems (SCADA), and various IoT devices. The key is to understand what you're looking for and formulate your query accordingly.

For more detailed query syntax and available fields, I recommend checking out the ScanSearch website directly.

Conclusion

ScanSearch offers a powerful and efficient way for developers, system administrators, and security professionals to gain insights into the internet's observable attack surface. By leveraging its internet-wide scanning data, you can quickly identify exposed services, pinpoint potential vulnerabilities, and audit your own infrastructure without the need for active scanning. It's a valuable tool in any developer's security toolkit for proactive defense and reconnaissance.

Catch expiring TLS certs and weak security headers in CI — with one small CLI

Billy — Wed, 08 Jul 2026 22:29:05 +0000

Your CI proves the app builds, the tests pass, and maybe that there are no known-vulnerable dependencies. Then it ships — behind a TLS certificate that will quietly expire in three weeks, serving responses with a security-headers grade of F.

Nobody notices until a browser does.

These two failure modes — an expiring certificate and weak HTTP security headers — are cheap to check and easy to forget. Here's a small, dependency-light CLI that checks them (plus DNS posture and subdomains) from your terminal, and a one-line GitHub Action that fails the build before your users find out.

Install

pip install "secably[dns]"

The core checks have zero dependencies; the optional [dns] extra pulls in dnspython for SPF/DMARC/DNSSEC. Everything runs locally — nothing about your domain is sent to a third party.

Check a TLS certificate

secably ssl example.com

TLS — github.com:443
--------------------
  Subject          github.com
  Issuer           Sectigo Public Server Authentication CA DV E36
  Valid until      2026-09-30T23:59:59+00:00  (84 days left)
  Protocol         TLSv1.3
  Cipher           TLS_AES_128_GCM_SHA256
  Trusted          yes
  SAN              github.com, www.github.com

It completes the handshake, reads the leaf certificate, and reports the issuer, validity window, days remaining and negotiated protocol. It also flags the things you actually care about: an expired or not-yet-valid certificate, a chain that doesn't verify, or a server still negotiating an obsolete protocol like TLS 1.0.

One implementation detail worth knowing if you build something similar: ssl.getpeercert() returns an empty dict unless the certificate verifies, so you can't use it to inspect an already-expired or self-signed cert — exactly the case you want to catch. The trick is to pull the certificate in DER form with getpeercert(binary_form=True) and decode that instead, then run verification separately just to set a "trusted" flag.

The CI-friendly part is the exit code:

secably ssl example.com --fail-if-expires 21

Exits non-zero if the certificate expires within 21 days. Drop it in a scheduled job and you get weeks of warning instead of an incident.

Grade your security headers

secably headers https://example.com

Security headers — https://example.com
--------------------------------------
  Grade: A  (90/100, HTTP 200)
  Present:
  • strict-transport-security
  • content-security-policy
  • x-frame-options
  • x-content-type-options
  • referrer-policy
  Missing:
  ✗ permissions-policy — Set a Permissions-Policy to restrict powerful browser features.
  Info leaks:
  ! server: nginx

It scores the response A–F across the six headers that matter most — HSTS, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy and Permissions-Policy — and calls out headers that leak your stack, like a verbose Server or X-Powered-By.

Same CI story:

secably headers https://example.com --fail-below B

Fails the build if the grade drops below B.

Bonus: DNS posture and subdomains

Two more checks that are handy for recon and hygiene:

secably dns example.com          # A/AAAA/MX/NS/TXT + SPF / DMARC / DNSSEC
secably subdomains example.com   # passive discovery via Certificate Transparency

dns tells you whether your domain can be spoofed (no SPF, no DMARC, no DNSSEC — the CLI warns on each). subdomains queries public Certificate Transparency logs — every name a CA has ever issued a certificate for — so there's no brute force and no traffic to the target. Add --resolve to get IPs, and --json to any command for a machine-readable object you can pipe into jq.

Put it in CI

You can wire the CLI into any pipeline, but on GitHub there's a ready-made action so you don't have to script the exit-code handling yourself:

name: Security check
on:
  schedule:
    - cron: '0 6 * * *'   # every morning
  workflow_dispatch:

jobs:
  secably:
    runs-on: ubuntu-latest
    steps:
      - uses: Secably/secably-security-check@v1
        with:
          url: your-domain.com
          fail-if-expires: 21
          headers-min-grade: B

That's the whole thing. The job runs entirely inside your runner, prints a job summary, and turns red — with a GitHub error annotation — the moment the certificate is close to expiry or the headers slip below your bar. Running it on a daily cron (not just on push) is the point: certificates expire on the calendar, not on your commits.

Where it runs

To be upfront about it: ssl, headers, dns and subdomains all run on your machine with no account and no rate limit. There's also a secably scan command for a deeper server-side website/port scan, which uses a free API key — but the four checks above need nothing.

A CI run is a snapshot at deploy time, though. Certificates still expire, new subdomains still appear, and ports still open up between deploys. If you want these same checks running continuously with alerts, that's what the hosted side at Secably does — but the CLI and the Action are open source and stand on their own.

CLI: github.com/Secably/secably-cli
Action: github.com/Secably/secably-security-check

If you add it to a pipeline, I'd genuinely like to hear which check caught something first — mine was an expiring cert on a staging box nobody was watching.

The Cheapest Dedicated Servers in 2026: A Price-Checked Shortlist

Billy — Mon, 06 Jul 2026 11:28:16 +0000

If you have ever tried to price a bare-metal server, you know the game: a big "from $X/mo" on the landing page, then the actually-orderable configs cost two or three times that, half of them are out of stock, and the cheap one ships "in 3–5 business days." This is a price-checked shortlist of where to rent a genuinely cheap dedicated server in 2026 — who each provider is for, roughly where pricing starts, and the catch.

I pulled the numbers for the last entry from a live, daily-refreshed dataset (linked at the bottom) so you can verify them instead of trusting a screenshot.

What "cheap" should actually mean

Before the list, four things that separate a real deal from bait:

The price you see is the price you pay. Watch for setup/install fees and "from" pricing that no in-stock config matches.
In stock now, not "provisioning in 5 days." A $16 box you can't deploy today isn't a $16 box.
Bandwidth and traffic caps. Unmetered 1 Gbps and a 30 TB cap are very different economics.
Location. A server two continents from your users is cheap and useless.

The shortlist

1. Kimsufi (OVH budget brand) — legacy boxes, rock-bottom

OVH's budget label recycles older Atom/Xeon hardware at the lowest headline prices in the market. Great for a cheap always-on box, a seedbox, or a lab. Catch: stock is famously thin, hardware is old, and support is minimal.

2. Scaleway Dedibox — cheap French metal

Long-running budget bare-metal out of Paris/Amsterdam, entry configs among the cheapest in the EU. Good if you specifically want France/NL and Scaleway's ecosystem. Catch: entry SKUs sell out and the cheapest tiers use aging CPUs.

3. Hetzner (server auction) — best price/performance in the EU

Not the lowest sticker, but the auction floor gets you modern Ryzen/EPYC with lots of RAM for what budget hosts charge for an Atom. The default answer for "cheap and fast" in Germany/Finland. Catch: auction inventory rotates, and it's EU-only locations.

4. Contabo — cheap high-spec, oversubscribed

Known for a lot of cores/RAM/disk per dollar. Fine for storage or batch workloads where you don't need guaranteed peak performance. Catch: reputation for oversubscription and variable I/O; read recent reviews for your region.

5. Leaseweb / enterprise carriers — not cheap, but predictable

When uptime SLAs and DDoS protection matter more than the invoice, the enterprise carriers start higher but deliver consistency. Catch: you'll pay for it, and it's overkill for a hobby box.

6. Valebyte — cheap and in-stock across 20 countries

Aggregates in-stock bare-metal across a wide provider network, so the differentiator is breadth + honesty: ~800 servers deployable now across 20 countries, prices are final (no "from $X" bait), and the whole catalog is published as a machine-readable dataset. Entry is $16/mo (Atom-class in France/NL), with modern Ryzen from ~$72 and EPYC from ~$160. Catch: it's an aggregator, so the cheapest tiers are still older CPUs — same as everyone else at $16.

Cheapest by region (live figures)

Where the boxes actually are and what the cheapest one costs, at the time of writing:

Region	In stock	Cheapest
France	241	$16/mo
Netherlands	120	$16/mo
Germany	87	$59/mo
United States	80	$41/mo
Poland	63	$54/mo
Singapore	29	$42/mo
Canada	28	$40/mo
Australia	18	$40/mo

France and the Netherlands are where the floor is; if you don't need a specific country, that's where the cheapest usable metal lives in 2026.

How to actually verify prices

The single most useful habit: check a machine-readable price list instead of marketing pages. The regional numbers above come from an open dataset that refreshes daily — dedicated-server-prices on GitHub — with CSV/JSON of specs, location, monthly price and order URL for every in-stock server. Clone it, sort by price, done. No "from $X," no stale screenshots.

Bottom line

Lowest sticker, don't care about hardware age → Kimsufi / Scaleway Dedibox.
Cheap and fast in the EU → Hetzner auction.
Lots of cores/RAM per dollar → Contabo.
SLA and DDoS matter more than price → Leaseweb.
Cheap, in-stock now, specific country → Valebyte.

Whatever you pick, sort by in-stock final price in your region — not the number on the hero banner.

Bypassing Telegram Blocks: How MTProto FakeTLS Proxies Actually Work

Billy — Sat, 04 Jul 2026 12:48:41 +0000

When an ISP or national firewall throttles or blocks Telegram, the usual advice is "use a VPN." But there's a lighter, Telegram-specific tool that is often more resilient and far simpler for end users: the MTProto proxy with FakeTLS.

DPI vs. Telegram

Deep Packet Inspection classifies traffic by its protocol fingerprint. A plain MTProto stream or a naive proxy is easy to spot and drop. The trick is to make the connection look like something completely ordinary.

How FakeTLS works

MTProto is Telegram's own transport protocol. FakeTLS wraps it so the wire looks like a normal TLS 1.3 handshake to a real website:

The client sends a TLS ClientHello with the SNI of a popular domain (e.g. www.microsoft.com).
Authentication is a per-user secret verified via an HMAC computed over the ClientHello (with the 32-byte random field acting as the digest).
If the HMAC is invalid (a censor's active probe), the proxy transparently proxies the prober to the real fronting domain — so an attacker who pokes the port just gets Microsoft's real certificate back.

To a censor, a real client's flow is indistinguishable from an HTTPS visit to that domain. That's what keeps it working where generic proxies are already blocked.

A failure mode nobody documents: clock skew

Here's a real-world gotcha we hit in production. FakeTLS embeds a timestamp for anti-replay. If the proxy host's clock drifts more than a couple of minutes, every legitimate client handshake is rejected:

invalid faketls client hello: incorrect timestamp. got=..., now=..., diff=2m54s

The nasty part: the proxy still accepts TCP, still answers TLS, still passes an openssl s_client masquerade check — it looks perfectly alive. It just silently falls back to domain-fronting instead of relaying, so every user on that host is broken at once. The fix is boring but essential: run NTP (chrony) on every proxy host. We shipped a box without it and spent an afternoon chasing "the proxy is down" tickets that were really a 3-minute clock drift.

MTProto proxy vs VPN

	MTProto proxy	VPN
Scope	Telegram only	whole device
Setup	one tap on a link	install + configure
Detectability	masked as TLS	often fingerprinted
Battery/speed	negligible / fast	heavier

For unblocking Telegram specifically, MTProto is usually the better tool.

Try it

Notes + setup (open): https://github.com/blureshot/free-telegram-proxy
A free, working MTProto proxy (24h, no signup, one-tap): https://t.me/NotifyGproxyBot?start=devto

Full disclosure: that bot is the free tier of a proxy service I work on (GProxy). But the mechanics above are protocol-level and vendor-neutral — any FakeTLS MTProto proxy (mtg, mtprotoproxy) works the same way, and the clock-skew lesson applies to all of them.

Need production-grade rotating & residential proxies with real uptime? Try *gproxy.net*.

A free proxy list that's actually checked (HTTP/SOCKS4/SOCKS5, updated every 30 min)

Billy — Sat, 04 Jul 2026 10:38:33 +0000

Every "free proxy list" you find is the same story: a giant wall of ip:port lines, 90% of them already dead, no idea which are HTTP or SOCKS, no idea where they exit. You paste 500 into your scraper and three of them work.

So I put up a small one that's actually checked: gproxynet/free-proxy-list. A checker validates each proxy, tags it with protocol, country and latency, and the repo is regenerated every 30 minutes. It's a rotating sample, not a 50k dump — the point is that what's in it was alive minutes ago, not last month.

Grab it

Plain text, one ip:port per line, split by protocol:

curl -s https://raw.githubusercontent.com/gproxynet/free-proxy-list/main/all.txt
curl -s https://raw.githubusercontent.com/gproxynet/free-proxy-list/main/socks5.txt

Or the structured version with protocol, country and latency:

curl -s https://raw.githubusercontent.com/gproxynet/free-proxy-list/main/proxies.json

[
  {"proxy":"103.156.224.66:8080","protocol":"http","country":"ID","latency_ms":4225,"checked_at":"2026-07-04T09:34:16Z"},
  {"proxy":"185.26.180.180:80","protocol":"http","country":"NL","latency_ms":340,"checked_at":"2026-07-04T09:41:02Z"}
]

Use it in Python

Filter to the protocol and speed you want:

import requests

proxies = requests.get(
    "https://raw.githubusercontent.com/gproxynet/free-proxy-list/main/proxies.json"
).json()

fast_socks5 = [
    p["proxy"] for p in proxies
    if p["protocol"] == "socks5" and (p["latency_ms"] or 9999) < 2000
]
print(len(fast_socks5), "fast SOCKS5 proxies")

If you're rotating them across a scraper, feed the list straight into a pool. I use proxyspin (a small rotating pool with health tracking + ban detection), which can load a URL directly:

from proxyspin import ProxyPool
from proxyspin.requests_adapter import RotatingSession

pool = ProxyPool.from_url("https://raw.githubusercontent.com/gproxynet/free-proxy-list/main/all.txt")
session = RotatingSession(pool)          # dead ones get benched automatically
print(session.get("https://httpbin.org/ip").json())

The honest part

These are public proxies. They're shared by strangers, they're slow, they die within minutes to hours, and you should never send anything sensitive through them. A checked list saves you the "which of these 500 is alive" step — it does not make public proxies reliable. They're great for testing, learning, one-off checks and throwaway requests.

The moment you need proxies that stay up — real scraping, account work, ad verification — you'll want dedicated ones (residential, mobile or datacenter) instead of public ones. But if a free, checked, auto-updating list is what you need today, it's there:

➡️ github.com/gproxynet/free-proxy-list

Bookmark the raw URL, poll it every so often, and you've always got a fresh handful to work with.

Need production-grade rotating & residential proxies with real uptime? Try *gproxy.net*.

Rotating proxies for Scrapy, Playwright and requests — one small library

Billy — Sat, 04 Jul 2026 10:28:29 +0000

If you scrape anything at scale, you know the drill: proxies die, sites start returning 403/429, and your run grinds to a halt. The classic Scrapy answer, scrapy-rotating-proxies, hasn't seen a real update in years — and it's Scrapy-only, so the moment you reach for Playwright or plain requests, you're rebuilding rotation from scratch.

I wanted one small proxy pool I could share across all three. So I wrote proxyspin: a rotating pool with health tracking, ban detection and sticky sessions, and thin adapters for Scrapy, Playwright and requests. Zero required dependencies, pure standard library.

pip install proxyspin

The pool

Everything is built around one object. Load proxies from a list, a file, or a URL:

from proxyspin import ProxyPool

pool = ProxyPool.from_file("proxies.txt", strategy="round_robin")
# or inline / from your provider's export endpoint:
pool = ProxyPool(["http://user:pass@gate1.example.com:8000", "10.0.0.2:8000"])
pool = ProxyPool.from_url("https://example.com/api/my-list.txt")

proxy = pool.get()        # -> Proxy; proxy.url is ready to use
pool.mark_failed(proxy)   # bench it after repeated failures
pool.mark_ok(proxy)       # reset its failure streak

It parses every common list format — host:port, host:port:user:pass, user:pass@host:port, scheme://user:pass@host:port — for HTTP, HTTPS, SOCKS4 and SOCKS5.

Rotation strategies

round_robin — cycle through healthy proxies in order (default)
random — pick one at random
sticky — keep returning the same proxy for a given key (a target domain, an account id, a worker name) until it goes unhealthy

The health model

Every proxy starts healthy. mark_failed bumps its failure streak; when the streak hits max_failures (default 2) the proxy is benched with exponential backoff (cooldown * 2**overshoot, base 60s, capped at 1h), then automatically rejoins rotation. mark_ok resets the streak. So dead proxies quietly drop out and recover on their own — you never manually prune the list.

Scrapy

Enable the middleware and point it at your proxies. Ban detection and retry-through-the-next-proxy are automatic:

# settings.py
DOWNLOADER_MIDDLEWARES = {
    "proxyspin.scrapy_middleware.ProxySpinMiddleware": 610,
}
PROXYSPIN_FILE = "proxies.txt"
PROXYSPIN_STRATEGY = "sticky"       # one proxy per target host
PROXYSPIN_BAN_CODES = [403, 429]    # these responses rotate the proxy
PROXYSPIN_MAX_RETRIES = 3

Any 403/429 (configurable) marks the proxy as failed and retries the request through the next healthy one. No spider code changes.

Playwright

Playwright takes a proxy per browser context, which is the natural rotation unit — great for one-IP-per-account flows:

from playwright.sync_api import sync_playwright
from proxyspin import ProxyPool
from proxyspin.playwright_helper import proxy_settings

pool = ProxyPool.from_file("proxies.txt", strategy="sticky")

with sync_playwright() as p:
    browser = p.chromium.launch()
    for account in accounts:
        context = browser.new_context(proxy=proxy_settings(pool, key=account.id))
        # each account keeps its own IP for the whole session

requests

A drop-in Session that rotates on every call and retries failures through another proxy:

from proxyspin import ProxyPool
from proxyspin.requests_adapter import RotatingSession

session = RotatingSession(ProxyPool.from_file("proxies.txt"))
print(session.get("https://httpbin.org/ip").json())   # new IP per call

Check a list first

Bad proxies waste run time. The bundled CLI tests a whole list concurrently and writes out the survivors:

proxyspin check proxies.txt --workers 100 --alive-out alive.txt

OK   45.155.10.4:8000        612 ms  HTTP 200
DEAD 91.10.77.2:3128                 TimeoutError
...
118/200 alive
wrote 118 proxies to alive.txt

Getting proxies to test with

Want to try it right now without your own proxies? Bootstrap straight from a live public list:

pool = ProxyPool.from_url("https://raw.githubusercontent.com/gproxynet/free-proxy-list/main/all.txt")

Fair warning: public proxies are unreliable by nature — they're shared, slow, and die within minutes. They're fine for kicking the tires, not for a real crawl. For production you'll want dedicated proxies (residential/mobile/datacenter); a pool of one gateway entry per endpoint is all proxyspin needs since rotation happens server-side.

Wrapping up

One pool, the same health model everywhere, and you stop babysitting dead proxies. The code is MIT-licensed and on GitHub — issues and PRs welcome. If you've been limping along on an unmaintained rotation middleware, give it a spin.

Need production-grade rotating & residential proxies with real uptime? Try *gproxy.net*.