Must Try Open-Source Python TLS Requests: Simplify Web Scraping, Bypass Cloudflare 403 Forbidden (WAF)

Earn Pham — Sat, 14 Dec 2024 06:39:10 +0000

TLS Requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS fingerprinting, anti-bot bypassing, and high performance.

Installation

To install the library, you can choose between two methods:

1. Install via PyPI

pip install wrapper-tls-requests

2. Install via GitHub Repository

pip install git+https://github.com/thewebscraping/tls-requests.git

Quick Start

Start using TLS Requests with just a few lines of code:

import tls_requests

response = tls_requests.get("https://httpbin.org/get")
print(response)         # Output: <Response [200 OK]>
print(response.status_code)  # Output: 200

Introduction

TLS Requests is a cutting-edge HTTP client for Python, offering a feature-rich, highly configurable alternative to the popular requests library.

Built on top of tls-client, it combines ease of use with advanced functionality for secure networking.

Key Benefits

Bypass TLS Fingerprinting: Mimic browser-like behaviors to navigate sophisticated anti-bot systems.
Customizable TLS Clients: Select specific TLS fingerprints to meet your needs.
Ideal for Developers: Build scrapers, API clients, or other custom networking tools effortlessly.

Why Use TLS Requests?

Modern websites increasingly use TLS Fingerprinting and anti-bot tools like Cloudflare Bot Fight Mode to block web crawlers. TLS Requests bypass these obstacles by mimicking browser-like TLS behaviors, making it easy to scrape data or interact with websites that use sophisticated anti-bot measures.

Unlocking Cloudflare Bot Fight Mode

Example:

import tls_requests

response = tls_requests.get("https://www.coingecko.com/")
print(response)  # Output: <Response [200]>

Key Features

Enhanced Capabilities

Browser-like TLS Fingerprinting: Enables secure and reliable browser-mimicking connections.
High-Performance Backend: Built on a Go-based HTTP backend for speed and efficiency.
Synchronous & Asynchronous Support: Seamlessly switch between synchronous and asynchronous requests.
Protocol Support: Fully compatible with HTTP/1.1 and HTTP/2.
Strict Timeouts: Reliable timeout management for precise control over request durations.

Additional Features

Internationalized Domain & URL Support: Handles non-ASCII URLs effortlessly.
Cookie Management: Ensures session-based cookie persistence.
Authentication: Native support for Basic and Function authentication.
Content Decoding: Automatic handling of gzip and brotli-encoded responses.
Hooks: Perfect for logging, monitoring, tracing, or pre/post-processing requests and responses.
Unicode Support: Effortlessly process Unicode response bodies.
File Uploads: Simplified multipart file upload support.
Proxy Configuration: Supports Socks5, HTTP, and HTTPS proxies for enhanced privacy.

Documentation

Read the full documentation at thewebscraping.github.io/tls-requests/.

DEV Community: Earn Pham