DEV Community: Sokol

Robotex: a real browser for your bots, behind a plain HTTP API

Sokol — Wed, 14 Oct 2026 08:57:54 +0000

Every bot that works with a website sooner or later hits one of three walls:

a captcha;
anti-bot protection (Cloudflare's "Just a moment…" page, Turnstile, reCAPTCHA);
heavy JavaScript that has to run before you can even submit a form.

A spider built on raw HTTP requests is helpless here. You need a real browser. But a browser is heavy: it eats half a gigabyte of RAM per window, it has to be hidden from detection, and it has to live next to your bot. Most of the time, though, the bot doesn't need the browser at all. It only needs it to get through the door.

That's the idea behind Robotex: a browser-as-a-service that you call only where there's no way around it.

The idea: the browser opens the door, the bot walks in

Robotex opens the page in real Chrome, passes the anti-bot checks, runs a declarative scenario (fill, click, wait, grab HTML or text) and returns:

the cookies, including cf_clearance;
the browser's own User-Agent;
optionally, the rendered HTML and a screenshot.

From there your bot keeps working with a plain HTTP client, which is fast and cheap. When the site shows a challenge again, the bot calls Robotex once more in the same session.

It's a service, not a library, so the bot can be written in anything that speaks HTTP.

A first request

The synchronous endpoint is the simplest way in:

POST https://app.robotex.sokolab.xyz/v1/solve
X-API-Key: <your key>
Content-Type: application/json

{
  "url": "https://example.com/login",
  "proxy": "http://user:pass@host:port",
  "captcha_type": "cloudflare",
  "actions": [
    {"type": "fill", "selector": "#email", "value": "user@example.com"},
    {"type": "fill", "selector": "#password", "value": "..."},
    {"type": "click", "selector": "#submit"},
    {"type": "wait_for", "selector": ".dashboard", "timeout": 30}
  ]
}

{
  "status": "ok",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64) ... Chrome/155.0.0.0 Safari/537.36",
  "cookies": [{"name": "cf_clearance", "value": "...", "domain": ".example.com"}],
  "actions": [
    {"type": "fill", "ok": true, "duration_ms": 41},
    {"type": "fill", "ok": true, "duration_ms": 37},
    {"type": "click", "ok": true, "duration_ms": 120},
    {"type": "wait_for", "ok": true, "duration_ms": 1830}
  ],
  "duration_ms": 9120
}

The response also sets a session_id cookie. That's a persistent Chrome profile on the server: cookies, local storage and a browser fingerprint. Send it back next time, and to the site you're still the same visitor, so the anti-bot check usually doesn't come back.

Using the result in Python

The important part is what you do after the browser:

import httpx

ROBOTEX = "https://app.robotex.sokolab.xyz"
PROXY = "http://user:pass@host:port"

robotex = httpx.Client(base_url=ROBOTEX, timeout=300, headers={"X-API-Key": "rbx_..."})


def solve(url: str) -> dict:
    data = robotex.post("/v1/solve", json={
        "url": url,
        "proxy": PROXY,
        "captcha_type": "cloudflare",
    }).json()  # session_id stays in robotex's cookie jar for the next calls
    if data.get("status") != "ok":
        raise RuntimeError(f"{data.get('code')}: {data.get('error')}")
    return data


solution = solve("https://example.com/projects")

site = httpx.Client(
    proxy=PROXY,                                     # the same proxy
    headers={"User-Agent": solution["user_agent"]},  # the same UA
    cookies={c["name"]: c["value"] for c in solution["cookies"]},
)
page = site.get("https://example.com/projects")
if page.status_code == 403:
    solution = solve("https://example.com/projects")  # same session, fresh cf_clearance

Two rules make this work: the same User-Agent and the same proxy. Cloudflare ties cf_clearance to the UA and usually to the IP. Change either and the cookie is worthless.

Lessons learned the hard way

Don't spoof the User-Agent

Early versions let clients pass their own user_agent. It seemed harmless: it's just a string. But you can only change the string. navigator.userAgentData, client hints and the engine's behaviour still belong to the real Chrome, and Cloudflare sees the mismatch. I measured it on Chrome 155 / Linux against a site behind Cloudflare:

User-Agent in the browser	Cloudflare
native (Linux Chrome 155)	passed 3 of 3, 7–9 s
Linux Chrome 147	failed 3 of 3
Windows Chrome 155	failed

So the parameter is gone (passing it returns 422), and the rule is: the service defines the browser's identity, the client adapts to the response. The same went for the window size. Each profile instead gets a plausible fingerprint (screen size and devicePixelRatio drawn from a real-world distribution) generated once and applied natively by Chrome, with no JS patches, since those broke the Cloudflare check.

Use real Chrome, not "Chromium with flags"

Robotex drives real Google Chrome through Patchright, an undetected Playwright fork without the CDP leaks anti-bots look for. Chrome runs headful under Xvfb, because headless mode is one more signal to hide.

Make the proxy's geo and the browser agree

Locale and timezone are derived from the proxy's exit IP. A German IP with a browser in America/New_York is a red flag. And no fingerprint will save you on a dirty IP: proxy quality matters more than any trick on the page.

Architecture v2: a queue between the API and the browsers

The first version ran Chrome inside the API process. When all browser slots were busy, a request simply got a 503. The current version splits it in two:

Client → api (FastAPI) ──task──▶ RabbitMQ ──▶ worker (dramatiq, Chrome + Xvfb)
            ▲                                         │
            └── GET /v2/tasks/{id} ◀── Redis ◀────────┘

api has no browser at all: it queues tasks and serves results;
RabbitMQ holds tasks until a worker has a free slot, so they wait instead of failing;
workers run Chrome, several windows each;
Redis keeps task statuses and results for a day. Proxy credentials and captcha keys never go there.

/v1/solve still exists and runs on top of the same queue, so old clients didn't change a line. New clients can go async:

POST /v2/tasks
X-API-Key: <your key>

{"url": "https://example.com/login", "actions": [...], "browser_idle_timeout": 10}

→ 202 {"task_id": "0b9c…", "status": "queued", "url": "/v2/tasks/0b9c…"}

Then poll GET /v2/tasks/{id} (with the same session_id cookie) until the status is done.

Keeping the browser window open

The feature I like most in v2 is browser_idle_timeout. With it, the session's Chrome window stays open for up to 120 idle minutes after the task. The next task with the same session_id is routed to the personal queue of the worker holding that window and runs right there: no Chrome start, no second anti-bot check. Leave out the url, and the actions run on the page that's already open.

That makes multi-step scenarios natural: log in, let the bot do its HTTP work, come back and continue filling the same form. When you're done, DELETE /v2/sessions/current closes the window and frees the slot.

What else it does

Captchas inside pages: the captcha step solves Turnstile, reCAPTCHA and hCaptcha through your own 2captcha key before the form is submitted.
Anti-bot pages: a detect → solve → verify loop for Cloudflare, before and during the scenario. Turnstile passes reliably.
Secrets stay out of logs: proxy credentials, captcha keys, fill values and cookie values are masked.

Try it

Robotex is free. You only pay for your own proxies and, if you need it, 2captcha.

Landing: robotex.sokolab.xyz
API and Swagger: app.robotex.sokolab.xyz/docs
API key: a test key for the demo instance is in the project boards
Source code: git.sokolab.xyz/s0k0l/robotex
Questions and bugs: Redmine forum and issues

Next on the roadmap: workers on several hosts, a second engine (Firefox via Camoufox) for sites where a Chrome fingerprint doesn't fit, and a bot that hands out personal API keys.

If you've fought Cloudflare from a scraper, I'd love to hear what worked for you, and what broke. And please use Robotex only on sites where you have the right to do so.

KeePass can't protect you from a stealer. So I built something that can.

Sokol — Wed, 27 May 2026 15:04:13 +0000

I built a hardware password manager because KeePass doesn't actually protect you

I'm a hobbyist developer. I work with sensitive data every day — accounts, crypto wallets, corporate systems, personal communications.

One day I had an uncomfortable realization.

The problem nobody talks about

I was using KeePassXC. Strong master password, encrypted database, local-only. I thought I was doing everything right.

Then I thought about what actually happens when I unlock my vault:

The encrypted database sits on disk — readable by any process with file access
The decrypted copy lives in RAM — readable by any process with memory access
The master password gets typed on a keyboard — interceptable by any keylogger
All of this happens inside an OS shared with dozens of other running programs

KeePass protects you from someone stealing your hard drive. It does not protect you from malware already running on your machine.

A stealer with a keylogger gets your master password when you type it, then grabs the database file. Game over. And you'll never know it happened.

The same is true for Bitwarden, 1Password, and every other software password manager. They're all prisoners of the OS they run on.

Cloud managers add another layer: you're now trusting a third-party server and its breach history.

Hardware keys like YubiKey solve part of the problem (2FA), but they don't replace a full offline password manager. They still require the target system to support their protocol.

I wanted something different.

What I actually wanted

A device that:

Stores passwords completely offline, on hardware
Never transfers the database to the computer — not even in encrypted form
Simply types the password like a keyboard (USB HID)
Works on any OS without installing software
Has no moving parts, no battery, no cloud

So I built it.

Introducing Protokey

Protokey is a compact USB device built on a microcontroller that acts as a "smart keyboard with memory."

You browse your vault on a 3.5" touchscreen → select an account → tap autofill → Protokey types your login, password, and Enter into whatever fi m,eld is focused. The computer sees only keystrokes. The database never leaves the device.

How it works technically

Hardware:

Panlee SC01 — 3.5" capacitive touchscreen (320×480), SD card slot, ESP32 microcontroller
Arduino with ATmega32u4 — handles USB HID pairing with the host computer
No physical buttons, no battery, no external power required

![Inside the device]d(https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d87jmgqxiikbntoyw92s.jpg)

Firmware & encryption:

Firmware, flash memory, bootloader, and user data are all encrypted
Master password stays in your head; derived key unlocks your data on the flash
Encryption keys for the firmware ship in the box with your device — they belong to you from day one
You can take full custody of your keys. If you want us to hold a copy for recovery, that's opt-in. If you want us to delete our copy, we will.

Unlike most hardware security products, we can't update your device without your explicit involvement. That's a feature, not a limitation.

The vault

Each record supports up to 16 fields. Field types:

Type	Purpose
text / word	plain text
login / email / phone	aggregation (fast search)
secret	hidden field with generator
TOTP	2FA time-based codes
tags	comma-separated, filterable
autofill template	`{field_name}`, `{tab}`, `{enter}`

The autofill template is the key feature. You define exactly what gets typed and in what order. Works for single-field logins, multi-step forms, desktop apps, terminal prompts — anything that accepts keyboard input.

Search and filtering happen in real time across visible fields, tags, and aggregation fields. Deep search (across all hidden fields) takes up to 3 minutes on 10,000 records — worth it when you need it.

Web interface

WiFi is used exclusively for device administration — never for password transmission.

Connect Protokey to your local network → access the web panel (HTTPS) to:

Create and edit records
Import/export backups
Update firmware
Manage TOTP entries

The WiFi credential setup and web panel access are done from the device's settings menu.

Physical threat protection

This one I'm quietly proud of.

Protokey has a duress system built into the PIN settings:

Second PIN — differs from your real PIN by one character. When entered, it switches the device into "panic mode": only shows records tagged with a specific decoy tag, sets the second PIN as the new primary, and wipes all duress settings — so there's no evidence a second PIN ever existed.
Blank password entry — by default, the device accepts any input at the unlock screen. You can require a specific master password, or leave it open and rely on PIN only.
Wipe after N attempts — standard brute-force protection.

The result: if someone demands access to your device, you hand them a plausible decoy vault. Your real records are in a hidden layer. The device shows no signs of having one.

Restoring from that state requires a backup. Keep one.

Current status

✅ Working MVP assembled from components ordered locally (Ozon)
✅ 3D-printed enclosure — 2cm thick, 3.5" wide, very light
✅ Full password manager on 10,000 records
✅ Autofill, TOTP, search, filters, web interface, duress protection

I understand the open-source expectation in this space. Here's where I stand: Protokey is a solo project with no funding. Opening the firmware immediately means someone with a factory in Shenzhen ships it for $40 while I'm still figuring out how to make the second batch. I'm choosing a middle path: the cryptography is documented and auditable, the firmware is available for review by security researchers, and I commit to open-sourcing if the project reaches sustainable production.

Why I'm posting this

I built Protokey because I wanted it for myself. Now I want to know if others feel the same way.

A few questions for you:

How do you store passwords today, and what's your biggest concern about it?
What's missing from current password managers?
Would you use a device like this?

If you're interested in following development — I'm setting up a Telegram channel.

Protokey — your passwords under your physical control.

Telegram: https://t.me/+-PpaF-4qGHNkZmY6