A single machine tops out somewhere around 60 concurrent BitBrowser profiles on 16GB of RAM. To sustain 500, you need eight to ten worker nodes running in coordination, a shared queue, health checks, a proxy allocator, and a way to redistribute work when a node dies at 3 a.m. That coordination layer is what Docker Swarm is good at.
This article covers the architecture that actually holds up in production. What runs inside Swarm services, what has to sit on the host, why Kubernetes is not the answer at this scale, and the specific gotchas that will burn a week of your life if you skip them.
What Docker Swarm gives you here
Docker Swarm is Docker's native container orchestrator. It groups a set of machines into a single logical cluster, schedules services across them, gives you an overlay network that spans hosts, and restarts anything that dies. It ships in the Docker CLI, needs no extra install, and boots to a working cluster in about 30 seconds with docker swarm init.
For a browser profile pool, that translates to: one command to add a machine, one command to scale a service from three replicas to fifty, and one YAML file that describes the whole architecture. No etcd, no kubectl, no Helm chart. For a 500-profile workload, this is the right amount of tooling.
Why "BitBrowser in a container" is the wrong question
BitBrowser is a full Chromium build wrapped in an Electron desktop app. It expects a GUI session, a display server, and direct GPU access for Canvas and WebGL fingerprint rendering. You cannot drop the installer into docker run and get a working profile.
You have three real options:
- Windows Server nodes with Docker Windows containers. Works, but base images run 3-5 GB and cold starts are slow. Fine for CI, painful for a live pool.
- Linux nodes with Wine + Xvfb inside a Linux container. Fragile. Fingerprint values drift because GPU passthrough into containers is inconsistent, and drift is the one thing you cannot tolerate on an antidetect stack.
- Bare-metal or VM worker nodes running the BitBrowser desktop client, with Docker Swarm coordinating services around them. This is the pattern that works.
The rest of the article assumes option 3.
The three-tier architecture
┌─────────────────────────┐
│ Control plane (Swarm) │
│ ─ dispatcher │
│ ─ proxy allocator │
│ ─ session tracker │
│ ─ metrics + alerts │
└────────────┬────────────┘
│ overlay network
┌──────────────────┼──────────────────┐
│ │ │
┌─────▼─────┐ ┌─────▼─────┐ ┌─────▼─────┐
│ Worker 01 │ │ Worker 02 │ .. │ Worker 10 │
│ BitBrowser│ │ BitBrowser│ │ BitBrowser│
│ LocalAPI │ │ LocalAPI │ │ LocalAPI │
│ :54345 │ │ :54345 │ │ :54345 │
└───────────┘ └───────────┘ └───────────┘
Control plane: three manager nodes running Swarm services. Cheap VMs, 2 vCPU / 4GB RAM each. They host the dispatcher, proxy allocator, metrics stack, and any queue you use (Redis, NATS, RabbitMQ).
Workers: eight to ten physical or bare-metal VMs running the BitBrowser desktop client. Each targets 50-60 concurrent profiles on 32GB RAM. GUI enabled through a lightweight X server (Xorg with nvidia-drm or intel-drm), locked to a single logged-in user per box.
Overlay network: Swarm's built-in mesh network stretches across every node. Control-plane services talk to each worker's LocalAPI at worker-hostname:54345 as if it were localhost.
The docker-compose stack
The control plane deploys as a Swarm stack. A trimmed version of the compose file:
version: "3.8"
services:
dispatcher:
image: registry.example.com/dispatcher:1.4.2
deploy:
replicas: 3
placement:
constraints: [node.role == manager]
restart_policy:
condition: any
delay: 5s
environment:
REDIS_URL: redis://queue:6379
WORKER_POOL: bitbrowser
MAX_INFLIGHT_PER_WORKER: 55
networks: [pool-net]
queue:
image: redis:7.4-alpine
deploy:
replicas: 1
placement:
constraints: [node.hostname == manager-01]
volumes:
- queue-data:/data
networks: [pool-net]
proxy-allocator:
image: registry.example.com/proxy-allocator:2.1.0
deploy:
replicas: 2
environment:
PROVIDER: proxy-seller
STICKY_SESSION_TTL: 600
networks: [pool-net]
metrics:
image: prom/prometheus:v2.55.0
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
networks: [pool-net]
ports:
- "9090:9090"
networks:
pool-net:
driver: overlay
attachable: true
volumes:
queue-data:
docker stack deploy -c stack.yml pool brings the whole control plane up. Every worker joins with docker swarm join --token <worker-token> manager-01:2377, then registers itself with the dispatcher on boot through a small local agent.
The worker agent
On each BitBrowser node, a small Go or Node service runs alongside the desktop client. Its job is minimal:
- Poll the dispatcher for pending tasks
- Translate task JSON into BitBrowser LocalAPI calls (
/browser/open,/browser/close,/browser/update) - Report profile health, RAM use, and any error codes back to the metrics endpoint
- Kill runaway Chromium processes when RSS crosses a threshold
Keep it under 400 lines. Every line you add to the worker agent is a line that has to be debugged remotely at 3 a.m.
Proxy allocation and session isolation
The proxy allocator is a Swarm service that sits between the dispatcher and your proxy provider. It hands each new profile a sticky residential IP with a defined TTL, holds a lease table in Redis, and refuses to double-assign the same IP to two profiles in the same account cluster.
Two hard rules:
- One IP per profile per session lifetime. Rotating mid-session breaks Facebook, TikTok, and half of the fintechs. If the IP dies, mark the profile stale and rebuild it on a new IP.
- Never route control-plane traffic through the same proxy pool as the profiles. Prometheus scraping through a residential IP is how you burn a $600 subscription in three days.
Where BitCloudPhone fits
Docker Swarm handles the browser side. For the mobile side, where the fingerprint budget is tighter and the app SDK is more suspicious, containerized emulators break instantly. Real device farms are the only path that stays uncaught.
For Android workloads (TikTok Shop US, Shopee, Lazada, Snapchat growth), BitCloudPhone Android exposes ARM instances running on physical device racks, addressable over a REST API that fits neatly into the same dispatcher pattern above. For iOS-first apps (Tinder, Bumble, BeReal, DoorDash), BitCloudPhone iOS does the same on real iPhone hardware. Treat them as another worker pool in the dispatcher, tag-routed by task type: browser, android, ios. The cost model for a hybrid pool is covered in this Puppeteer-grid-on-cloud-phones architecture writeup.
Scaling math for 500 concurrent
Sizing that has held up across three production pools:
| Component | Spec | Cost /month |
|---|---|---|
| 10 worker nodes | 32GB RAM, 8 vCPU, 500GB SSD | $1,600 |
| 3 manager nodes | 4GB RAM, 2 vCPU, 40GB SSD | $90 |
| Residential proxies | 500 sticky IPs, 300GB traffic | $700 |
| Object storage (profile backups) | 200GB | $5 |
| Total | ~$2,395 |
That supports a steady 500 open profiles with roughly 15% headroom. Push the concurrent count higher and RAM becomes the ceiling well before CPU. Split across more nodes rather than fatter ones. A dead 32GB box loses 60 profiles; a dead 128GB box loses 240.
Failure modes you will actually hit
Four things break in the first month:
- Xvfb OOM at ~55 profiles per node. X buffers grow unbounded. Set a hard memory limit on the X server process and restart it via systemd when it crosses the line. The BitBrowser client reconnects; profiles survive.
-
Docker overlay network MTU mismatch. Default MTU is 1500; some cloud providers cap effective MTU at 1450 after encapsulation. Symptoms are silent packet drops on large API responses. Set
--opt com.docker.network.driver.mtu=1400on the overlay network at create time. - Redis becoming a single point of failure. The dispatcher's queue lives there. Run Redis with AOF persistence, back it up hourly, and rehearse the restore path. A lost queue means every worker is idle until you rebuild it.
- Proxy provider rate limits. Most providers cap new session creation at 20-30 per minute per account. If the dispatcher fires 500 new profiles at once, you starve. Add a token bucket in the proxy allocator sized to the provider's real limit.
Monitoring what matters
Three metrics predict every incident:
- Profile launch latency p95. If it climbs from 4 seconds to 15, a worker is failing without alerting.
- Proxy failure rate per pool. Above 3%, the pool is being rotated too aggressively or an IP block is starting.
- Worker agent heartbeat gap. Any node silent for over 60 seconds gets drained.
Prometheus scrapes each worker agent's /metrics endpoint every 10 seconds. Grafana dashboard, three panels, no more. If you cannot explain a spike from the dashboard alone, the dashboard has too much on it.
FAQ
Can I use Kubernetes instead of Docker Swarm for this?
Yes, but you probably should not. K8s adds three layers of abstraction (Deployments, StatefulSets, Ingress) that a 500-profile pool does not need. Swarm's compose file is 60 lines; the K8s equivalent is 400. Only switch if you already run K8s for other reasons.
Do I need GPU on the worker nodes?
For fingerprint fidelity, yes. Integrated GPUs (Intel UHD, AMD Vega mobile) are enough. Discrete GPUs give you a slight fingerprint diversity boost but the cost jump is not worth it under 1,000 profiles.
How do I handle profile backups?
The BitBrowser desktop app supports export to a shared folder. Mount an S3-compatible bucket on each worker with s3fs or rclone mount, schedule a daily export, keep 7 days of rolling snapshots. Restore is a copy back into ~/.bitbrowser/profiles/.
What happens when a worker node dies mid-session?
The dispatcher's inflight table detects the missing heartbeat, marks all profiles on that worker as stale, and requeues any in-progress task. Sessions that were mid-run on the failed node are lost. There is no way around that. Design your automation to be idempotent and short-lived.
Can I run this on my own hardware instead of cloud?
Yes. A rack of used Dell R730s with 128GB each will run 200 profiles per box and pay for itself in three months versus cloud. Colo costs and IP reputation are the tradeoffs.
Where to take it from here
Start with two worker nodes and one manager. Get the dispatcher, worker agent, and proxy allocator talking end to end. Prove you can create, launch, use, and close a single profile through the full stack. Only then add nodes.
Every "scale to N" post I have written started as a working 3-machine cluster that got copy-pasted a lot. The architecture above is the same. Start small, prove it, then multiply.
Affiliate disclosure: This post contains affiliate links. If you sign up for a paid BitBrowser or BitCloudPhone plan through the links above, I may earn a small commission at no additional cost to you. All infrastructure numbers above come from pools I run on my own paid accounts.
Top comments (0)