I Built a Headless Browser Toolkit With Chrome DevTools Protocol

I Built a Headless Browser Toolkit With Chrome DevTools Protocol — Here Is the Full Architecture

I needed to post to Twitter, Reddit, and dev.to from one codebase. Twitter's API has rate limits. Reddit bans bots. dev.to is fine. So I wrote 54 lines of JavaScript that automates the whole thing — no API keys, no OAuth, just a headless Chrome doing what you'd do manually.

It's the browser automation equivalent of raw SQL. You skip all the abstraction layers and go straight to what actually works.

The Problem

Every platform says "use our API." So you do. Then:

• Twitter API tiers are restrictive and expensive for indie devs. Free tiers barely cover casual use.
• Reddit moderation is aggressive toward automation — accounts that post programmatically without human review get flagged quickly.
• Gumroad's API covers product management, but it's limited compared to what you can do through their web interface. For complex product setups with multiple files and variants, browser automation is sometimes faster.
• Google kills APIs every 18 months.

Meanwhile, the actual browser sitting on your machine can do everything a logged-in user can do. That's the gap I went after.

Puppeteer is maintained by the Chrome team and works fine, but the CDP docs are scattered across Chrome DevTools protocol pages that haven't been updated since 2020. Selenium is ancient — WebDriver is a different paradigm entirely. Cypress is for testing, not automation.

I wanted something that did one thing well: give me the browser so I can control it programmatically. 54 lines later, I had it.

The Solution — cdpp.js

Here's the entire wrapper:

const { chromium } = require('playwright')
let browser, page

async function connect(url) {
  browser = await chromium.launch({ headless: true })
  page = await browser.newPage()
  if (url) await page.goto(url)
}

Five lines to launch a browser. Playwright handles Chromium, Firefox, and WebKit — I only use Chromium. No config needed. No playwright.config.js. No test runners.

async function goto(url) { await page.goto(url) }
async function pageText() { return await page.textContent('body') }

Navigation and content reading. pageText() pulls all visible text from the page — useful for scraping, link checking, or verifying content posted correctly.

async function clickByText(text) {
  await page.click(`text="${text}"`)
}

Instead of brittle CSS selectors or XPath queries, I click by what the user sees. clickByText("Tweet") finds the first visible element containing that text and clicks it. Playwright's auto-wait handles the timing — no setTimeout hacks.

async function fillInput(selector, value) {
  await page.fill(selector, value)
}

async function setFileInput(selector, filePath) {
  await page.setInputFiles(selector, filePath)
}

Form filling and file upload. fillInput clears the field first, then types. setFileInput handles the <input type="file"> element that's invisible to normal click automation.

async function screenshot(path) {
  await page.screenshot({ path, fullPage: true })
}

async function evalInTab(js) {
  return await page.evaluate(js)
}

Screenshots for debugging or documentation. evalInTab runs arbitrary JavaScript inside the page context — the same as opening DevTools console and typing code.

async function disconnect() {
  await browser.close()
}

Cleanup. Releases the Chromium process.

54 lines total. Just functions that call Playwright.

Real Example: Posting a Tweet

Here's how twitter-poster.js uses the toolkit:

const cdp = require('./cdpp')

async function postTweet(text) {
  await cdp.connect()
  await cdp.goto('https://x.com/compose/post')
  await cdp.fillInput('[data-testid="tweetTextarea_0"]', text)
  await cdp.clickByText('Tweet')
  await cdp.screenshot('./tweet-posted.png')
  await cdp.disconnect()
}

That's the core. Navigate to the compose screen, fill the text area, click Tweet, screenshot for confirmation.

No API key. No OAuth. No rate limits. It works because the browser is already logged in — the session cookies are in the Chrome profile. Same way you'd post a tweet manually.

The real version handles login state detection, waits for the compose modal to fully load, and retries if Twitter's React app hasn't rendered the textarea yet. But the core flow is exactly this.

The API tiers make automation painful. CDP sidesteps those limits — you post as fast as the page loads.

Real Example: Filling a Gumroad Form

Okay but real talk — Gumroad's API covers product management, but it's limited for complex setups. Their internal endpoints require CSRF tokens that rotate. So I automate the form directly.

await cdp.goto('https://app.gumroad.com/products/new')
await cdp.fillInput('#product_name', 'My New Product')
await cdp.fillInput('#product_description', 'Product description here')
await cdp.fillInput('#product_price', '29')
await cdp.setFileInput('input[type="file"]', './cover.png')
await cdp.clickByText('Publish')

This fills 15+ fields — name, description, price, permalink, cover image, content files. The file upload uses setFileInput which handles the hidden <input type="file"> that Gumroad uses.

The full script processes a product definition from a JSON file and fills every field automatically. Before this, creating a product on Gumroad took 20 minutes of manual clicking. Now it takes 30 seconds.

This is how CDP works in practice: you use the exact same interface the human uses, sidestepping missing or rate-limited APIs.

The Contrast: CDP vs REST API

Look, I know what you're thinking — isn't this just reinventing the wheel? Not really. Here's the tradeoff, concrete:

dev.to posting — REST API (129 lines):

const response = await fetch('https://dev.to/api/articles', {
  method: 'POST',
  headers: { 'api-key': API_KEY, 'Content-Type': 'application/json' },
  body: JSON.stringify({
    title: 'My Post',
    body_markdown: content,
    tags: ['webdev', 'javascript']
  })
})

Clean. Well-documented. Rate-limited to 30 articles/30 seconds, which nobody hits. The API handles markdown rendering, tag validation, canonical URLs — everything.

Twitter posting — CDP (200+ lines):

Navigate. Wait for React to render. Find the textarea by data-testid attribute. Type content character by character (because React synthetic events don't trigger on page.fill). Click Tweet. Handle the confirmation modal. Screenshot.

When to use which:

	CDP	REST API
Speed	Slow (full page load)	Fast (single HTTP call)
Reliability	Breaks when UI changes	Stable with versioning
Auth	Browser session (cookies)	API key / OAuth token
Rate limits	Browser speed only	Platform-defined
Capabilities	Anything a user can do	Only what the API exposes
Maintenance	UI changes = your code breaks	API changes = you update

My rule: If the platform has a good API, use it. dev.to, GitHub, Slack — all have clean REST APIs. Use them.

If the platform's API is rate-limited, expensive, or missing features — use CDP. Twitter, Reddit, Gumroad, LinkedIn. The browser doesn't care about your API tier.

When Should I Use CDP vs REST API?

Use CDP when:

• The platform has no API or a severely limited one
• You need to perform actions that the API doesn't expose
• API rate limits are blocking your automation
• You need to use a logged-in session (no API key available)
• The platform actively discourages automation (Reddit, LinkedIn)

Use REST API when:

• The platform has a well-documented, stable API
• You need speed (CDP is 10-100x slower per request)
• You're building something that needs to run at scale
• The API covers everything you need

Use both when:

• You want reliability for the core flow (API) but need to handle edge cases (CDP)
• You're migrating from one approach to the other

The toolkit supports both patterns. devto-poster.js uses the REST API. twitter-poster.js uses CDP. Same codebase, same patterns, different transport layer.

Why Not Just Use Puppeteer?

Puppeteer is fine. It works. But:

• Playwright has better auto-wait. Puppeteer requires manual waitForSelector calls. Playwright waits for elements to be actionable before clicking. Fewer race conditions, less debugging.

• Playwright supports multiple browsers natively. If you ever need to test in Firefox or WebKit, you're one line away. Puppeteer is Chromium-only.

• Playwright's text= selector is magic. page.click('text=Submit') finds visible text. Puppeteer requires you to write your own XPath or use page.evaluate to find elements by content.

• Playwright has faster feature velocity and broader browser support. Puppeteer remains maintained by the Chrome team with a CDP-first focus, but Playwright's API is more developer-friendly for automation tasks.

If you already have a Puppeteer codebase, don't rewrite it. But for new projects, Playwright is the better default.

The Architecture

cdpp.js (54 lines)
├── connect() → Playwright chromium.launch()
├── Navigation: goto(), pageText()
├── Interaction: clickByText(), fillInput(), setFileInput()
├── Observation: screenshot(), evalInTab(), listTargets()
└── Cleanup: disconnect()

Everything builds on top of this. twitter-poster.js imports cdpp. reddit-poster.js imports cdpp. verify-links.js imports cdpp. gumroad-upload.js imports cdpp.

One wrapper. Multiple tools. No shared state beyond the browser instance.

The listTargets() function is worth mentioning — it enumerates all open tabs and frames. Useful when you need to interact with popups or iframes without knowing their selector in advance.

The Reality Check

Browser automation violates the Terms of Service of most platforms. Account termination is a real risk. Use at your own risk.

Anti-bot systems (Cloudflare, DataDome, behavioral analysis) detect and block headless Chrome. Canvas fingerprinting, navigator.webdriver detection, and timing analysis are real. You'll hit walls on sites that invest in bot protection.

The 54 lines are the wrapper. Production scripts like twitter-poster.js are 200+ lines with retry logic, error handling, and session management. Don't expect to copy-paste the wrapper and have it work on every site.

CDP doesn't scale — one browser instance = one session. For scale, consider cloud browser services like Browserless or Browserbase.

Takeaway

Playwright + a thin CDP wrapper worked for me. Your mileage may vary.

If a platform's API is rate-limited, expensive, or missing features, browser automation is a viable fallback. The browser is the most universal interface we've built — every platform has a web version, and CDP lets you control it programmatically.

For new projects, start with the platform's API if one exists. When it doesn't — or when it's too restrictive — a small wrapper around Playwright gets the job done.

Found this useful? I'm building a PWA dev toolbox — check it out → TinyCoder Web Toolbox