DEV Community: Vitalii Holben

How to Handle Anti-Bot Measures When Taking Screenshots Programmatically

Vitalii Holben — Thu, 03 Sep 2026 12:33:16 +0000

How to Handle Anti-Bot Measures When Taking Screenshots Programmatically

You send a request. The page loads. The screenshot comes back blank, or shows a CAPTCHA, or captures a "Please verify you're human" wall. This is one of the most common problems when building any screenshot pipeline.

Here's what's actually happening and how to deal with it.

Why headless browsers get flagged

Bot detection works by looking for patterns that differ from real users. Headless Chrome has several tells:

navigator.webdriver returns true by default
Missing Chrome-specific properties like window.chrome
Inconsistent screen dimensions (no monitor attached means no GPU info)
Mouse events fire at pixel-perfect coordinates with no jitter
Font fingerprints differ from headed browsers

Modern detection services (Cloudflare, Akamai, Datadome) look for combinations of these signals, not individual flags. Spoofing one without the others often makes the fingerprint more suspicious, not less.

The practical spectrum of detection

Most sites fall into one of three categories:

No active detection — a basic bot check via User-Agent string at most. Simple fix: set a realistic UA.

Passive fingerprinting — loads a detection script, collects signals, blocks on second or third visit. You'll see this on news sites, e-commerce, media platforms.

Active challenges — Cloudflare Turnstile, hCaptcha, reCAPTCHA v3 score-based. These require real interaction or a solving service.

Know which category your target falls into before spending time on it.

Fixes that work for most cases

1. Use a stealth plugin

For Playwright, playwright-extra with puppeteer-extra-plugin-stealth patches the most common fingerprinting vectors:

npm install playwright-extra puppeteer-extra-plugin-stealth

import { chromium } from 'playwright-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';

chromium.use(StealthPlugin());
const browser = await chromium.launch();

This handles navigator.webdriver, window.chrome, and several other flags automatically.

2. Set realistic headers

await page.setExtraHTTPHeaders({
  'Accept-Language': 'en-US,en;q=0.9',
  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
  'sec-ch-ua': '"Chromium";v="120", "Google Chrome";v="120"',
  'sec-ch-ua-mobile': '?0',
  'sec-ch-ua-platform': '"macOS"',
});

3. Use a real Chrome binary, not Chromium

Chromium's font set differs from Chrome. Detection services track which fonts are available. Running headful Chrome via executablePath gives you a more convincing fingerprint than the bundled Chromium.

4. Add realistic delays and mouse movement

// Move mouse before clicking
await page.mouse.move(100 + Math.random() * 50, 200 + Math.random() * 30);
await page.waitForTimeout(300 + Math.random() * 500);
await page.click('#target');

Perfectly timed, pixel-precise interactions are a dead giveaway. Add jitter.

When the above isn't enough

Some sites run heavy fingerprinting that's hard to spoof at the browser level. Options at this point:

Residential proxies — rotate real residential IPs. The IP reputation matters as much as the browser fingerprint. Datacenter IPs are often pre-blocked.

Browser profiles — maintain persistent browser profiles with cookies, browsing history, and localStorage. A "fresh" headless browser looks different from a browser with 3 weeks of history.

Playwright with persistent context:

const context = await chromium.launchPersistentContext('./user-data-dir', {
  headless: false, // or true with stealth
});

The ethical and legal line

There's a difference between taking screenshots of public pages for archival, testing, or display purposes — which is generally fine — and bypassing authentication or rate limits to scrape data at scale. The former is what most screenshot tools do. The latter runs into Terms of Service issues and, in some jurisdictions, legal risk.

Worth being clear on which side of that line your use case sits before investing in bypass techniques.

What to monitor

Once you have a working pipeline, track failure reasons explicitly:

const response = await page.goto(url, { waitUntil: 'networkidle2' });
if (response.status() === 403 || response.status() === 429) {
  // Bot detection hit — log, rotate IP, back off
}

const pageTitle = await page.title();
if (pageTitle.toLowerCase().includes('captcha') || pageTitle.includes('verify')) {
  // Challenge page — flag for manual review
}

Silent failures — a 200 status but a CAPTCHA screenshot — are harder to catch. Use a content hash comparison or check for expected elements before saving the screenshot.

The problem doesn't go away entirely, but being systematic about detection and response turns a reliability nightmare into something manageable.

How to Add Visual Regression Testing to Your CI Pipeline

Vitalii Holben — Mon, 31 Aug 2026 11:15:13 +0000

How to Add Visual Regression Testing to Your CI Pipeline

Your CSS change looked fine in the browser. You merged it. The deployment went out. Then someone noticed the pricing page hero section collapsed on mobile. The button text wrapped. The testimonial card lost its shadow.

Visual regression testing catches these before they reach production. Here's how to set it up without overcomplicating your pipeline.

The Concept

Take screenshots of key pages before and after a code change. Compare them pixel by pixel. If the difference exceeds a threshold, fail the build.

It's the visual equivalent of unit tests — you define what "correct" looks like, and the system tells you when something deviates.

Baseline Screenshots

First, capture reference screenshots of your pages in a known-good state. These are your baselines:

# capture-baselines.sh
PAGES=(
  "/"
  "/pricing"
  "/features"
  "/docs/getting-started"
  "/blog"
)

for page in "${PAGES[@]}"; do
  slug=$(echo "$page" | sed 's/\//_/g' | sed 's/^_//')
  [ -z "$slug" ] && slug="index"
  
  curl -s "https://screenshotrun.com/api/v1/screenshot?\
url=https://staging.yourapp.com${page}&\
width=1280&\
format=png&\
full_page=false" \
    -H "Authorization: Bearer $SCREENSHOT_API_KEY" \
    -o "baselines/${slug}.png"
    
  echo "Captured baseline: $slug"
done

Store baselines in your repository or artifact storage. They update only when you intentionally approve visual changes.

CI Comparison Script

On each pull request, capture the same pages from your preview/staging environment and compare against baselines:

#!/usr/bin/env python3
"""
visual_regression.py — compare current screenshots against baselines.
Exit code 1 if any page exceeds the diff threshold.
"""

import os
import sys
import requests
from pathlib import Path

try:
    from PIL import Image
    import numpy as np
except ImportError:
    print("pip install Pillow numpy")
    sys.exit(1)

API_URL = "https://screenshotrun.com/api/v1/screenshot"
API_KEY = os.environ["SCREENSHOT_API_KEY"]
STAGING_URL = os.environ.get("STAGING_URL", "https://staging.yourapp.com")
THRESHOLD = float(os.environ.get("VISUAL_DIFF_THRESHOLD", "2.0"))

PAGES = {
    "index": "/",
    "pricing": "/pricing",
    "features": "/features",
    "docs": "/docs/getting-started",
    "blog": "/blog",
}


def capture(url: str) -> bytes:
    params = {
        "url": url,
        "width": 1280,
        "format": "png",
        "full_page": False,
        "delay": 2,
    }
    resp = requests.get(
        API_URL,
        params=params,
        headers={"Authorization": f"Bearer {API_KEY}"},
        timeout=60,
    )
    resp.raise_for_status()
    return resp.content


def compare_images(baseline_path: str, current_bytes: bytes) -> float:
    baseline = np.array(Image.open(baseline_path).convert("RGB"))
    current = np.array(Image.open(
        __import__("io").BytesIO(current_bytes)
    ).convert("RGB"))

    # Resize if dimensions don't match
    if baseline.shape != current.shape:
        h = min(baseline.shape[0], current.shape[0])
        w = min(baseline.shape[1], current.shape[1])
        baseline = baseline[:h, :w]
        current = current[:h, :w]

    diff = np.abs(baseline.astype(int) - current.astype(int))
    changed_pixels = np.any(diff > 25, axis=2).sum()
    total_pixels = diff.shape[0] * diff.shape[1]
    return (changed_pixels / total_pixels) * 100


def main():
    baselines_dir = Path("baselines")
    failures = []

    for slug, path in PAGES.items():
        baseline_file = baselines_dir / f"{slug}.png"
        if not baseline_file.exists():
            print(f"  SKIP {slug}: no baseline")
            continue

        url = f"{STAGING_URL}{path}"
        print(f"  Checking {slug} ({url})...")

        img_bytes = capture(url)
        diff_pct = compare_images(str(baseline_file), img_bytes)

        if diff_pct > THRESHOLD:
            failures.append((slug, diff_pct))
            print(f"  FAIL {slug}: {diff_pct:.2f}% changed (threshold: {THRESHOLD}%)")

            # Save the current screenshot for review
            Path("diffs").mkdir(exist_ok=True)
            (Path("diffs") / f"{slug}_current.png").write_bytes(img_bytes)
        else:
            print(f"  OK   {slug}: {diff_pct:.2f}% changed")

    if failures:
        print(f"\n{len(failures)} page(s) exceeded visual diff threshold.")
        sys.exit(1)

    print("\nAll pages passed visual regression check.")


if __name__ == "__main__":
    main()

GitHub Actions Integration

name: Visual Regression
on: [pull_request]

jobs:
  visual-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      
      - name: Install dependencies
        run: pip install requests Pillow numpy
      
      - name: Run visual regression tests
        env:
          SCREENSHOT_API_KEY: ${{ secrets.SCREENSHOT_API_KEY }}
          STAGING_URL: ${{ env.PREVIEW_URL }}
          VISUAL_DIFF_THRESHOLD: "2.0"
        run: python visual_regression.py
      
      - name: Upload diff artifacts
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: visual-diffs
          path: diffs/

When the check fails, reviewers can download the diff artifacts and see exactly which pages changed and how.

Choosing What to Test

Don't screenshot every page. Focus on:

Revenue-critical pages — pricing, checkout, signup
High-traffic pages — homepage, main landing pages
Component-heavy pages — pages that use many shared components, so a single CSS change cascades

Skip pages with dynamic content (dashboards, feeds) unless you can seed them with consistent test data.

Managing Baseline Updates

When you intentionally change a page's appearance, the visual test will fail. That's expected. The workflow:

PR changes CSS/layout
Visual test fails (expected)
Review the diff artifacts to confirm changes look correct
Update baselines: ./capture-baselines.sh
Commit new baselines
Visual test passes

Some teams automate baseline updates with a bot comment — reply "update baselines" on the PR, and a workflow captures fresh screenshots and commits them.

Threshold Tuning

A 0% threshold catches everything, including subpixel rendering differences between environments. Too noisy.

A 5% threshold misses subtle changes — a shifted button, a truncated label. Too loose.

Start at 1-2% and adjust based on your false positive rate. Different pages might need different thresholds — a text-heavy docs page is more stable than a marketing page with animations.

Cost and Performance

Each screenshot API call takes 2-5 seconds. Testing 10 pages adds under a minute to your CI pipeline. The cost is minimal compared to the engineering time spent debugging visual bugs that shipped to production.

The tricky part isn't the technology — it's the discipline of maintaining baselines and reviewing diffs. Treat visual tests like any other test: if they fail, investigate before merging.

Using ScreenshotRun for the capture step keeps the infrastructure simple — no need to maintain your own headless browser pool in CI. Send a URL, get a screenshot, compare locally.

How to Build a Link Preview Thumbnail Service in Node.js

Vitalii Holben — Tue, 25 Aug 2026 12:42:03 +0000

Ever built a link aggregator, bookmarking tool, or dashboard that shows URLs? At some point you need thumbnails for those links. Users paste a URL, your app should show a preview — like Slack, Twitter cards, or Notion bookmarks.

You could self-host a browser instance and manage Puppeteer yourself. I did that for about a year. Scaling it was painful, and the memory leaks were real.

Here's how I rebuilt it with a screenshot API instead.

What we're building

A simple Express service that:

Accepts a URL
Returns a thumbnail image (cached)
Handles errors gracefully

The whole thing is under 80 lines.

Setup

mkdir link-thumbs && cd link-thumbs
npm init -y
npm install express node-cache axios sharp

The service

const express = require("express");
const NodeCache = require("node-cache");
const axios = require("axios");
const sharp = require("sharp");
const crypto = require("crypto");

const app = express();
const cache = new NodeCache({ stdTTL: 86400 }); // 24h cache

const API_KEY = process.env.SCREENSHOT_API_KEY;
const API_URL = "https://screenshotrun.com/api/screenshot";

async function getThumbnail(url, width = 1280, thumbWidth = 400) {
  const cacheKey = crypto
    .createHash("md5")
    .update(url + "-" + width + "-" + thumbWidth)
    .digest("hex");

  const cached = cache.get(cacheKey);
  if (cached) return cached;

  const response = await axios.get(API_URL, {
    params: {
      url,
      width,
      format: "png",
      full_page: false,
    },
    headers: { Authorization: "Bearer " + API_KEY },
    responseType: "arraybuffer",
    timeout: 30000,
  });

  // resize to thumbnail
  const thumb = await sharp(response.data)
    .resize(thumbWidth, null, { fit: "inside" })
    .webp({ quality: 80 })
    .toBuffer();

  cache.set(cacheKey, thumb);
  return thumb;
}

app.get("/thumb", async (req, res) => {
  const { url, w } = req.query;

  if (!url) {
    return res.status(400).json({ error: "url parameter required" });
  }

  // basic URL validation
  try {
    new URL(url);
  } catch {
    return res.status(400).json({ error: "invalid url" });
  }

  try {
    const thumb = await getThumbnail(url, 1280, parseInt(w) || 400);
    res.set("Content-Type", "image/webp");
    res.set("Cache-Control", "public, max-age=86400");
    res.send(thumb);
  } catch (err) {
    console.error("Failed: " + url, err.message);
    res.status(502).json({ error: "screenshot failed" });
  }
});

app.listen(3000, () => console.log("Thumbnail service on :3000"));

Using it

# get a thumbnail
curl "http://localhost:3000/thumb?url=https://github.com" -o github.webp

# custom width
curl "http://localhost:3000/thumb?url=https://dev.to&w=600" -o devto.webp

In your frontend, just use it as an image source:

<img 
  src="/thumb?url=https://example.com" 
  alt="example.com preview"
  loading="lazy"
/>

Why not self-host Puppeteer

I ran a self-hosted setup for a year. Here's what I dealt with:

Memory. Each Chromium instance eats 200-400MB. With 10 concurrent requests, you're looking at 4GB just for the browser processes. My 8GB droplet would OOM about once a week.

Zombie processes. Browsers crash. Tabs hang. Pages with infinite scroll or heavy JavaScript would occasionally lock up a tab. I wrote a watchdog script to kill stuck processes, which itself had bugs.

Font rendering. Missing fonts on Linux servers produce garbage screenshots. I installed a font pack, but CJK sites still looked wrong. Kept finding edge cases for months.

Timeouts. Some pages just take forever. A 30-second timeout seems generous until you hit a page that loads 47 third-party scripts and triggers a cookie consent modal that blocks rendering.

A screenshot API handles all of this. Browser pool management, font libraries, timeout handling, retries — someone else's problem. My service just makes HTTP calls and resizes images.

Improvements worth adding

Rate limiting. Without it, someone will feed your service 10,000 URLs and you'll burn through API credits. I use express-rate-limit — 10 requests per minute per IP is reasonable for most use cases.

const rateLimit = require("express-rate-limit");
app.use("/thumb", rateLimit({ windowMs: 60000, max: 10 }));

Persistent cache. NodeCache is in-memory — restarts kill it. For production, swap to Redis or just write thumbnails to disk with the URL hash as filename. Disk cache is underrated for images.

Fallback image. When a screenshot fails (site is down, blocked by firewall, etc.), return a generic placeholder instead of an error. Your UI shouldn't break because one link preview failed.

const FALLBACK = await sharp({
  create: { width: 400, height: 300, channels: 4, background: "#f0f0f0" }
}).webp().toBuffer();

Queue for bulk requests. If you're generating thumbnails for an import of 500 bookmarks, don't fire 500 concurrent API calls. Use a simple queue — bull or even just a promise pool with concurrency of 5.

What about Open Graph images?

OG images are great when they exist. But in practice:

~30% of URLs have no OG image at all
Many OG images are generic company logos, not page-specific
Some are broken CDN links
Quality varies wildly — some are 100x100 pixels

I use OG images as a first try, falling back to a screenshot when OG is missing or low quality. Best of both worlds — fast when OG exists, accurate when it doesn't.

Production numbers

Running this for a bookmarking app with ~2K daily active users:

Average thumbnail generation: 2-3 seconds (cold),

Generating PDFs From Web Pages Without a PDF Library

Vitalii Holben — Wed, 12 Aug 2026 13:11:02 +0000

Most PDF generation tutorials start with "install this PDF library" and then you spend an hour fighting with fonts, CSS support, and layout quirks. There's a simpler path if your content already lives on a web page.

The idea

Instead of rendering HTML to PDF inside your app, render the page in a real browser and print it to PDF. Chrome's DevTools Protocol has Page.printToPDF — same engine that powers "Save as PDF" in your browser, but automated.

Quick setup with Puppeteer

const puppeteer = require('puppeteer');

async function pageToPdf(url, outputPath) {
  const browser = await puppeteer.launch({ headless: 'new' });
  const page = await browser.newPage();

  await page.goto(url, { waitUntil: 'networkidle0' });

  await page.pdf({
    path: outputPath,
    format: 'A4',
    margin: { top: '20mm', bottom: '20mm', left: '15mm', right: '15mm' },
    printBackground: true
  });

  await browser.close();
}

pageToPdf('https://example.com/invoice/123', 'invoice.pdf');

That's it. Full CSS support, web fonts, flexbox, grid — everything renders exactly as it does in Chrome.

Things that tripped me up

Print stylesheets matter. If the page has @media print rules, they'll apply. This is actually useful — hide navigation, sidebars, cookie banners. But if you're not expecting it, your PDF might look nothing like the page.

@media print {
  nav, footer, .cookie-banner { display: none; }
  .content { width: 100%; margin: 0; }
}

waitUntil: 'networkidle0' isn't always enough. Pages with lazy-loaded images or JS-rendered charts might need explicit waits:

await page.waitForSelector('.chart-container canvas', { timeout: 5000 })
  .catch(() => console.log('chart not found, proceeding anyway'));

I learned this the hard way — generated 200 invoices with blank chart sections before noticing.

Headers and footers. The headerTemplate and footerTemplate options exist but they're fiddly. They use a tiny isolated context with limited CSS. For anything complex, I just bake the header into the page HTML itself.

When this approach makes sense

Invoices, reports, receipts — anything you already display on a web page
Generating PDFs from user-created content (blog posts, documentation)
Any situation where you want the PDF to match the web version pixel-for-pixel

Where it doesn't: high-volume generation (thousands per minute) where the browser overhead adds up, or cases where you need PDF-specific features like form fields or digital signatures.

Running it without managing Chrome

If you don't want to deal with keeping a Chrome instance running on your server — especially in containerized environments where Chromium dependencies are a pain — screenshot APIs like ScreenshotRun handle the browser part. You send a URL, get back a PDF or PNG. Offloads the rendering infrastructure entirely.

One more thing

The page.pdf() method has a scale option. Default is 1. Setting it to 0.8 or 0.9 can help fit content that's slightly too wide for the page. Beats reworking your CSS for print.

await page.pdf({
  path: 'output.pdf',
  format: 'A4',
  scale: 0.85,
  printBackground: true
});

Not something I see mentioned often but it's saved me from the "content gets cut off on the right edge" problem more than once.

How to Screenshot Dynamic Content That Changes on Every Page Load

Vitalii Holben — Mon, 03 Aug 2026 19:47:31 +0000

Some pages look different every time you load them. Randomized hero banners, A/B test variants, personalized product recommendations, rotating testimonials. If you're automating screenshots for testing or documentation, this is a problem.

You capture a baseline. You capture again tomorrow. The diff lights up red — not because anything broke, but because the carousel rotated and a different testimonial is showing.

The two types of dynamic content

There's server-side dynamic (different HTML on each request) and client-side dynamic (same HTML, JavaScript shuffles things around after load). The fix is different for each.

Server-side: you need to control the request. Pass a seed parameter, set a cookie, or hit a specific URL variant. If the page personalizes based on geo-IP, you need to pin your request to a consistent location.

Client-side: you need to run JavaScript before or after the page renders to freeze the randomization.

Freezing Math.random

A lot of front-end randomization uses Math.random(). Override it before the page loads:

await page.evaluateOnNewDocument(() => {
  let seed = 42;
  Math.random = () => {
    seed = (seed * 16807) % 2147483647;
    return (seed - 1) / 2147483646;
  };
});

Now every "random" value is deterministic. The carousel always shows the same slide. The A/B test always picks the same variant. Your screenshots become comparable.

Handling date/time displays

Pages that show "Posted 3 hours ago" or "Last updated: August 3, 2026" will always diff because the timestamp changes. Two options:

Option 1: Freeze the clock

await page.evaluateOnNewDocument(() => {
  const fixed = new Date('2026-01-15T12:00:00Z');
  const OrigDate = Date;
  Date = class extends OrigDate {
    constructor(...args) {
      if (args.length === 0) return super(fixed);
      return super(...args);
    }
    static now() { return fixed.getTime(); }
  };
});

Option 2: Mask the regions

If freezing the clock breaks page logic (timers, session management), mask the date regions in your diff comparison instead. Mark those areas as "ignore" in your visual diff tool.

I prefer option 1 for screenshot capture and option 2 for visual regression testing. Depends on whether you need the screenshot to look realistic or just be consistent.

CSS animations and transitions

An animated element captured mid-transition looks different every time. Kill all animations before capture:

await page.addStyleTag({
  content: `
    *, *::before, *::after {
      animation-duration: 0s !important;
      animation-delay: 0s !important;
      transition-duration: 0s !important;
      transition-delay: 0s !important;
    }
  `
});

This one saved me a ton of debugging. Had a landing page where a fade-in animation on the hero image made every screenshot slightly different in opacity. Took me a while to figure out it wasn't a rendering bug.

Lazy-loaded images and intersection observers

Content that loads on scroll won't appear in a screenshot if the viewport never scrolls to it. For full-page screenshots, scroll the page programmatically first:

await page.evaluate(async () => {
  await new Promise((resolve) => {
    let totalHeight = 0;
    const distance = 300;
    const timer = setInterval(() => {
      window.scrollBy(0, distance);
      totalHeight += distance;
      if (totalHeight >= document.body.scrollHeight) {
        clearInterval(timer);
        window.scrollTo(0, 0);
        resolve();
      }
    }, 100);
  });
});

Scroll to the bottom, trigger all the lazy loads, scroll back to top, then capture. Not elegant but it works reliably.

When you can't control the content

Sometimes you're screenshotting third-party pages you don't own. You can't inject seeds or override Math.random. In that case, accept that the screenshots will vary and adjust your comparison threshold.

Instead of pixel-perfect diffs, use structural comparison. Did the layout change? Did elements move? Is text content different? A 2-3% pixel difference from a rotated banner is noise. A 15% difference from a missing navigation bar is a real issue.

The goal isn't identical screenshots — it's catching meaningful changes while ignoring expected variation.

Building a Visual Changelog for Your Web App

Vitalii Holben — Sun, 26 Jul 2026 22:28:15 +0000

You ship a frontend change, merge the PR, deploy to production. Two days later your PM asks "what exactly changed on the pricing page last Tuesday?" and you're digging through git logs trying to reconstruct what the page looked like before and after.

Git tracks code. It doesn't track what the rendered page actually looked like to users.

A visual changelog fixes that. Capture a screenshot of key pages after every deploy, store them with timestamps, and you've got a browsable history of how your app looked at any point. Here's how to set one up.

The basic idea

After each deploy, a script hits a list of URLs and saves screenshots with metadata:

URL
Timestamp
Git commit hash
Deploy environment

You end up with a folder (or database) of timestamped screenshots you can scroll through. "What did the homepage look like three deploys ago?" becomes a five-second lookup instead of a twenty-minute archaeology project.

A minimal implementation

Here's a Node.js script that captures screenshots of a list of pages using Playwright:

const { chromium } = require('playwright');
const fs = require('fs');
const path = require('path');

const PAGES = [
  { name: 'homepage', url: 'https://yourapp.com' },
  { name: 'pricing', url: 'https://yourapp.com/pricing' },
  { name: 'dashboard', url: 'https://yourapp.com/dashboard' },
];

const COMMIT = process.env.GIT_COMMIT || 'unknown';
const TIMESTAMP = new Date().toISOString().replace(/[:.]/g, '-');
const OUTPUT_DIR = path.join(__dirname, 'changelog', TIMESTAMP);

async function capture() {
  fs.mkdirSync(OUTPUT_DIR, { recursive: true });

  const browser = await chromium.launch();
  const context = await browser.newContext({
    viewport: { width: 1440, height: 900 },
  });

  for (const page of PAGES) {
    const tab = await context.newPage();
    await tab.goto(page.url, { waitUntil: 'networkidle' });
    await tab.screenshot({
      path: path.join(OUTPUT_DIR, `${page.name}.png`),
      fullPage: true,
    });
    console.log(`captured ${page.name}`);
  }

  // save metadata
  fs.writeFileSync(
    path.join(OUTPUT_DIR, 'meta.json'),
    JSON.stringify({ commit: COMMIT, timestamp: TIMESTAMP, pages: PAGES.map(p => p.name) })
  );

  await browser.close();
}

capture().catch(console.error);

Run this as a post-deploy hook. In a GitHub Actions workflow:

- name: Capture visual changelog
  run: node capture-changelog.js
  env:
    GIT_COMMIT: ${{ github.sha }}

The problem with running your own browser

This works. I used a setup like this for about four months. Then I hit the issues:

Memory. Chromium eats 300-500MB per instance. On a CI runner with 4GB RAM, capturing 15 pages sequentially takes a while. Doing it in parallel kills the runner.

Font rendering. Screenshots taken on Ubuntu CI runners look different from macOS or Windows. If you're comparing screenshots across deploys and your CI runner image updated, every single screenshot shows a "diff" that's really just font antialiasing.

Flaky waits. doesn't mean "the page looks right." SPAs that fetch data on mount will show loading spinners in half your captures. You end up writing per-page wait logic and maintaining it forever.

After dealing with all three I switched to hitting a screenshot API (I use ScreenshotRun for this) instead of running Playwright locally. Same script structure, but instead of launching a browser you make an HTTP call:

const fetch = require('node-fetch');
const fs = require('fs');

async function captureViaApi(url, outputPath) {
  const response = await fetch(
    `https://api.screenshotrun.com/capture?` +
    `url=${encodeURIComponent(url)}&width=1440&height=900&full_page=true`,
    { headers: { 'Authorization': 'Bearer YOUR_API_KEY' } }
  );
  const buffer = await response.buffer();
  fs.writeFileSync(outputPath, buffer);
}

No browser to manage, consistent rendering environment, and it handles the wait logic for you. Tradeoff is you're paying per screenshot, but for a changelog that captures 10-20 pages per deploy it's negligible.

Making the changelog browsable

A folder of PNGs isn't great for anyone except the person who set it up. Build a simple viewer.

The laziest version that actually works: generate an HTML file that lists screenshots grouped by deploy.

function generateIndex(changelogDir) {
  const deploys = fs.readdirSync(changelogDir)
    .filter(d => fs.existsSync(path.join(changelogDir, d, 'meta.json')))
    .sort()
    .reverse();

  let html = '';

  for (const deploy of deploys) {
    const meta = JSON.parse(
      fs.readFileSync(path.join(changelogDir, deploy, 'meta.json'))
    );
    html += `<h2>${deploy} — ${meta.commit.slice(0, 8)}</h2>`;

    for (const page of meta.pages) {
      html += `
        <h3>${page}</h3>
        <img src="${deploy}/${page}.png" style="max-width: 100%; border: 1px solid #ccc;" />
      `;
    }
  }

  html += '';
  fs.writeFileSync(path.join(changelogDir, 'index.html'), html);
}

Deploy this to a static hosting bucket or serve it from an internal tool. PMs and designers can now browse the visual history without asking engineering.

Side-by-side diffing

The real power comes when you compare consecutive deploys. For each page, show the previous and current screenshot next to each other.

You can get fancier with pixel-level diffing using a library like :

const pixelmatch = require('pixelmatch');
const { PNG } = require('pngjs');

function diffScreenshots(beforePath, afterPath, diffPath) {
  const before = PNG.sync.read(fs.readFileSync(beforePath));
  const after = PNG.sync.read(fs.readFileSync(afterPath));
  const diff = new PNG({ width: before.width, height: before.height });

  const mismatchedPixels = pixelmatch(
    before.data, after.data, diff.data,
    before.width, before.height,
    { threshold: 0.1 }
  );

  fs.writeFileSync(diffPath, PNG.sync.write(diff));
  return mismatchedPixels;
}

If is above zero, something changed visually. You can even fail the deploy or send a Slack notification when unexpected visual changes are detected.

What pages to track

Don't capture everything. Start with pages that matter most:

Landing page and pricing — these change often and have business impact
Auth flows — login, signup, password reset. Regressions here directly cost you users
Core product screens — the 3-5 pages users spend the most time on
Email templates — if you render them as HTML, screenshot those too

I started with 6 pages and now track about 20. Add pages when something breaks that you wish you'd been tracking. Don't over-engineer it upfront.

What I'd skip

Pages with lots of dynamic content (dashboards with live data, feeds with user-generated content). The screenshots will always look "different" and you'll stop paying attention.
Admin panels. Unless you've got non-technical admins who report UI bugs, probably not worth it.
Every single marketing page. Track the ones that convert, not the ones that exist.

The whole setup takes maybe an afternoon. The first time someone asks "when did the pricing page change" and you pull it up in ten seconds, it pays for itself.

How to Know When Your Cron Job Silently Stops Running

Vitalii Holben — Wed, 22 Jul 2026 14:50:00 +0000

Had a backup script running every night at 2 AM for about six months. Worked great. Then one day I needed to restore something and discovered the backups stopped three weeks ago. The cron entry was still there. No errors in syslog. The script just... wasn't running.

Turned out a server update changed the PATH and the script couldn't find pg_dump anymore. It failed silently because I was redirecting stderr to /dev/null like a genius.

The core problem

Cron doesn't care if your job succeeds. It cares if it runs. And even "runs" is generous — if cron fires the command and it exits immediately with an error, cron considers that a success.

Most setups have zero monitoring on whether scheduled jobs actually complete. You find out they broke when the thing they were supposed to do didn't happen, which could be days or weeks later.

Dead man's switch — the simplest fix

The idea is backwards from normal monitoring. Instead of checking if something fails, you check if it doesn't report success.

At the end of your cron script, ping a URL:

#!/bin/bash
pg_dump mydb > /backups/mydb_$(date +%F).sql

if [ $? -eq 0 ]; then
    curl -fsS --retry 3 https://your-monitoring/ping/abc123 > /dev/null
fi

If the monitoring service doesn't receive that ping within the expected window (say, 30 minutes after 2 AM), it alerts you.

That's it. No agent to install, no daemon running, no config files. One curl at the end of your script.

What trips people up

Forgetting to check the exit code. If you just put the curl at the end without the if check, it pings even when the job fails. Defeats the entire purpose.

Timeout too tight. Your job usually takes 5 minutes but sometimes takes 20. If your monitoring window is 10 minutes, you'll get false alerts. Set it to 2x your worst case.

Multiple schedules. A job that runs every hour needs a different timeout than one that runs daily. I've seen people set up monitoring for a daily job and use a 1-hour timeout — so it alerts 23 times before the next run.

Swallowed output. Stop doing 2>/dev/null on cron jobs. Send stderr somewhere useful:

*/5 * * * * /path/to/script.sh >> /var/log/myjob.log 2>&1

DIY vs hosted

You can build this yourself. A small endpoint that logs pings, a checker that runs every minute and compares last-ping time against expected schedule, and a notification sender. Maybe 200 lines of code.

I did that once. It worked until the monitoring server itself went down and nobody noticed because nothing was monitoring the monitor. Classic.

Hosted services handle the reliability part — redundant infrastructure, multiple notification channels, escalation if you don't acknowledge. Worth it if you have more than a handful of jobs to track.

The minimum I'd set up for any project

Every cron job pings a URL on successful completion
Every ping has a timeout matched to the job's schedule
Failed pings send notifications to Slack or email, not just a dashboard
Cron output goes to a log file, not /dev/null

Takes maybe 15 minutes per job. Saves you from the "wait, when did this stop working?" conversation that nobody enjoys.

How to Build Rich Link Previews Without Maintaining a Browser Instance

Vitalii Holben — Fri, 17 Jul 2026 10:46:36 +0000

The problem with link previews

You paste a URL in Slack and get a nice preview card. Title, description, maybe a thumbnail. Users expect the same thing in your app.

The "standard" approach is parsing Open Graph tags. Fetch the page, grab og:title, og:image, done. Works for maybe 60% of URLs. The rest either don't have OG tags, have broken ones, or the image URL is a relative path that resolves to nothing.

So you think: I'll just screenshot the page and use that. Spin up Puppeteer, navigate to the URL, take a screenshot, crop it, serve it. Now you're maintaining a headless browser in production.

Why self-hosted Puppeteer gets painful

I ran this setup for about four months. Here's what went wrong:

Memory leaks. Chromium tabs that don't close properly. After a few hundred screenshots, the container would eat 2GB+ of RAM and start timing out. Had to add a cron job that restarted the service every 6 hours.

Font rendering. Pages with custom fonts looked wrong because the server didn't have them installed. You can install common font packages but you'll never have everything.

SPA timing. React and Vue apps need you to wait for JS to finish rendering. waitUntil: 'networkidle0' works sometimes. Other times you need custom wait logic per site.

Scaling. Each screenshot blocks a browser tab for 2-10 seconds. Want to handle 50 concurrent requests? That's 50 Chromium instances. Good luck with your cloud bill.

The API approach

After the third OOM incident, I moved to ScreenshotRun and the implementation got boring — in a good way. One HTTP call, get back an image.

async function getLinkPreview(url) {
  const apiUrl = `https://api.screenshotrun.com/capture`;

  const response = await fetch(apiUrl, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.SCREENSHOT_API_KEY}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      url: url,
      viewport: { width: 1200, height: 630 },
      format: 'webp',
      wait_for: 'networkidle',
      clip: { x: 0, y: 0, width: 1200, height: 630 }
    })
  });

  return response.url; // CDN URL to the screenshot
}

That's it. No browser pool, no memory management, no font issues.

Caching strategy

You don't want to screenshot the same URL every time someone loads a page. I cache the preview image URL in Redis with a 24-hour TTL:

async function getCachedPreview(url) {
  const cacheKey = `preview:${hashUrl(url)}`;
  const cached = await redis.get(cacheKey);

  if (cached) return cached;

  const imageUrl = await getLinkPreview(url);
  await redis.setex(cacheKey, 86400, imageUrl);

  return imageUrl;
}

For most apps, a 24-hour cache is fine. Users rarely share the same URL twice within minutes, and sites don't change their layout that often.

OG tags as fallback

I still parse OG tags first. If the page has a decent og:image (actual URL, not a relative path, responds with 200), I use that. Screenshots are the fallback for pages without good metadata.

async function getPreview(url) {
  const og = await parseOGTags(url);

  if (og.image && await isValidImage(og.image)) {
    return {
      title: og.title,
      description: og.description,
      image: og.image,
      source: 'og'
    };
  }

  // Fall back to screenshot
  const screenshot = await getCachedPreview(url);
  return {
    title: og.title || new URL(url).hostname,
    description: og.description || '',
    image: screenshot,
    source: 'screenshot'
  };
}

Numbers after switching

Response time: ~800ms for cache miss (API screenshot), ~5ms for cache hit
Memory: from 2GB+ (self-hosted Chromium) down to baseline Node.js (~150MB)
Reliability: zero OOM crashes in 3 months vs. weekly restarts before
Cost: roughly $15/month for our volume vs. $80/month for the beefy container

Not every link needs a screenshot. But for the ones that do, offloading the browser to an API saved us from a maintenance headache that kept growing.

Handling Lazy-Loaded Content in Automated Screenshots

Vitalii Holben — Sun, 12 Jul 2026 09:24:23 +0000

You set up Puppeteer, navigate to a page, call page.screenshot(), and the bottom half of your image is blank placeholder boxes. Welcome to lazy loading.

Most modern sites defer images and heavy content until the user scrolls. Your headless browser never scrolls. So those elements never load.

Here's how to deal with it.

The scroll trick

The most common fix is to programmatically scroll down the page before taking the screenshot:

async function scrollToBottom(page) {
  await page.evaluate(async () => {
    const delay = ms => new Promise(r => setTimeout(r, ms));
    const distance = 300;

    while (window.scrollY + window.innerHeight < document.body.scrollHeight) {
      window.scrollBy(0, distance);
      await delay(150);
    }

    window.scrollTo(0, 0);
  });
}

await page.goto("https://example.com", { waitUntil: "networkidle2" });
await scrollToBottom(page);
await page.waitForTimeout(1000);
await page.screenshot({ fullPage: true });

The 150ms delay between scrolls gives IntersectionObserver-based lazy loaders time to trigger. Too fast and you'll scroll past elements before they start loading.

That final waitForTimeout after scrolling back to top lets any remaining images finish rendering. Not elegant, but necessary.

Why networkidle2 isn't enough

You'd think waitUntil: "networkidle2" would handle this. It waits until there are no more than 2 network connections for 500ms. But lazy-loaded images haven't even been requested yet at that point — they're waiting for a scroll event that never happens.

networkidle2 only helps with content that loads on page init. For scroll-triggered content, you need the scroll.

The loading="eager" override

Some sites use the native loading="lazy" attribute. You can override it before images load:

await page.evaluateOnNewDocument(() => {
  Object.defineProperty(HTMLImageElement.prototype, "loading", {
    set: function(val) { this.setAttribute("loading", "eager"); },
    get: function() { return "eager"; }
  });
});

await page.goto("https://example.com");

evaluateOnNewDocument runs before any page script, so it intercepts lazy loading before it kicks in. This won't work for custom JS-based lazy loaders (which check scroll position or use IntersectionObserver), but it handles the native HTML attribute.

Dealing with IntersectionObserver loaders

Libraries like lazysizes, lozad, or custom implementations use IntersectionObserver. The scroll trick usually works, but here's a more reliable approach — force all observed elements to intersect:

await page.evaluate(() => {
  // Find all images with data-src (common lazy pattern)
  document.querySelectorAll("img[data-src]").forEach(img => {
    img.src = img.dataset.src;
    if (img.dataset.srcset) img.srcset = img.dataset.srcset;
  });

  // Same for background images
  document.querySelectorAll("[data-bg]").forEach(el => {
    el.style.backgroundImage = `url(${el.dataset.bg})`;
  });
});

Brute force, but it works when you know the lazy loading pattern. The downside is it's brittle — every site structures their lazy loading differently.

Infinite scroll pages

Social feeds, product listings, search results — pages that load more content as you scroll. You usually don't want the entire feed. Cap it:

async function scrollWithLimit(page, maxScrolls = 10) {
  let previousHeight = 0;
  let scrollCount = 0;

  while (scrollCount < maxScrolls) {
    await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight));
    await page.waitForTimeout(2000);

    const currentHeight = await page.evaluate(() => document.body.scrollHeight);
    if (currentHeight === previousHeight) break;

    previousHeight = currentHeight;
    scrollCount++;
  }

  window.scrollTo(0, 0);
}

The 2-second wait is generous but safe. Some APIs take a while to respond, and if you scroll again before new content renders, you'll think you've hit the bottom when you haven't.

When scrolling isn't practical

Some pages are just hard to capture reliably. Heavy SPAs with virtual scrolling, pages that require authentication state, sites with aggressive bot detection that block headless browsers.

At some point the complexity of handling every edge case exceeds the value of doing it yourself. I've used ScreenshotRun for projects where I needed consistent captures without babysitting the rendering pipeline — it handles the lazy loading, wait conditions, and viewport stuff on their end.

Not always necessary, but worth knowing the option exists when your homegrown setup starts eating more time than it saves.

Quick checklist

Before you take a full-page screenshot:

Scroll the full page height with delays between steps
Wait 1-2 seconds after scrolling for images to finish loading
Override loading="lazy" if the site uses native lazy loading
Set a realistic viewport width (not the 800x600 default)
Use fullPage: true — otherwise you only capture the viewport area

Most screenshot issues come down to timing. The browser is ready before the content is. When in doubt, wait longer.

A Developer Guide to Viewport Sizes for Screenshots

Vitalii Holben — Wed, 08 Jul 2026 10:35:46 +0000

const viewports = [
  { width: 1920, height: 1080 },  // desktop full HD
  { width: 1366, height: 768 },   // most common laptop
  { width: 375, height: 812 },    // iPhone X/11/12
];

That's probably 80% of what you need. But if you're automating screenshots at scale — for visual regression, PDF generation, social previews — "probably" isn't good enough, and the edge cases will find you.

The viewport problem nobody warns you about

I wasted an entire afternoon once trying to figure out why a client's dashboard screenshots looked fine on my machine but came out wrong in CI. The page had a sidebar that collapsed at 1024px. My local browser was 1440px wide, the headless Chrome in CI defaulted to 800x600.

800x600. In 2026.

That's the default viewport for headless Chromium if you don't set one. Puppeteer uses it too unless you override it in launch() or page.setViewport(). Playwright defaults to 1280x720, which is slightly better but still catches people off guard.

// Puppeteer — you HAVE to set this
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setViewport({ width: 1366, height: 768 });

# Playwright (Python) — set in browser context
context = browser.new_context(viewport={"width": 1920, "height": 1080})

Which sizes actually matter

Here's the thing — the "right" viewport depends on what you're screenshotting and why. There's no universal list.

For visual regression testing, match your actual user base. Pull your analytics. If 40% of your traffic is 1366x768 laptop users and 35% is mobile, test those two. Maybe throw in 1920x1080 for good measure. Three viewports covers most projects.

For social preview images (og:image, Twitter cards), you want exactly 1200x630. Not approximately. Exactly. Some tools crop from center, some from top-left, and if your viewport doesn't match the output ratio you'll get weird results.

For PDF generation from HTML, it's different again. You're not matching a real screen — you're matching a paper size. A4 at 96dpi is roughly 794x1123 pixels. Letter is 816x1056.

// PDF-oriented viewport
await page.setViewport({ width: 794, height: 1123 });
await page.pdf({ format: 'A4', printBackground: true });

Device emulation vs. viewport size

Setting the viewport width to 375px doesn't make your page behave like an iPhone.

Two things are missing: device pixel ratio and the user agent string. A real iPhone has a DPR of 3, so a 375px wide screen actually renders at 1125 physical pixels. If your page serves different images based on srcset or uses window.devicePixelRatio in JS, you'll get different results.

await page.setViewport({
  width: 375,
  height: 812,
  deviceScaleFactor: 3
});
await page.setUserAgent(
  'Mozilla/5.0 (iPhone; CPU iPhone OS 16_0 like Mac OS X)...'
);

Playwright has a nicer API for this — device descriptors built in:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    iphone = p.devices["iPhone 13"]
    browser = p.chromium.launch()
    context = browser.new_context(**iphone)

That handles viewport, DPR, user agent, and touch events all at once. I wish Puppeteer had something this clean out of the box.

The full-page vs. visible-area trap

This trips people up constantly. When you set a viewport to 1366x768 and take a full-page screenshot, the width stays at 1366 but the height extends to cover the entire scrollable content. Your viewport height setting gets effectively ignored.

That's usually fine. Unless your page has sticky elements, lazy-loaded images, or scroll-triggered animations.

Sticky headers will repeat in every "fold" of a full-page screenshot in some tools. Lazy images below the fold won't load because the browser never actually scrolled. And scroll-triggered CSS animations will stay in their initial state.

Fixes:

// Force-scroll to trigger lazy loading
await page.evaluate(async () => {
  await new Promise(resolve => {
    let totalHeight = 0;
    const distance = 400;
    const timer = setInterval(() => {
      window.scrollBy(0, distance);
      totalHeight += distance;
      if (totalHeight >= document.body.scrollHeight) {
        clearInterval(timer);
        resolve();
      }
    }, 100);
  });
});

// Then scroll back to top
await page.evaluate(() => window.scrollTo(0, 0));

// Now take the screenshot
await page.screenshot({ fullPage: true });

Not elegant. Works though.

Responsive breakpoints worth testing

Skip the generic "test every device" advice. In practice, the breakpoints that catch bugs are the ones where your CSS layout shifts. Look at your own stylesheets:

/* These are YOUR breakpoints — test at these widths */
@media (max-width: 1024px) { /* tablet layout kicks in */ }
@media (max-width: 768px)  { /* mobile nav appears */ }
@media (max-width: 480px)  { /* single column */ }

Test at the breakpoint and one pixel above it. 1024 and 1025. 768 and 769. That's where the visual glitches hide — elements half-collapsed, overlapping text, buttons that are technically visible but pushed off to the side.

I usually test at these widths for a typical marketing site:

1920 — full desktop
1366 — laptop (this is the one people forget)
1024 — tablet landscape / small desktop
768 — tablet portrait
375 — mobile

Five viewports. Covers the real-world spread without going overboard.

Retina and HiDPI output

If you're generating screenshots for documentation, marketing pages, or app stores, you probably want 2x output even from desktop viewports. Set deviceScaleFactor: 2 and your 1366x768 viewport produces a 2732x1536 image.

await page.setViewport({
  width: 1366,
  height: 768,
  deviceScaleFactor: 2
});

File sizes roughly quadruple. Keep that in mind if you're storing thousands of screenshots.

Quick reference

Use case	Width	Height	DPR
Desktop screenshot	1920	1080	1
Laptop	1366	768	1
Tablet	768	1024	2
Mobile (iPhone-ish)	375	812	3
Social preview (og:image)	1200	630	1
PDF (A4)	794	1123	1
Retina desktop	1440	900	2

Don't treat this table as gospel — check your own analytics and test the breakpoints that match your CSS. That's what actually catches bugs.

How to Automate OG Image Generation for Your Blog Using a Screenshot API

Vitalii Holben — Thu, 02 Jul 2026 09:48:48 +0000

Every blog post needs an OG image. Without one, your links look blank on Twitter, LinkedIn, and Slack — just a plain URL that nobody clicks. Most developers solve this by spinning up a headless browser, loading an HTML template, taking a screenshot, and uploading it somewhere. It works, but now you're maintaining a Puppeteer instance, dealing with font rendering quirks, and burning server resources on something that should be simple.

There's a faster approach: design your OG images as HTML templates and let a screenshot API handle the rendering.

The Idea: HTML Templates as OG Images

Think of your OG image as a tiny webpage. You already know HTML and CSS. Build a 1200×630 template with your blog title, author name, maybe a gradient background — whatever fits your brand. Host it or pass it as raw HTML. Then call an API to screenshot it. Done.

A basic template might look like this:

<div style="width:1200px;height:630px;display:flex;align-items:center;
  justify-content:center;background:linear-gradient(135deg,#1a1a2e,#16213e);
  font-family:Inter,sans-serif;padding:60px">
  <div style="color:#fff;text-align:center">
    <h1 style="font-size:48px;margin:0">{{title}}</h1>
    <p style="font-size:24px;color:#8892b0;margin-top:20px">{{author}} · {{date}}</p>
  </div>
</div>

Replace the placeholders on your server, then send the resulting HTML (or a URL pointing to it) to the API.

Calling the API

With ScreenshotRun, a single curl request captures the rendered template as a PNG:

curl -X POST "https://api.screenshotrun.com/v1/screenshot" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://yourblog.com/og-template?title=My+Post+Title",
    "viewport_width": 1200,
    "viewport_height": 630,
    "format": "png"
  }'

The response gives you the image file. Save it to your CDN, set the og:image meta tag, and you're done. No browser to manage, no Chrome binary eating RAM on your CI server.

Wiring It Into Your Build

If you publish with a static site generator or a CMS, hook this into your build step or post-publish webhook. For each new post:

Generate the template URL with the post's title and metadata.
Call the screenshot API.
Upload the returned image to S3 or your storage of choice.
Set <meta property="og:image"> in the page head.

The whole pipeline runs in a few seconds per post. No headless browser dependencies, no Docker containers for rendering. Your CI stays clean.

Why Not Just Use Puppeteer Directly?

You can. But managing Chromium in production is annoying. Font loading breaks between environments. Lambda has size limits. Cold starts add latency. A screenshot API offloads all of that — you send a request, you get an image. Someone else worries about the browser.

If you want to see more patterns like this, ScreenshotRun has an OG image generator use-case page with additional examples and template ideas.

Quick Tips

Keep your template at 1200×630. That's the standard OG image size.
Use web-safe fonts or Google Fonts loaded via <link> in your template.
Cache aggressively. OG images rarely change after publish.
Test with the Twitter Card Validator and Facebook's Sharing Debugger before going live.

That's the whole setup. HTML template, one API call, and your blog posts stop looking naked when someone shares them.

How to take screenshots of password-protected pages with a screenshot API

Vitalii Holben — Fri, 10 Apr 2026 13:07:05 +0000

Not every page you need to screenshot is open to the world. Sometimes you need a screenshot of a service admin panel, an internal dashboard, a staging server, or a page behind basic auth. And that's where the problem starts: you send the URL to a screenshot API, and what comes back is a screenshot of the login form. The headless browser on the API side doesn't know your credentials and visits the page as a brand new user.

I want to walk through how to handle this. As an example, I'll use my own project fixheaders.com, a security headers scanner. It has a Filament admin panel, and I periodically need a screenshot of the dashboard showing scan counts without logging in manually every time.

I'll cover three authentication methods: cookies, custom HTTP headers, and Basic Auth. Each method comes with code examples in cURL, Node.js, and PHP.

Why a regular screenshot request won't work

When you ask a screenshot API (or Playwright, or Puppeteer) to capture a URL, the headless browser opens it as a completely new visitor. No session, no cookies, no saved credentials. If the page requires a login, the server sees an unauthenticated request and redirects to the login form. Or just shows it directly.

Here's what happens when you send a regular request to the fixheaders.com admin URL without any cookies:

curl -X POST https://screenshotrun.com/api/v1/screenshots \
  -H "Authorization: Bearer $SCREENSHOTRUN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://fixheaders.com/admin"}'

We wait a few seconds and download the result:

curl -H "Authorization: Bearer $SCREENSHOTRUN_KEY" \
  https://screenshotrun.com/api/v1/screenshots/SCREENSHOT_ID/image \
  -o screenshot-no-auth.png

Open the file and you see this:

The login form. This is exactly what the headless browser sees when it visits a protected URL without authentication. It's the same thing that happens when you open a private URL in an incognito window. The browser has no cookies, so the server treats you as a stranger.

The fix is to give the headless browser the same credentials your regular browser already has. There are a few ways to do that, depending on how the target site handles authentication.

Method 1: pass session cookies (the most common case)

Most web apps (Laravel, Django, Rails, WordPress, Filament, any admin panel) use cookie-based sessions. When you log in, the server creates a session and sends a cookie. Every subsequent request includes that cookie, and the server knows who you are.

The idea is simple: grab the session cookie from your browser and pass it to the screenshot API. The headless browser will send that cookie with the request, the server will see a valid session, and you'll get a screenshot of the actual dashboard, not the login page.

Step 1: find your session cookie

Open the site you want to screenshot (in my case, the fixheaders.com admin panel). Log in normally. Then open DevTools → Application → Cookies (in Chrome) or Storage → Cookies (in Firefox).

Look for the session cookie. In Laravel apps it's usually called laravel_session or a custom name from config/session.php. In my case it's fixheaders-session. The name varies across frameworks. In some apps it's PHPSESSID, in Django it's sessionid, in Rails it's _yourapp_session.

Copy the cookie value. It'll look something like this:

eyJpdiI6IkxMNk1DVjZhN0FKWjZ2a3...long_base64_string

Step 2: pass it to the screenshot API

Here's how to send that cookie with a screenshotrun API request.

cURL:

curl -X POST https://screenshotrun.com/api/v1/screenshots \
  -H "Authorization: Bearer $SCREENSHOTRUN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://fixheaders.com/admin",
    "cookies": [
      {
        "name": "fixheaders-session",
        "value": "eyJpdiI6IkxMNk1DVjZhN0FKWjZ2a3...",
        "domain": "fixheaders.com"
      }
    ]
  }'

The cookies parameter takes an array of objects. Each one needs a name, value, and domain. The domain tells the browser which site to send the cookie to. Without it, the cookie might not get attached to the request.

Download the result the same way, using the ID from the response:

curl -H "Authorization: Bearer $SCREENSHOTRUN_KEY" \
  https://screenshotrun.com/api/v1/screenshots/SCREENSHOT_ID/image \
  -o screenshot-with-auth.png

And here's what we get this time:

The actual dashboard. Scan counts, average score, recent checks. Exactly what I see when I'm logged in through the browser. One cookie made all the difference.

Node.js:

const response = await fetch("https://screenshotrun.com/api/v1/screenshots", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://fixheaders.com/admin",
    cookies: [
      {
        name: "fixheaders-session",
        value: "eyJpdiI6IkxMNk1DVjZhN0FKWjZ2a3...",
        domain: "fixheaders.com",
      },
    ],
  }),
});

const { data } = await response.json();
console.log(data.id);

PHP (Laravel):

If you're working with PHP and haven't set up the screenshot API yet, I covered the full setup process in the PHP screenshot tutorial. Here's the short version for authenticated pages:

$response = Http::withToken(env('SCREENSHOTRUN_API_KEY'))
    ->post('https://screenshotrun.com/api/v1/screenshots', [
        'url' => 'https://fixheaders.com/admin',
        'cookies' => [
            [
                'name' => 'fixheaders-session',
                'value' => 'eyJpdiI6IkxMNk1DVjZhN0FKWjZ2a3...',
                'domain' => 'fixheaders.com',
            ],
        ],
    ]);

$screenshotId = $response->json('data.id');

After the request completes, poll the screenshot status or use a webhook to get notified when it's ready.

What can go wrong with cookies

Session expiration. This is the biggest one. Session cookies have a lifetime. In Laravel the default is 2 hours. After that, the session is invalid and the screenshot API will get the login page again. If you're automating this, you need to refresh the cookie periodically. One option is to write a script that logs in via HTTP (POST to the login endpoint with credentials), grabs the fresh session cookie from the Set-Cookie response header, and uses that for the screenshot request.

Multiple cookies. Some apps require more than one cookie. CSRF tokens, "remember me" cookies, or multi-cookie session setups. If the screenshot comes back as a login page even though you're sending the session cookie, check whether the site sets additional cookies during login. You can pass up to 20 cookies in a single screenshotrun request, which covers pretty much any setup.

HttpOnly and Secure flags. Session cookies are often marked as HttpOnly (can't be read by JavaScript) and Secure (only sent over HTTPS). These flags don't affect the screenshot API. The headless browser handles them the same way a regular browser does. Just make sure you're using HTTPS URLs.

SameSite cookies. If the cookie has SameSite=Strict, it might not get sent in certain cross-origin scenarios. With a screenshot API, the headless browser navigates directly to the URL (it's a top-level navigation, not a cross-origin fetch), so SameSite=Strict cookies are included normally.

Method 2: custom HTTP headers (API tokens and Bearer auth)

Not every protected page uses cookies. Internal tools, API documentation portals, and headless CMS interfaces sometimes protect pages with custom HTTP headers: an API token, a Bearer token, or a custom X-Auth-Token header.

If the page you need to screenshot checks for a specific header instead of (or in addition to) cookies, you can pass custom headers through the screenshot API.

cURL:

curl -X POST https://screenshotrun.com/api/v1/screenshots \
  -H "Authorization: Bearer $SCREENSHOTRUN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://internal-tool.example.com/dashboard",
    "headers": {
      "X-Auth-Token": "your-internal-token-here"
    }
  }'

Node.js:

const response = await fetch("https://screenshotrun.com/api/v1/screenshots", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://internal-tool.example.com/dashboard",
    headers: {
      "X-Auth-Token": "your-internal-token-here",
    },
  }),
});

PHP:

$response = Http::withToken(env('SCREENSHOTRUN_API_KEY'))
    ->post('https://screenshotrun.com/api/v1/screenshots', [
        'url' => 'https://internal-tool.example.com/dashboard',
        'headers' => [
            'X-Auth-Token' => 'your-internal-token-here',
        ],
    ]);

The headers parameter accepts an object where keys are header names and values are header values. You can send up to 20 custom headers per request.

One thing to watch out for: the Authorization header in the headers parameter is the one sent to the target page, not to the screenshotrun API itself. The API key for authenticating with screenshotrun goes in the top-level Authorization header of the HTTP request. The headers object is what the headless browser sends when loading the target URL. These are two different things, and it's easy to mix them up.

Method 3: Basic Auth for staging and htpasswd-protected sites

If you've ever password-protected a staging server with Nginx's auth_basic or Apache's .htpasswd, you know how it works. The browser shows a modal dialog asking for a username and password. Headless browsers can't interact with that dialog, so the request just fails or returns a 401.

The fix is simple: send the credentials as a Basic Auth header. The browser won't see the dialog at all because the server gets the credentials before it has a chance to ask for them.

cURL:

curl -X POST https://screenshotrun.com/api/v1/screenshots \
  -H "Authorization: Bearer $SCREENSHOTRUN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://staging.example.com",
    "headers": {
      "Authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ="
    }
  }'

The Basic dXNlcm5hbWU6cGFzc3dvcmQ= part is just username:password encoded in Base64. You can generate it from the terminal:

echo -n "username:password" | base64
# Output: dXNlcm5hbWU6cGFzc3dvcmQ=

Node.js:

const credentials = Buffer.from("username:password").toString("base64");

const response = await fetch("https://screenshotrun.com/api/v1/screenshots", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://staging.example.com",
    headers: {
      "Authorization": `Basic ${credentials}`,
    },
  }),
});

PHP:

$credentials = base64_encode('username:password');

$response = Http::withToken(env('SCREENSHOTRUN_API_KEY'))
    ->post('https://screenshotrun.com/api/v1/screenshots', [
        'url' => 'https://staging.example.com',
        'headers' => [
            'Authorization' => 'Basic ' . $credentials,
        ],
    ]);

This method works for any site that uses HTTP Basic Authentication. It's common on staging environments, dev servers, and some internal tools. If you're monitoring your staging after deployments, I wrote a separate article about why screenshots catch things logs miss that covers the reasoning behind this.

Combining cookies with other capture options

All three authentication methods work alongside the regular screenshot parameters. You can take a full-page screenshot of an authenticated dashboard in dark mode, at a mobile viewport, with cookie banners blocked. All in one request:

curl -X POST https://screenshotrun.com/api/v1/screenshots \
  -H "Authorization: Bearer $SCREENSHOTRUN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://fixheaders.com/admin",
    "cookies": [
      {
        "name": "fixheaders-session",
        "value": "your-session-cookie-value",
        "domain": "fixheaders.com"
      }
    ],
    "full_page": true,
    "dark_mode": true,
    "device": "mobile",
    "format": "webp"
  }'

I covered the full list of parameters (dark mode, device presets, CSS injection and element selectors) in the API documentation. If you haven't looked at the other capture options yet, the cURL examples article walks through each one with copy-paste commands.

Automating the login (when cookies expire too fast)

If you want to capture authenticated screenshots on a schedule (say, every hour for a dashboard you embed somewhere), manually copying cookies won't cut it. Sessions expire, and your screenshot will revert to a login page.

The workaround is to automate the login step itself. Instead of grabbing cookies from a browser by hand, write a script that logs in programmatically, extracts the session cookie, and passes it to the screenshot API.

Here's a Node.js example using plain fetch (if you need more context on the Node.js setup, check out the full Node.js tutorial):

async function getSessionCookie(loginUrl, email, password) {
  const response = await fetch(loginUrl, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ email, password }),
    redirect: "manual",
  });

  const setCookie = response.headers.get("set-cookie");
  if (!setCookie) {
    throw new Error("No Set-Cookie header in login response");
  }

  const match = setCookie.match(/(\w+[-_]session)=([^;]+)/);
  if (!match) {
    throw new Error("Could not find session cookie in Set-Cookie header");
  }

  return { name: match[1], value: match[2] };
}

// Usage
const cookie = await getSessionCookie(
  "https://fixheaders.com/login",
  "your@email.com",
  "your-password"
);

const response = await fetch("https://screenshotrun.com/api/v1/screenshots", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://fixheaders.com/admin",
    cookies: [
      {
        name: cookie.name,
        value: cookie.value,
        domain: "fixheaders.com",
      },
    ],
  }),
});

The login endpoint depends on your app. Laravel's default route is POST /login with email and password fields (plus a CSRF token, more on that below). Django uses POST /accounts/login/. WordPress uses POST /wp-login.php with log and pwd fields.

The CSRF token problem

Most frameworks protect login forms with CSRF tokens. If you just POST to /login without a valid token, you'll get a 419 (Laravel) or 403 (Django/Rails). You need to first GET the login page, extract the CSRF token from the HTML or cookies, then include it in the POST request.

In Laravel, the CSRF token lives in a cookie called XSRF-TOKEN (URL-encoded) and needs to be sent back as an X-XSRF-TOKEN header or as a _token form field:

async function laravelLogin(baseUrl, email, password) {
  // Step 1: GET the login page to grab the CSRF cookie
  const loginPage = await fetch(`${baseUrl}/login`);
  const csrfCookie = loginPage.headers
    .get("set-cookie")
    ?.match(/XSRF-TOKEN=([^;]+)/)?.[1];

  if (!csrfCookie) {
    throw new Error("Could not extract CSRF token");
  }

  const csrfToken = decodeURIComponent(csrfCookie);

  // Step 2: POST login credentials with the CSRF token
  const response = await fetch(`${baseUrl}/login`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-XSRF-TOKEN": csrfToken,
      Cookie: `XSRF-TOKEN=${csrfCookie}`,
    },
    body: JSON.stringify({ email, password }),
    redirect: "manual",
  });

  // Step 3: extract the session cookie from the response
  const sessionCookie = response.headers
    .get("set-cookie")
    ?.match(/(\w+[-_]session)=([^;]+)/);

  if (!sessionCookie) {
    throw new Error("Login failed — no session cookie returned");
  }

  return { name: sessionCookie[1], value: sessionCookie[2] };
}

This is more involved than just copying a cookie from DevTools. But it runs unattended, and the session is always fresh. I use a similar approach inside screenshotrun's own renderer when I need to test authenticated pages during development.

When to use which method

Cookies — for any standard web app with form-based login. This is the most common case: admin panels, dashboards, CMS backends, SaaS tools. If you log in through a form in your browser and the site remembers you via a cookie, this is your method.

Custom headers — for internal tools and API-protected pages. If the page checks for an X-Auth-Token, a Bearer token, or any non-standard header, use the headers parameter. This also works for API documentation portals that require authentication headers.

Basic Auth — for staging environments and htpasswd-protected sites. If the browser shows a native username/password dialog (not an HTML form), the site uses HTTP Basic Authentication. Pass the credentials as a Base64-encoded Authorization header.

Some setups combine multiple methods. A staging server might have Basic Auth at the Nginx level and cookie-based sessions at the application level. In that case, pass both — the headers parameter with the Basic Auth header and the cookies parameter with the session cookie. The screenshotrun API sends everything together with the request.

Security notes

Don't hardcode credentials. Use environment variables or a secrets manager. If you're storing session cookies or passwords in your codebase, you're one accidental commit away from leaking them.

Use dedicated accounts. If you're automating dashboard screenshots, create a read-only user account specifically for this purpose. Don't use your admin credentials in scripts.

Be careful with screenshot storage. Authenticated dashboards might contain sensitive data: revenue numbers, user emails, internal metrics. If you're using screenshot caching or storing images long-term, think about who has access to those files.

Session cookies are temporary credentials. Treat them like passwords. Don't log them, don't put them in URLs, don't send them over unencrypted connections.

What I ended up doing with fixheaders

Back to the original problem. I needed to periodically check fixheaders.com scan stats without logging in manually every time.

Here's what I built: a scheduled task in Laravel (schedule:run) that fires every hour. It logs into fixheaders using the automated approach above, takes a screenshot of the dashboard, and saves it. I can open the latest screenshot any time and see the numbers without any extra steps.

Could I have built a proper API integration instead? Sure. But that would have taken way longer than the 20 minutes this approach took. Sometimes a screenshot is the fastest way to pull data out of a system that doesn't have an API for what you need.

If you want to try screenshotrun for something like this, the free plan gives you 200 screenshots per month, more than enough for hourly dashboard captures. The cookies and headers parameters are available on the Pro plan and above.

I hope this saves you some time next time you need to screenshot a page behind a login. If you hit edge cases with specific frameworks (NextAuth, Passport, Sanctum — they all have their quirks), feel free to reach out and I'll try to help.