We QA'd 100 Websites Automatically with Playwright — Here's How

#playwright #testing #automation #webdev

When you build 100 websites in one day, manual QA is not an option. You need automated testing that catches every broken image, dead link, and JavaScript error across all 100 sites.

We built a Playwright-based QA pipeline that runs 118 checks per site. Here's the system.

What We Check

Round 1: Basic Health

HTTP status code (must be 200)
Page loads within 5 seconds
No JavaScript console errors
All images load (no broken <img> tags)
No 404 resources in network tab

Round 2: Content & UX

Navigation links work (no dead links)
Language toggle switches all text (we have bilingual sites)
Mobile responsive — no horizontal overflow
Footer links present and functional
Meta tags exist (title, description, OG tags)

Round 3: SEO

JSON-LD structured data present and valid
Canonical URL set correctly
Robots meta tag allows indexing
Sitemap.xml accessible
Images have alt text

The Code Pattern

async def check_site(url, page):
    results = []

    response = await page.goto(url, wait_until='networkidle')
    results.append(('HTTP Status', response.status == 200))

    # Check for console errors
    errors = []
    page.on('console', lambda msg: errors.append(msg.text) if msg.type == 'error' else None)

    # Check all images
    broken = await page.evaluate('''() => {
        return [...document.images].filter(img => !img.complete || img.naturalWidth === 0).length
    }''')
    results.append(('Broken Images', broken == 0))

    # Check mobile overflow
    overflow = await page.evaluate('''() => {
        return document.documentElement.scrollWidth > document.documentElement.clientWidth
    }''')
    results.append(('No Horizontal Overflow', not overflow))

    return results

Running at Scale

async def qa_all_sites(sites):
    async with async_playwright() as p:
        browser = await p.chromium.launch()

        for site in sites:
            page = await browser.new_page(viewport={'width': 1280, 'height': 720})
            desktop_results = await check_site(site, page)

            # Mobile viewport
            await page.set_viewport_size({'width': 375, 'height': 812})
            mobile_results = await check_site(site, page)

            await page.close()

Results Dashboard

After each run, we generate a report:

✅ 98/100 sites passed all checks
⚠️ 2 sites have warnings:
  - site-042: 1 image slow to load (>3s)
  - site-087: meta description too long (162 chars)
❌ 0 critical failures

What We Learned

Don't resize base64 images with regex — We tried truncating long base64 strings. It corrupted images. Use PIL to actually resize the image file instead.
Test bilingual content properly — element.textContent strips inner HTML. If you have <span data-en>Hello</span>, checking textContent on the parent gets both languages concatenated. Query the spans directly.
Hamburger menus and language toggles conflict on mobile — They fight for the same top-right corner. Pick one UI pattern and stick with it.
localStorage persists between test runs — Clear it between sites or your language toggle tests will give false positives.

The Bottom Line

100 sites. 118 checks each. 11,800 automated tests. Zero broken images, zero 404s, zero JS errors in production.

Manual QA would have taken a week. Playwright did it in 12 minutes.

Built by HEY!BOSS — see all 100 sites live.

DEV Community