How to screenshot your whole website using Playwright API.

Hello, in this article I'll be showing you on how to screenshot your whole website with multiple devices using playwright API and how to divide your pages into multiple sections to take micro-screenshots of your page.

Github repository can be found here.

What is playwright?

Playwright enables reliable end-to-end testing for modern web apps. You can test multiple devices, screenshot, test interaction of a page/component and it's very easy to use!

For testing purposes, I've selected dev.to to test the script with the following routes as a fake api.

export const CONSTANTS = {
    origin : 'http://dev.to',
    paths : [
        '/',
        '/about',
        '/guides',
        '/software-comparisons',
        '/devteam/if-youre-interested-in-webassembly-and-dont-get-enough-depth-here-on-dev-read-this-pfl',
    ]
}

The purpose of the script will be to enter on each slug and screenshot the following page on mobile and desktop viewport.
This comes in handy if you need to get a lot of images from different websites for social media, check to see if hydration works (e.g. SSG), save the content of pages from a domain by having a list with the paths and so on.

Bonus
For each page, we'll check if we have the element '.crayons-story' by using page.locator. It's similar with 'document.querySelectorAll' but it returns an array of the selectors found as a reference instead of a DOM element.
If we find at least one element, we'll screenshot the DOM element and save it to a different folder.

Main Function

async function screenshotPageHandler({isMobile = false, isJavascriptEnabled = false}){

    // Get Pages Paths
    let paths = await getPagePaths();
    // If we don't have any paths, we stop the process
    if(!paths) throw new Error('paths not found');

    // We initialize playwright via a custom function
    const {page,browser} = await getPlaywrightPageAPI(isJavascriptEnabled,isMobile);

    // We loop through each path
    for(let i = 0; i < paths.length; i++){
        // We get the slug of the page
        let pageSlug = clientSlug(paths[i]);

        await page.goto(pageSlug,{timeout: 8000});
        await page.waitForLoadState('networkidle');

        // find the element to screenshot
        let locators = page.locator('.crayons-story');

        for(let j = 0; j < await locators.count(); j++){
            // since we are dealing with an infinite scroll, we'll be breaking the loop after 50 articles. Otherwise it'll be an infinite loop.
            if(j >= 50 ) break;

            let locator = locators.nth(j);
            // find more details about the element, we'll use the first anchor tag for the section name.
            let fullUrl = await locator.locator('.crayons-story__hidden-navigation-link').evaluate(el => el.href);
            if(!fullUrl) continue;

            // we'll convert the full url to a slug
            let sectionName = new URL(fullUrl).pathname;
            // we'll get the screenshot path 
            let sectionSlug = getScreenshotSectionLocalPath(isJavascriptEnabled,isMobile,sectionName)
            // we'll check if the element is visible, otherwise we'll continue to the next section;
            let isSectionVisible = await locator.isVisible();
            if(!isSectionVisible) continue;

            // we'll take the screenshot of the element
            await locator.screenshot({path : sectionSlug });
        }

        // we'll get the screenshot path
        let slug = getScreenshotLocalPath(paths[i],isJavascriptEnabled,isMobile);
        // we'll take the screenshot of the page
        await page.screenshot({ path: slug , fullPage: true });
    }

    await browser.close();
}

For more details, the rest of the helpers function can be found in the repository.

Thank you for reading,
Mihai