DEV Community: Mirek Ciastek

How to Run TestCafe CLI on BrowserStack in Docker Container

Mirek Ciastek — Sun, 17 May 2020 17:06:13 +0000

Every application needs testing and most of them need End-to-End (E2E) testing. The quickest way to have E2E tests is to do them manually, but repetitive manual work costs far too much for a company. Automation can help reduce the costs of running End-to-End tests and increase the confidence of a team at the same time. It’s true that configuring an E2E test runner and including it into our development workflow isn't an easy task, but once you're done, you never look back. We benefit from more confidence in our code with every single test written, and ensure that the combined pieces of an application, or system, work in perfect harmony.

End-to-end tests are must-have

When you develop for one of the leading real estate websites in Spain, you need to be sure you’re ready to handle huge traffic and make your site ready for various devices and browsers. That's a real challenge at Kyero.com and because of that we decided to automate E2E testing. There are a few good reasons why.

First of all, End-to-End tests give you the best guarantee that your application works well from a user perspective. They verify the complete system flow.

Next, going automated takes off a tedious burden of manual testing. It's more consistent and much faster than doing it manually.

Finally, using automated testing in a development workflow, allows a team to be more confident in their work. Creating an E2E testing process can be time consuming, so we need to use an easy and ready to use tool, like TestCafe.

What is TestCafe

As you may guess, TestCafe is a tool for writing and running End-to-End tests using JavaScript and TypeScript. It's quite an advanced environment for testing in JS and even has a dedicated IDE called TestCafe Studio. This tool also allows you to write tests according to the PageObject pattern, which encapsulates UI specific structure and makes creating tests easier and more convenient for a developer.

Like many other tools on the market, TestCafe supports testing in a browser environment out of the box. However, unlike some tools, TestCafe allows you to run your tests against different browsers like Chrome, Firefox and even Opera. When you need to support older browsers, for example IE11, you can use one of the providers offering automated testing services like BrowserStack or SauceLabs. Another cool feature is a dedicated CLI, that helps adding this test runner to your CI/CD pipeline. Finally, if you are a fan of containerization you can use an image to run TestCafe in a Docker container.

Customisation gives flexibility

It looks like TestCafe has all we need to use it in a Dockerized environment, right? Unfortunately, it's not that simple. Like any other tool, it requires some customisation before you can get the best out of it. At Kyero, all development environments are fully dockerized. Yet we need to run the E2E test both in Docker and locally on a software developer’s machine, in order to easily debug any problems. To fulfil these requirements, we need to customise how TestCafe runs.

First of all, TestCafe's default CLI gives you the minimum required options to run tests against different browsers (Chrome and Firefox in this case). Running tests in a Docker container and in local OSes requires different kinds of browsers. In Docker you need headless browsers or external testing services to run tests, because Docker doesn't support a GUI out of the box. Locally, on your machine, you can use any browser you have installed. Then, we want to add an option to run tests in live mode for debugging purposes. Finally, we need to create our own Docker image to support automated testing services, because the default images doesn't support them.

Creating Docker image

As we're dealing with launching browsers in a Docker container, it's configuration isn't that trivial. We need to install and then compile a few libraries to run browsers in headless mode. If you are familiar with creating a Docker image, you may know that we need to specify what image we're going to use as a base. Here we want to use alpine:edge, because Alpine is a secure and lightweight Linux distribution, and we need an edge version to make the image work.

FROM alpine:edge

Next let's create a user and directory where our CLI will run (copy-paste from official docker-node image). It's a good practice to create a user, because default Docker's user has root privileges, which can be very risky in case of security breach.

RUN addgroup -g 1000 node \
    && adduser -u 1000 -G node -s /bin/sh -D node

RUN mkdir -p /app

Now, we need to install all required packages - browsers (Chromium and Firefox), Node.js with NPM and Yarn and others.

# Install required dependencies (with Chromium and Firefox)
RUN apk update && \
    echo "http://dl-cdn.alpinelinux.org/alpine/edge/main" >> /etc/apk/repositories && \
    echo "http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories && \
    echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories &&  \
    apk add --no-cache --purge -u \
    nodejs nodejs-npm yarn \
    chromium firefox \
    xwininfo xvfb dbus eudev ttf-freefont fluxbox procps xvfb-run \
    nss freetype freetype-dev harfbuzz \
    ca-certificates wget

We're using BrowserStack to run tests on various other browsers. We need to allow this service to access our local server by using BrowserStack Local Testing. There are a few steps to do, before we are ready to install this package. First we need to install glibc, because Alpine distribution lacks this package.

# Get glibc
RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub
RUN wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.29-r0/glibc-2.29-r0.apk
RUN apk add glibc-2.29-r0.apk

# Get glibc-bin and glibc-i18n (required by BrowserStack Local)
RUN wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.29-r0/glibc-bin-2.29-r0.apk
RUN wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.29-r0/glibc-i18n-2.29-r0.apk
RUN apk --update add glibc-bin-2.29-r0.apk glibc-i18n-2.29-r0.apk
RUN /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8

# Required by Chromium and Firefox
RUN apk add libstdc++

Now we're ready to install BrowserStack Local Testing package.

# Get BrowserStack Local
RUN wget https://www.browserstack.com/browserstack-local/BrowserStackLocal-linux-x64.zip \
    && unzip BrowserStackLocal-linux-x64.zip \
    && chmod +x BrowserStackLocal \
    && mv BrowserStackLocal /usr/local/bin \
    && rm BrowserStackLocal-linux-x64.zip

Next step is to set proper environment variables to make Chromium visible to TestCafe.

ENV CHROME_BIN=/usr/bin/chromium-browser \
    CHROME_PATH=/usr/lib/chromium/

After that we need to copy an entrypoint script and make it executable. Then we clear apk's cache.

# Copy install-deps script that installs Node.js dependecies
# Practically `yarn install` command
COPY install-deps /usr/local/bin/
RUN chmod +x /usr/local/bin/install-deps

# Clear cache
RUN rm -rf /var/cache/apk/*

Finally, we can launch testing tool. Here we're using xvfb-run to run test command in virtual framebuffer, which is required to run browsers in headless mode.

WORKDIR /app

# Switching to non-root user
USER node

# Required for TestCafe
EXPOSE 1337 1338

# Install Node.js dependecies
ENTRYPOINT [ "install-deps" ]

CMD xvfb-run --server-num=99 --server-args='-ac -screen 0 1024x768x16' yarn test

If it's more convenient for you here's a gist with full Dockerfile of described image. Now, let's move to creating a custom CLI for our test runner.

Custom CLI for TestCafe

Launching tests in TestCafe programmatically is super simple, thanks to its nice API. Apart from builtin CLI, this tool provides a Node.js package that you can use to create your own test runner. It gives you the same options as CLI, but embedded into a useful interface. In my example I'm going to use TypeScript, but don't worry if you're not familiar with its syntax. It's basically the same as JavaScript's syntax, plus type definitions.

First of all, we need to define what browsers we want to test against. As I mentioned earlier, TestCafe allows us to perform fully automated tests on BrowserStack. This feature can be very useful, when you need to support any browser that cannot be easily installed in a Docker image. In this case, we want to test our app in Internet Explorer 11 🤭.

import isDocker from 'is-docker';

const runsInContainer = isDocker();

// In Docker we can use only headless browsers
const firefox = (runsInContainer) ? 'firefox:headless' : 'firefox';
const chrome = (runsInContainer) ? 'chromium:headless' : 'chrome';

const localBrowsers = [firefox, chrome];
const remoteBrowsers = ['browserstack:ie@11.0:Windows 10'];

const browsers = [...localBrowsers, ...remoteBrowsers];

Next, let's define a basic test runner, with some default options. Here we want to skip all JS runtime errors, because we don't need to stop tests when any error occurs (from the external service's script). When creating a test runner, we should define ports to which it will respond (remember the ports that we exposed in Dockerfile?). Then we run tests asynchronously (here with async/await syntax for simplicity). After all tests are done, we should end the runner's process.

import createTestCafe from 'testcafe';

const RUNNER_OPTIONS = {
  skipJsErrors: true,
  skipUncaughtErrors: true,
};

const runTests = async (browsers: string[] | string) => {
  const testcafe = await createTestCafe('localhost', 1337, 1338);
  const runnerFactory = testcafe.createRunner.bind(testcafe);
  const runner = runnerFactory();

  await runner.src('test.ts')
    .browsers(browsers)
    .run(RUNNER_OPTIONS);

  return testcafe.close();
};

runTests(browsers); // run our tests!

Our CLI works fine with local browsers, even with those used in a Docker's image. Unfortunately, it's not enough. Let's say that our app needs to be compatible with IE11. For our convenience, TestCafe supports several automated test providers. The only thing we need to do is to install the specific provider addon - testcafe-browser-provider-browserstack in this case. Then TestCafe knows that when we provide a specific browser name with the browserstack prefix, it should connect to BrowserStack and run our tests remotely.

Running in watch mode

Sometimes it's comfortable to run tests in watch mode, especially when you want to work in a TDD manner. Guess what, TestCafe has that too. At this point, we're adding another feature to our custom CLI and it gets less flexible. As you may know, any good CLI supports using options and I think that ours should too.

There are plenty of libraries for parsing commands arguments, but I decided to go with yargs. It has a simple and declarative API, so let's give it a try. We would like to allow people to run tests in watch mode, pass a directory name where tests are placed and maybe pass a browser's name directly.

import yargs from 'yargs';

const argvConfig = yargs.strict().options({
  watch: {
    alias: 'w',
    describe: 'Runs tests in live mode (only for local browsers)',
    type: 'boolean',
  },
  path: {
    alias: 'p',
    describe: 'Path or glob pointing to test files',
    default: 'spec/**/*.ts',
    type: 'string',
  },
  browser: {
    alias: 'b',
    describe: 'Runs test for specific browser',
    type: 'string',
  },
});

const argv = argvConfig.parse();

Now, we need to alter our test runner to support the new options.

/**
 * Beginning of the file
 */

const browsers = (() => {
    // Passed browser takes precedence over others
    if (argv.browser) {
        return argv.browser;
    }

    // We use only local browsers in watch mode
    if (argv.watch) {
        return localBrowsers;
    }

    return [...localBrowsers, ...remoteBrowsers];
})();

// Missing exported TestCafe type from 'testcafe' 😢
// @ts-ignore
const getRunner = (testcafe) => (
  (argv.watch) ? testcafe.createLiveModeRunner.bind(testcafe) : testcafe.createRunner.bind(testcafe)
);

const runTests = async (browsers: string[] | string) => {
  const testcafe = await createTestCafe('localhost', 1337, 1338);
  const runnerFactory = getRunner(testcafe);
  const runner = runnerFactory();

  await runner.src([argv.path])
    .browsers(browsers)
    .run(RUNNER_OPTIONS);

  return testcafe.close();
};

runTests(browsers); // run our tests!

Thanks to these small amends we can launch our CLI with different options. We're using TypeScript, so we should compile the code with tsnode before running.

# watch mode
tsnode cli.ts -w

# custom path
tsnode cli.ts -p "somewhere_else/*.ts"

# other browser
tsnode cli.ts -b "opera"

Now we're ready to launch our first test.

import { Selector } from 'testcafe';

fixture('Kyero Page')
  .page('https://www.kyero.com/');

test('Has page title', async (t) => {
  const title = Selector('title');

  await t.expect(title.textContent).eql('Property for sale and rent in Spain - Kyero.com');
});

Here are previews of our test running locally, in Docker container and using BrowserStack automated tests.

Result of tests running on local machine

Tests running in Docker container

Tests running locally in watch mode

Report of our tests displayed in BrowserStack app

Preview of result in Chrome

And there it is, our custom TestCafe CLI that supports running tests both locally and on BrowserStack. If you need a recap, you can visit the Github gist where I’ve placed all the code presented in this article.

Compatible, Reliable and Experimental Image Lazy-loading in Web Browser

Mirek Ciastek — Tue, 01 Oct 2019 11:05:19 +0000

In the last few years, web developers have become more aware of performance and loading issues in web applications. The complexity of web apps has increased so much, that making an application overwhelm a user’s device is pretty easy. Luckily for us, browser vendors noticed this risk, and they started to educate us about how to care more about performance and why it’s important for web users.

Getting the best performance and load times can be very challenging. We can minify, split and load on-demand scripts and styles, but there are other resources that we can’t split and they are usually very important for us.

Images, the topic of this article, are often very heavy resources, especially if we aim to provide the best quality content for high-resolution screens. Imagine that you need to load all the content of a very long page, full of scripts, complex styles and high-resolution images in less than 1 second on average? How you would tackle it?

What is lazy-loading and why it’s useful

You probably know that loading on demand can be the right path to speeding up your website. The rule is simple, if it’s not used don’t load it. Everything that is not visible or not used by the app, should be loaded later. This allows you to save some bandwidth and CPU cycles for more important stuff, like delivering the main content as fast as possible or fetching necessary data.

Here comes lazy-loading, a well-known technique of loading images only when they are required. To make use of lazy-loading properly, you need to define what resources are critical. The basic criterion is “above the fold”. In other words, if you want to know what needs to be loaded, just check if the resource is used or present in the viewport at the first load.

How lazy-loading is done today

Modern web technologies give us plenty of possibilities on how to deal with lazy-loading. The task seems to be very easy, we simply need to check if a certain image is present in the browser’s viewport, which means the user sees the image at the right moment.

To make an image lazy-load, first we need to disable automatically loading the image file by the browser. Simply, we replace src attribute with its equivalent data-src. Here’s a sample markup, that we can use in our lazy-loading feature.

<img data-src="path/to/image.png" alt="" />

Now, I would like to show you three approaches of implementing lazy-loading in your web app — a compatible, a reliable and an experimental approach. Let me break them down in the next sections.

The compatible approach

As we know, the main criterion for lazy-loading an image is its visibility in the viewport. A task that seems to be pretty simple, but requires some computations to be done, like calculating an element’s bounding box, the size of the viewport and the position of the element relative to the viewport.

First, we need to get an element’s bounding box measurements using the getBoundingClientRect method. Preferably we should do it once, on the first load, as constant reading can hurt performance. Next, we should check if any part of the element is present within the viewport’s coordinates. Lastly, we need to repeat the previous steps in a loop, to get the result in real-time. Let’s see the code.

First, the image loading handler.


const loadImage = (src) => {
  const img = new Image();

  return new Promise((resolve, reject) => {
    img.onload = () => resolve(src);
    img.onerror = reject;

    img.src = src;
  });
};

Then, let’s take care of checking element’s presence in the viewport.

const isInViewport = ({ top, height, windowHeight }) => {
  const scroll = window.scrollY || window.pageYOffset;
  const boundsTop = top + scroll;

  const viewport = {
    top: scroll,
    bottom: scroll + windowHeight,
  };

  const bounds = {
    top: boundsTop,
    bottom: boundsTop + height,
  };

  return (bounds.bottom >= viewport.top && bounds.bottom <= viewport.bottom)
    || (bounds.top <= viewport.bottom && bounds.top >= viewport.top);
};

Finally, we put everything in the scroll event handler and update measurements on demand.

import throttle from 'lodash/throttle';

const images = [...document.querySelectorAll('img')];

let windowHeight = window.innerHeight;

// We need to store images' sizes in a WeakMap
// to get them later in scroll handler
const imagesSizes = new WeakMap();

// This method allows to get top and height of each image
// and store them in WeakMap
const getImagesSizes = () => {
  images.forEach((image) => {
    const { top, height } = image.getBoundingClientRect();
    imagesSizes.set(image, { top, height });
  });
};

const onScroll = () => {
  images.forEach(async (image) => {
     // If image has been already loaded, bail out
     if (image.classList.contains('loaded')) {
       return;
     }

     const { top, height } = imagesSizes.get(image);

     // We use isInViewport method from previous example
     if (isInViewport({ top, height, windowHeight }) {
       try {
         // We use loadImage method from previous example
         await loadImage(image.src);
         image.classList.add('loaded');
       } catch (error) {
         console.error(error);
       }
     }
  });
};

// When window dimensions changed, update sizes
const onResize = () => {
  windowHeight = window.innerHeight;
  getImagesSizes();
};

getImagesSizes();

window.addEventListener('scroll', throttle(onScroll));
window.addEventListener('resize', onResize);

The scroll event listener is the most convenient way of checking an element’s visibility for any user’s interaction. What’s more, it’s a natural way of doing it, because the page needs to scroll if a user wants to see the next images.

You can imagine that performing any complex computations or operations in a scroll handler can easily kill your app. There are a few tricks that I used in my example, which helps avoid such mistakes. You probably already know the throttle method (check lodash docs), which decreases the number of a function’s calls. Additionally, I decided to read an element’s dimensions once on load (using WeakMap as a storage), and then update them only on a resize event to avoid too many requests for size computations.

This approach gives you the best support in comparison to others, but it’s not that easy to implement. Luckily it’s not the only way to do it, because recently we’ve got a nice new API which simplifies this process a lot.

The reliable approach

I’m pretty sure that you’ve heard about Intersection Observer API. This API has been around for about a year and is already supported by all major modern browsers (according to “Can I use” stats). What’s more, the Intersection Observer API is an Editor’s Draft. This means that shortly it will become a recommendation, which I’m really looking forward to.

What Intersection Observer does is observe if any part of a certain element is visible in the viewport. It works similarly to our custom script based on scroll but does it better, with less performance impact and in fewer lines. Let’s take a look at the example.

const images = [...document.querySelectorAll('img')];

const onIntersection = (entries, observer) => {
  entries.forEach(async (entry) => {
    if (entry.isIntersecting) {
      try {
        // We use loadImage method from previous example
        await loadImage(entry.target.src);
      } catch (error) {
        console.error(error);
      } finally {
        // When image has been loaded
        // stop observing the image
        observer.unobserve(entry.target);
      }
    }
  });
};

const observer = new IntersectionObserver(onIntersection);

// Start observing every image
images.forEach((image) => observer.observe(image));

You will have noticed that with Intersection Observer we don’t need to write any code for checking if an element is in the viewport. We simply use isIntersecting flag, and in the if block we run our image loader, from the previous section. You only need to remember about removing elements from the observed collection, just after the image is loaded. Moreover, I recommend using only one observer for all images in the active page.

Intersection Observer is something that was first seen a few years back. It facilitates working with lazy-loading patterns and it has a developer-friendly API. You may think that there’s nothing better out there… well, there is, but it’s still an experimental technology.

The native approach

Lazy-loading is such a common solution that browser vendors decided to build it into browsers. The result of their efforts is loading attribute - a native way of lazy-loading resources (not only images but also iframes). This proposal makes lazy-loading even more straightforward to implement than using the Intersection Observer API. At last, we don’t care about how it’s working, we can just use it, just like any other HTML feature.

Unfortunately for us, this attribute is only currently supported in Chrome 76+ (refer to “Can I use” table). There is a long way to go before it is an accepted standard or even a recommendation.

This is how the future of lazy-loading in browsers looks like.

<img src="path/to/image.png" loading="lazy" alt="" width="500" height="350">

Recommendations

I presented you with three approaches to implement lazy-loading images in web applications. All of them have pros and cons, but there is no single solution for this problem. Nevertheless, there is one that I can recommend to you in good conscience.

Between all presented solutions, I’d recommend using Intersection Observer with polyfill for old browsers. Intersection Observer API is the one that gives you a reliable solution with simple usage, albeit not being supported by all browsers. If you target modern browsers you don’t need to worry about that support, otherwise, use a polyfill to get better coverage.

I decided not to recommend the first and last approach, because the first one seems to be unnecessary, especially when Intersection Observer has acceptable browser support. Finally, loading attribute is still an experimental feature. It is supported by only one browser on the market and doesn’t give you enough flexibility compared to the first and second approach.