Starchart: Playwright and Chakra UI Setup

The past week I worked on setting up Playwright, and Chakra UI with starchart. Chakra UI is needed so that their components could be used to build the web pages, and Playwright is to be used for end-to-end testing.

Chakra UI Setup

Setting up Chakra UI was relatively simple. I followed their instructions for the most part. I also found this example to be really helpful. My process for this went like this: I looked at the instructions in Chakra UI docs for the client, server and root files, and compared them with the existing code in starchart. Then, I also referenced them against the example code.

Not an ideal process to follow, but since I do not understand Chakra UI or Remix that well yet, I felt that this was the best way to do the setup. It worked, and I created a PR. Based on some feedback, I added some basic components (instead of using html tags) to the index page so some manual testing could be done.

Now, if you followed the PR link, you would have seen that it was merged but had a failing check. And that is what was addressed later while adding Playwright. Since the project was setup using the Remix blues stack, it shipped with cypress for end-to-end testing. The cypress folder had some tests but these were not modified when the app was setup since we intended to use Playwright. This resulted in the typecheck job failing, since it was also checking the cypress folder: tsc && tsc cypress

Playwright Setup

Setting up Playwright took longer than I had expected. First, I made a list of node modules that needed to be removed. These included cypress, @faker-js/faker, @testing-library/cypress, and eslint-plugin-cypress. These were only used by cypress so I uninstalled them using npm uninstall.

Next, I deleted the cypress folder, and the configuration file for cypress (cypress.config.ts). I installed playwright by using npm init playwright@latest, and following the prompts (docs). This also generated a config file for playwright, some simple tests in the specified test folder, and some more detailed examples of tests. I deleted the folder containing the examples since it was simply for reference, and added some basic tests.

For the configuration, I ended up using the defaults for most of the options. I also uncommented the configuration for mobile viewpoints, branded browsers, and output directory for test artifacts; and added the following based on some feedback:

baseURL: `http://localhost:${process.env.PORT}`
...
video: 'on-first-retry'

Setting a baseURl allows navigating to a page with a relative URL:

await page.goto('/');

The second configuration records a video of the test on the first retry after it fails.

Trying to Setup CI

So far, this process was pretty simple. Installing playwright also auto generates a GitHub workflow file. I tried adding a new job based on this auto generated file, and updated some npm scripts that were previously being used for cypress to run the tests. However, to run the tests, the app must be started. And to do that, the mysql database container must be running. I tried using the defined docker script, which uses docker-compose up -d to start the container but prisma db push failed with the following error:

I tried a bunch of things to try and fix this. I posted about this in Slack, and found out that mysql can take a while to start. So, I tried adding a wait before running the tests based on this discussion. However, this still resulted in the same error. I even tried doing prisma db pull before prisma db push. However, I got a different error this time:

This was really strange, since the database should have been setup. Still I did not know why this was failing.

Using Service Containers

Since I was basically out of ideas on how to fix this, I tried completely reworking the job by using a service container for mysql instead:

  E2E-Test:
    timeout-minutes: 15
    runs-on: ubuntu-latest
    services:
      mysql:
        image: mysql:8
        ports:
          - 3306:3306
        env:
          MYSQL_DATABASE: starchart
          MYSQL_USER: starchart
          MYSQL_PASSWORD: starchart_password
          MYSQL_ROOT_PASSWORD: root_password

    steps:
    ...

And the tests worked! I am still not sure why they were failing before. Maybe docker-compose doesn't work with GitHub actions?
With this, I marked the PR ready for review, and the broken CI was fixed.