Solution to Challenge 4 - Accessibility testing

Challenge #4 is done — here's my solution to Practice Real-World Testing Scenarios for QA: Challenge 4 - Accessibility testing

1) Manual testing

I added a video on how to use VoiceOver on Mac and how VoiceOver in Safari behaves differently from Chrome, with and without VoiceOver enabled; for example, it is better for navigating the page with Tab, but in Chrome it has the same behavior.

With NVDA on Windows, I remember that you can change some behavior with a keyboard shortcut because has similar behavior when is enabled some keyboard shortcuts are different.

With a screen reader, you need to ensure that the info is useful for a blind person. For example, there are some gaps:

• The user menu is not accessible with the keyboard.
• When you click the top charts table, the user doesn't realize it opens a detailed chart.
• When you click on the four charts, the user doesn't hear anything about the bar chart or its content.
• When you click the chart or the customer name in the first 2 charts, the user can't navigate to the rows with that customer's invoice details.

Accessibility testing requires a lot of patience and attention to detail, including ensuring that the screen reader is visible, that focus is visible, and that keyboard navigation or assistive technology follows a logical order.

Another useful course focused that I took to learn more about how to design accessible web sites since the beginning is: Accessibility Foundations

2) Testing with extensions

I used various browser extensions to help me detect accessibility issues such as color contrast issues, missing H1 headers, or incorrect header order.

Different extensions show different issues, but in general I tried:

Accessibility Insights

This extension includes:

• Automated checks: Quick check about common issues and generates a summary of the issues, along with tips on how to fix them.

In this example the errors are related to color contrast

• Color: Changes your website to grayscale and provides tips on testing and changing your design so it doesn't rely solely on color; for example, for error messages, it is also good to add an icon and text.

• Headings: Highlight the headings of your page

• Accessible names: Display the text that a screen reader will read.

• Landmarks: Display the different landmarks of your page. This is also helpful for screen readers to understand the structure of your website.

• Tab stops: Add a number and line to check the order of the Tab elements.

• Needs review: Additional tests that can't be automated with any browser extension

silktide

Silktide includes a that explains the accessibility rules in a fun way.

Their extension offers:

• Screen reader: You can listen and read the text that the screen reader will say.

• Dislexia: You can see how a person with dyslexia reads your text.

• Impaired vision: You can simulate conditions such as cataracts and myopia.

• ** Color blindness**: You can simulate different forms of color blindness because some people are unable to see specific colors.

• Landmarks: Display the main landmarks of the page.

• Focus order: You don't need to manually press tab several times display the focus order automatically

You can also try other of the popular chrome extensions

3) Automation Testing

You can add a eslint rule for accessibility in the front end project. For angular you can follow this article: Angular ESLint Rules for Accessible HTML Content

Selenium

For selenium you can use axe-core that is a popular library to check the accessibility of your website.

Deque example with c#

Cypress

Cypress offers a paid option to assess your website's accessibility.

You can also check the open source wick-a11y that a new version was released some days ago

You can also use the deque npm package for cypress

Playwright

Playwright includes a section for accessibility with @axe-core/playwright to check the common accessibility issues, and also offers a snapshot testing option that compares the main HTML elements of your website

I worked on a project that required accessibility testing some years ago, and at that time the options were limited; Cypress, TestMu AI didn't exist, so I use the playwright axe core and I created a detailed custom html report with a summary of the accessibility errors, including links to the deque documentation, a video highlighting the elements with the error, a description of each error, and a copy option to streamline issue reporting.

This year after a suggestion from Sebastian the author of wick-a11y I decided to create a new npm package and with the help of AI tools like Claude, Gemini I created a npm that connects to Azure Devops to create the bugs.

To add into your project you need to install the package

npm install snap-ally --save-dev

You need to set up the report path, and you can customize the colors for different error severities that are added to the video and to your Azure DevOps project, organization, and area to create the bugs.

import { defineConfig } from '@playwright/test';

export default defineConfig({
    reporter: [
        [
            'snap-ally',
            {
                outputFolder: 'a11y-report',
                // Optional: Visual Customization
                colors: {
                    critical: '#b91c1c',
                    serious: '#c2410c',
                    moderate: '#a16207',
                    minor: '#1e40af',
                },
                verbose: true,      // Show in terminal
                consoleLog: true,   // Show in browser console
                // Optional: Azure DevOps Integration
                ado: {
                    organization: 'your-org',
                    project: 'your-project',
                    areaPath: 'your-project\\your-team', // Optional: Define where bugs should be created
                },
            },
        ],
    ],
});

And you can add the steps to navigate to your website to get a better bug report and you only need to call the checkAccessibility function

import { test } from '../fixtures';
import { DashboardPage } from '../pages/DashboardPage';
import { checkAccessibility } from 'snap-ally';
import { ServersPage } from '../pages/ServersPage';

test.describe('Accessibility Testing', {
    tag: ['@Accessibility'],
}, () => {
     test.use({ storageState: '.auth/admin.json' });

    // eslint-disable-next-line playwright/expect-expect
    test('Dashboard Page', async ({ page, locale }, testInfo) => {
        test.skip(testInfo.project.name !== 'English', 'Accessibility checks only run against the English locale');
        const dashboardPage = new DashboardPage(page, locale);
        await dashboardPage.goTo(); 
        await dashboardPage.waitForChartsAreVisible();
        await checkAccessibility(page, testInfo, { 
            consoleLog: true,
        });
    });

You can see the full code on my Testing Dojo repository

Thanks for following along these challenges. Testing is all about continuous learning, so don't hesitate to ask questions or share your feedback below.

If this helped you in any way, feel free to share it with the community.