Challenge #3 is done — here's my solution to Challenge 3: Visual Testing with API Mocking
I'm not a big fan of mock APIs for end-to-end tests, but when testing a dashboard with daily API changes, you need to ensure the charts are displayed correctly. One time, after some charts, one chart wasn't displayed. In another Google Maps integration, with some changes, one button overlaps another button, so I think visual testing is good for detecting those issues automatically. One approach is to mock the API to return the same data and save an image snapshot to compare the current website against it. If the UI changes, you need to update the base snapshot.
Different paid tools include visual testing tools like Percy by BrowserStack, Applitools Eyes, and Sauce Labs (Sauce Visual).
I previously wrote an article: Visual Testing with Playwright
The charts have patterns to be accessible, allowing people with low vision to see the charts without relying on colors. Accessibility testing will be the next challenge.
Playwright Visual Testing
Playwright includes a built-in free option to compare screenshots
await expect(page).toHaveScreenshot()
I created a visual helper to store snapshots with 2 methods: one for the whole page and another for a specific element. You can set the timeout to check the screenshot and the difference in pixels or as a percentage, as allowed. By default, I set a very minor difference only to check any change in the design.
import { expect, Locator, Page, test } from '@playwright/test';
export class VisualHelper {
constructor(private page: Page) {
}
/**
* Check full page snapshot
* @param snapshotName Snapshot name
* @param timeout Max timeout
* @param maxDiffPixels Max number of differing pixels allowed
*/
async checkPageSnapshot(snapshotName: string, timeout = 5_000, maxDiffPixels = 100) {
const stepDescription = 'Compare snapshot: ' + snapshotName + ' with maxDiffPixels: ' + maxDiffPixels;
await test.step(stepDescription, async () => {
await expect(this.page).toHaveScreenshot(snapshotName, {
timeout: timeout,
maxDiffPixels: maxDiffPixels
});
});
}
/**
* Check element snapshot
* @param element Element to check the snapshot
* @param snapshotName Name of the snapshot
* @param timeout Max timeout
* @param maxDiffPixels Max number of differing pixels allowed
*/
async checkElementSnapshot(element: Locator, snapshotName: string, timeout = 5_000, maxDiffPixels = 100) {
const stepDescription = 'Compare snapshot: ' + snapshotName + ' with maxDiffPixels: ' + maxDiffPixels;
await test.step(stepDescription, async () => {
await expect(element).toHaveScreenshot(snapshotName, {
timeout: timeout,
maxDiffPixels: maxDiffPixels
});
});
}
}
To mock the API, I created an API with fixed data, for example:
{
"Amount": 56046.00,
"AverageDaysPastDue": 35
}
To mock the API, I added this code:
/**
* Mock an api
* @param description Description for the HTML reporter
* @param url API URL to mock
* @param jsonData JSON that will be returned
* @param status HTTP status code to return (defaults to 200)
*/
async mockApi(description: string, url: string, jsonData: any, status = 200) {
await test.step(description, async () => {
await this.page.route(`**${url}`, async route => {
await route.fulfill({
status: status,
contentType: 'application/json',
body: JSON.stringify(jsonData),
});
});
});
}
To mock the summary, I created this function on the DashboardPage.ts
async mockSummary() {
const stepDescription = 'Modify the summary with fixed data';
await test.step(stepDescription, async () => {
await this.apiHelper.mockApi(
stepDescription,
'/api/collection/summary',
summary,
);
});
}
The most basic example is:
import { expect, test } from '../fixtures';
import { DashboardPage } from '../pages/DashboardPage';
import top5Delay from '../data/mocks/top5Delay.json';
import top5Total from '../data/mocks/top5Total.json';
import top10Limit1 from '../data/mocks/top10Limit1.json';
test.describe('Check Dashboard', () => {
let dashboardPage: DashboardPage;
test.use({ storageState: '.auth/admin.json' });
test.beforeEach(async ({ page, locale }) => {
dashboardPage = new DashboardPage(page, locale);
await dashboardPage.mockAllApis();
await dashboardPage.goTo();
await dashboardPage.waitForChartsAreVisible();
});
// eslint-disable-next-line playwright/expect-expect
test('Should show dashboard', {
tag: ['@VisualTesting', '@Dashboard'],
}, async () => {
await dashboardPage.checkPageSnapshot();
});
});
With this approach, if there is a minor change on the charts, you will see an error like this:
The difference is that the previous version displayed the details when you clicked on a segment. I changed it to add a table with the table details to make it more accessible.
This is the current version:
To update the snapshots you can execute this command:
npx playwright test --update-snapshots
Unit testing
Unit tests are important because they allow developers to catch bugs early. With .NET EF, you can connect to any database using the same LINQ query code. With this approach, you can add predefined fixed data to check the data returned by the APIs. Usually, developers create unit tests. I added it only as an example; if there is a unit or integration test that compares against the database, it may not be necessary to create another API test for that purpose. You can check that the API returns the correct number of items maximum 5 and the body response.
How to set up an in-memory database.
var options = new DbContextOptionsBuilder<MicrosipContext>()
.UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString())
.Options;
using var isolatedContext = new MicrosipContext(options);
For example to add some invoice and data to test I added a function
/// <summary>
/// for GetTotalByInvoiceAsync/GetTotalByClientAsync to pick it up.
/// </summary>
private static void AddInvoice(PaymentsContext context, int invoiceId, int clientId, decimal amount, DateTime dueDate)
{
context.Invoices.Add(new Invoices
{
Id = invoiceId,
ClientId = clientId,
ConceptId = 1,
InvoiceNumber = $"V{invoiceId:00000000}",
Date = DateTime.Now,
Cancel = "N",
Status = "A",
});
context.Ledger.Add(new Ledger
{
InvoiceId = invoiceId,
Cancel = "N",
Estatus = "A",
Type = "P",
Date = DateTime.Now,
Amount = amount,
Tax1 = 0,
Tax2 = 0,
Tax2 = 0,
Discount = 0
});
}
With this, you can have invoices without payments to check whether the API returns the correct info.
[Fact]
public async Task GetTotalByClient_WithTwoInvoices_AggregatesTotalAndAverageOverdueDays()
{
// Own in-memory database, independent of the shared PaymentsContext fixture and of
// GetDetailed/GetTotalByInvoiceAsync's own tests, so this test's expected values come
// from hand-picked numbers rather than from another DAO method's output.
var options = new DbContextOptionsBuilder<PaymentsContext>()
.UseInMemoryDatabase(databaseName: Guid.NewGuid().ToString())
.Options;
using var isolatedContext = new MicrosipContext(options);
const int clientId = 700;
isolatedContext.Clientes.Add(new Clientes { ClienteId = clientId, Nombre = "Test Client", Estatus = "A", SujetoIeps = "N" });
const decimal invoice1Amount = 1000m;
var invoice1DueDate = DateTime.Now.AddDays(-30);
const decimal invoice2Amount = 2000m;
var invoice2DueDate = DateTime.Now.AddDays(-10);
AddInvoice(isolatedContext, invoiceId: 1, clientId: clientId, amount: invoice1Amount, dueDate: invoice1DueDate);
AddInvoice(isolatedContext, invoiceId: 2, clientId: clientId, amount: invoice2Amount, dueDate: invoice2DueDate);
isolatedContext.SaveChanges();
var collectionDAO = new CollectionDAO(isolatedContext, new ParameterMock());
var totals = await collectionDAO.GetTotalByClientAsync();
var expectedTotal = invoice1Amount + invoice2Amount;
var overdueDays1 = (DateTime.Now.AddDays(1) - invoice1DueDate).Days;
var overdueDays2 = (DateTime.Now.AddDays(1) - invoice2DueDate).Days;
var expectedAverageDaysOverdue = (int)Math.Round((overdueDays1 + overdueDays2) / 2.0, MidpointRounding.AwayFromZero);
var clientTotal = Assert.Single(totals);
Assert.Equal(clientId, clientTotal.ClientId);
Assert.Equal(expectedTotal, clientTotal.Total);
Assert.Equal(expectedAverageDaysOverdue, clientTotal.DaysOverdue);
}
For integration tests that need to compare against the database, you can use test containers. I didn't include it here, but here is an interesting article: Test Container best practices
API Testing
Postman Testing
Postman includes an option to visualize API results with a chart. You can ask the AI option integrated in Postman to create the chart, or you can write the HTML template code and use the function visualize to display the chart on the Visualization tab.
var template = `
<canvas id="myChart" height="200"></canvas>
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.5.0/Chart.min.js"></script>
<script>
var ctx = document.getElementById('myChart');
pm.getData(function (err, value) {
var myChart = new Chart(ctx, {
type: 'bar', // Changed chart type to bar
data: {
labels: value.response.map(item => item.Client), // Used map to extract 'Client' values
datasets: [{
label: [],
backgroundColor: ["#003f5c", "#58508d", "#bc5090", "#ff6361", "#ffa600"],
borderWidth: 1,
data: value.response.map(item => item.Total) // Used map to extract 'Total' values
}]
},
options: {
legend: { display: false },
title: {
display: true,
text: 'Top 5 Total'
},
scales: {
yAxes: [{
ticks: {
beginAtZero: true
}
}]
}
}
});
});
</script>`;
const response = pm.response.json();
pm.visualizer.set(template,{
response: response
});
In this case, I use both a bar and pie chart, but it can be any type.
Another option to validate the JSON schema is Zod, but if you are using Newman in your pipeline, Zod does not work; you can use the Postman CLI.
With Zod, you can validate the body response like this:
// Zod schema validation
const { z } = pm.require('npm:zod');
const itemSchema = z.object({
ClientId: z.number(),
Client: z.string(),
Total: z.number(),
DaysOverdue: z.number()
});
I create a general function to reuse in different tests.
CheckArrayZodSchema: function(itemSchema) {
const { z } = pm.require('npm:zod');
// Wrap in z.array so the whole response array is validated
const responseSchema = z.array(itemSchema);
const response = pm.response.json();
// Validate data safely
const validation = responseSchema.safeParse(response);
pm.test("Response matches Zod schema", () => {
pm.expect(validation.success).to.be.true;
});
// Log helpful details if validation fails
if (!validation.success) {
console.error("Zod Validation Failures:", JSON.stringify(validation.error.format(), null, 2));
}
}
RestAssured
For RestAssured, I only created the basic tests that check the response body and status code.
[Test]
public async Task GetTop5Total_WithValidUser_ReturnsTop5ClientsWithBiggestDebt()
{
var response = Given()
.Header("Authorization", $"Bearer {AuthToken}")
.When()
.Get($"{BaseUrl}/{CollectionEndpoint}/top-5-total")
.Then()
.StatusCode(200)
.DeserializeTo<List<TopClient>>();
await Assert.That(response?.Count).IsEqualTo(5);
}
Restsharp
With RestSharp, I use the same approach as in other tests.
[Test]
public async Task GetTop5Total_WithValidUser_ReturnsTop5ClientsWithBiggestDebt()
{
var client = ApiClient.Create(Configuration, AuthToken);
var response = await client.GetAsync<List<TopClient>>($"{CollectionEndpoint}/top-5-total");
await Assert.That(response.StatusCode).IsEqualTo(200);
await Assert.That(response.Data?.Count).IsEqualTo(5);
}
Thanks for following along these challenges. Testing is all about continuous learning, so don't hesitate to ask questions or share your feedback below.
If this helped you in any way, feel free to share it with the community.
Top comments (0)