DEV Community: Alicia Marianne 🇧🇷

O que uma usina nuclear tem a ver com o seu processo de QA?

Alicia Marianne 🇧🇷 — Sun, 05 Apr 2026 12:02:52 +0000

A gente sabe que testar e validar um software antes de ir para produção é importante. Mas você já parou para pensar no peso real que isso carrega?

Recentemente, estava revendo a série Chernobyl, e ela me fez refletir sobre muita coisa — especialmente sobre a forma como encaro minha área, sendo QA, e sobre a responsabilidade que ela traz. Resolvi compartilhar isso com vocês.

Para quem não conhece, Chernobyl é uma minissérie dramática lançada em 2019 que retrata o desastre nuclear ocorrido na usina de mesmo nome, na então União Soviética, em 26 de abril de 1986. A história acompanha os eventos logo após a explosão do reator número 4 — o caos, as tentativas do governo soviético de esconder a gravidade do acidente e o enorme esforço de cientistas, bombeiros, militares e trabalhadores que arriscaram, e muitas vezes perderam, suas vidas para evitar uma catástrofe ainda maior. A série também segue o cientista Valery Legasov, que tenta descobrir a verdadeira causa do acidente e expor a verdade por trás da tragédia.

Mas o ponto aqui vai além da série.

O que mais me chamou atenção foi o quanto aquela tragédia conversa com algo que vivemos diariamente no desenvolvimento de software: a responsabilidade nas decisões.

O que aconteceu em Chernobyl?

Mesmo sabendo que existem elementos dramatizados, dá para aprender muita coisa com esse desastre. Não sou física nuclear, mas vou tentar resumir o que aconteceu — porque foi justamente essa parte que mais me fez refletir sobre responsabilidade e tomada de decisão.

Na madrugada do dia 26 de abril de 1986, os operadores da usina realizavam um teste de segurança no reator 4. O objetivo era validar se, em caso de queda de energia, as turbinas ainda conseguiriam gerar eletricidade por alguns segundos — tempo suficiente até que os geradores de emergência fossem acionados. No papel, o teste parecia simples.

O problema é que, para executá-lo, diversos sistemas de segurança foram desativados e a potência do reator foi reduzida para um nível muito abaixo do ideal. Foi aí que tudo começou a sair do controle.

O reator utilizado era do tipo RBMK, um modelo com uma falha crítica de projeto: em determinadas condições, quanto mais vapor era gerado dentro do sistema, maior ficava a potência do reator. Em vez de estabilizar, ele se tornava cada vez mais instável. Para piorar, o teste foi conduzido sob forte pressão da liderança, mesmo diante de sinais claros de que não era seguro continuar.

Ao longo da série, vemos os próprios operadores levantando preocupações — que foram ignoradas. Quando perceberam que a situação era crítica, acionaram o botão de desligamento de emergência. Em teoria, esse comando deveria encerrar a reação. Mas, por uma falha no design das barras de controle, o efeito inicial foi o oposto: a potência disparou. Em poucos segundos, temperatura e pressão subiram de forma descontrolada.

O resultado foram duas explosões que destruíram o topo do reator, expuseram o núcleo à atmosfera e liberaram uma enorme quantidade de material radioativo. O incêndio que se seguiu espalhou radiação por boa parte da Europa, transformando Chernobyl no maior desastre nuclear da história.

O que mais me impactou foi perceber que a tragédia não aconteceu por um único erro. Ela foi consequência de uma cadeia de decisões ruins: falhas técnicas ignoradas, riscos mal avaliados e pessoas que não foram ouvidas.

E o que isso tem a ver com software?

Depois de assistir à série, comecei a fazer um paralelo com a nossa área. Porque, no fim, quantas vezes um incidente em produção também não nasce da mesma forma?

Nem sempre o problema vem de um único bug. Muitas vezes, ele é o resultado de uma sequência de decisões tomadas sem o devido cuidado: um requisito mal definido, um risco não mapeado, uma validação superficial, uma entrega apressada — ou um alerta levantado pelo time que acabou sendo ignorado.

Foi aí que a série me fez enxergar algo que vai além do contexto dela: quando a pressa fala mais alto do que a análise, o custo quase sempre aparece depois.

O maior problema: risco tratado de forma rasa

Um dos maiores desafios que vejo hoje no desenvolvimento de software é justamente o planejamento e o levantamento de riscos feitos de maneira superficial.

Tenho certeza de que você vai concordar: não tem coisa pior do que refazer algo ou ficar apagando incêndios que poderiam ter sido discutidos antes.

Vivemos num mundo onde tempo é dinheiro. E é exatamente por isso que qualidade precisa estar presente desde o início — não como uma etapa final, mas como parte do processo.

Como QA, vou compartilhar duas coisas que considero essenciais.

Ouça sua equipe

Independentemente da sua posição no time, ouça as pessoas ao seu redor.

Ninguém conhece melhor o produto do que quem o construiu, testou e convive com ele diariamente. Antes de uma nova funcionalidade entrar ou de um teste ser executado, converse com o time.

Escute quem desenvolveu. Escute o produto. Escute suporte. Escute quem está mais próximo do usuário.

Muitas vezes, o risco já foi identificado por alguém. Ele só não foi ouvido.

Analise antes de agir

Entender o impacto de uma mudança não é só uma questão técnica — é também uma questão de negócio.

Por exemplo: você vai melhorar a query de uma API. Em teoria, isso pode não alterar nenhuma regra de negócio. Mas como isso afeta o usuário final? A performance realmente melhorou? Existe algum impacto colateral? Como vamos medir se essa mudança foi positiva ou negativa?

Nem toda melhoria técnica gera melhoria de produto. E esse olhar crítico é parte fundamental do papel de QA.

Conclusão

No fim, Chernobyl me fez refletir sobre algo que vai muito além de uma série: a responsabilidade por trás de cada decisão que tomamos no dia a dia.

A gente não está lidando com um reator nuclear. Mas ainda assim lida com impacto real — no usuário, no negócio e no próprio time. Um risco ignorado, um teste mal planejado ou uma decisão tomada sem ouvir a equipe podem não gerar uma catástrofe, mas certamente geram problemas que poderiam ter sido evitados com mais atenção, diálogo e análise.

Para mim, ser QA vai muito além de encontrar bugs.

É questionar antes que o problema aconteça. É analisar cenários com olhar crítico. É antecipar riscos. É provocar conversas importantes dentro do time. É ajudar a construir decisões mais seguras e conscientes.

No final, qualidade não é apenas sobre software funcionando.

É sobre responsabilidade, colaboração e cuidado com tudo aquilo que criamos.

Your Swagger Doc Is More Than Documentation — It's a Test Suite Waiting to Happen

Alicia Marianne 🇧🇷 — Thu, 19 Feb 2026 15:31:47 +0000

In a previous article, I showed how we can use Swagger and AI to generate test cases automatically. This time, we're going to explore another powerful use for Swagger: using it as the source of truth for contract tests.

Understanding Contract Testing

Before jumping into code, let's understand what contract testing actually means in the context of APIs.

According to the ISTQB (International Software Testing Qualifications Board) syllabus:

Contract testing is a type of interface testing that verifies that a service provider and a service consumer conform to a shared contract specifying request and response formats, data types, status codes, and interaction rules.

In plain terms, contract testing ensures that:

The API provider delivers responses in the format it promised
The consumer sends requests in the expected format
Both sides can evolve independently without breaking each other

Think of it like a formal agreement between two people. The contract testing job is simply to check: "Is everyone still honoring the deal?"

Why does this matter?

Imagine your team changes the response of an endpoint — maybe they rename a field, change a data type, or remove a property. Without contract tests, that kind of change can silently break consumers who depend on that contract. Contract testing catches this early, before it reaches production.

Why Use Swagger for This?

There are several tools for contract testing, such as:

Pact – Consumer-driven contract testing
Spring Cloud Contract – JVM-based microservices
Swagger / OpenAPI – API schema validation
Postman – Can validate schema contracts

Some teams adopt an API First approach, where API specifications are defined using OpenAPI before the API is even built. This prevents teams from being blocked during development and improves communication between frontend, backend, and QA.

That's precisely why using Swagger as our source of truth makes a lot of sense — if the contract is already documented, why not automate the validation against it?

What We're Building

Here's a quick overview of what we'll build:

A Swagger loader that fetches and caches the OpenAPI document
A schema extractor for responses
A schema extractor for requests (request body validation)
A validator that checks JSON payloads against those schemas
NUnit tests that tie everything together

We'll be using the public FakeRESTApi as our API under test, and the NSwag package to parse the Swagger document.

Step 1 — Loading the Swagger Document

The first thing we need is a way to fetch and cache the OpenAPI document. We don't want to download it on every test run — that would be slow and fragile.

public static async Task<OpenApiDocument> LoadDocument(string url)
{
    _cachedDocument = await OpenApiDocument.FromUrlAsync(url);
    return _cachedDocument;
}

Why cache it? The Swagger document usually doesn't change during a test run. Fetching it once and reusing it keeps your tests fast and avoids unnecessary network calls.

Step 2 — Extracting the Response Schema

With the document loaded, we need to navigate through it and find the specific schema for a given path, HTTP method, and status code.

private static JsonSchema GetSchemaResponse(
    OpenApiDocument doc,
    string path,
    string method,
    int statusCode,
    string contentType = "application/json")
{
    if (!doc.Paths.TryGetValue(path, out var pathItem))
        throw new Exception($"Path '{path}' not found in the swagger provided");

    if (!pathItem.TryGetValue(method, out var op))
        throw new Exception($"The method {method} isn't available for the path {path}");

    var statusKey = statusCode.ToString();

    if (!op.ActualResponses.TryGetValue(statusKey, out var response))
        throw new Exception($"Response {statusCode} isn't available for this path {path}");

    if (response.Content == null || !response.Content.Any())
        throw new Exception($"Response {statusCode} for {path} has no content.");

    var mediaType = response.Content
        .FirstOrDefault(c =>
            c.Key.StartsWith(contentType, StringComparison.OrdinalIgnoreCase));

    if (mediaType.Value == null)
        throw new Exception(
            $"The content type {contentType} isn't available for this path {path}. " +
            $"Available: {string.Join(", ", response.Content.Keys)}");

    return mediaType.Value.Schema;
}

Notice that we're throwing descriptive exceptions at every possible failure point. This is intentional — when a test fails, you want to know exactly why without digging through stack traces.

Step 3 — Extracting the Request Body Schema

Contract testing isn't only about what the API returns — it's also about what it receives. Let's add a method to extract the schema for request bodies:

private static JsonSchema GetSchemaRequest(
    OpenApiDocument doc,
    string path,
    string method,
    string contentType = "application/json")
{
    if (!doc.Paths.TryGetValue(path, out var pathItem))
        throw new Exception($"Path '{path}' not found.");

    if (!pathItem.TryGetValue(method, out var operation))
        throw new Exception($"Method '{method}' not found for '{path}'.");

    var requestBody = operation.RequestBody;

    if (requestBody == null)
        throw new Exception($"No request body defined for {method} {path}");

    var mediaType = requestBody.Content
        .FirstOrDefault(c =>
            c.Key.StartsWith(contentType, StringComparison.OrdinalIgnoreCase));

    if (mediaType.Value == null)
        throw new Exception($"Content type {contentType} not found for request.");

    return mediaType.Value.Schema;
}

Step 4 — Validating the JSON Payload

Now that we have the schema, we need to validate a real JSON payload against it. This is where the actual contract check happens:

private static void ValidateSchema(string bodyResponse, JsonSchema schema)
{
    var json = JToken.Parse(bodyResponse);
    var validator = new JsonSchemaValidator();

    var errors = validator.Validate(json, schema);
    if (errors.Count == 0) return;

    var msg = string.Join("", errors.Select(e => $"{e.Path} -> {e.Kind}"));
    throw new Exception($"The response isn't valid against the schema: {msg}");
}

If there are any violations — wrong types, missing required fields, unexpected values — this will surface all of them in a single readable message.

Step 5 — The Main Validation Entry Point

Let's wrap everything into one clean public method that the tests will call:

public async Task ContractValidation(
    string swaggerUrl,
    string path,
    string method,
    int expectedStatusCode,
    string bodyResponse,
    object? requestBody = null)
{
    // Load (or use cached) Swagger document
    var doc = await LoadDocument(swaggerUrl);

    // Validate request body if provided
    if (requestBody != null)
    {
        var requestSchema = GetSchemaRequest(doc, path, method.ToLower());
        var requestJson = JsonConvert.SerializeObject(requestBody);
        ValidateSchema(requestJson, requestSchema);
    }

    // Validate the API response
    var responseSchema = GetSchemaResponse(doc, path, method.ToLower(), expectedStatusCode);
    ValidateSchema(bodyResponse, responseSchema);
}

The requestBody parameter is optional — if you're testing a GET endpoint, you simply don't pass it, and only the response will be validated.

Step 6 — Writing the Tests

Now let's put it all together in NUnit tests:

using NUnit.Framework;
using SwaggerIntegration.Common;
using Assert = NUnit.Framework.Assert;

namespace SwaggerIntegration.Tests;

[TestFixture]
public class CheckSwaggerIntegration
{
    private SwaggerContractValidator _validator;
    private Requests _requests;

    private readonly string _baseUrl = "https://fakerestapi.azurewebsites.net/";
    private string _swaggerUrl;

    [SetUp]
    public void Setup()
    {
        _requests = new Requests(_baseUrl);
        _validator = new SwaggerContractValidator();
        _swaggerUrl = $"{_baseUrl}swagger/v1/swagger.json";
    }

    // Validates that the GET response matches the Swagger schema
    [Test]
    public async Task CheckSwaggerGetMethod()
    {
        var response = await _requests.GetAsync<string>("api/v1/Activities");
        Assert.That(response.StatusCode, Is.EqualTo(System.Net.HttpStatusCode.OK));

        await _validator.ContractValidation(
            _swaggerUrl,
            "/api/v1/Activities",
            "get",
            200,
            response.Content);
    }

    // Validates both the request body and the response against the Swagger schema
    [Test]
    public async Task CheckSwaggerPostMethod()
    {
        var bodyRequest = new
        {
            id = 0,
            title = "Test Activity",
            dueDate = DateTime.UtcNow,
            completed = true
        };

        var response = await _requests.PostAsync<string>("api/v1/Activities", bodyRequest);
        Assert.That(response.StatusCode, Is.EqualTo(System.Net.HttpStatusCode.OK));

        await _validator.ContractValidation(
            _swaggerUrl,
            "/api/v1/Activities",
            "post",
            200,
            response.Content,
            bodyRequest);
    }

    // This test intentionally sends an invalid body (id as string instead of int)
    // It should fail the contract validation, proving our validator works
    [Test]
    public async Task CheckSwaggerPostMethodInvalidBody()
    {
        var bodyRequest = new
        {
            id = "Sending as string", // Wrong type — should be int
            title = "Test Activity",
            dueDate = DateTime.UtcNow
        };

        var response = await _requests.PostAsync<string>("api/v1/Activities", bodyRequest);

        await _validator.ContractValidation(
            _swaggerUrl,
            "/api/v1/Activities",
            "post",
            200,
            response.Content,
            bodyRequest);
    }
}

Note on the invalid body test: This test is expected to fail — that's the point. It demonstrates that our validator is actually catching type mismatches. In a real project, you might want to assert the exception explicitly using Assert.ThrowsAsync to make the intent clear and keep the test green in CI.

Conclusion

Using Swagger as the foundation for contract testing is a smart move for teams that already follow an API First approach — or want to start. Instead of writing assertions manually for every field in every response, you let the OpenAPI spec do the heavy lifting.

What we built here is lightweight, reusable, and easy to plug into any existing test project:

One method handles both request and response validation
Descriptive errors tell you exactly what broke and where
Schema caching keeps things fast
Works for any endpoint defined in your Swagger doc

The best part? When the API contract changes, you update the Swagger spec — and your tests automatically reflect the new expectations. No need to hunt down and update dozens of individual assertions.

Future Improvements

There's a lot of room to evolve this approach. Here are some ideas worth exploring:

1. Support for path parameters and query strings Currently the approach focuses on request bodies and responses. A natural next step is validating path parameters (e.g., /api/v1/Activities/{id}) and query string types against the Swagger spec.

2. Schema versioning and drift detection Automatically compare two versions of a Swagger document (e.g., current vs. production) and generate a diff report. This could be used as a pre-merge gate to detect breaking changes before they reach consumers.

3. Integration with CI/CD pipelines These tests are already a great fit for pipelines, but you could take it further by generating a contract validation report as an artifact — showing which endpoints were validated, which passed, and which failed.

4. Support for multiple content types The current implementation defaults to application/json. Extending it to handle multipart/form-data, application/xml, and others would make it applicable to a wider range of APIs.

6. Auto-generating test cases from Swagger Going full circle with the first article in this series — the Swagger document could be used to automatically generate contract test skeletons for every endpoint and status code defined in the spec, reducing the manual effort to near zero.

You can check this full project here.

API vs Event Streaming: O Que Muda Para Quem Testa?

Alicia Marianne 🇧🇷 — Thu, 12 Feb 2026 20:50:04 +0000

✨ Oi, gente!

Recentemente, tenho pensado bastante em como contribuir para a comunidade de QA e ajudar aqueles que estão começando ou que já estão há algum tempo na área. Também quero relatar mais sobre os desafios de ser uma brasileira morando em terras lusitanas 🇵🇹.

O que me levou a querer voltar — na verdade, começar — a escrever na minha língua materna?

Bom, se eu fosse dizer de bate-pronto: saudade. 💛

Sim, aqui em Portugal fala-se português também, porém, como boa mineira que sou, sinto falta do nosso jeitinho de falar e escrever as coisas. Acredito que falar sobre o que sei na minha língua materna me deixa mais próxima de casa.

Outra razão que me fez querer começar a escrever em português foi voltar a ser mais ativa em comunidades de desenvolvimento (He4rt 💜). Tenho visto muitas mulheres entrando para a área, e algumas não são fluentes em inglês ou não confiam no que sabem para se arriscar em conteúdos em outros idiomas.

Com isso, eu pensei: por que não focar por um tempo em escrever em português? E cá estamos! ✍️

E, para que isso não seja apenas um comunicado, mas algo útil, já vou começar falando sobre serviços de mensageria (Event Streaming) e por que é importante que um QA (ou aspirante a QA) entenda pelo menos o básico sobre isso.

📡 Em termos técnicos

Event Streaming (Transmissão de Eventos) é a prática de capturar dados em tempo real a partir de fontes como bancos de dados, sensores, dispositivos móveis, serviços na nuvem e aplicações de software, na forma de fluxos de eventos; armazenar esses fluxos de forma duradoura para recuperação posterior; manipular, processar e reagir a esses eventos tanto em tempo real quanto retrospectivamente; e encaminhá-los para diferentes destinos, conforme necessário.

Lindo, né? 😅

Mas como podemos entender isso na prática?

🧠 Vamos imaginar o seu corpo

Sim, ele mesmo!

Você sente aquela dor no estômago 🍽️, que é um sinal de que está com fome. Esse sinal “fala” para o seu cérebro:

— Querido, estou precisando de nutrientes. Quando eles chegam?

Normalmente, você para o que está fazendo e vai comer algo. Mas perceba: seu estômago precisa esperar até o momento certo. Ainda assim, seu corpo continua funcionando normalmente. Ele não para.

No mundo do software, isso significa que, sempre que existir um sinal ou uma modificação, essa informação será interpretada sem que outras funcionalidades do sistema sejam interrompidas.

🛒 Exemplo prático: compras online

Você faz o seu pedido
Realiza o pagamento
O sistema de pagamento processa esse pagamento
Ele publica um evento informando que o pagamento foi aprovado
O sistema de notificação consome esse evento
E o cliente recebe a confirmação de que a compra será entregue 📦

Tudo isso pode acontecer sem que um sistema precise “esperar parado” pelo outro.

🔧 Algumas ferramentas de Event Streaming

Hoje, existem várias ferramentas no mercado que trabalham com esse modelo de comunicação baseada em eventos. Algumas das mais conhecidas são:

Apache Kafka 🐘 (uma das mais populares no mercado)
RabbitMQ 🐰
AWS Kinesis

Cada uma tem suas particularidades, mas todas trabalham com o conceito de produtores, consumidores e eventos trafegando por tópicos ou filas.

E aqui começa a parte interessante para nós, QAs. 👀

🔄 Event Streaming vs API

Você pode se perguntar:

“Alicia, mas qual a diferença disso para uma API? Eu consigo fazer o mesmo apenas chamando uma API.”

Conseguir, você consegue. Mas será que é a melhor opção? 🤔

Depende.

Normalmente, sistemas que utilizam apenas APIs (geralmente síncronas) funcionam assim: fazem uma requisição, esperam a resposta, e só depois continuam o processamento. Isso pode deixar o sistema mais acoplado e dependente.

Já em um sistema de Event Streaming (assíncrono), não é necessário esperar uma resposta para continuar o fluxo. O sistema segue funcionando enquanto os eventos são processados em paralelo.

E, principalmente, não queremos que o usuário fique esperando uma resposta para continuar usando o software ⏳.

🧪 E onde o QA e automação entram nisso?

Quando falamos de sistemas orientados a eventos, surgem novas perguntas de teste:

O evento foi realmente publicado?
O formato (schema) está correto?
O consumidor está processando corretamente?
O que acontece se o consumidor falhar?
Existe retry?

Percebe como o nível de complexidade aumenta?

Testar Event Streaming não é só validar payload. Envolve entender fluxo, consistência eventual, ordenação de mensagens, tolerância a falhas e comportamento assíncrono e partir dessa compreensão, é possível pensar em automações, do tipo:

Produzir eventos automatizados para testes
Validar consumo de mensagens
Testes de contrato (como com Schema Registry no Kafka)
Testes de integração ponta a ponta
Validação de mensagens em filas durante pipelines de CI/CD

✅ Conclusão

Entender mensageria e Event Streaming não é apenas “coisa de backend” — é conhecimento essencial para qualquer QA que queira testar sistemas modernos com mais profundidade. 💡

Quando compreendemos como os eventos trafegam, como os sistemas se comunicam de forma assíncrona e onde podem ocorrer falhas, ampliamos nossa capacidade de análise, criamos melhores cenários de teste e nos tornamos profissionais mais estratégicos.

Escrever sobre isso em português é, para mim, uma forma de compartilhar conhecimento, fortalecer a comunidade e também me sentir mais próxima de casa. 💛

E esse é só o começo. 🚀

[Boost]

Alicia Marianne 🇧🇷 — Thu, 05 Feb 2026 16:49:56 +0000

vitoriazzp

Jan 30

8 formas de fazer networking em Tecnologia

#productivity #networking #tutorial #braziliandevs

Comments 4

3 min read

Rewriting My First NUnit API Tests: Cleaner, Faster, Better

Alicia Marianne 🇧🇷 — Mon, 26 Jan 2026 20:08:21 +0000

Have you ever caught yourself thinking: “What am I even doing?” Well, that’s exactly how I feel right now. I was looking back at my previous tutorials and thought: “Why not review them, critique them, and make a better version?” And here we are. In this first article, I’ll revisit my initial tutorial: NUNit and C# - Tutorial to automate your API Tests from scratch.

I remember this project was born out of a real need at work. We had a lot of APIs with Postman tests, and we decided to migrate them to NUnit and C#. Now, looking back after over two years, I see many ways to improve it.

Review of the tests that will be automated and the project

Starting from the tests that will be done. For this new project, we will use the https://dummyjson.com/ just to test some concepts, since the API are simple, and after that, we will refactor the tests for NASA API Tests.
For the new endpoint we will automate the following tests:

Endpoint	Test	Expected results
/users	Get all users	Should return status 200 and a list with user information
/users/add	Create a new user	Should return status 200 and user data information

The main goal is to make the project more robust, easier to replicate across scenarios, improve request handling, make it reusable for different projects, enable future CI/CD integration, and ensure sensitive data are protected.

Creating the automated tests

The first major improvement is how we set up API requests. In my first article, I had a separate method for each API request. Now, I’ve made them more abstract and maintainable:

using RestSharp;

namespace ApiTestsWithNUnit.Common;

public class Requests(string baseUrl)
{
    private readonly RestClient _client = new(baseUrl);

    public async Task<RestResponse<T>> GetAsync<T>(
        string endpoint,
        Dictionary<string, string>? headers = null)
    {
        var request = new RestRequest(endpoint, Method.Get);

        if (headers != null)
        {
            foreach (var header in headers)
                request.AddHeader(header.Key, header.Value);
        }

        return await _client.ExecuteAsync<T>(request);
    }

    public async Task<RestResponse<T>> PostAsync<T>(
        string endpoint,
        object? body = null,
        Dictionary<string, string>? headers = null)
    {
        var request = new RestRequest(endpoint, Method.Post);

        if (body != null)
            request.AddJsonBody(body);

        if (headers != null)
        {
            foreach (var header in headers)
                request.AddHeader(header.Key, header.Value);
        }

        return await _client.ExecuteAsync<T>(request);
    }
}

And for the tests, they will be used in the following way:

using System.Net;
using ApiTestsWithNUnit.Entities;
using FluentAssertions;
using NUnit.Framework;
using RestSharp;

namespace ApiTestsWithNUnit.Tests;

public class Users
{
    private Common.Requests _api;
    private Dictionary<string, string> _headers;

    [SetUp]
    public void Setup()
    {
        _api = new Common.Requests("https://dummyjson.com");
        _headers  = new Dictionary<string, string>
        {
            { "Content-Type", "application/json" }
        };
    }

    [Test]
    public async Task GetAllUsers()
    {
        var response = await _api.GetAsync<object>("/users");
        response.Should().NotBeNull();
        response.StatusCode.Should().Be(HttpStatusCode.OK);
        var users = response.Content;
        users.Should().NotBeNull();
    }

    [Test]
    public async Task CreateNewUser()
    {
        var bodyRequest = new UsersRequestBody
        {
            firstName = "John",
            lastName = "Doe",
            email = "john.do@example.com",
            age = 20
        };

        var response = await _api.PostAsync<object>("/users/add", bodyRequest, _headers);
        response.Should().NotBeNull();
        response.StatusCode.Should().Be(HttpStatusCode.Created);
        var users = response.Content;
        users.Should().Contain(bodyRequest.firstName);
        users.Should().Contain(bodyRequest.lastName);
        users.Should().Contain(bodyRequest.email);
    }

}

Updating the NASA API tests, we will make two changes, first, move all the validations to a new class called CheckBodyResponse.cs:

using FluentAssertions;
using LearningNUnit.BackEnd.Entities;

namespace ApiTestsWithNUnit.Common;

public class CheckBodyResponse
{
    public void CheckBodyResponseNasaApi(NasaApiEntity response)
    {
        response.Date.Should().NotBeNullOrEmpty();
        response.Explanation.Should().NotBeNullOrEmpty();
        response.Media_type.Should().NotBeNullOrEmpty();
        response.Service_version.Should().NotBeNullOrEmpty();
        response.Title.Should().NotBeNullOrEmpty();
        response.Url.Should().NotBeNullOrEmpty();
    }

}

And refactoring our tests, they will look like:

using System.Net;
using System.Text.Json;
using ApiTestsWithNUnit.Common;
using ApiTestsWithNUnit.Config;
using ApiTestsWithNUnit.Entities;
using FluentAssertions;
using LearningNUnit.BackEnd.Entities;
using NUnit.Framework;
using RestSharp.Serializers;

namespace ApiTestsWithNUnit.Tests;

public class NasaApiTest
{
    private Common.Requests _api;
    private Dictionary<string, string> _headers;
    private string _api_key;
    private CheckBodyResponse CheckBodyResponse = new();

    [SetUp]
    public void Setup()
        {
        _headers  = new Dictionary<string, string>
        {
            { "Content-Type", "application/json" }
        };
        _config = TestConfig.GetConfig();
        string nasa_url= "https://api.nasa.gov";
        _api = new Common.Requests(nasa_url);
        _api_key = $"api_key=API_KEY";
    }

    [Test]
    public async Task SearchApodSucess()
    {
        var response = await _api.GetAsync<string>($"/planetary/apod?{_api_key}", _headers);
        response.StatusCode.Should().Be(HttpStatusCode.OK);
        if (response.Content != null)
        {
            var data = JsonSerializer.Deserialize<NasaApiEntity>(
                response.Content,
                new JsonSerializerOptions { PropertyNameCaseInsensitive = true }
            );
            CheckBodyResponse.CheckBodyResponseNasaApi(data);
        }
    }
    [Test]
    public async Task SearchApodWithDate()
    {
            var queryParameters = "date=2023-05-01";
            var response = await _api.GetAsync<string>($"/planetary/apod?{_api_key}&{queryParameters}", _headers);
            response.StatusCode.Should().Be(HttpStatusCode.OK);
            if (response.Content != null)
            {
                var data = JsonSerializer.Deserialize<NasaApiEntity>(
                    response.Content,
                    new JsonSerializerOptions { PropertyNameCaseInsensitive = true }
                );
                CheckBodyResponse.CheckBodyResponseNasaApi(data);
            }
    }

    [Test]
    public async Task SearchApodWithDateWrongFormat()
    {
        var queryParameters = "date=2023/05/01";
        var response = await _api.GetAsync<string>($"/planetary/apod?{_api_key}&{queryParameters}", _headers);
        response.StatusCode.Should().Be(HttpStatusCode.BadRequest);
    }

    [Test]
    public async Task SearchApodWithStartDateEndDate()
    {
        const string startDate = "2023-05-01";
        const string endDate = "2023-06-01";
        var queryParameters = $"start_date={startDate}&end_date={endDate}";
        var response = await _api.GetAsync<string>($"/planetary/apod?{_api_key}&{queryParameters}", _headers);
        response.StatusCode.Should().Be(HttpStatusCode.OK);
    }

    [Test]
    public async Task SearchApodWithStartDateBiggerThanEndDate()
    {
        const string startDate = "2023-12-01";
        const string endDate = "2023-11-01";
        var queryParameters = $"start_date={startDate}&end_date={endDate}";
        var response = await _api.GetAsync<string>($"/planetary/apod?{_api_key}&{queryParameters}", _headers);
        response.StatusCode.Should().Be(HttpStatusCode.BadRequest);
    }
    [Test]
    public async Task SearchApodWithInvalidToken()
    {
        var token = "invalidToken";
        var response = await _api.GetAsync<string>($"/planetary/apod?{token}", _headers);
        response.StatusCode.Should().Be(HttpStatusCode.Forbidden);
    }
}

Improvements in This Version:

Setup notation: Makes tests cleaner and avoids repeating variables.
Abstract requests: One method for multiple endpoints instead of duplicating code.
Easier maintenance: The tests are more readable and modular.
Making the tests asyncronos.

Beware of sensitive data

Testing user data safely is always tricky. Here, we’ll leverage Rider/Visual Studio’s .runsettings file:

.runsettings is an XML file that allows you to configure test parameters, environments, timeouts, and other TestRunner behaviors in Visual Studio or dotnet test.

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <TestRunParameters>
    <Parameter name="BaseUrl" value="https://dummyjson.com" />
    <Parameter name="UserName" value="John" />
    <Parameter name="LastName" value="do" />
    <Parameter name="Email" value="john.do@example.com" />
    <Parameter name="ApiToken" value="YOUR_API_KEY" />
    <Parameter name="NasaBaseUrl" value="https://api.nasa.gov" />
  </TestRunParameters>
  <RunConfiguration>
    <TargetFrameworkVersion>net7.0</TargetFrameworkVersion>
    <ResultsDirectory>TestResults</ResultsDirectory>
  </RunConfiguration>
</RunSettings>

⚠️ Do not commit this file to version control! Include it in .gitignore.

We can then create a config class to handle local vs. CI environments:

using NUnit.Framework;

namespace ApiTestsWithNUnit.Config;

public class Config
{
    public string BaseUrl { get; set; } = string.Empty;
    public string UserName { get; set; } = string.Empty;
    public string LastName { get; set; } = string.Empty; 
    public string Email { get; set; } = string.Empty;
    public string ApiToken { get; set; } = string.Empty;
    public string NasaBaseUrl { get; set; } = string.Empty;
}

public class TestConfig
{
    public static Config GetConfig()
    {
        // Check if it is running in the CI
        bool isCi = Environment.GetEnvironmentVariable("GITHUB_ACTIONS") == "true";

        if (isCi)   
        {
            return new Config
            {
                BaseUrl = Environment.GetEnvironmentVariable("BASE_URL") 
                          ?? throw new Exception("BASE_URL not found in the CI"),
                UserName = Environment.GetEnvironmentVariable("API_KEY") 
                           ?? throw new Exception("USER_NAME not found in the  CI"),
                LastName = Environment.GetEnvironmentVariable("LAST_NAME") 
                           ?? throw new Exception("LAST_NAME not found in the  CI"),
                Email = Environment.GetEnvironmentVariable("LAST_NAME") 
                           ?? throw new Exception("EMAIL not found in the  CI"),
                ApiToken = Environment.GetEnvironmentVariable("API_TOKEN") 
                           ?? throw new Exception("API_TOKEN not found in the  CI"),
                NasaBaseUrl = Environment.GetEnvironmentVariable("NASA_BASE_URL") 
                           ?? throw new Exception("NASA_BASE_URL not found in the  CI")
            };
        }
        return new Config
        {
            BaseUrl = TestContext.Parameters["BaseUrl"] ?? "",
            UserName = TestContext.Parameters["UserName"] ?? "",
            LastName = TestContext.Parameters["LastName"] ?? "",
            Email = TestContext.Parameters["Email"] ?? "",
            ApiToken = TestContext.Parameters["ApiToken"] ?? "",
            NasaBaseUrl = TestContext.Parameters["NasaBaseUrl"] ?? ""
        };
    }
}

This lets us run the same tests locally or in CI/CD pipelines seamlessly.

Updating our tests, they will look like this:

{}

namespace ApiTestsWithNUnit.Tests;

public class NasaApiTest
{
    private Common.Requests _api;
    private Dictionary<string, string> _headers;
    private Config.Config _config;
    private string _api_key;
    private CheckBodyResponse CheckBodyResponse = new();

    [SetUp]
    public void Setup()
    {
        _headers  = new Dictionary<string, string>
        {
            { "Content-Type", "application/json" }
        };
        _config = TestConfig.GetConfig();
        _api = new Common.Requests(_config.NasaBaseUrl);
        _api_key = $"api_key={_config.ApiToken}";
    }

    [Test]
    public async Task SearchApodSucess()
    {
        var response = await _api.GetAsync<string>($"/planetary/apod?{_api_key}", _headers);
        response.StatusCode.Should().Be(HttpStatusCode.OK);
        if (response.Content != null)
        {
            var data = JsonSerializer.Deserialize<NasaApiEntity>(
                response.Content,
                new JsonSerializerOptions { PropertyNameCaseInsensitive = true }
            );
            CheckBodyResponse.CheckBodyResponseNasaApi(data);
        }
    }

 }

To use the runsettings file in Rider, you need to:

File / Settings → Build, Execution, Deployment → Unit Testing → Test Runner
Go to Use specific .runconfig/.testsettings settings file and select your .runsettings file.

In the Visual Studio you go to:

Go to Test → Configure Run Settings and check if Visual Studio is using the .runsettings file.

Adding Allure report to the tests

We will be using the Allure report to see the results of our tests, you can check how to install here - Allure install.

To add Allure to the project, is pretty simple. You can go directly through Nuget Packages management of your IDE and Search for Allure and install the latest version, or you can run:

dotnet add ⟨PATH TO PROJECT⟩ package Allure.NUnit

Now let's start to add some notations to our project:

Let's add to our test class the notation [AllureNunit] so we can use the features of allure.

Here are some notations that will be useful for this example:

Notation	Description
AllureSeverity	The severity of the test
AllureOwner	The responsible for the test
AllureFeature	The feature that will be tested
AllureBefore	Setup that is made before the test

This are some of notations that can make the report more clear, so the tests will looks like:

{}

[TestFixture]
[AllureNUnit]
[AllureFeature("Nasa API Tests")]
public class NasaApiTest
{
    private Common.Requests _api;
    private Dictionary<string, string> _headers;
    private Config.Config _config;
    private string _api_key;
    private CheckBodyResponse CheckBodyResponse = new();

    [SetUp]
    [AllureBefore("Loading the test variables")]
    public void Setup()
    {
        _headers  = new Dictionary<string, string>
        {
            { "Content-Type", "application/json" }
        };
        _config = TestConfig.GetConfig();
        _api = new Common.Requests(_config.NasaBaseUrl);
        _api_key = $"api_key={_config.ApiToken}";
    }

    [Test]
    [AllureOwner("@m4rri4nne")]
    [AllureSeverity(SeverityLevel.critical)]
    [AllureDescription("Search for APOD")]
    public async Task SearchApodSucess()
    {
        var response = await _api.GetAsync<string>($"/planetary/apod?{_api_key}", _headers);
        response.StatusCode.Should().Be(HttpStatusCode.OK);
        if (response.Content != null)
        {
            var data = JsonSerializer.Deserialize<NasaApiEntity>(
                response.Content,
                new JsonSerializerOptions { PropertyNameCaseInsensitive = true }
            );
            CheckBodyResponse.CheckBodyResponseNasaApi(data);
        }
    }
  }

To configure the allure we will need to add a few steps before run again the tests. First, we will create an json file, with the minimal information needed for Allure:

{
  "allure": {
    "directory": "allure-results"
  }
}

This file should be created on the root of the project. And after that, add to the .csproj the follwing information:

<ItemGroup>
  <None Update="allureConfig.json">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </None>
</ItemGroup>

And create a global setup to always register the executions, in a AllureConfig.cs file:

using NUnit.Framework;
using Allure.Net.Commons;

[SetUpFixture]
public class AllureSetup
{
    [OneTimeSetUp]
    public void GlobalSetup()
    {
        AllureLifecycle.Instance.CleanupResultDirectory();
    }
}

After that, remember to: Clean, restore and build your project before run the tests.
After running the tests you can see the results using this command:

allure serve ./bin/Debug/net8.0/allure-results

And there is our report:

You can check the notations from allure to add more that can be useful in your context.

Final thoughts

It was fun revisiting a project I built a few years ago. There are more improvements I could make, but I’ve focused on clarity, maintainability, and introducing Allure reports.

One key takeaway: .runsettings for environment variables is great, but for large projects, managing many variables can get tricky. A future improvement could be to orchestrate credentials via JSON files and add to CI/CD workflow.

You can explore the complete solution in my Git repository.

From Swagger to Tests: Building an AI-Powered API Test Generator with Python

Alicia Marianne 🇧🇷 — Sat, 03 Jan 2026 21:23:08 +0000

Working as a QA with APIs can be… well, kind of a nightmare sometimes. APIs are always changing, endpoints get added, status codes get updated, and keeping your tests in sync feels like chasing a moving target.

If you only look at your task board, it’s easy to lose track of what actually changed and what still needs testing.

In the projects I worked on, we had Swagger available for the API. And I thought: wait a minute… why not use AI and Swagger to save time generating tests?

And that’s how this little project started. In this post, I’ll walk you through how I did it, the challenges I faced, and some cool things you can do next.

The Idea

The goal was simple: take the Swagger spec and extract all the useful info, like:

HTTP methods
Expected status codes
Query parameters
Request bodies

…and then generate both positive and negative test scenarios automatically.

For example, for a simple GET /users/{id} endpoint, I wanted the output to look like this:

GET /users/{id}
✔ Scenario: Retrieve a user with a valid ID  
✔ Scenario: Validate 404 for user not found  
✘ Scenario: Missing ID parameter  
✘ Scenario: Invalid format for ID

To make this work nicely, I used AI to create the scenarios based on the endpoint’s Swagger specification, following a template I defined.

About the project

Stack

Python – fast, easy to parse data, integrate stuff
Rich / Typer (CLI UX) – because a pretty CLI makes life better
Gemini AI – super simple Python integration for AI prompts
dotenv – to keep the AI keys safe

Project Structure

api-test-generator/
├── README.md                   # Documentation of the project
├── requirements.txt            # Dependências Python
├── main.py                     # Main function
│
├── output/                         # Folder with generated tests 
│   ├── get_Books.txt
│   ├── post_Books.txt                  
│
├── functions/                       # Main functions of the project 
│   ├── navigation.py                # CLI navigation
│   ├── read_swagger.py              # Read files and URL swaggers 
│   └── test_generator.py            # Generate tests and save them in the files
│
└── assets/                    # theme and example for the project 
    ├── swaggerexample.json
    └── theme.py

How it works

┌──────────────────────────────┐
│          User / QA           │
│   (CLI Interaction - Rich)   │
└──────────────┬───────────────┘
               │
               ▼
┌──────────────────────────────┐
│        CLI Interface         │
│   (Typer + Rich Menu)        │
└──────────────┬───────────────┘
               │
               ▼
┌──────────────────────────────┐
│   Swagger/OpenAPI Loader     │
│  - URL, Manual, or Local JSON│
│  - Validation & Parsing      │
└──────────────┬───────────────┘
               │
               ▼
┌──────────────────────────────┐
│   API Specification Parser   │
│  - Endpoints                 │
│  - Methods                   │
│  - Parameters                │
│  - Responses / Status Codes  │
└──────────────┬───────────────┘
               │
               ▼
┌──────────────────────────────┐
│        Gemini AI API         │
│   (Test Case Generation)     │
└──────────────┬───────────────┘
               │
               ▼
┌──────────────────────────────┐
│     Output Generator         │
│  - Text file export (.txt)   │
│  - Structured scenarios      │
└──────────────────────────────┘

So basically: user interacts with CLI → loads Swagger → parses specs → builds a prompt → sends to AI → AI returns tests → saves to file.

Code Highlights

The test generator

The core idea here was: extract as much info as possible from Swagger so the AI could generate meaningful tests.

Here’s the main function I wrote:

def test_generator(path, method, swagger_data):  
    print(f"Generating tests for {method.upper()} {path}...")  
    details = swagger_data["paths"][path][method]  
    request_body = ""  
    parameters = ""  
    # Getting information about the endpoint  
    if 'tags' not in details:  
        endpoint_name = path  
    elif len(details['tags']) == 0:  
        endpoint_name = path  
    else:  
        endpoint_name = details['tags'][0]  
    if 'requestBody' in details:  
        request_body = details['requestBody']  
    if 'parameters' in details:  
        parameters = details['parameters']  
    prompt = (f"Generate positive and negative tests for this endpoint:{path} for the method {method.upper()}"  
              f"considering the following specifications: "             
              f"Name of the endpoint: {endpoint_name}"  
              f"Request body: {request_body}"  
              f"Query Parameters: {parameters} and return the tests following this template: {theme.PROMPT_TEMPLATE}")  
    test_scenario = ai_connection(prompt)  
    print(f"Exporting tests to file...")  
    export_to_file(test_scenario, method, endpoint_name)

Connecting to Gemini AI

Connecting to the AI is simple: create a client, set the model, and pass the prompt:

def ai_connection(prompt):  
    load_dotenv()  
    api_key = os.getenv("GOOGLE_API_KEY")  
    client = genai.Client(api_key=api_key)  

    response = client.models.generate_content(  
        model="gemini-2.5-flash",  
        contents=prompt  
    )  
    return response.text

And voilà. The AI returns something like:

POST /api/v1/Books  
✔ Scenario: Successfully create a new book with all valid fields  
✔ Scenario: Successfully create a new book with only mandatory fields  
✔ Scenario: Successfully create a new book using 'text/json; v=1.0' content type  

✘ Scenario: Fail to create book due to missing 'title' field  
✘ Scenario: Fail to create book due to missing 'author' field  
✘ Scenario: Fail to create book due to missing 'isbn' field  
✘ Scenario: Fail to create book with an 'isbn' that already exists (conflict)  
✘ Scenario: Fail to create book due to invalid 'isbn' format (e.g., too short, non-numeric where expected)  
✘ Scenario: Fail to create book due to 'publication_year' being a string instead of an integer  
✘ Scenario: Fail to create book due to empty request body  
✘ Scenario: Fail to create book due to malformed JSON in request body  
✘ Scenario: Fail to create book with an empty 'title' string  
✘ Scenario: Fail to create book with an empty 'author' string

Challenges & Lessons Learned

Honestly, the hardest part was cleaning up Swagger data and building prompts that make sense for the AI.
Another challenge was designing a workflow that actually works in a CLI without feeling clunky.
But in the end, it was super fun, and I learned a lot about AI-assisted testing.

What’s Next

While building this, I started dreaming about all the things I could do next:

Automatically generate Postman collections from these tests
Integrate with test management tools like Zephyr or Xray
Make it a service that monitors Swagger and updates tests whenever endpoints change The possibilities are endless.

Conclusion

This project really showed me that AI + OpenAPI = massive time saver.

Instead of manually writing dozens of tests for every endpoint, I now have an automated system that generates both positive and negative scenarios in minutes.

Next steps? Think bigger: integrate it with CI/CD pipelines, plug it into test management tools, or even make it monitor APIs in real-time. Smarter, faster, and way less painful API testing—sounds like a win to me.

If you want to check out the full project, explore the code, or try it yourself, it’s all on my GitHub: API Test Generator.

Dive in, experiment, and see how much time you can save!

From Genin to Kage - Understanding the Test Pyramid with Naruto

Alicia Marianne 🇧🇷 — Tue, 23 Dec 2025 23:35:23 +0000

It has been a long time since I last wrote an article, and in order to break this hiatus, I decided to return to the basics and revisit core concepts that are essential for QAs and developers. I will start with the Test Pyramid, using an analogy with the ninja hierarchy in Naruto to make the concept more intuitive and relatable.

Before exploring the Test Pyramid, it is important to briefly understand what Naruto is. Naruto is a Japanese manga and anime series created by Masashi Kishimoto. The story follows Naruto Uzumaki, a young ninja from the Hidden Leaf Village who grows up lonely and socially isolated because a powerful fox spirit, the Nine-Tails, is sealed within him. Despite this, Naruto dreams of becoming the Hokage, the leader of his village, as a way to earn recognition and prove his value.

Within this universe, ninjas are classified into different levels according to their experience and responsibilities:

Academy Students
Genin
Chūnin
Jōnin
Kage
ANBU

But how does this hierarchy relate to the Test Pyramid? The Test Pyramid was introduced and popularized around 2009 by Mike Cohn and, in essence, it is a model used to classify automated tests with the goal of achieving better test coverage, improved software delivery performance, and, ultimately, higher software quality. The pyramid is composed of three main layers:

Unit tests
Integration tests
End-to-end (E2E) tests

Let us now examine each layer of the Test Pyramid using the ninja ranks from Naruto as a metaphor.

Unit Tests = Academy Students and Genin

Academy Students are children in training who are not yet official ninjas. They learn the fundamentals, such as chakra control, village rules, teamwork, and basic jutsu like transformation, substitution, and cloning. Genin, in turn, are newly graduated ninjas who begin wearing their village headbands and performing simple missions, typically ranked D or C. They operate in teams under the supervision of a jōnin, focusing on gaining experience, discipline, and strengthening their basic skills.

In the context of software testing, unit tests closely resemble these early ninja ranks. Unit tests validate small, isolated pieces of code—such as functions or methods. Individually, they may seem simple, but when combined, they provide significant value, fast feedback, and broad test coverage. Just like Genin missions, they are numerous, lightweight, and form the foundation of the entire testing strategy.

Integration Tests = Chūnin

Chūnin are intermediate-level ninjas who demonstrate not only combat ability, but also intelligence, leadership, and emotional maturity. Usually promoted through the Chūnin Exams, they are capable of leading squads, making tactical decisions, and carrying out more complex missions ranked C or B. They represent the operational backbone of the village.

Similarly, integration tests validate the interaction between different components of a system, such as communication between services, databases, or external APIs. These tests take longer to execute than unit tests and require more setup, but they are essential for ensuring that different parts of the application work correctly together. Like Chūnin missions, there are fewer integration tests than unit tests, but they remain a critical layer of the pyramid.

End-to-End (E2E) Tests = Jōnin

Jōnin are highly experienced and versatile ninjas, capable of handling high-risk missions (rank A or S) independently and mentoring the next generation. They possess deep technical knowledge, excellent chakra control, and advanced tactical skills.

End-to-end tests align well with the Jōnin role. These tests simulate real user behavior and validate complete system flows, such as login processes or checkout journeys. While they provide high confidence, they are also more complex, slower to execute, and more expensive to maintain. For this reason, they should be used sparingly, just as Jōnin are assigned only to the most critical missions.

Manual Testing = Kage

The Kage is the most powerful and respected ninja in a village, responsible for strategic, political, and military decisions, as well as the village’s protection in times of crisis. A Kage possesses vast experience and oversees all other ranks.

In this analogy, the Kage represents manual testing. Although the Test Pyramid focuses on automation, manual testing remains essential. Without it, the entire strategy can fail. Manual tests help QAs and developers decide what should be automated, at which layer, and what is better left unautomated. They prevent low-value automation and ensure that testing efforts remain aligned with business goals.

ANBU = Non-Functional Tests

ANBU is not a traditional rank, but a special force that reports directly to the Kage. Its members perform sensitive missions such as espionage, infiltration, and protection of critical information, operating discreetly and often without recognition.

This role aligns well with non-functional tests, such as performance, load, and security testing. These tests focus on the technical qualities of a system rather than its functionality. While they may not be executed as frequently as other tests, their impact is significant, as they directly influence system stability, reliability, and trustworthiness.

Conclusion

Just as a village relies on a strong foundation of Genin, capable Chūnin, elite Jōnin, wise Kage leadership, and specialized ANBU forces, a healthy testing strategy depends on a solid base of unit tests, supported by integration tests, a small number of E2E tests, informed manual testing, and essential non-functional validations. When these elements are well balanced, teams can deliver software with confidence, quality, and sustainability.

Browser-based performance testing with K6

Alicia Marianne 🇧🇷 — Mon, 04 Aug 2025 21:28:28 +0000

Is pretty common link perfomance testing with backend applications, but what if I say to you that you can automate and get key metrics that will help you improve the performance of your frontend applications? That is what we are going to learn in this tutorial.

Key concepts for Browser performance

Before we go to the hands on, we need to understand what is the browser performance testing.

Browser-based load testing verifies the frontend performance of an application by simulating real users using a browser to access your website.

This type o testing provides a way to mesure user experience and find issues that are difficult to catch on the protocol level. When running the tests, we will be able to answers questions like:

When my application is receiving thousands of simultaneous requests from the protocol-level, what happens to the frontend?
How can I get metrics specific to browsers, like total page load time?
Are all my elements interactive on the frontend?
Are there any loading spinners that take a long time to disappear?

How we will use for this tutorial k6, we need to understand some concepts about the framework itself:

Browser type: A browser script must have the options.browser.typefield set in the options object.
Asynchronous operations: Browser interactions (like navigation and clicks) are asynchronous, so your test must use async/await. Refer toAsynchronous operations for more details.
Locators: The Locator API can be used to find and interact with elements on a page, such as buttons or headers. There are other ways to find elements, such as using the Page.$() method, but the Locator API provides several benefits, including finding an element even if the underlying frame navigates, and working with dynamic web pages and SPAs such as React.

Setup of the project

Now that we know the key concepts, we can start to create our project. For this tutorial, we will use the e-commerce application Swag Labs and get the metrics for the main workflow:
Login → Add items to the cart → CheckOut
For it you need to:

Have a latest version of node and npm installed
Install K6

Project structure

k6-tests/       
├── libs/                   
│   └── addToCart.js  
│   └── auth.js   
│   └── checkOut.js            
├── pages/                    
│   └── CheckOutOverview.js 
│   └── CheckoutPage.js 
│   └── DashboardPage.js 
│   └── LoginPage.js 
│   └── SuccessPage.js        
├── tests/                  
│   └── buyItemsPeformanceGlith.js
│   └── buyItemsUserA.js              
├── utils/
│   └── k6-options.js

Creating the tests

Is important to mention that k6 runs only one test per time

The basic structure of a K6 project is:

Imports
Define options
Setup
Default function: The default exported function is the entry point for the test script. It will be executed repeatedly the number of times you define with the iterations option. In this function, using async/await, add the functions that open a browser page, interacts with the elements on the page, takes a screenshot, and performs a check.

Knowing it, we can write our tests in the following model:
buyItemsUserA.js

import {performLogin} from '../libs/auth.js';
import {addItemsToCart} from '../libs/addToCart.js';
import {browserOptions} from '../utils/k6-options.js';
import {checkOut} from '../libs/checkOut.js';


export const options = browserOptions;


export default async function () {
    const page = await performLogin('standard_user', 'secret_sauce');
    await addItemsToCart(page);
    await checkOut(page);

    await page.close();
}

Now, we need to understand what which function does.

browserOptions

To avoid write for all tests the same configuration, we will create on the utils folder the file k6-options.js where we will add the basic configuration for our test:

export const browserOptions = {
    scenarios: {
        ui: {
            executor: 'shared-iterations',
            vus: 1,
            iterations: 15,
            options: {
                browser: {
                    type: 'chromium',
                },
            },
        },
    },
};

Here we define the number of interactions we want, the number of instances, and the type of browser that will be used.

performLogin

On the lib folder we will create a function that will be responsible to perform the login and return the page context to our tests:

import { browser } from 'k6/browser';
import { check } from 'k6';
import { LoginPage } from '../pages/LoginPage.js';

/**
 * Do the login and return the page authenticated.
 * @param {string} username
 * @param {string} password
 * @param {string} url
 * @returns {Promise<Page>}
 */
export async function performLogin(username, password, url = 'https://www.saucedemo.com/') {
    const page = await browser.newPage();
    const loginPage = new LoginPage(page);
    await loginPage.goto(url);
    await loginPage.login(username, password);

    const loginMessage = await loginPage.getHomePageMessage();
    check(page, {
        'Login done with success': () => loginMessage === 'Products',
    });

    return page;
}

The performLogin will call the LoginPage.js class that contain the locators, and methods for the login:

export class LoginPage {
    constructor(page) {
        this.page = page;
        this.usernameField = page.locator('#user-name');
        this.passwordFiled = page.locator('#password');
        this.loginButton = page.locator('#login-button');
        this.homePageHeader = page.locator('.title');
    }

    async goto(){
        await this.page.goto('https://www.saucedemo.com/');
    }

    async login(username, password) {
        await this.usernameField.type(username);
        await this.passwordFiled.type(password);
        await this.loginButton.click();
    }

    async getHomePageMessage(){
        return this.homePageHeader.textContent();
    }
}

addItemsToCart

The method addItemsToCart will be responsible to add items to the cart and make sure that they were added with success using the DashboardPage.js methods

import { check } from 'k6';
import {DashboardPage} from '../pages/DashboardPage.js';

/**
 * Receives the page context and add items to the cart
 * @param {Page} page
 */
export async function addItemsToCart(page) {
    const dashboardPage = new DashboardPage(page);

    await dashboardPage.addCart();
    let count = await dashboardPage.getCartCount();
    check(page, {
        'Items': () => count === '3',
    });
}

DashboardPage.js

export class DashboardPage {
    constructor(page) {
        this.page = page;
        this.addCartBackPack = page.locator('[data-test="add-to-cart-sauce-labs-backpack"]');
        this.addCartboltTShirt = page.locator('#add-to-cart-sauce-labs-bolt-t-shirt');
        this.addCartonesieRip = page.locator('#add-to-cart-sauce-labs-onesie');
        this.cartIcon = page.locator('[data-test="shopping-cart-badge"]');
    }

    async addCart(){
        await this.addCartBackPack.click();
        await this.addCartboltTShirt.click();
        await this.addCartonesieRip.click();
    }

    async getCartCount(){
        await this.cartIcon.isVisible();
        return await this.cartIcon.textContent();
    }

}

checkOut

The checkOut method make all the process of checkout:

import {CheckoutPage} from "../pages/CheckoutPage.js";
import {SuccessPage} from "../pages/SuccessPage.js";
import {CheckOutOverview} from "../pages/CheckOutOverview.js";

/**
 * Receives the page context and add items to the cart
 * @param {Page} page
 */
export async function checkOut(page) {
    const checkoutPage = new CheckoutPage(page);
    const checkoutOverview = new CheckOutOverview(page);
    const successPage = new SuccessPage(page);


    await checkoutPage.checkOut('John', 'Doe', '12345');
    await checkoutOverview.confirmCheckOut();
    await successPage.getSuccessMessage();
}

For checkout, different from te other pages, we have three different pages when doing it:

First one to access the cart and confirm the items
Second to add the information
Third page of Success So for it, we will create three pages as well: CheckOutPage.js - Click on the cart and redirect to the CheckOut Overview page

export class CheckoutPage {

    constructor(page) {
        this.page = page;
        this.cartIcon = page.locator('[data-test="shopping-cart-link"]');
        this.checkoutButton = page.locator('[data-test="checkout"]');
        this.checkInformation = page.locator('[data-test="title"]');
        this.firstName = page.locator('#first-name');
        this.lastName  = page.locator('#last-name');
        this.zipCode = page.locator('#postal-code');
        this.continueButton = page.locator('[data-test="continue"]');
    }

    async goCheckoutPage(){
        await this.cartIcon.click();
        await this.checkoutButton.click();
        return await this.checkInformation.textContent();
    }

    async addInformation(firstName, lastName, zipCode){
        await this.firstName.type(firstName);
        await this.lastName.type(lastName);
        await this.zipCode.type(zipCode);
        await this.continueButton.click();
    }

    async checkOut(firstName, lastName, zipCode){
        await this.goCheckoutPage()
        await this.addInformation(firstName, lastName, zipCode);

    }

}

CheckOutOverview.js- Confirm the informations and what is going to be the payment method

import { check } from 'k6';

export class CheckOutOverview{
    constructor(page){
        this.page = page;
        this.chekoutOverview = page.locator('[data-test="title"]');
        this.finishButton = page.locator('[data-test="finish"]');
    }

    async confirmCheckOut(){
        const checkOutOverviewTitle = await this.chekoutOverview.textContent();
        check(this.page, {
            'CheckoutOverview': () => checkOutOverviewTitle === "Checkout: Overview",
        })
        await this.finishButton.click();
    }
}

SuccessPage.js - Confirmation of the payment

import { check } from 'k6';

export class SuccessPage {
    constructor(page){
        this.page = page;
        this.successMessage = page.locator('[data-test="complete-header"]');
        this.fullMessage = page.locator('[data-test="complete-text"]');
        this.backHomeButton = page.locator('[data-test="back-to-products"]');
    }

    async getSuccessMessage(){
        const successMessageText = await this.successMessage.textContent();
        const fullMessageText = await this.fullMessage.textContent();
        check(this.page, {
            'Success short message': () =>  successMessageText === "Thank you for your order!",
            'Success full message': () => fullMessageText === "Your order has been dispatched, and will arrive just as fast as the pony can get there!",
        })
        await this.backHomeButton.click();
    }
}

Running the tests and generating reports

To run the tests with K6 you can just open the terminal and run the following command:

k6 run ./tests/buyItemsUserA.js

We will see a lot of information and different metrics, to help us to understand the results, we can use the dashboard provided by k6. For it, you just need to add on the command that K6 will open a local server with the results of your execution in real time.

K6_WEB_DASHBOARD=true K6_WEB_DASHBOARD_OPEN=true k6 run ./tests/buyItemsUserA.js

If you want only the summary of the execution, you can ask k6 to generate the html report:

k6 run ./tests/buyItemsUserA.js K6_WEB_DASHBOARD=true K6_WEB_DASHBOARD_EXPORT=results/report.html

The metrics

Now that we have the tests, the results, we need to understand the results provided by k6. The metrics provided by K6 are based on Core Web Vitals
The vitals are composed of three important metrics to help a user experience when using your web application:
- Longest content display (LCP): measures loading performance. To offer a good user experience, LCP needs to occur within 2.5 seconds of the page starting to load.
- Interaction to Next Paint (INP): measures interactivity. To offer a good user experience, pages need to have an INP of 200 milliseconds or fewer.
- Cumulative Layout Shift (CLS):measures visual stability. To offer a good user experience, pages need to maintain a CLS of 0.1 or less

These are the metrics that are stable now on the Core Web Vitals page; They help us understand which areas of our frontend application need optimization.

Existing browser measures, such as Load and DOMContentLoaded times, no longer accurately reflect user experience very well. Relying on these load events does not give the correct metric to analyze critical performance bottlenecks that your page might have. Google’s Web Vitals is a better measure of your page performance and its user experience.

For this test, if we only run once, we had the following results:

If we run more, like 5 times:

A quick analyze from the results, is that when we add more interactions, our browser_web_vital_lcp, responsible to measures the time it takes for the largest content element on a page to become visible is less than 1 second, what is acceptable, but how it incresed from 362ms to 799ms is important to investigate the number of users that will pass more than 2 seconds to load the content and from there, improve the software.,
The more important of this reports, is to have them running with a number of interations and virtual users expected for the product that will be tested and from there, plan improvements.

Conclusion

This tutorial guided us through creating our first browser-based performance test using k6 browser. We also explored how to structure the project using the Page Object Model for better maintainability and scalability. Finally, we learned how to generate dashboards to help analyze the performance test results effectively.

You can check this project here.

Write Cypress Commands Once, Use Everywhere: Building a Reusable NPM Package

Alicia Marianne 🇧🇷 — Thu, 03 Jul 2025 19:10:31 +0000

We already know that cypress allow us to create commands that can be used in different tests, but this is on the context of the project, imagine that you have different teams but they have similar workflows, like, authentication. Each team will need to create a command or a test itself related to authentication, but what if I tell you that you can just create it once and use in different projects? In this article we will see how create our own npm package that can be used in any of your cypress projects.

What we are going to learn

For this tutorial we will use tests that I used on this article API Testing with Cypress - Part I . During this tutorial we will:

How to create the common project library
Implement the first common features to this project
Publish our npm common project on NPM
Using this common in our project

Setup of the project

Let's start our project creating a new project:

mkdir cypress-common
cd cypress-common
npm init -y

Installing the dependencies:

npm install cypress typescript --save-dev

Our project will have the following structure:

cypress-common/
├── src/
│   ├── utils/
│   │   └── requestService.ts
│   │   └── responseValidation.ts
│   │   └── constractValidation.ts
│   ├── contracts/
│   │   └── bookContracts.json
│   ├── mocks/
│   │   └── example.ts
│   ├── pages/
│   │   └── login.ts
│   └── index.ts
├── tsconfig.json
├── package.json

Adding the configuration of TypeScript:

{

    "compilerOptions": {
        "target": "es6",
        "lib": ["es6", "dom"],
        "moduleResolution": "node",
        "strict": true,
        "esModuleInterop": true,
        "resolveJsonModule": true,
        "declaration": true,
        "outDir": "./dist",
        "types": ["cypress"]
    },
    "include": ["src/**/*.ts"]
}

To use some methods from cypress we will need to install it in our project:


npm install cypress --save-dev

Creating the features of our common library

requestService.ts

First feature that we will add in our common, is the request feature. For this feature, we need to receive the method, url and the body if needed, for it, we will create a type called BaseRequestOptions that is a custom type alias that defines the shape of an object:

type BaseRequestOptions = {
    method: Cypress.HttpMethod;
    url: string;
    body?: any;
};

Now we can create the class and our method:

type BaseRequestOptions = {
    method: Cypress.HttpMethod;
    url: string;
    body?: any;
};

export class RequestService {
    request({ method, url, body = null }: BaseRequestOptions): Cypress.Chainable<Cypress.Response<any>>{
        return cy.request({
            method,
            url,
            body,
            failOnStatusCode: false,
        });
    }
}

Basically this class provides a wrapper around Cypress’s cy.request() command. When called, it:

Sends an HTTP request using cy.request() with the given method, url, and optional body.
Sets failOnStatusCode: false so Cypress does not fail the test automatically if the response status is an error(like 4xx or 5xx).
Returns a Cypress.Chainable<Cypress.Response<any>>, which means you get back a Cypress chainable object wrapping the HTTP response, so you can continue chaining .then(), .should(), etc., in your test.

contractValidation.ts

The ContractValidator class validates that a given object has all the required properties defined in a contract schema and that each property matches its expected type (string, number, boolean, object, or array).This is a good way to make sure that your APIs are respecting the contract defined.

type SupportedTypes = 'string' | 'number' | 'boolean' | 'object' | 'array';
type ContractSchema = Record<string, SupportedTypes>;
export class ContractValidator {
    private schema: ContractSchema;
    constructor(schema: ContractSchema) {
        this.schema = schema;
    }

    validate(response: Record<string, any>): void {
            for (const key in this.schema) {
                const expectedType = this.schema[key];
                // Check property exists
                expect(response, `Response should have property '${key}'`).to.have.property(key);
                const value = response[key];
                const actualType = Array.isArray(value) ? 'array' : typeof value;
                // Check type matches expected type
                expect(actualType, `Property '${key}' should be of type '${expectedType}', but got '${actualType}'`).to.equal(expectedType);
            }
        }
}

responseValidation.ts

The ResponseValidation class verifies that all properties of a ResponseBody object from a request match exactly with those in a corresponding response, using equality checks for each field

type ResponseBody = {
    id: number,
    title: string,
    description: string,
    pageCount: number,
    excerpt: string,
    publishDate: string
}



export class ResponseValidation{

    responseValidation(requestBody: ResponseBody, responseBody: ResponseBody): void{
        for (const key in requestBody) {
            // Check that the property exists in the response
            expect(responseBody, `Response should have property '${key}'`).to.have.property(key);
            // Check that values match
            expect(responseBody[key as keyof ResponseBody], `Property '${key}' should match`).to.equal(requestBody[key as keyof ResponseBody]);
        }
    }
}

Using and publishing your common project

Now that we have our common ready, we need to build it and start to use in our cypress projects.

Build and run locally

First, go to you package.json and make add to it or update:

{
    "main": "dist/index.js",
    "scripts": {
        "build": "tsc"
    },
}

After that, you can run the command npm run build It runs the TypeScript compiler (tsc) to transpile TypeScript code (.ts / .tsx) into JavaScript (.js).

When it is done, you need to run npm link this way you will be able to add to your project.

Publishing on NPM

First you need to created your account on npmjs. After that, on you terminal run:

npm login

After do the login, make sure that your package.json contains:

{
    "name": "@m4rri4nne/cypress-common",
    "version": "1.0.0",
    "main": "dist/index.js",
    "publishConfig": {
        "access": "public"
    }
}

⚠️ --access public is mandatory on the 1st publication if the package has a scope (like @m4rri4nne/). If you've removed the scope, --access public is optional, but you can leave it in.

After that run:


npm publish --access public

After that you can check you package created on your profile on npmjs.

Using in our project

Now that we have the common package done, you can add to your cypress project running: .


npm install @m4rri4nne/cypress-common

Or if you are running it locally, after make the link, run in your project:


npm link @m4rri4nne/cypress-common

Changing the POST method

Refactoring our existing tests using the common:


import {RequestService, ContractValidator, bookContract, ResponseValidation } from '@m4rri4nne/cypress-common';

// Declaring the depencencies 

const requestService = new RequestService();
const validator = new ContractValidator(bookContract);
const responseValidation = new ResponseValidation();
const baseUrl = "https://fakerestapi.azurewebsites.net/api/v1/Books"

describe('Testing POST Method', ()=>{

    it('Create a book with success', ()=>{
        const body = {
            id: 100,
            title: "string1",
            description: "string1",
            pageCount: 100,     
            excerpt: "string",
            publishDate: "2023-10-14T18:44:34.674Z"
        }
        requestService.request({
            method: 'POST',
            url: baseUrl,
            body: body
        }).then((response) =>{
            expect(response.status).to.equal(200)
            validator.validate(response.body)
            responseValidation.responseValidation(body, response.body)
        })
    })
})

You can check the other refactoring here.

Why create a library instead using cypress commands?

I think the only possible answer to that question is a big Depends.
If we take the execution of tests that have been refactored, we can see a significant reduction in test execution time

But is it worth developing a project from scratch simply for a few milliseconds? That will depend on your project, your time available to maintain it. But there are a few advantages I want to highlight that might make you consider it:

Reusing code between projects: You create customised commands, request services, validators, etc. only once in the common project. Different teams and projects can install via NPM and use it, ensuring consistency.
Strong typing with TypeScript: Avoids passing the wrong parameters to functions and commands due validation at build time. Facilitates maintenance, because TS itself shows where your code breaks if you change something.
Single, consistent standard: All teams follow the same standard for requests, validations, commands, etc. Easy to apply good practices (e.g. failOnStatusCode, response schemas, logs).
Reduction of duplication: No need to recreate the same login commands, data creation, mocks, etc. in each Cypress project. Updates and bug fixes are done in one place.
Better maintenance and evolution: If a flow changes (e.g. the company login), you adjust it in the common project and all the projects that use it already benefit from updating the version.

These were one of the main reasons that encouraged me to try to create this and I hope it motivates you too. I still intend to add more things to this common and if you have any questions or ideas, just get in touch with me.

You can see the common project here

From Theory to Impact: Implementing Shift-Left from Scratch

Alicia Marianne 🇧🇷 — Tue, 24 Jun 2025 20:19:44 +0000

Shift-left, do you know what it is? After help my team implement this methodology from scratch, I've decided to write about what is the methodology, the benefits and practices that were essential for this methodology to work and improve not only the product we were working on, but our daily lives as QA and developers.

Introduction to Shift-left testing

Shift-left testing is an approach in software development that emphasizes moving testing activities earlier in the development process for improved software quality, better test coverage, continuous feedback and a faster time to market.

This practice have a lot of benefits, like:

Cost Savings: Early detection of defects through automated unit tests is cheaper than fixing issues that reach customer environments.
Increased Efficiency: Continuous testing leads to quicker issue resolution and a smoother development cycle.
Early Bug Detection: Finding bugs early in development minimizes costs and effort.
Faster Development: Testing alongside development reduces delays from last-minute fixes.
Developer Wellbeing: Effective automated testing and Continuous Integration (CI) give developers confidence to deploy frequently, reducing stress and making error resolution faster.
Resilient Architecture: Prioritizing testability leads to better software design and easier maintenance.
Greater Overall Quality: Enhances customer experience by reducing incidents and improving software health through continuous feedback.
Better Collaboration: Encourages communication between developers and testers from the start.

But as not everything is perfect, this methodology has some limitations:

Increased Initial Investment: Requires significant upfront costs for tools and training, which may be a barrier for some organizations.
Quality of Automated Tests: Over-reliance on automation can lead to issues if tests are not well-designed, resulting in false positives or negatives.
Limited Scope: Focuses on early detection but may overlook aspects like usability and accessibility, which need different testing methods.
Resource Intensive: Continuous testing can demand significant resources, including skilled personnel and infrastructure, which may not be feasible for all teams.
Code Audits: Code auditing is essential before implementing shift-left processes. Regular code audits during software development help ensure quality; if done poorly, they can hinder the smooth execution of code testing and ultimately impact overall software quality.
Difficulty in Managing Test Data: Creating and maintaining relevant test data can be challenging, impacting test effectiveness.

Context

For security reasons, I won't give details of how the process was before or after this methodology in the company I work for, but I will use the traditional software development model as an example.

If we look at the traditional software development model, we can see that the testing starts in the very end or the the extreme right of the pipeline:

As we can see, at this moment, the QA will only be validating what was developed, in the end of the process. And there's some problems with this practice:

Excessive regressions - How is needed to make sure the stability of the system, each deploy in the test environment, demands one regression, even with automated tests, it will take time.
If there's one critical bug, all the work will be blocked to be deployed => impact directly final users
Difficult debug bugs and issues - How we deployed a complete feature, if a bug appears, debug it will cost more time for the developers, because it could be added in any one of the features of the package.

As you can see, a process without shift-left testing have a lot a problems in a long term for projects that pass through a lot of changes.

Is important to mention, sometimes, have only the test phase on the end of the complete development, can be enough for projects with close scope or with few changes

How start to implement shift-left

The main goal of shift-left testing is have the test phase starts early in the development and there's some actions that can me made to help it. Probably you already know the test pyramid:

The Test Pyramid is a software engineering concept that proposes a balanced distribution of the different types of automated tests in a system. The idea is to maximize test efficiency, minimizing maintenance costs and execution time.

Understanding the test pyramid, in the shift-left we abuse this concept.
How we want earlier results, unit, integration and E2E testing will be a big ally on it.

In addition, I wanted to talk about other points that I think are extremely important when we implement this methodology and they made the difference for my team.

INVEST method

The acronym INVEST helps to remember a widely accepted set of criteria, or checklist, to assess the quality of a user story Basically, it says that one user story should be:

“I” ndependent (of all others)
“N” egotiable (not a specific contract for features)
“V” aluable (or vertical)
“E” stimable (to a good approximation)
“S” mall (so as to fit within an iteration)
“T” estable (in principle, even if there isn’t a test for it yet)

Use this acronym when writing or breaking us, it is essential for shift-left. With it, we can visualize and deliver value easily and quickly, and the main thing in my opinion: reduce retesting.

With INVEST	Without INVEST
As a registered user, I want to be able to reset my password via a link sent to me by e-mail, so that I can regain access to my account if I forget my password.	The system must allow secure password resets.

If we look to the criteria without INVEST, we can see that this statement:

May depend on security, authentication, validation, etc.
Is very vague. ‘secure password resets’ can mean many things.
Don't have enough details to estimate.
May involve multiple components (token, backend, UI).
No clear criteria: how do you know if it's ‘secure’?

Use invest help to understand more about the feature and define the possible tests that will be needed, avoiding redundancies, guesswork during development and frustrated expectations.

CI/CD - Continuos integration and Continuos deploy

CI/CD helps organizations avoid bugs and code failures while maintaining a continuous cycle of software development and updates. As apps grow larger, features of CI/CD can help decrease complexity, increase efficiency, and streamline workflows.

You can read more about CI/CD here.

Clear communication

For me, this is the BIG and most important change to have this methodology working correctly.
This process brings the ones that are responsible to the testing close to the ones responsible to the development, is a good opportunity to discuss improvements, future implementations and prioritization.
In my context, I think this was the most important feature in making this methodology work. Being closer to the developers during development helped me understand more about the product and foresee possible risks depending on the modifications that would be included. As well as helping them to understand what is expected of QAs, how what they do affects the quality of the product as a whole, not just what they are involved in.

Conclusion

As we can see, shift-left is a good practice, but having automated tests alone is not enough to make it work. I confess that during the time I spent with the team implementing this, I realized that the way we write what we're going to do, the infrastructure we have and, most importantly, our communication as a team are essential if we're to deliver things continuously and with quality, without our lives as developers being overburdened by too many bugs or excessive retesting.

That's it for now. If you've made it to the end of this article/report, let me know if you already use or have used this methodology and the challenges you've had with it.

Sources used

Tips to write automated tests with High-quality

Alicia Marianne 🇧🇷 — Fri, 22 Mar 2024 13:27:38 +0000

Automated tests is something very common in our days, but only write automated tests isn't enough if you want to stand out in the job market, during this article, we will see some definitions that can help us write high-quality automated tests.

Summary

Why write high-quality automated tests
Overview of SOLID Principles
Characteristics for high-quality automated tests
- Test Independency
- Test Maintainability and Reusability
- Readability
- API Usability
- Extensibility
- Learning Curve
- KISS
Conclusion

Why write high-quality automated tests

Before study a topic or try to understand any definition, we need first ask ourselves why we should spend time writing high-quality automated tests? Create automated tests is already an task that get a lot of our time and now we need to think in make them with quality?

I like to think that the effort from today it's a gift for my future self.

We know that automated tests are wonderful to help the development team during the project, a lot of problems are avoid due the automations but, what make a lot of automations become a little monster is the maintenance and keep it updated due the changes of the project, based on it, we already have a good reason to think in the quality of our automated tests.

We will see a lot of concepts during this article that can help us improve the quality of our automated tests.

Overview of SOLID Principles

First, let's do a quick overview about SOLID principles. SOLID is a mnemonic acronym for five design principles, the goal of this principles is to help us writing more understandable, flexible and maintainable software:

SRP - Single Responsibility Principle: Every Software module should have only one reason to change
OCP - Open/Closed Principle: Software entities(classes, modules, functions, etc) should be open for extension but closed for modification
LSP - Liskov Substitution Principle: You should be able to use any derived class instead of a parent class and have it behaved in the same manner without modification
ISP - Interface Segregation Principle: Clients should not be forced to implement interfaces they don't use. Instead of one fat interface, many small interfaces are preferred based on groups of methods, each one serving one sub-module
DIP - Dependency Inversion Principle: High-level classes should not depend on low-level classes. Both types should be created based on abstractions. These abstractions shouldn't depend on any low-level details. All details should use the abstractions instead

We won't go more deep on this concepts because this isn't the focus of this article and knowing the definition of each one, we will be able to understand the characteristics that will be mentioned later.

If you want to go deep and understand better the SOLID principles, you can check it in this articles:

Era SOLID o que me faltava - PT BR
The S.O.L.I.D Principles in Pictures

Characteristics for high-quality automated tests

Now that we know the SOLID principles, we can start to look on the characteristics that our automated tests should have to achieve a high-quality, and they are:

Test Independency
Test Maintainability and Reusability
Readability
API Usability
Extensibility
Learning Curve
KISS

Test Independency

The principle states that each test should be completely independent and self-sufficient. Any dependency on other tests or third-party services that cannot be controlled should be avoided at all costs. We get access to many benefits if we integrate the principle in the development of our automated test:

Resilience - developing our tests in a non-dependent way makes them more resilient if some other test or 3rd party services fail to setup/delete the required data or leave the system in a state that is not expected.
Faster test development - speed-up the long-term maintenance and test development since we don’t need to make a long analyzation session wondering what will happen if we change/delete this test.
Random run order - allows us to be able to run the tests in random order and in different subsets.
Parallel testing - connected to the previous point, not only we can run the tests in random order, but we can run them in parallel, which cannot happen in the case where the tests should be run in a particular order, because they depend on each other to set up required data.

Test Maintainability and Reusability

We can say that Maintainability and Reusability are a couple in the world of automated tests. We don't have maintainability without reusability.

Reusability is related to DRY(Don't Repeat Yourself) principle where the most basic idea is to reduce long-term maintenance cost by removing all unnecessary duplication. There's different ways and techniques that can be used to improve the reusability of your tests. The Page objects model and the SOLID principles are examples that can help you to have your tests more reusable(OCP principle, DIP principle) and with this, your maintainability will be improved was well.

When you have a good reusability, where you are able to extend or reuse what you already have created in your automation, the better the maintainability is, will be easier for you and your team support the existent code, accommodate new requirements or just fix some bugs.

Readability

Is related to how easy your test is to read. If you look at a method in your code and you are not able to response the question: "What does this method do?" , then your method most probably has more than one purpose, therefore it doesn't follow the Single Responsibility principle.
So, when creating your methods, variables, think in meaningful names, avoid abbreviations.

API Usability

The API is the specification of what you can do with a software library. API usability means how easy it is for you as a user to find out what the methods do and figure out how to use them. If we are talking about a test library - how much time is needed for a new user to create a new test? It is related to the learning curve. In the programming community, we sometimes use another term for the same thing called syntactic sugar. It describes how easy it is to use or read some expressions. It sweetens the programming languages for humans. The programming statements become more concise and clearer.

Extensibility

"Did you write your tests in a way that if you add this new functionality, you would not affect your existing tests?"

When creating automated tests and thinking in the Usability and Maintainability we can create share libraries that can be used by multiple teams, so we should pay attention to how one change will affect what already exists.

Learning Curve

"How easy is it for someone to learn by himself?"

The learning curve is tightly connected to API usability, but it means something a bit different. If a new member joins your team, is he able to learn by himself how to use your test automation framework or he needs to read the documentation if it exists? Or you have a mentoring program where you need to teach these new members yourself every time how to use your code?

KISS

"Keep It Simple Stupid". The below quote from Martin Fowler best describes the principle.

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”

This principle is the criteria that will define if your automated test have high-quality or not. When you have the previous ones implemented, your code became simple and easy to learn.

Conclusion

This article was an introduction of some characteristics that we should consider to have a high-quality automated tests. The main goal to use this is to have an code that is easy to maintain, read and learn with all the benefits that automation can bring to our daily lives. If you want to go more deep in this, I recommend the book Design Patterns for High-Quality Automated Tests from Anton Angelov, which this article was based on.

I hope this content will be useful for you.

If you have any questions, feel free to reach out to me!

API Testing with Cypress - Part I

Alicia Marianne 🇧🇷 — Sun, 03 Mar 2024 17:29:51 +0000

Did you know cypress? It's a famous framework for test automation that enables the developers, QAs write faster, easier and more reliable tests. In this tutorial, we will learn what is cypress, how we install it in our project and how to automate API testing.

Summary

About cypress
- Quick overview about the test types
Setup of the project
Automating tests
Using commands on cypress
Conclusion

About cypress

Cypress is a front end testing tool for modern web. It enables you write faster, easier and more reliable tests.

With cypress you can automate different types of tests:

E2E
Component
Integration
Unit

The main features of cypress are:

Easy debug
Cross Browser testing
Network traffic control
Screenshots, videos and test replay

Quick overview about test types

Lets do a quick overview of the types of tests:

End-to-End(E2E): Test the entire application or system as a whole, focus on simulate real user scenarios. The main purpose of this type of test is to validate that all the components, integration are working as expected
Integration Tests: Validates the integration between components or systems.
Component Tests: Similar to Integration, but focus on testing larger units of code. Verifies the correct behavior of larger sections of the application.
Unit Tests: Focuses on testing individual components or functions in isolation.

Setup of the project

Starting this tutorial, let's create our project. The requirements to use cypress is have Node installed with version 18.x or 20.xˆ. To see the version of your node, open the terminal and type:

node --version

If you have the node, on the folder of your project, run:

npm init -y

And after that, install the cypress:

npm install cypress --save-dev

Now, we need to open the cypress, to do it, run:

npx cypress open

After opening the launchpad choose e2e testing and accept the configuration. After that, your project will look like:

|--cypress/
|  |-- downloads
|  |-- e2e
|  |-- fixtures
|  |-- support
|      |-- commands.js
|      |-- e2e.js
|-- node_modules
|-- cypress.config.js
|-- package.json

Automating tests

For this tutorial, we will use the same tests cases and API used on my Postman article, you can check here.

Now, lets create our test file. How we are going to automate different methods, we will create one file for each of them. The first will be the GET method, inside of e2e folder, we will create a new file called GETMethod.cy.js:


|-- e2e
|  |-- GETMethod.cy.js

The basic structure of a test of cypress, will be :

describe('Name of the feature', ()=>{
    it('Name of the test itself', ()=>{
        expect(true).to.equal(true)
    })
})

How we are making a GET method, we will change the name of our feature to Testing GET method and create our first test that will be Search all books with success.

To make a request on cypress, we will use the command cy.request() and we need to inform to this command basically this:

The method: GET, POST, PUT (if we don't pass, cypress will use the default method - GET)
The url
The body - if its needed In our test we will inform the method so we can see the difference between the features. You can learn more about this command here

Knowing this, our first test will look like:

describe('Testing GET method', ()=>{
    it('Search all books with success', ()=>{

        cy.request('GET','https://fakerestapi.azurewebsites.net/api/v1/Books').then((response)=>{
            expect(response.status).to.equal(200)
            response.body.forEach(element => {
              expect(element).to.have.property('id')
              expect(element).to.have.property('title')
              expect(element).to.have.property('description')
              expect(element).to.have.property('pageCount')
              expect(element).to.have.property('excerpt')
              expect(element).to.have.property('publishDate')
            });
        })
    })
})

Using the same logic to the other ones, our test Feature will be:

describe('Testing GET method', ()=>{
    it('Search all books with success', ()=>{

        cy.request('GET','https://fakerestapi.azurewebsites.net/api/v1/Books').then((response)=>{
        expect(response.status).to.equal(200)
        response.body.forEach(element => {
          expect(element).to.have.property('id')
          expect(element).to.have.property('title')
          expect(element).to.have.property('description')
          expect(element).to.have.property('pageCount')
          expect(element).to.have.property('excerpt')
          expect(element).to.have.property('publishDate')
        });
        })
    })
     it('Search for a specific books with success', ()=>{
        cy.request('GET','https://fakerestapi.azurewebsites.net/api/v1/Books/10').then((response)=>{
          expect(response.status).to.equal(200)
          expect(response).to.have.property('id')
          expect(response).to.have.property('title')
          expect(response).to.have.property('description')
          expect(response).to.have.property('pageCount')
          expect(response).to.have.property('excerpt')
          expect(response).to.have.property('publishDate')
        })
      })
    it('Search for a specific books with invalid Id', ()=>{
        cy.request('GET','https://fakerestapi.azurewebsites.net/api/v1/Books/10AAA').then((response)=>{
          expect(response.status).to.equal(400)
        })
    }) 
})

Using commands on cypress

Before we start automate the other methods, lets improve our tests. We can make our test easy to read using the commands of cypress. The custom commands on cypress allows you create, and overwrite existing commands. For our tests, we can create two different commands:

One that will be responsible to make all our requests - GET, POST, DELETE, PUT
One to validate the contract of the API

Let's start with the command that will make all our requests. All commands on cypress will be added on commands.js file, and to add a command, we use the following structure:

Cypress.Commands.add('NameOfTheCommand', (variables)=>{
    //Actions of this command 
})

For our test, we will create the command BaseRequest. We know that for this command, we need to have the url, the method and the body request(for POST and PUT methods), and to allow our bad scenarios, we will pass in this command the parameter failOnStatusCode as true, so it will looks like:

Cypress.Commands.add('BaseRequest', (method, baseUrl, bodyRequest='')=>{
    cy.request({
        'method': method,
        'url': baseUrl,
        'body': bodyRequest == ''? null : bodyRequest,
        'failOnStatusCode': false
    })
})

For the method responsible to validate the contract of the API, we will need to receive on it the response body of the request, so it will be:

Cypress.Commands.add('ContractValidation', (response)=>{
    expect(response).to.have.property('id')
    expect(response).to.have.property('title')
    expect(response).to.have.property('description')
    expect(response).to.have.property('pageCount')
    expect(response).to.have.property('excerpt')
    expect(response).to.have.property('publishDate')
})

Now, we can update our tests:

describe('Testing GET method', ()=>{
    const baseUrl = "https://fakerestapi.azurewebsites.net/api/v1/Books"
    it('Search all books with success', ()=>{
        cy.BaseRequest('GET', baseUrl).then((response)=>{
            expect(response.status).to.equal(200)
            response.body.forEach(element => {
                cy.ContractValidation(element)
            });
        } )
    })
    it('Search for a specific books with success', ()=>{
        const url = baseUrl + "/10"
        cy.BaseRequest('GET', url).then((response)=>{
            expect(response.status).to.equal(200)
            expect(response.body.id).to.equal(10)
            cy.ContractValidation(response.body)
        })
    })
    it('Search for a specific books with invalid Id', ()=>{
        const url = baseUrl + "/10AAA"
        cy.BaseRequest('GET', url).then((response)=>{
            expect(response.status).to.equal(400)
        })
    })
})

Using the commands for the other methods:
DELETE:

describe('Testing DELETE Method', ()=>{
    const baseUrl = "https://fakerestapi.azurewebsites.net/api/v1/Books"
    it('Delete a book with success', ()=>{
        cy.BaseRequest('DELETE', baseUrl + "/100").then((response)=>{
            expect(response.status).to.equal(200)
        })
    })

    it('Update a book passing invalid ID', ()=>{
        cy.BaseRequest('DELETE', baseUrl + "/invalid").then((response)=>{
            expect(response.status).to.equal(400)
        })
    })
})

For the PUT and POST Method, we can create another command that will validate the content of the body response and compare with what was sent:

Cypress.Commands.add('ResponseValidation', (requestBody, responseBody)=>{
    expect(responseBody.id).to.equal(requestBody.id)
    expect(responseBody.title).to.equal(requestBody.title)
    expect(responseBody.description).to.equal(requestBody.description)
    expect(responseBody.pageCount).to.equal(requestBody.pageCount)
    expect(responseBody.excerpt).to.equal(requestBody.excerpt)
    expect(responseBody.publishDate).to.equal(requestBody.publishDate)
})

And our tests will be:

describe('Testing PUT Method', ()=>{
    const baseUrl = "https://fakerestapi.azurewebsites.net/api/v1/Books"
    const body = {
        id: 100,
        title: "string1 updated",
        description: "string1 updated",
        pageCount: 100,
        excerpt: "string",
        publishDate: "2023-10-14T18:44:34.674Z"
    }
    it('Update an book with success', ()=>{
        cy.BaseRequest('PUT', baseUrl + "/100", body).then((response)=>{
            expect(response.status).to.equal(200)
            cy.ContractValidation(response.body)
            cy.ResponseValidation(body, response.body)
            })
    })
    it('Update a book passing invalid ID', ()=>{
        cy.BaseRequest('PUT', baseUrl + "/invalid", body).then((response)=>{
            expect(response.status).to.equal(400)
        })
    })
    it('Update a book with empty body request', ()=>{
        const body = {}
        cy.BaseRequest('PUT', baseUrl + "/100", body).then((response)=>{
            expect(response.status).to.equal(200)
            cy.ContractValidation(response.body)
        })
    })
})

describe('Testing POST Method', ()=>{
    const baseUrl = "https://fakerestapi.azurewebsites.net/api/v1/Books"
    it('Create a book with success', ()=>{
        const body = {
            id: 100,
            title: "string1",
            description: "string1",
            pageCount: 100,
            excerpt: "string",
            publishDate: "2023-10-14T18:44:34.674Z"
        }
        cy.BaseRequest('POST', baseUrl, body).then((response)=>{
            expect(response.status).to.equal(200)
            cy.ContractValidation(response.body)
            cy.ResponseValidation(body, response.body)
        })
    })

    it('Create a book without body request', ()=>{
        cy.BaseRequest('POST', baseUrl).then((response)=>{
            expect(response.status).to.equal(415)
        })
    })

    it('Create a book with empty body request', ()=>{
        const body = {}     
        cy.BaseRequest('POST', baseUrl, body).then((response)=>{
            expect(response.status).to.equal(200)           
            cy.ContractValidaton(response.body)     
        })
    })
})

Conclusion

During this tutorial we learn what is the cypress framework and how we can use it to automate our API Testing. Is important to notice that is only one alternative of how you can automate your API testing using commands, but as we saw, use this practice help us to make our test more simple, easy to maintain and readably.

You can access the project used in this tutorial here.

I hope this content will be useful for you.

If you have any questions, feel free to reach out to me!