Dynamic Tests in Cypress: To Loop or Not To Loop

Use Case for Balancing Code Duplication and Complexity for Effective Test Automation

(Cover image from pexels.com by Pixabay)

---

ACT 1: EXPOSITION

It's interesting how divided the automation world is about the use of loops when implementing tests. Some prefer to put the direct assertions in the test, even if it means repeating the same code again and again with just changes to the asserted data. They believe this approach is clearer. Others are definitely against duplicating code and are firm proponents of the DRY (Don't Repeat Yourself) principle to the fullest extent.

However, I believe that any radical perspective in either approach will never be optimally effective, as each has its justified use cases. In my opinion, for code clarity and easy maintenance of your test suite, overusing the duplication of almost identical assertions can be as counterproductive as having complicated loops that require PhD in Theoretic Computer Science to understand. 🤯

But wouldn’t it be amazing if we could find a way of creating Dynamic Tests in Cypress that would be super easy not just to implement but also to maintain, or even scale effortlessly if needed?

---

ACT 2: CONFRONTATION

There is one particular case (although not the only one) that I believe would benefit from the approach of Dynamic Tests: testing APIs with multiple possible combinations of inputs that have to be validated.

Our Case:

I have an API where you provide in the request body up to 4 parameters: account, ipAddress, phone, and email.

Depending on certain combinations of valid or invalid values of those parameters, the request will be approved or rejected. The logic implemented in the backend that we need to test is: if the account is valid or the ipAddress is valid or both the phone and email are valid, then the request is approved; otherwise, it is rejected.

So we will need to test various case scenarios (Input -> Output):

• We provide a valid account -> Approved
• We provide an invalid account -> Rejected
• We provide a valid ipAddress -> Approved
• We provide an invalid ipAddress -> Rejected
• We provide a valid phone and a valid email -> Approved
• We provide an invalid phone and a valid email -> Rejected
• We provide a valid phone and an invalid email -> Rejected
• We provide an invalid phone and an invalid email -> Rejected

Easy, right?

But you really do not know the logic that was implemented in the backend (especially in black box testing).

What should happen if we provide in the same request a valid account and an invalid ipAddress? According to our specification above, it should be approved.

But how can you ensure this is how it was implemented if you do not test it? Maybe, just maybe, the backend developer made an error, and in the event that both parameters are provided in the body (account and ipAddress), both have to be valid in order to approve the request (in other words, they implemented a logic AND instead of an OR).

So we will also need to test:

• We provide a valid account and a valid ipAddress -> Approved
• We provide a valid account and an invalid ipAddress -> Approved
• We provide an invalid account and a valid ipAddress -> Approved
• We provide an invalid account and an invalid ipAddress -> Rejected

... and so on. You get it, right?

Let's now write the code using plain assertions without using any kind of loop operation. The code would look something like this:

File: /e2e/apiValidation.cy.js

const method = 'POST'
const url = '/api/endpoint'

describe('API Tests without Loops', () => {

  it('should approve with a valid account', () => {
    cy.request({
      method,
      url,
      body: { account: 'valid-account#' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'approved');
    });
  });

  it('should reject with an invalid account', () => {
    cy.request({
      method,
      url,
      body: { account: 'invalid-account#' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'rejected');
    });
  });

  it('should approve with a valid ipAddress', () => {
    cy.request({
      method,
      url,
      body: { ipAddress: 'VA.LID.IP' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'approved');
    });
  });

  it('should reject with an invalid ipAddress', () => {
    cy.request({
      method,
      url,
      body: { ipAddress: 'IN.VA.LID.IP' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'rejected');
    });
  });

  it('should approve with a valid phone and a valid email', () => {
    cy.request({
      method,
      url,
      body: { phone: '(555) Valid-Phone', email: '[e-mail]' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'approved');
    });
  });

  it('should reject with an invalid phone and a valid email', () => {
    cy.request({
      method,
      url,
      body: { phone: '(555) Invalid-Phone', email: '[e-mail]' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'rejected');
    });
  });

  it('should reject with a valid phone and an invalid email', () => {
    cy.request({
      method,
      url,
      body: { phone: '(555) Valid-Phone', email: '[e-mail]' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'rejected');
    });
  });

  it('should reject with an invalid phone and an invalid email', () => {
    cy.request({
      method,
      url,
      body: { phone: '(555) Invalid-Phone', email: '[e-mail]' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'rejected');
    });
  });

  it('should approve with a valid account and a valid ipAddress', () => {
    cy.request({
      method,
      url,
      body: { account: 'valid-account#', ipAddress: 'VA.LID.IP' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'approved');
    });
  });

  it('should approve with a valid account and an invalid ipAddress', () => {
    cy.request({
      method,
      url,
      body: { account: 'valid-account#', ipAddress: 'IN.VA.LID.IP' }
    }).then((response) => {
      expect(response.status).to.eq(200);
      expect(response.body).to.have.property('status', 'approved');
    });
  });

  // And many more tests according our test plan
  // [...]

});

Now think about maintaining this suite, especially if, for whatever business reasons, "the product" decides to change the validation logic so that when a valid account is provided, a valid ipAddress also needs to be provided to approve the request.

This could become quite a mess. 🧻

What if... we create a data structure that includes the mapped combinations for both approved and rejected requests, along with the names of the use cases they represent? We could also put that data structure in a Cypress fixture. Why not?

I know... but just bear with me for now.

File: /fixtures/apiValidationUseCases.json

[
  {
    "nameUseCase": "Valid account",
    "requestBody": {"account": "valid-account#"},
    "responseBody": {"status": "approved"}
  },
  {
    "nameUseCase": "Invalid account",
    "requestBody": {"account": "invalid-account#"},
    "responseBody": {"status": "rejected"}
  },
  {
    "nameUseCase": "Valid ipAddress",
    "requestBody": {"ipAddress": "VA.LID.IP"},
    "responseBody": {"status": "approved"}
  },
  {
    "nameUseCase": "Invalid ipAddress",
    "requestBody": {"ipAddress": "IN.VA.LID.IP"},
    "responseBody": {"status": "rejected"}
  },
  {
    "nameUseCase": "Valid phone and valid email",
    "requestBody": {"phone": "(555) Valid-Phone", "email": "[e-mail]"},
    "responseBody": {"status": "approved"}
  },
  {
    "nameUseCase": "Invalid phone and valid email",
    "requestBody": {"phone": "(555) Invalid-Phone", "email": "[e-mail]"},
    "responseBody": {"status": "rejected"}
  },
  {
    "nameUseCase": "Valid phone and invalid email",
    "requestBody": {"phone": "(555) Valid-Phone", "email": "[e-mail]"},
    "responseBody": {"status": "rejected"}
  },
  {
    "nameUseCase": "Invalid phone and invalid email",
    "requestBody": {"phone": "(555) Invalid-Phone", "email": "[e-mail]"},
    "responseBody": {"status": "rejected"}
  },
  {
    "nameUseCase": "Valid account and valid ipAddress",
    "requestBody": {"account": "valid-account#", "ipAddress": "VA.LID.IP"},
    "responseBody": {"status": "approved"}
  },
  {
    "nameUseCase": "Valid account and invalid ipAddress",
    "requestBody": {"account": "valid-account#", "ipAddress": "IN.VA.LID.IP"},
    "responseBody": {"status": "approved"}
  },

  // And the rest of all our use cases
  // [...]
]

If you need to add more use cases to test in the future or even change the logic of the conditions for approval or rejection, it would be fairly simple to do in this JSON data structure, as the human brain is able to process mapped data in a list or tabular form quite easily.

Now we only need to write our test suite in the following manner:

File: /e2e/apiValidationDynamic.cy.js

import useCases from '../fixtures/apiValidationUseCases.json';

const method = 'POST'
const url = '/api/endpoint'

describe('API Tests with Dynamic Fixtures', () => {
  for (let { nameUseCase, requestBody, responseBody } of useCases) {

    it(`Use Case: ${nameUseCase}`, () => {
      cy.request({
        method,
        url,
        body: requestBody
      }).then(response => {
        expect(response.status).to.eq(200);
        expect(response.body).to.deep.equal(responseBody)
      });
    });
  }
});

Notice that the .it() function is within the loop, which is key!

This way, if one use case fails, the test suite will continue running tests for the rest of the use cases instead of aborting at the first test that fails.

Maybe, just maybe, you have just started looking at Dynamic Tests and loops with kinder eyes? 😍

---

ACT3: RESOLUTION

At this point, the choice of whether to embrace loops or not is in the capable hands of our diligent QA engineers.

While loops can turn a mess into a neatly organized suite of tests, every scenario is unique, and sometimes sticking with plain assertions might feel just right. After all, finding the perfect balance between simplicity and efficiency is an art.

So, whether you decide to loop or not to loop, remember that the ultimate goal is to ensure robust, maintainable tests. And hey, a little bit of code repetition never hurt anyone... except maybe the next person maintaining your code! 😉

Don't forget to follow me, leave a comment, or give a thumbs up if you found this post useful or insightful.

Happy reading!