DEV Community: Wioletta Mikłasewicz

How can chat GPT help in the QA world?

Wioletta Mikłasewicz — Wed, 08 Feb 2023 20:36:38 +0000

Recently, We have heard a lot about chat GPT and its usage in the IT world, I was impressed with the possibilities given with that tool. Of course, we need to be aware that everything that AI produces needs to be verified by a human, especially in the beginning of this beautiful journey, but why not use it if we have this possibility? In this article my main concerns are focused on what we can achieve in the QA world by using AI, so let’s check some options:

Automated tests generation – this one sounds promising, we could have tests written by AI, so I have asked chat GPT to generate simple cypress test for Google search functionality with usage Page Object Pattern. As an output we have:

Query: Write cypress test for google search functionality using page object pattern

// google-search.page.js
const searchInput = 'input[name="q"]';
const searchButton = 'input[type="submit"]';

class GoogleSearchPage {
  visit() {
    cy.visit('https://www.google.com/');
  }

  search(searchTerm) {
    cy.get(searchInput).type(searchTerm);
    cy.get(searchButton).click();
  }
}

export default new GoogleSearchPage();

// google-search.spec.js
import GoogleSearchPage from './google-search.page';

describe('Google Search', () => {
  it('should display search results for a query', () => {
    const searchTerm = 'Cypress';
    GoogleSearchPage.visit();
    GoogleSearchPage.search(searchTerm);
    cy.contains(searchTerm).should('be.visible');
  });
});

It looks quite good, we have selectors on separated page, also helpers are extracted, so in test we can use them without code duplication. To improve it we could use before or beforeEach clause to extract repeatable test parts e.g. GoogleSearchPage.visit();

describe('Google Search', () => {
  before(() => {
    GoogleSearchPage.visit();
  })
  it('should display search results for a query', () => {
    const searchTerm = 'Cypress';
    GoogleSearchPage.search(searchTerm);
    cy.contains(searchTerm).should('be.visible');
  });
});

I would say chat GPT can generate basic tests or templates, which is great as in that way we can speedup work, but still on the human side is to keep an eye on the good practices, follow the newest trends, and make sure that solution proposed by AI is the best for the current project. Also, in case of more advanced tests, there might be issues with providing the right data to generate proper test.

Test suggestions - let say I would like to have some test suggestions from chat GPT on how to test Google search functionality, as an output we have:

Query: Suggest me how I can test search functionality on google page

To test the search functionality on Google, you can follow these steps:

1. Open a web browser and go to the Google homepage.
2. Verify that the search box is present and accessible.
3. Enter a query in the search box and hit the search button.
4. Check that the search results are relevant to the query entered.
5. Verify that the search results are displayed in the correct format and order.
6. Test for correct spelling suggestions and "did you mean" functionality.
7. Try searching for various combinations of keywords, phrases, and special characters.
8. Test the search functionality with different types of queries such as short queries, long queries, and queries with special characters.
9. Check the search results for accuracy and completeness.
10. Test the search functionality with edge cases such as search queries that contain only stop words.
11. Verify that the search functionality returns appropriate results for empty or invalid queries.
12. Check the search results pagination and make sure it works correctly.

Note that Google is a highly sophisticated search engine and has many advanced features such as auto-complete, personalization, and more. Consider testing these advanced features as well, depending on the scope of your testing.

We can notice that as an output we have received a set of functional tests proposals, which is good. That can be used as a start for test cases creation or inspiration for exploratory testing, but there's no mention about non-functional tests e.g. response time, how this will look like on different web browsers etc., Let's then try to precise our query to chat GPT.

Query: Suggest me non-functional tests for google search functionality

Non-functional tests for the Google search functionality may include:
1. Performance testing: This includes testing the response time for search queries and verifying that the search results are returned quickly.
2. Scalability testing: This involves testing the search functionality under heavy load conditions to verify that it can handle an increased number of concurrent users.
3. Security testing: This includes testing the search functionality for vulnerabilities such as SQL injection, cross-site scripting, and other security threats.
4. Usability testing: This involves testing the search functionality from the user's perspective to ensure that it is user-friendly and easy to use.
5. Compatibility testing: This involves testing the search functionality on different browsers and devices to verify that it works correctly on all platforms.
6. Accessibility testing: This involves testing the search functionality for accessibility for users with disabilities, such as users with visual, auditory, or motor impairments.
7. Internationalization testing: This involves testing the search functionality for compatibility with different languages and cultures.

These are general examples of non-functional tests that can be performed on the Google search functionality. The exact tests to be performed will depend on the specific requirements and constraints of the testing project.

The output looks nice, we have a list of non-functional tests for Google search functionality, noticeable thing is that chat GPT listed types of non-functional testing and assigned a potential test for it, so the answer is a bit general, but still, I think it's a quite good base for extending the tests to be more granular.

Test cases generation - writing test documentation and maintenance can be really time consuming, especially, when product is continuously changing. Having that in mind would be awesome if chat GPT could write some test cases for us, so let's see:

Query: Write test case to visit google site

Here is a sample test case to visit the Google website:

Test Case: Visit Google Website

Pre-condition:

    The Internet connection is stable
    A web browser is installed

Steps:

1. Open a web browser
2. In the address bar, type "www.google.com" and press enter
3. Wait for the website to load
4. Verify that the website title is "Google"
5. Verify that the website URL is "https://www.google.com/"

Expected Result:

 - The Google website should load successfully
 - The website title should be "Google"
 - The website URL should be "https://www.google.com/"

Generated test case looks good, we can see it has pre-condtion, steps, and expected results, so all required parts. Anyway, I think the main limitation (for now) here is that chat GPT can access google page, so creating these steps doesn't require any additional rights. If we would like to use it in our system, we would need to provide full access to it, but still, it quite awesome!
Testing is not only about checking the positive cases, let's see now how chat GPT will interpret it for us:

Query: Write negative test case to visit google page

Here is a sample negative test case to visit the Google website:

Test Case: Visit Google Website with Invalid URL

Pre-condition:

- The Internet connection is stable
- A web browser is installed

Steps:

1. Open a web browser
2. In the address bar, type "google" (without the "www." and ".com") and press enter
3. Wait for the website to load

Expected Result:

- An error page should be displayed with a message such as "This site can’t be reached" or "Page Not Found"
- The website title should not be "Google"
- The website URL should not be "https://www.google.com/"

And again, it looks good!

To summarize, chat GPT is a great tool, we can use it to speedup work or even get more inspiration in our QA world. There's a one important thing we need to remember when using it, you need to know how to ask right questions and be able to verify the output. From my point of view it's a more advanced browser, which can give you what you want - "if you need to ask, you will never know, if you know you just need to ask".

A few words about exploratory testing

Wioletta Mikłasewicz — Tue, 17 Jan 2023 12:51:12 +0000

Exploratory testing is a type of software testing that is focused on exploring the system and discovering new defects and issues. It is an interactive and dynamic testing technique that allows testers to use their creativity and curiosity to find defects that might have been missed by other types of testing.

That technique is often used when there is little or no documentation or requirements for the system being tested. It is also used when the system is complex or the testing goals are not clearly defined.

During exploratory testing, testers will typically use a combination of testing techniques, including manual testing, automated testing, and exploratory testing tools. They will also use their knowledge and experience to identify potential issues and test the system in ways that were not previously planned or specified.

One of the key benefits of exploratory testing is that it allows testers to uncover defects early in the development process, which can save time and money in the long run. It also allows testers to identify areas of the system that need further testing and to develop new test cases based on the defects that are found.

We still need to remember, that exploratory testing is a powerful technique for discovering defects and issues in software, however, it may not be sufficient in certain situations.

One of the main limitations of exploratory testing is that it is highly dependent on the skills and experience of the tester.
It relies on the tester's ability to identify potential issues and test the system in ways that were not previously planned or specified. In situations where the tester lacks the necessary expertise or experience, exploratory testing may not be effective.

Another limitation of exploratory testing is that it may not be as thorough as other types of testing. It is focused on discovering new defects and issues, but it may not cover all possible scenarios or test cases. For example, if a software is built to be compliant with certain regulations, exploratory testing may not cover all the regulations or test cases required to assure compliance.

Additionally, in situations where there are strict deadlines or the system is mission-critical, exploratory testing may not be feasible due to the time and resources required. In these cases, other types of testing such as regression testing or automated testing may be more appropriate.

Overall, while exploratory testing is a valuable technique for discovering defects and issues, it should be used in conjunction with other types of testing to ensure that the software is thoroughly tested and meets the required quality standards.

Good QA wanted – why in these times is really difficult to find a skilled QA?

Wioletta Mikłasewicz — Mon, 02 Jan 2023 17:12:25 +0000

In this article, I will try to find a reason why these times it’s really hard to find a skilled QA on the market.

While I was doing multiple interviews for QA/Test Automation Engineers I noticed that a significant part of the candidates was not able to explain their skills from CV based on their experience. I would even say that over-skilled CVs are some kind of problem for hiring managers as the expectation from a CV is not met with reality, which definitely increases the hiring costs and wasting the time on both sides.
If you want to have a good QA in your team, look at the single achievements in the past/current projects, and ask the technical/soft questions about how this was done, and why – don’t focus on the title and experience years. Look for creative problem solvers with good technical and communication skills who are increasing their knowledge constantly, not just for QA on paper.

Today the IT industry is experiencing a total boom – everyone wants to be in, unfortunately, a lot of people are treating a testing job as a good start, even when not feeling that this is the right way, they usually think – you don't need to know a programming language, just click around the application and find some bugs, hah easy thing – no, it’s NOT. That kind of thinking is producing a lot of QAs on the market that are doing their job mechanically, this is reflecting on the project quality, team cooperation, and overall thinking about QA as a part of a development team. Very often I came across the opinion that testers are redundant in the team, do not bring a specific value to the project, or developers can cover some testing aspects, etc., today I know this opinion is unfortunately based on the bad quality testers and I agree with that.

My other observation is a lack of good communication from the QA side, we should think about that role as a bridge between development and project, that requires exceptional communication skills and an understanding of the processes and product, instead of this, really often we can notice a judging QA who’s trying to find a guilty or don’t see the issues in the QA processes. In addition, communication skills are a must when speaking about bugs to the developers and product managers, that’s a mix between tech and soft language – not easy, but also not impossible.

During my whole career, I met the QAs who have been introduced to me as super seniors but when I started to work with them I was disappointed, there was a lack of technical skills, especially in the test automation area, and the quality of work was unacceptable in some aspects. How this could happen?
In my opinion, the process of interviewing QAs is not focused on real skills but on asking the same questions, expecting to solve unreal cases, and not looking at the ability for problem-solving. It’s really hard to verify all these during an interview, but all can be noticeable during daily work. If you notice that your QA is not performing as you expected, don’t be afraid to talk about improvements. I would go even further, don't be afraid to fire an employee if you see they don't meet your expectations. Keeping a bad-performing QA in the team without any signs of progress may lead your team rockstars will find a better place.

And the last thing, but in my opinion the most critical – money saving on good people. How often do you hear that we need someone for the project, ideally senior level, but in reality, we are looking for junior/mid as in that way we can save some money? Unfortunately, this is false thinking, finally, this ends with team frustration, and opening a recruitment process from the beginning, which equals more money spent. Setting up the project expectations and needed skills is crucial for hiring good people and having a dream team.

How do not automate tests?

Wioletta Mikłasewicz — Sat, 17 Sep 2022 10:15:17 +0000

It’s not easy to create excellent automated tests, I would say it’s even more difficult when you join the company and see that a bunch of stuff needs to be rebuilt, to create at least a good base to write tests. In this article, I would like to describe my observations based on several projects I had an opportunity to work with, and I will try to find the holy grail of rules for creating automated tests.

Wrong automation framework
The decision on the automation framework is the most crucial as it will affect the teamwork on automated tests for a long-term period. Ok, based on what usually the automation frameworks are chosen?
There are two main rules I can observe:
– The one I know, for example, from my last job
– The one which is the most famous at the moment
I am not saying this is bad, but, not enough. The automation framework should be considered at the first point for project needs, there’s no point in pickup the framework with the issues for document upload if in our project it’s base functionality. Also, we should ensure that the framework will support the programming language used in a project. Why? Because, QAs are not the only ones in a team, and quality is a team effort, so we need to make sure the dev team will be able to handle potential changes in the tests repository. If that option is not possible, just make sure, that chosen automation framework has good documentation and community, basically, that’s also one of the crucial points to consider.

Incorrect tests hierarchy
The most often test hierarchy we can observe in companies is Testing Ice Cream Cone, so we have very few unit tests, some UI tests, and lots of Manual Tests.

The Testing Ice Cream Cone approach can be easily identified where the QAs have created an automated test suite, but the development team has not, or there are very limited. As a result, tests will be very long-running and flakey because the development team has not helped build the suite or architect the application in a way that makes it easy to test. The new check-ins by the developers break the product functionality regularly and they are relying on the QAs to report the defects to fix them.
The above scenario can be fixed with reversed test pyramid, where the development team is the base for automation.

In that way, we have complete coverage at a low-level point, and the tests are fast, so developers can receive quickly feedback about their changes. Unit tests also prevent adding not checked changes to the app, so constant breaking changes can be avoided.
The E2E tests (UI), shouldn’t cover a significant part of code, as there are time-consuming and might require a huge effort in maintenance. Rather than on UI tests, the QA team should focus on AP/Components tests to find potential issues at an early stage, and in specific places in the code.

Lack of automation scope (strategy)
As I mentioned at the beginning of this article writing good automated tests is not easy, but there’s one more thing, which is even more difficult – defining what should be automated. At the first glance, that seems to be easy, but from my experience, it’s not. Well-designed tests shouldn’t be created without a definition of what should be automated as a priority, what are our core functionalities, what can wait, which tests require a lot of effort etc.
Usually, automation is done ad hoc, that not cover all crucial functionalities, and it’s really hard to track test coverage. A good approach, in that case, is to at least create a high-level document about the automation scope, ideally share it with the team, so everyone is aware of the automation scope, and eventually propose some changes.

Ignoring automated tests results
Cast the first stone who checked all test reports from the automated tests pipeline. Not checking test reports is unfortunately common practice, the tests are executed, but the results are not verified by QAs, or this is done from time to time. So why are we writing automated tests and then ignoring their results? If the tests are flaky and fail from time to time, they should be fixed and treated seriously with each run. If not why create them?

Maintenance hell
Automated tests sometimes can give you a headache, especially when it comes to maintenance. Anyway, there are a few ways, that could help you and your team with maintaining tests.
Firstly, make sure that UI tests are using predefined selectors, ideally, if that is agreed upon between developers and QAs, thanks to that, you will avoid test failures where the selector has been changed without the knowledge of the QA team.
Secondly, split reusable code functions, organize them, and make them adjustable, so you will avoid multiple places in a code, which handle the same thing, but in a slightly different way. That might be a set of helpers in your tests repository or a separate code repository, that is shared between multiple repositories.
Thirdly, keep UI selectors in one place, so you will avoid eventual updates in multiple places, I not saying everyone needs to follow the Page Object Pattern, but make sure your code is well organized, especially the one which can change often.
Fourthly, make sure your tests will be understandable for everyone, even after a few years, and do not forget about documenting and commenting code. Remember that automated tests might require the whole team to contribute, make sure this journey will be easy and smooth, not hell.

As we can see creating good automated tests is not an easy thing, but it’s still possible, just remember, each project is different and might require various approaches, do not follow blindly the things you know from previous jobs, but be open-minded and flexible. Don’t be afraid of changes, and write code in that way, so everyone will be able to continue your masterpiece.

How to not get bored as QA

Wioletta Mikłasewicz — Thu, 07 Jul 2022 20:40:26 +0000

Everyone has a career point when they feel burnt out, or their current duties become boring. Personally, I think there is nothing worse than a bored QA at work, the reason is very simple, QA should be inquisitive by nature, it helps to find problems but more importantly, prevents them.
So, how not get bored as QA at work and take care of career development?

If you are a QA that is not programming and doesn’t know basic DevOps things, that is an excellent point to start your development. Manual testers/QAs are needed but the future at some point will require more automation engineers with the QA mindset, and the ability to set up new test environments as well as pipelines. Understanding the programming will open you to new ways of testing as you will start to understand the code written by developers, and you will be able to point to the root cause of the issue.

Bring a new quality to your bug reports, everyone knows what the bug description should look like, but how many QAs are trying to put there more than the template? When you found a bug try to do the previously mentioned root cause analysis and include these details in the report. In that way, you will get extra knowledge about the app you are testing, and for sure developers will love you more.

Make test reports great. Usually, when I see a typical test report, it is so monotonous (unless you will see a prod bug there). Personally, I don’t like when QAs treat the test reports as a document, which is only output from the tests, remember, we are doing it for a purpose. Do you know there are great libraries for the graphical presentation of the data? Why not use them? Add more colors, play with the data, and make sure your team loves test reports.

Did you know that we have so many project patterns which can be used for test automation setup, so you don’t have to stick with one or two for your all projects? We tend to use things we know, which also applies when setting up automation, as a result of that we don’t choose the best option for the project, continuously using the same patterns etc. Next time before you will be designing an automation framework explore what’s in the QA world, maybe in your case a better option will consider two approaches and create something custom, or just pick up a totally new design pattern. Just explore.

Take ownership. Don’t stick with the same things over and over, be proactive and try to put yourself into new initiatives etc., As QAs we have a great opportunity to try something new really often, we are a bridge between product and development teams, which gives a thousand of new challenges.

Analyze the QA process continuously. Let’s say you have a setup of the QA process in your team/company. That’s great, but after some time you start to notice that number of bugs is increasing and work is not going smoothly as previous. What happened then? Maybe some processes have been updated in the meantime, but you didn’t consider how this will impact the current QA process? Maybe the product you are working on just reached the next level? There are so many things that can affect the QA process, trust me, you will never get bored once you realize it. QA process is like a child, it’s changing, needs a lot of attention, your time, and drawing conclusions.

Take a step back, and get rest. Being a QA is not easy, there’s no option that you will be a superhero/bug buster all the time. Even the best need to take a rest, and then read that article again.

As you can see, there are so many ways you can bring yourself to the next QA level and never get bored, just take a chance.

5 Reasons why you don't need test cases

Wioletta Mikłasewicz — Sun, 01 May 2022 20:17:59 +0000

Before I start to convince you that test cases are not needed let’s see what they are, so, test cases are a descriptive form of the tests we want to perform, they usually contain steps, expected output, some preconditions and other information about the environment etc., In theory, this sounds informative and as a thing that can improve testing, since all tests are defined and written down.

I have been working on various projects and a significant part of them had test cases, today I remember it as a nightmare. Just imagine this situation, you have a feature to test, based on that we need to create test cases, it requires time. After that, we need to test a feature based on the test cases we created or write some automated tests. Sounds like a fairly long process, isn't it? You may want to say, ok, so why don’t you create test cases before the feature is ready? Good point, I was trying to do that, but usually, acceptance criteria are not ideal, designs are not attached, so created test cases will touch only a high level of the functionality, there is also a high risk that something will change and you will spend extra time to correct all test cases. Let’s now consider how our life would look without test cases.

Reason 1 – Documentation
When we are creating test cases the main reason is to have test documentation, so something which will allow us to go there, get knowledge about application, and speed up with testing. Now imagine, that project documentation contains all information, and the acceptance criteria are written well. That would give you all the required information about the application and the whole team could benefit from that. I know, that might sound a bit idyllic, but as QA we should keep an eye on all aspects of developed software, If we take care of documentation, why do we need to duplicate it?

Reason 2 – Maintenance
Let’s imagine that you have an application and with each iteration, it’s growing, according to that, you are adding new test cases, more, more, and more. Then an application has been updated so you need to update all required test cases. Over time the number of test cases increased drastically and you need to hire another QA to help in the maintenance. Sounds familiar?

Reason 3 – Automate and save time in the future
We all know that test automation is time-consuming, but the benefits from it are significant. Well designed automated tests will bring more value to the test process rather than manual test cases.

Reason 4 – Use Gherkin language properly
The main point of the Behaviour Driven Development approach is to create a communication layer between the product and dev teams, so they can communicate in an effective way. Not all product team members will understand code created by devs, also devs need some structured information about features they need to create. Unfortunately, I didn’t see in my whole career a project where this approach would be used properly. Usually, it’s just some additional lines in a set of automated tests, not used by anyone or just by the QA team. Now, what would happen If you use the BDD approach as it should be? Test cases won’t be needed, you will get all with Given, When, Then nomenclature.

Reason 5 – Don’t do something just because someone told you how it needs to be done
When I was starting my career I went through a number of courses where each contained a chapter about test cases, when you think about testing, I am sure you also think of test cases. Why? Because someone in a past told everyone how the test process should look? But, we know that each project is different, so why we should implement the same pattern everywhere? Just be creative, see what’s the better solution for project needs, what’s the resources, do you have time to write test cases every time and maintain them?

To summarize, depending on the project the approach should be different, we should always consider if we have time to create and maintain a set of the test cases, but in my opinion, well-designed project documentation and/or automated tests will cover the same thing as test cases and save your time.