How can chat GPT help in the QA world?

Recently, We have heard a lot about chat GPT and its usage in the IT world, I was impressed with the possibilities given with that tool. Of course, we need to be aware that everything that AI produces needs to be verified by a human, especially in the beginning of this beautiful journey, but why not use it if we have this possibility? In this article my main concerns are focused on what we can achieve in the QA world by using AI, so let’s check some options:

Automated tests generation – this one sounds promising, we could have tests written by AI, so I have asked chat GPT to generate simple cypress test for Google search functionality with usage Page Object Pattern. As an output we have:

Query: Write cypress test for google search functionality using page object pattern

// google-search.page.js
const searchInput = 'input[name="q"]';
const searchButton = 'input[type="submit"]';

class GoogleSearchPage {
  visit() {
    cy.visit('https://www.google.com/');
  }

  search(searchTerm) {
    cy.get(searchInput).type(searchTerm);
    cy.get(searchButton).click();
  }
}

export default new GoogleSearchPage();

// google-search.spec.js
import GoogleSearchPage from './google-search.page';

describe('Google Search', () => {
  it('should display search results for a query', () => {
    const searchTerm = 'Cypress';
    GoogleSearchPage.visit();
    GoogleSearchPage.search(searchTerm);
    cy.contains(searchTerm).should('be.visible');
  });
});

It looks quite good, we have selectors on separated page, also helpers are extracted, so in test we can use them without code duplication. To improve it we could use before or beforeEach clause to extract repeatable test parts e.g. GoogleSearchPage.visit();

describe('Google Search', () => {
  before(() => {
    GoogleSearchPage.visit();
  })
  it('should display search results for a query', () => {
    const searchTerm = 'Cypress';
    GoogleSearchPage.search(searchTerm);
    cy.contains(searchTerm).should('be.visible');
  });
});

I would say chat GPT can generate basic tests or templates, which is great as in that way we can speedup work, but still on the human side is to keep an eye on the good practices, follow the newest trends, and make sure that solution proposed by AI is the best for the current project. Also, in case of more advanced tests, there might be issues with providing the right data to generate proper test.

Test suggestions - let say I would like to have some test suggestions from chat GPT on how to test Google search functionality, as an output we have:

Query: Suggest me how I can test search functionality on google page

To test the search functionality on Google, you can follow these steps:

1. Open a web browser and go to the Google homepage.
2. Verify that the search box is present and accessible.
3. Enter a query in the search box and hit the search button.
4. Check that the search results are relevant to the query entered.
5. Verify that the search results are displayed in the correct format and order.
6. Test for correct spelling suggestions and "did you mean" functionality.
7. Try searching for various combinations of keywords, phrases, and special characters.
8. Test the search functionality with different types of queries such as short queries, long queries, and queries with special characters.
9. Check the search results for accuracy and completeness.
10. Test the search functionality with edge cases such as search queries that contain only stop words.
11. Verify that the search functionality returns appropriate results for empty or invalid queries.
12. Check the search results pagination and make sure it works correctly.

Note that Google is a highly sophisticated search engine and has many advanced features such as auto-complete, personalization, and more. Consider testing these advanced features as well, depending on the scope of your testing. 

We can notice that as an output we have received a set of functional tests proposals, which is good. That can be used as a start for test cases creation or inspiration for exploratory testing, but there's no mention about non-functional tests e.g. response time, how this will look like on different web browsers etc., Let's then try to precise our query to chat GPT.

Query: Suggest me non-functional tests for google search functionality

Non-functional tests for the Google search functionality may include:
1. Performance testing: This includes testing the response time for search queries and verifying that the search results are returned quickly.
2. Scalability testing: This involves testing the search functionality under heavy load conditions to verify that it can handle an increased number of concurrent users.
3. Security testing: This includes testing the search functionality for vulnerabilities such as SQL injection, cross-site scripting, and other security threats.
4. Usability testing: This involves testing the search functionality from the user's perspective to ensure that it is user-friendly and easy to use.
5. Compatibility testing: This involves testing the search functionality on different browsers and devices to verify that it works correctly on all platforms.
6. Accessibility testing: This involves testing the search functionality for accessibility for users with disabilities, such as users with visual, auditory, or motor impairments.
7. Internationalization testing: This involves testing the search functionality for compatibility with different languages and cultures.

These are general examples of non-functional tests that can be performed on the Google search functionality. The exact tests to be performed will depend on the specific requirements and constraints of the testing project.

The output looks nice, we have a list of non-functional tests for Google search functionality, noticeable thing is that chat GPT listed types of non-functional testing and assigned a potential test for it, so the answer is a bit general, but still, I think it's a quite good base for extending the tests to be more granular.

Test cases generation - writing test documentation and maintenance can be really time consuming, especially, when product is continuously changing. Having that in mind would be awesome if chat GPT could write some test cases for us, so let's see:

Query: Write test case to visit google site

Here is a sample test case to visit the Google website:

Test Case: Visit Google Website

Pre-condition:

    The Internet connection is stable
    A web browser is installed

Steps:

1. Open a web browser
2. In the address bar, type "www.google.com" and press enter
3. Wait for the website to load
4. Verify that the website title is "Google"
5. Verify that the website URL is "https://www.google.com/"

Expected Result:

 - The Google website should load successfully
 - The website title should be "Google"
 - The website URL should be "https://www.google.com/"

Generated test case looks good, we can see it has pre-condtion, steps, and expected results, so all required parts. Anyway, I think the main limitation (for now) here is that chat GPT can access google page, so creating these steps doesn't require any additional rights. If we would like to use it in our system, we would need to provide full access to it, but still, it quite awesome!
Testing is not only about checking the positive cases, let's see now how chat GPT will interpret it for us:

Query: Write negative test case to visit google page

Here is a sample negative test case to visit the Google website:

Test Case: Visit Google Website with Invalid URL

Pre-condition:

- The Internet connection is stable
- A web browser is installed

Steps:

1. Open a web browser
2. In the address bar, type "google" (without the "www." and ".com") and press enter
3. Wait for the website to load

Expected Result:

- An error page should be displayed with a message such as "This site can’t be reached" or "Page Not Found"
- The website title should not be "Google"
- The website URL should not be "https://www.google.com/"

And again, it looks good!

To summarize, chat GPT is a great tool, we can use it to speedup work or even get more inspiration in our QA world. There's a one important thing we need to remember when using it, you need to know how to ask right questions and be able to verify the output. From my point of view it's a more advanced browser, which can give you what you want - "if you need to ask, you will never know, if you know you just need to ask".