How to Test Software in this AI Era with KaneAI

Most developers would agree that writing tests is more stressful than writing code itself.

We often try to skip Testing (I've been there!), But trust me, that's a recipe for disaster when your code hits production. Yes, testing can be a pain, but it's absolutely vital for keeping your website running smoothly.

I learned this lesson the hard way during my recent web project. Like many developers, I thought I could get away with quick manual testing - big mistake! So I decided to pick up Selenium and start writing test scripts.

After spending hours wrestling with these scripts, I had a thought: "There's got to be a better way to do this." I mean, we're living in the age of AI - surely there must be a smarter solution that could make testing less of a headache.

There was indeed a solution, and that was KaneAI.

You're probably wondering what KaneAI is. Well, KaneAI is a generative AI native test agent. Let me break down that definition: KaneAI uses the power of generative AI to create tests based on the natural language input provided by the user.

Apart from its natural language test generation capabilities, it also takes advantage of other LambdaTest tools like HyperExecute and Test Manager, which make it even more robust.

In this Article, I'll walk you through how I used KaneAI to test my website, LogoAI, and show you how you can do the same for your projects!

Pre-KaneAI Testing

Before we dive into how KaneAI can be used for testing, let me share a bit about the experience that led me to search for a better way to handle testing. I recently built a web platform called LogoAI. LogoAI allows users to generate logos using AI models and a set of presets.

When I first started building LogoAI, I decided not to write any tests because I wanted to keep things simple. But manually testing a website is anything but simple. LogoAI had multiple pages, and I constantly had to navigate through them after every code change to make sure nothing had broken.

Clearly, this wasn’t sustainable, especially if I wanted the website to scale.

Automated Tests with Code

After manual testing failed me, I decided to automate my testing process. To do this, I turned to writing code with testing frameworks like Selenium and Playwright.

At first, it was going alright, but after writing my initial tests for the first version of the LogoAI, those tests quickly became stale. I had to go back and forth between writing tests and development. There were other challenges I encountered besides the stale test.

Here are a few of them:

1. Frequent Locator Breakage: Every time I made changes to the website’s UI, many of my test locators (CSS selectors, XPath expressions) became invalid. This meant I had to constantly update my test scripts, adding extra maintenance effort.
2. Flaky Tests: Some tests would pass in one run but fail in another, especially when dealing with dynamically loaded elements. I had to take the edge cases into account when writing my test scripts.
3. Time-Consuming Updates: With every UI modification, I had to manually review and adjust my tests, which slowed down my entire development.

With all of these challenges I faced and the ones I could foresee coming, I decided to give KaneAI a shot.

What is KaneAI?

KaneAI is a generative AI-native test agent built on top of modern large language models. With it, you can write your tests using natural language, and it will generate the appropriate test.

You can think of KaneAI as a quality assurance AI agent that you communicate with. It ensures that your website, application, and endpoints work as expected.

Changes after using KaneAI in testing

When I set out to find a new testing approach, I had two main criteria that I felt were essential for my new testing approach.

First, I needed a testing approach that leverages artificial intelligence, and second, one built on a solid foundation.

After thorough research, I discovered KaneAI, the Generative AI native test agent developed by LambdaTest. KaneAI is designed to assist in generating tests using natural language.

Core Features of KaneAI

KaneAI checked all the boxes for what I was looking for. Beyond just meeting my two initial criteria, it also has several powerful features that changed how I approached testing forever.

• Natural language test generation: KaneAI can generate test scripts based on prompts provided by the user. These prompts are given in a step-by-step format, where each step includes a specific prompt and results in a corresponding test case.
• Intelligent Test Planner: KaneAI doesn’t just execute tests based on step-by-step instructions from the user. It can also generate test steps from high-level objectives. For example, a user can simply instruct KaneAI to “test the login workflow,” and KaneAI will intelligently generate the relevant test steps to cover that functionality.
• Auto-Healing: KaneAI comes with a really handy feature called Auto-Heal, which helps keep your test scripts running even when your app changes. Maybe an element ID or XPath has been updated. Normally, that would break your test. But with Auto-Heal, KaneAI steps in and tries to find the element dynamically during the test, using the original natural language instruction as a guide. If needed, KaneAI can also use its visual query, which identifies elements visually rather than relying on locators like ID or XPath.
• Multi-language code export and 2-way test editing: Another feature of KaneAI that I found interesting is the fact that the generated test scripts could be exported as code in any language of your choice. It also supports multiple test frameworks. With this you can go back and forth with both test code and generated code.
• Integration with HyperExecute for cross-platform testing: KaneAI also integrates perfectly with HyperExecute. This enables it to perform parallel test execution across different browsers, operating systems, and devices.
• Integration with Test Manager: KaneAI also integrates perfectly with Test Manager. Using it, users can measure all the generated tests from KaneAI.
• Module creation and reusable test flows: It also supports the creation of reusable test modules. Essentially thanks to these test modules you can reuse test scripts.

With all my criteria met and KaneAI’s powerful features at my disposal, I was super excited to dive into testing with it.

Using KaneAI for Web Testing

Let’s get started with using KaneAI for testing. In this section we would be testing my logo generation website: LogoAI. This means we would be taking advantage of KaneAI web testing functionality.

Creating First Web Test

Let’s get started with testing. We’ll create a simple test that navigates to LogoAI, signs into the platform, and browses through the gallery.

To begin, go to KaneAI’s site: kaneai.lambdatest.com, where you can access their web agent.

Once KaneAI is open, click on the “Create a Web Test” button to begin web testing.

Clicking this option takes you to a page where you can enter your test prompts and use an integrated web browser to view the execution in real time.

Alright, let’s get started. Here’s the workflow we’ll be testing:

• Start by navigating to the LogoAI homepage
• Sign in to the platform
• Go to the Gallery page
• Browse through the available logo designs

Signing In with KaneAI

Let’s walk through the sign-in process. To begin, I entered the prompt: Go to logoai.in

KaneAI launched the site and displayed the rendered page in real-time.

Next, we would instruct KaneAI to click on the Signin button:

Immediately, we do that, and KaneAI will execute the action. Next, we would instruct KaneAI to enter our email address and click on the continue button.

This would navigate us to the next option, where we can instruct KaneAI to e*nter our password*

With this, we would be redirected to the generate page of logoAI:

Now it’s time to test the Gallery workflow.

Exploring the gallery with KaneAI

Now, let’s instruct KaneAI to click on the gallery after we have provided our instructions; we will be redirected to the gallery page. Then we will also instruct it to scroll down the page

It executed that. Next, let’s test if the See More button is working by instructing KaneAI to click on it.

With this, we can see that the LogoAI gallery workflow runs perfectly, with nine test scripts executed. Now you can click on the Finish Test button.

This would save our tests:

We can view the summary of the saved tests, the code, the test runs, and the version history in the test manager console.

Test Logo Generation using KaneAI

We’ve explored the gallery page using KaneAI. Now let’s test the logo generation feature of LogoAI, this time using a new test case. Since we’re starting fresh, we’ll need to go through the entire sign in process again. This introduces a lot of repetition for a process we’ve already completed.

KaneAI can help reduce this kind of repetition by utilizing modules. Modules enable test reusability. You can think of KaneAI modules as similar to JavaScript modules, but instead of storing reusable logic, KaneAI modules store reusable test steps.

To get a better understanding of modules, let’s walk through the entire login process again. We'll go through the following steps:

• Go to LogoAI.in
• Click on the sign-in button
• Input the email
• Input the password

When we are done with this, we will click on the Pause Test button.

This will pause our test execution, and then we can begin selecting the tests we want in our module.

When we are done selecting, we can click the Create Module button. Upon clicking the Create Module button, we will be prompted to enter a module name and description. We will name our module LogoAI Login.

Once that is done, our module is saved. We could use the module right now, but instead, let's start a new test.

Once we have our new test, instead of typing a new test step, we will type /, and different options will pop up. These options are known as actions in KaneAI. We will select the Module action:

Once we have done that, we will see all our created modules. We will then select LogoAI Login and click the Add to Queue button.

This will begin executing all the tests stored in the module. With this module, we will never have to re-implement the login flow. If, for any reason, there’s a failure in the login flow, we will know something has broken and needs to be fixed.

Let’s continue our logo generation test using this module. In LogoAI, to generate a logo, we need to specify a brand name and a couple of presets. Let’s generate a logo using the brand name KaneAI.

Set the style preset to Technology. We will then prompt KaneAI to generate the logo and ensure that the logo is successfully generated.

KaneAI is able to perform visual queries, so it can verify that an image was indeed loaded. All we have to do now is save the test.

With this, I was always able to ensure that my login and logo generation workflows worked as expected.

Viewing and Generating Test Scripts

If you want to view the generated code for your test, you can navigate to your test in the Test Manager and view the code there.

By default, KaneAI uses Selenium, but you can generate code in other supported languages or frameworks that are available within KaneAI.

In the image above, I am generating JavaScript code using the playwright framework. These generated codes can be taken outside the KaneAI platform and be used for whatever purpose you want.

Executing the generated Test scripts

When your tests have been saved, you can always run them at any time via HyperExecute by clicking on the Run on HyperExecute that would run the test scripts.

You can select which operating system or browser you wish your test to run on.

Using KaneAI for App Testing

You can easily test your Android and iOS applications using KaneAI, and the process is essentially the same for both platforms. Application tests can be generated using Appium.

To begin, simply upload the APK (for Android) or IPA (for iOS) file to KaneAI, and then start testing just as we did with the web application.

For this demonstration, we’ll test the Proverbial sample application.

To begin our application testing, we will navigate to KaneAI on our web browser just like we did previously but this time we will click on the “Create App Test” button. This would bring up a modal that would give us the option of selecting whether we want to test an Android or iOS application.

You can also choose which mobile device you want to run your test on.

Let’s perform our test on iOS using the iPhone 16 Pro Max. Now that we’ve selected our OS and device, let’s upload our application. There are three options available for the Proverbial app.

The "Sample App.ipa" shown on the screen is actually the Proverbial app. However, if you prefer to upload a different application, you can provide the IPA file directly via a URL.

Here’s the URL for the Proverbial IPA file: https://prod-mobile-artefacts.lambdatest.com/assets/docs/proverbial_ios.ipa

You can also upload an IPA file directly from your system.

If you followed both methods above, you should see three applications displayed on KaneAI:

Select Sample App.ipa, then click Start Testing.

This will launch the iPhone emulator. Once the Proverbial app is running, we’ll provide our first prompt instructing KaneAI to click the Toast button.

This will trigger a toast notification.

Next, we’ll instruct KaneAI to click the Notification button. This will trigger a notification in the app.

Now that we've seen how KaneAI can control mobile apps just like web pages, let’s try out its visual query feature on the mobile app.

Clicking the Color button in the Proverbial app changes the color of the welcome text. KaneAI’s visual query feature was able to verify that this change occurred.

Clicking the Text button in the Proverbial app changes the welcome text to Proverbial. KaneAI's visual query feature was also able to detect this as well.

KaneAI can also navigate through the application. Instructing KaneAI to click the Geolocation button will navigate the app to a new view.

Instructing KaneAI to go back will return us to the home page.

With this we can then click the Finish Test button.

Once clicked we will be redirected to a page where we can save our tests.

This was a quick rundown of how KaneAI can be used for mobile application testing. All of these test procedures apply the same when testing Android applications as well.

Generating Appium Code with KaneAI

Once your tests are saved, you can generate Appium test code in either Python or Java.

Simply head over to the Test Manager, where you’ll find a summary of your test session and access the generated code under the Code tab.

You can choose to either view the code directly or download it for later use.

You can generate new code by clicking the Generate Code button.

Next, we'll generate the test code using Java.

Once the code is generated, you can view and download it, just as you did with the Python version.

When test code is generated, KaneAI automatically runs validation using HyperExecute. The generated code can then be run on different environments or executed directly on HyperExecute.

Additional KaneAI Features

We’ve seen how KaneAI can be used to test web pages. Now let’s explore some other use cases. In this section, we’ll look at how KaneAI can be applied to the following:

We’ve seen how I used KaneAI to test LogoAI. Now, let’s move on to some other use cases of KaneAI.

Lazy Loaded Image Testing

In the logo generation test we conducted, we used the visual query feature of KaneAI to test if our logo image was generated successfully. Let’s take this feature and apply it to testing lazy-loaded images.

In this example, we’ll use a different website: Ecommerce Playground, which is a demo e-commerce website.

Here’s the homepage of the site. Let’s begin testing the lazy-loaded images. This is also a great opportunity to see how KaneAI handles multi-step instructions. In the previous section, we provided only one instruction at a time.

We will instruct KaneAI to go to the e-commerce site and click on the blog button. Let’s see how it performs:

KaneAI was able to divide our instruction into two steps and successfully navigated to the blog page.

Okay, let’s test our lazy loading. We would instruct KaneAI to click on laptops and ensure all the images loaded successfully:

It worked perfectly, and it was able to check that all the images loaded successfully.

Let’s test it further, we will prompt it to go down the page and ensure all the images will load.

KaneAI was able to scroll down the page and confirm that all our images had loaded successfully. This section demonstrates how KaneAI can be used for lazy image testing by leveraging its visual query feature.

Adding JavaScript to KaneAI Testing Steps

KaneAI doesn’t just support writing tests with natural language. It also offers the ability to write tests using JavaScript code. When testing websites, you can easily add a test step that runs JavaScript code, which can access the DOM or make API calls.

The JavaScript code snippet can activate the same modules that are available through the / command. When you select "JavaScript," a modal will pop up where you can enter your code.

API Testing With KaneAI

KaneAI can also be used to test backend APIs. This allows you to manage all your tests, both frontend and backend, within the KaneAI application. To test backend endpoints, you use the / command and then select API as the action you wish to perform.

When API is selected, a modal will pop up we will then provide the API endpoint we wish to test. For this example, we would be utilizing this endpoint: https://jsonplaceholder.typicode.com/users

Once we have provided the endpoint, click on the “Add in steps” button. This would add the API test case as a step.

When API is selected, a modal will pop up. We will then provide the API endpoint we want to test. For this example, we will use the following endpoint: https://jsonplaceholder.typicode.com/users

Once we have provided the endpoint, click the Add in steps button. This will add the API test case as a step.

Once the step has been added, we can press Enter.

This would make a request to the endpoint. If the request is sucessful KaneAI would indicate it and then it would save the API response as a variable.

We can check the variable from the top right button with the labeled {x}

This will send a request to the endpoint. If the request is successful, KaneAI will indicate it and save the API response as a variable.

You can view the variable by clicking the {x} button in the top right corner.

Then, in the chat input, you can access the variable by typing {{, which will display all available variables.

Since the API response is in JSON format, KaneAI provides an interface that lets us easily navigate through it and select the value we want by clicking. Let’s confirm that the status of our response is 200.

We can also access the response body of the API in a similar fashion.

Let’s perform a test to see if the first object's email matches the email we provide.

Both the status and email tests pass successfully. You can then save the test, just like we did with the web test.

Handling Secrets With KaneAI

In the previous section, we discussed how variables are used within KaneAI. To enhance security, KaneAI also supports secret variables. These secret variables utilize HashiCorp Vault to securely store and manage sensitive information such as tokens, passwords, certificates, and API keys.

Let’s walk through how you can create and use secrets in KaneAI. To create a secret, use the / command and then select secrets.

This will bring up a modal where we can provide the key and value for our secret. We saved the secret as SECRET_KEY. To access the secret in KaneAI, simply type {{secrets.user.SECRET_KEY}}. This way, KaneAI can securely access the secret while it remains protected in the vault.

Simulating different locations with KaneAI

Let’s say you want to test how your website loads and performs in different regions. How can you go about that? Well, with KaneAI, you can simulate different geographic regions using its advanced geolocation support. KaneAI also supports tunneling and dedicated proxies.

Integrating KaneAI with Your CI/CD Pipeline

KaneAI can easily be integrated into your CI/CD pipeline, whether you're using Jenkins or GitHub Actions. Through HyperExecute, KaneAI offers APIs that can be called directly from within your pipeline. These APIs can be accessed through the Test Manager.

Smart Variables

KaneAI supports smart variables. These are dynamic placeholders that adapt in real time during test execution. They’re similar to the variables we talked about in the API testing section, but with a key difference: smart variables are predefined by KaneAI and automatically updated based on the current environment or context.

You can access smart variables while writing your tests by typing {{ in the Web/App Agent.

Smart variables in KaneAI fall into a few categories:

• Date & Time:

  Variables that provide the current time context.

  • {{smart.current_date}}
  • {{smart.current_day}}
  • {{smart.current_month}}
  • {{smart.current_year}}
  • {{smart.current_timestamp}}

• User/System Information:

  These give system-level data.

  • {{smart.os_type}}
  • {{smart.os_version}}
  • {{smart.browser_name}}
  • {{smart.browser_version}}
  • {{smart.device_type}}

• Randomized Values:

  Useful for generating unique data each time a test runs.

  • {{smart.random_email}}
  • {{smart.random_username}}
  • {{smart.random_name}}
  • {{smart.random_number}}
  • {{smart.random_string}}

With smart variables, your tests can become more adaptable and robust. Whether you're testing across different systems or need dynamic data like the current date or a randomly generated email, smart variables help you handle it with ease.

Module Versioning

Earlier, we talked about how multiple test steps can be grouped and saved as modules for reuse. KaneAI not only supports this feature but also adds version control for modules. This means you can easily track changes and revert to previous versions of your tests whenever needed.

To dive deeper into module versioning, check out the KaneAI documentation on module versioning.

Upload and Download files

KaneAI supports uploading files during a test session. For example, if you're testing a form that requires file input and you want to make sure everything works as expected, KaneAI has you covered. This feature allows you to upload different types of files, making it easier to test real-world scenarios.

It supports the following file types and formats:

• Images: JPG, JPEG, PNG, GIF (Max size: 10 MB)
• Videos: MP4 (Max size: 50 MB)
• Documents: XLS, XLSX, DOC, DOCX, PDF, CSV, TXT (Max size: 15 MB)

Files can also be downloaded during test sessions, which means KaneAI supports both uploading and downloading workflows, making your tests more comprehensive and flexible.

To learn more about this feature, check out the KaneAI documentation on uploading and downloading files.

Assertions

During a testing session in KaneAI, you can easily control how your test steps behave, specifically whether the session should continue running or stop if a test step fails. This is managed through assertions. KaneAI offers two types of assertions:

• Hard Assertions
• Soft Assertions

When hard assertions are enabled, KaneAI will halt the session if a test step fails. On the other hand, if soft assertions are used, the session will continue running even after a failure. This flexibility allows you to choose the level of strictness that best suits your testing goals.

To learn more about this feature, check out the KaneAI documentation on assertions.

Scheduled Test Runs

KaneAI lets you schedule test runs just like you’d schedule a cron job. If there’s a test you want to run every night, every Monday morning, or even every few hours, KaneAI makes it easy to set that up. You can schedule a test run via the test manager.

To learn more about this feature, check out the KaneAI documentation on schedule test runs.

Different Prompts used in KaneAI

KaneAI prompts mirrors how a developer naturally thinks about writing tests. This makes the process of test creation in KaneAI simple and efficient.

In this section, we'll explore some example prompts used in KaneAI and see how they correspond to equivalent code in popular testing frameworks like Selenium.

The navigation prompts

If you want to navigate to a particular website with KaneAI, you can simply prompt it with the word go or open. So, for example, if we want to navigate to Google, we would just instruct KaneAI this way:  Go to Google.com or Please open google.com.

The “please” doesn’t matter; the KaneAI agent is smart enough to understand your intent.

This instruction is equal to its coding counterpart, which can be written in selenium as:

# go to 'google.com'
driver.get("https://google.com")

The click prompt

Clicking is the most common action performed on a website or app, and performing clicks in KaneAI is quite easy. All you have to do is use the click and then specify the element you want to click on. In our first example, when using KaneAI, we said to click on the signup button and provided there’s a signup button on the page. KaneAI would perform that action successfully.

Here’s the generated Selenium code that corresponds to that prompt:

# Step - 2 : Click on 'Sign In' button in the top right corner
element_locators = ['//html/body/div[1]/div[1]/header/nav/div[2]/button', "//header//nav//div//button[contains(normalize-space(.), 'Sign In')]", "//nav//div//button[contains(normalize-space(.), 'Sign In')]", "//button[contains(@class, 'inline-flex') and contains(normalize-space(.), 'Sign In')]", "//div//button[contains(normalize-space(.), 'Sign In')]"]
element = get_element(driver,element_locators)

try:
        actions.move_to_element(element).click().perform()
except:
        element.click()
driver.implicitly_wait(6)

The type prompt

In order to type in KaneAI, you can use the following words: type, enter or send keys.

Synonyms of these words can also work, so if you want to type an email you can say Select the email input and type [e-mail].

Here’s the generated Selenium code that corresponds to that prompt:

# Step - 3 : Type '[e-mail]' in the email address input field
element_locators = ['//html/body/div[4]/div/div/div/div/div[1]/div[2]/form/div/div[1]/div/div[1]/input', "//div[contains(@class, 'cl-internal-11m7oop')]//input[@name='identifier']", "//div[contains(@class, 'cl-internal-11m7oop')]//input[contains(@placeholder, 'Enter your email address')]", "//div[contains(@class, 'cl-internal-11m7oop')]//input[@id='identifier-field']"]
element = get_element(driver,element_locators)

try:
        element.click()
except:
        driver.execute_script("arguments[0].click();", element)
driver.execute_script("arguments[0].value = '';", element)
if element.get_attribute("pattern") and '[0-9]{2}' in element.get_attribute("pattern"):
        for char in '[e-mail]':
            element.send_keys(char)
else:
        element.send_keys('[e-mail]')
driver.implicitly_wait(6)

You can also use the clear prompt to clear an input after typing.

The scroll prompt

To scroll across the page, we use the scroll prompt. We use this in combination with the direction we wish to scroll in. Here’s the generated Selenium code that corresponds to that prompt:

# Step - 8 : scroll down through the page
driver.execute_script("window.scrollTo(0, document.body.scrollHeight)")
time.sleep(1)
driver.implicitly_wait(6)

To learn more about KaneAI prompts, please refer to the documentation.

Conclusion

My experience using KaneAI as my testing suite was nothing short of exceptional. It significantly accelerated the development process of LogoAI. The natural language test generation capabilities of KaneAI simplified the entire testing process. I no longer had to spend so much time writing tests with code or battling flaky tests and broken locators. KaneAI solves all of that, thanks to its use of natural language processing.

KaneAI essentially represents what it means to apply Generative AI to software testing, built on a solid foundation. Using KaneAI boosted my productivity tenfold while building LogoAI. I hope you’ll give it a try and see how it can take your next build to a whole new level.

Comments

[Debajyati Dey]

Bruh can't read this much now. Bookmarking for later.

[Arindam Majumder]

Sure,

It's indeed an in-depth Guide!

But, After reading you'll understand how helpful it could be!

[Debajyati Dey]

Okayyy