Testing Webservices with Azure DevOps

Testing is not a common scenario but i will get more important in complex systems. So I worked several times on projects that uses an API to starting a large process. But the API sometimes doe not work or does not start any process and so on. Especcially after a deployment this will be a common scenario. To prevent this, there comes n integration test to the rescue.

Let me show you, how I created a simple API integrationtest running with newman in Azure DevOps.

What is AzureDevOps

Azure DevOps is a cloud-based platform for managing your application development pipeline. It provides tools for version control, continuous integration, continuous deployment, and more. With Azure DevOps, you can automate your development processes and ensure that your code is always in a releasable state.

What is Newman?

Newman is a Node.js-based tool that enables you to run Postman collections from the command line. Postman collections are a set of API requests that can be organized, documented, and shared with others. With Newman, you can run these collections as part of your continuous integration (CI) pipeline, ensuring that your API remains functional and error-free.

Using Newman in Azure Devops

To integrate Newman with Azure DevOps, you'll need to create a build pipeline in Azure DevOps that runs the Newman tests. Here's how:

Prepare dependendcies

Before you can use Newman, you'll need to install Node.js and npm on your Azure DevOps build agent. I prefer to upload a package.json in my repo like this:

After that you must export your Postman Collection that you will need to execute. Also you should export an environment setting. So that you can switch the environments, by selecting these variable file.

Create a build pipeline

In Azure DevOps, create a new build pipeline and choose the "Empty Job" template. Add a new task to the pipeline and select the "Node.js Tool Installer" task. Use this task to install the version of Node.js and npm that you want to use.

1. Add a npm Task

Use this task to run the npm run install command. This will install the Newman command that you can execute on your own need.

1. Add a CMD-Task

Now you must execute the postman file that you have available. Let's assume your collection is called collection.json and to support some environment variables we use the environment-file environment.json. So the cmd to execute will look like this:

You will notice that I use more parameters. In this case, I will export the test result to a file called report.xml. As a format, it will take the Junit Format. So that I can use it later in Azure DevOps to display it in the test results tab from the build itself.

1. Publish the test report

Finally, I want to publish the test report so that it's accessible from Azure DevOps. To do this, add a "Publish Build Artifacts" task to the pipeline and specify the XML report as the artifact to publish.

This will look like this in my definition:

With these steps, you have a working build pipeline that runs your API tests with Newman and generates a test report. You can now integrate this pipeline into your DevOps workflow and run your API tests as part of your continuous integration process. My full definition looks like this for now, you can use it as regular basis if you want:

Displaying the testreults

Since you published the testrsult in your build definition, you will be able to view the testresults in your build itself. Here is an example:

If you click on this text you will be able to see the details of the testresults

As you see, we integrated our Test results now in the Build Pipeline, so that we can use it in every position of the CI/CD pipeline. Simply integrate it into the Release to execute an integration test after a deployment succeeds.

Conclusion

By using Newman with Azure DevOps, you can ensure the quality and reliability of your API by running automated tests as part of your DevOps pipeline. With the ability to generate test reports, you can quickly see the results of your tests and identify any issues that need to be addressed. With a few simple steps, you can integrate Newman with Azure and it's pipeline. So when you ever have a complex system, you will be able to test the "hotpath" injected by an API call, like an order Processing, now by newman.