A number of tools and frameworks exist to support automated testing of API endpoints. A relatively new player in the area is the open source tool Karate. In this post I want to review some of the basics around API testing, talk about what sets Karate apart from other tools, and walk through some examples.
After you walk through this post, feel free to checkout my Karate Demo on YouTube.
Automated API testing
The test pyramid visually depicts test types and gives some general guidelines on how many of each to create. We want lots of unit tests and a small number of end-to-end UI tests.
API testing is one layer of the test pyramid. API tests validate the application from the API level, and do not include the user interface(UI). Because of that, API tests will typically run faster and be less brittle.
In the past I’ve used tools such as Postman and REST Assured to do automated API testing. Karate is a tool I’ve recently started working with that gives a new spin to API testing.
Gherkin and API testing
Gherkin is a description of expected application behavior from an end-user perspective, and can be used with Cucumber to automate UI-based scenarios. But what would a gherkin-based API test scenario look like?
In the past I've used Ruby/Cucumber and .Net/Specflow to implement automated API tests, using Gherkin to describe acceptance criteria just like we do with UI scenarios. The only difference is instead of using business domain language like we do with UI tests, we use API domain language to define the scenario. For example:
Feature: sample API test
Scenario: get all users
Given endpoint 'https://jsonplaceholder.typicode.com/users'
When performing a 'GET'
Then the HTTP status of the response is 200
The intent of this scenario is probably pretty obvious. When we perform a GET operation against the path, we expect the response status to be 200.
If I'm doing this without Karate, finishing the Gherkin is just the first step. From there I need to develop code that will implement the behaviors defined in the Gherkin. In our basic example, I'd need to write code to make a GET request to the defined path, write code to verify the response code, and of course consider other things like error handling, reporting, etc.
Karate can expedite many of these steps, because it is tailored to the domain you are working in, API testing.
What is Karate
Karate is an open source tool that you leverage in a Java codebase to build automated API tests. A couple things that make it stand out to me:
- Karate supports familiar Gherkin-like syntax when defining API test scenarios, which means they are very readable. You can define your test specs with a business-readable language, just like you define your UI scenarios with Gherkin.
- Karate is a Domain Specific Language (DSL), which is tailored for API testing. That means it provides the code implementations of the steps defined in the Gherkin. It’s actually possible to define and execute API test specs without writing any code!
Karate has many additional features, but I want to focus on these critical basics for this post.
OK enough talking, let’s see Karate in action!
Hello World with Karate
We are going to use a Maven archetype to generate our first Karate-based API tests. To do that, open a terminal window, navigate to the location where you want the new project folder created, and enter:
mvn archetype:generate \
-DarchetypeGroupId=com.intuit.karate \
-DarchetypeArtifactId=karate-archetype \
-DarchetypeVersion=0.9.3 \
-DgroupId=com.leadingedje \
-Dversion=1.0-SNAPSHOT \
-DartifactId=karateblog
If things work out, you should now have a new folder named "karateblog".
From here you can open the project in your favorite Java IDE. I am using IntelliJ IDEA with JetBrains plugins "Cucumber for Java" and "Gherkin".
The project already has a the feature file "users.feature" found in karateblog/src/test/java/examples/users. The first scenario should look something like this:
Background:
* url 'https://jsonplaceholder.typicode.com'
Scenario: get all users and then get the first user by id
Given path 'users'
When method get
Then status 200
* def first = response[0]
When path 'users', first.id
And method get
Then status 200
This scenario is similar to the previous scenario we discussed, but in addition this scenario will make a 2nd API call to get details for one of the customers returned in the first API call. We expect the response to be 200 for that 2nd call also.
This is just like a UI scenario, except we are using domain specific terminology for API interactions such as “path”, “method”, “status”, and "response". This is terminology specific to the API testing domain.
With Karate we are ready to run this test. No additional API test framework code is needed.
In IntelliJ IDEA you can right-click on that scenario and click "Run Scenario...". The scenario runs and the results are displayed in the Run window, including the data that was returned from each GET request.
So there, we have created and executed an API test scenario without any Java code!
Setting up JSON Server
This test is pointing to https://jsonplaceholder.typicode.com, a good collection of mock API endpoints you can use for testing. Unfortunately the site will not persist any data changes during a POST request, so I am going to quickly setup my own endpoints locally, using an open source tool called JSON server.
To get that setup on your machine:
- Install json-server
npm install -g json-server - Create a file in your project folder called db.json and paste the following into it:
{
"users": [
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "Sincere@april.biz",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
},
"phone": "1-770-736-8031 x56442",
"website": "hildegard.org",
"company": {
"name": "Romaguera-Crona",
"catchPhrase": "Multi-layered client-server neural-net",
"bs": "harness real-time e-markets"
}
}
]
}
- From the command line
json-server --watch db.json - You should now have a API endpoint at http://localhost:3000/users
- Change the background of the first scenario to point to your local endpoint and verify your tests still works:
Background:
* url 'http://localhost:3000'
With JSON Server you can start building your test automation components before the developer implements the API. You can easily mock the expected responses and once the API is implemented, your tests can point to that implementation instead of JSON Server. Development and test automation can happen in parallel.
Testing a POST
Let's take a look at an example that tests an HTTP POST. You should have a 2nd scenario in your user.feature file. Replace that scenario with:
Scenario: create a user and then get it by id
* def new_user_json =
"""
{
"name": "Test User",
"username": "testuser",
"email": "test@user.com",
"address": {
"street": "Has No Name",
"suite": "Apt. 123",
"city": "Electri",
"zipcode": "54321-6789"
}
}
"""
Given path 'users'
And request new_user_json
When method post
Then status 201
* def id = response.id
* print 'created id is: ' + id
When path 'users', id
And method get
Then status 200
And match response contains new_user_json
This scenario is a little more interesting. We want to add a new user so the first thing to do is define the JSON payload.
Once we do that, it's easy to post that data to our endpoint, verify the response status, and then GET the newly added customer and confirm the data returned matches that data we added.
If you look at db.json you'll see your newly added data. This is all managed by JSON Server.
Again, we wrote no Java code to implement this test. It's all baked-in with Karate.
Wrap-up
So hopefully that gives you a taste of the power of Karate. In my next post I am going to cover some more advanced Karate topics such as calling java code, data driven tests, parallel tests, reading files, and performance testing. Happy testing!
Top comments (7)
How to pass an access token that answered the first service and use it in the header of the second service and use the access token that answered the second service in the header of the third service and the same with the fourth, fifth ...
Hi, see if this helps github.com/intuit/karate/blob/mast...
more or less, but I need to do them in different scenarios, karate could not handle this as global variables
I would also suggest you take a look at the documentation on the Karate page github.com/intuit/karate. I have not run into a problem yet that was not answered somewhere there. The doco is very mature and complete.
You might also want to check out this informative talk from the creator of Karate. It was just recorded today. youtube.com/watch?v=yu3uupBZyxc&fe...
Hi Dennis,
Excellent post, I will be waiting for what you say with Karate for tests of performance
I test this karate api but few loopholes there because some functions not working.