DEV Community: Sabarish Sasidharan

Should you use BDD for API testing?

Sabarish Sasidharan — Sat, 07 May 2022 23:53:47 +0000

BDD is a collaboration practice: you describe business rules, examples of these rules, and acceptance criteria that illustrate these rules. Some of these acceptance criteria can be potentially automated.

Detailed API testing (e.g. testing CRUD functionalities, error codes, JSON format etc.) is rarely of much interest to the business, so it generally makes little sense to use Gherkin to express them - it just adds an extra layer of complexity. Lower level tools such as RestAssured do the job just fine. And testing REST APIs with any JavaScript test library is pretty trivial. So if your team finds it more efficient to write REST API tests with a tool like Karate, that's great! But it ain't BDD, and it's not the only game in town.

The only exception is where you might use Cucumber as a communication tool between developers across teams, as a more expressive way to describe and document API contracts. But you can do this with contract testing tools as well, though it may not be as readable.

Let me know your thoughts in the comments!

Wiremock on Heroku cloud!

Sabarish Sasidharan — Thu, 17 Sep 2020 18:41:17 +0000

Wiremock: WireMock is an HTTP mock server. At its core it is web server that can be primed to serve canned responses to particular requests (stubbing) and that captures incoming requests so that they can be checked later (verification).

A core feature of WireMock is the ability to return canned HTTP responses for requests matching criteria.

It can be used as a library by any JVM application, or run as a standalone process either on the same host as the system under test or a remote server.

Heroku: Heroku is a cloud platform as a service (PaaS) supporting several programming languages

The Heroku network runs the customer's apps in virtual containers which execute on a reliable runtime environment. Heroku calls these containers "Dynos." These Dynos can run code written in Node, Ruby, PHP, Go, Scala, Python, Java, or Clojure. Heroku also provides custom buildpacks with which the developer can deploy apps in any language. Heroku lets the developer scale the app instantly by either increasing the number of dynos or by changing the type of dyno the app runs in.

Setup Wiremock on Heroku

1) Create a free Heroku account (https://signup.heroku.com/dc)
2) Install Heroku CLI - https://devcenter.heroku.com/articles/heroku-cli
3) Create an application in the Heroku cloud (You can use CLI or the UI dashboard)
4) Using Heroku CLI install the java plugin: $ heroku plugins:install java
5) Deploy Wiremock:
a. Using Build Pack: Refer https://github.com/gpratte/wiremock-deploy-heroku to use the Energized Work downloadable jar buildpack.
b. Without Build Pack:
i. Refer https://github.com/sabariish/wiremock-deploy-heroku and obseve the Procfile.
ii. In the deploy tab for the Heroku application created connect the the above git hub repo
iii. Hit Deploy Branch or setup automatic deploys
6) Access the Wiremock deployed @ https://{appname}.herokuapp.com/__admin/mappings

See https://github.com/sabariish/wiremock/blob/master/src/test/java/wiremock/mock/ExampleTest.java for couple of tests which run on a hosted wiremock instance.

exampleTest – Runs a simple test which sets up a mock service (stateless rest call)
exampleScenario – Example of a stateful request

References: http://wiremock.org/docs/

Service Virtualization for Digital Transformation

Sabarish Sasidharan — Wed, 29 Jul 2020 00:09:33 +0000

Testing in a modern microservices architecture is a nightmare of dependencies. You’re ready to deploy, but the Payment Gateway Sandbox is down, the Legacy Mainframe team is 'refreshing data,' and the Auth API isn't even built yet.

Service Virtualization (SV) isn't just a fancy mock; it’s the difference between shipping today and waiting two weeks for a dependency to wake up.

Why?

It addresses how to shift left efficiently in an iterative development methodology by providing a way out from:

Unavailable Services (External systems)
Dependent Services (under development)
Unavailable Data (Need to generate test data to test the scenario under construction)
Inconsistent Environment (No environment to deploy complete ecosystem)

Service Virtualization allows us to create a simulated version of a service helping to develop system interactions independently in an isolated environment. Simply playback an expected response and see how the system works. Configure complex test conditions using stubbed responses which otherwise would be near impossible to test in a live environment due to several dependent factors.

Most of the time we are unable to test an edge case scenario due to unavailability of data. Service Virtualization allows us to generate and stub responses which would help emulate such edge case scenarios. Moreover the onus is a stable and reliable contract generated to test such scenarios consistently every release.

What to virtualize?

The service is costly to use.
The service is available only for limited timeframes.
The service is needed by multiple teams.
The service is being developed.

If the answers to these questions is a Yes, then such services should be considered for Virtualization.

Can I virtualize every system interaction?

Yes. However, be cautious while doing so! If the virtualization is at the services layer, sure, but if you are having to execute scripted and compiled code outside of your application then consider not to.

Avoid keeping counts in a virtual service. Let's consider a banking domain use case where you want to validate that a user’s account activity screen gets updated when you transfer money between two accounts. What you can simulate easily is a user's experience of transfer money, or validate account activity screen displays the required field as per requirement in isolation. However you will need extra code to simulate that the amount which was transferred is getting reflected in the user’s account activity. This involves managing states of the services which does transfer money and the one which is used to get account details information. In case of stubs, the account details service stub will always send the response with a balance when the response was recorded. You can update the value in the response based on the state captured from transfer money, but that means you are writing additional code to update which should not be the intention of service virtualization.

Can you do this? Yes. But I would not recommend managing states between services. Break down the use case and validate the intent of the service and the behaviour in isolation.

Mocks vs Virtual Service:

Although mocks and virtual service are used interchangeably, there are differences between the two. Both serve the same purpose but in different forms and phases.

Benefits:

Reduced dependency on internal teams or third parties.
Isolate behaviors in a shorter time frame
Eliminate the need to build infrastructure for every feature/scrum team per release
Enabling CI by enabling continuous test data and continuous test environments
Run realistic and quantifiable Performance tests

Challenges:

Tools:

Examples of few tools available for Service Virtualization in the market:

Tool Comparisons: https://trafficparrot.com/Service_virtualization_and_stubbing_tools_comparison.html
https://www.softwaretestinghelp.com/service-virtualization-tools/

Achieving effective Continuous Testing with Behavior Driven Development

Sabarish Sasidharan — Mon, 29 Jun 2020 22:52:40 +0000

Continuous Testing

The goal of continuous testing is to provide fast and continuous feedback on the level of business risk, allowing for a rapid go/no go decision on the feature delivery.

Motivation:
• Risks are exposed as soon as they are introduced
• Prevent those problems from progressing to the next stage of SDLC
• Reduce time and effort for finding and fixing defects

Challenges:
One of the major impediments achieving an efficient continuous testing outcome is Test Design.
Business/PO’s make go/no go decision based on the user journey. The disconnect happens when:

tests are developed without an understanding of the business context.
tests designed either based on requirements or to test the unit of code.

Testing should be a collaborative effort of Dev, QA, and Business, aligned with priorities of the line of business. Remember: Quality is everybody’s responsibility!

What is BDD?
BDD enables software teams to work in collaboration deriving business value. This allows us to build a shared understanding of the work being done. Teams use domain-specific language and produce system documentation based on this shared understanding.

Example:
Feature: Consumer Credit Card Application
Scenario: Reject credit card application if the applicant’s credit score is less than 450
Given An individual applied for a credit card from an affiliate corporate site
When the individual has a credit score of less than 450
Then reject the credit card application with an error message
And show suggestions for secured-card options.

https://cucumber.io/docs/bdd/?sbsearch=What%20is%20BDD describes in detail the essential three-step activity as an iterative process to achieve an executable specification deriving business value. Note the Discovery and Formulation phases. At the end of the formulation phase, there should be multiple readable acceptance criteria as part of the Definition of Ready for the user story.

Anti-Pattern:
Feature: CreditCardApplication API
Scenario: Validate submitCreditCardApplication returns ‘LOW_CREDIT_SCORE’ code for credit score < 450
Given the client sends submitCreditCardApplication a message
And getCreditScore api returns applicant.score.value less than 450
Then return status code 404 with error code “LOW_CREDIT_SCORE”
And the response has section applicant.misc.suggestions

The above is an anti-pattern because the scenario does not describe the business value. A product owner or a business unit owner may find difficulty at deriving the value it provides to the product even though it is ultimately testing the obvious rules (per dev/qa technical team members).
Most often the Dev/SDET/QE ends up writing the anti-pattern as the intention is to test the underlying code or business rule.

How does Continuous Testing benefit from BDD?
Ideally, all tests should be executed across all non-production test environments. All tests need to be run at some point in the build pipeline, but not all tests need to be run all the time.
Testing must begin early; it must be logically componentized and must be executed continuously.

BDD feature files allow the business to pick on tests that prioritize the business value and those can serve as the minimum tests which should be run on a per commit basis. If the tests are too technical to read and do not throw light on the underlying business value then you end up with the temptation of running all the tests or do not understand what the failure of the test means to the product, hence makes it difficult to make an informed and quick go/no go decision. You still run all your Unit Tests, but the user journey tests can be prioritized in a local/dev environment. A similar Test selection needs to be done for each upstream environment the build is promoted.

BDD way allows the team to compartmentalize the application and derive logical tests focusing user journey.

Artifacts produced by Continuous Testing process:
• List of actionable tasks
• Metrics on the quality of the process leading to continuous improvement

Continuous testing should be able to achieve predictability. It should help understand the impact of a change "early". Predictability, in turn, will pave the way to be faster in repetitions.

Gotcha's: (For another day)
• Heavy reliance on manual and/or automated “GUI” tests. (Emphasize on API Tests)
• Test execution process (Lack of Dynamic Test Automation)
• Test environment and Test data constraints (Service Virtualization);
• Test infrastructure