Currently I am working in a project as QA where we are doing re-model of Sql database to new shiny Graph db and writing a new rest service on top of it.
Greenfield “yay” . I have been hearing a lot about Test Pyramid for quite sometime and now is the time when will be witnessing how it is built with all ups and downs.
In test pyramid most of the automated test should be covered at unit level, then some at integration and few for end to end journeys.
Unit testing for most of my career was black box. Also as a gate keeper of App(this statement is debatable, lets park this for future), trusting test cases which are majority in number, which you pretend to understand and also have not written, is not very comfortable position to be in.
Hence I wanted to keep my eyes ear open for anything which is not caught by unit test. And my wish was answered very soon.
It all started with a small ad hock change request from a BA in our team.
Requirement
BA: Hey can we please change the “id” to “key” in the response.
Dev: Sure.
QA: will be tested in a jiffy.
Lets look at some pseudo code
Implementation
Lets assume we are building simple App which returns Employee details. Pseudo code will look something like below.
- Scala case class as model
- Serialization and deserialization
- Neo4j connection
- Query to get Employee
**case class** Employee(id: String, firstName:String, lastName:String)
**implicit val** employeeFormat = jsonFormat3(Employee)
def getEmployee(id:String): Employee {
//read neo4j data and return case class
query = "match (e:Employee {id:id})
//some code to parse query result and make Employee model
return Employee(e)
}
Minimal test case & seed test data for above code will be.
**def** testEmployee():Unit={
//spin in-memory neo4j
//add create (e1:Employee {id:'1', firstName:'Joe', lastName:'Doe')
//val employee = getEmployee("1")
//assert employee model with "key"
Everything is good so far, developer actually failed test cases by changing “id” to “key” in assertion of testEmployeModel test case, fixed code and “Yay” TDD.
The Problem
While doing e2e testing we loaded data in test db.
create (e1:Employee {key:’1', firstName:’Joe’, lastName:’Doe’)
Hit the service and “internal server error”.
Though the fix for this bug was pretty straightforward, but it kept me thinking what could have gone wrong here. We have written unit test case for this requirement, even development was done TDD way, now how come this bug?
If you know the answer then continue else go through the implementation one more time and see if you can figure out the problem.
The Fix
Change query in getEmployee method to
query = "match (e:Employee {key:id}) return Employee(e)
The Miss
There are 2 fold miss in this.
- Requirement understanding
- Test Fixture
Requirement from BA was never understood/explained clearly as implementation was easy and never discussed the “why” behind it. Consumer of the service wants “key” instead of “id” but also it was the database value which was going to change in the data-ingestion part.
If some more effort was spent in understanding the test case, and test data in particular for a unit test case before making any change to code. This problem must have been caught and fixed then and there itself.
But now imagine once the project matures, these test fixtures would be present in thousands of test how would someone know what to fix/update.
Its better to have no test case then a test case giving false positive results.
Have you ever experienced this problem of updating or forgetting to update hundreds of test fixture for any change in functionality, if yes any solution/strategy to handle. Do share in comment section.
Top comments (0)