I.Why test pyramid
Companies are investing a lot of money into automated testing, but they are
not seeing improved quality
not seeing improved productivity
not seeing improved user sentiment
One of the main reasons for this is the over reliance on browser based end to end tests.
If a tech team has a visible build radiator than it might look like this:
Long running tests suites; 30 mins-4 hours, sometimes days
Flaky tests fails 1 in 10 or even 1 in 3 runs
Too much time by QA and devs creating and maintaining regression tests; rather than finding and fixing problems
Environments broken or not reflecting production; therefore negating the usefulness of the regression tests
II.What is a test pyramid
In 2012 Martin Fowler wrote an article about the test pyramid, this was a concept developed by Mike Coen.
The test pyramid is a tool to fix the problem of over-reliance on long-running UI tests.
The pyramid says that tests on the lower levels are cheaper to write and maintain, and quicker to run. Tests on the upper levels are more expensive to write and maintain, and slower to run. Therefore you should have lots of unit tests, some service tests, and very few UI tests.
I often see companies test suite looking like an ice cream cone.
Testing ice cream cone
Here they have very few unit tests, some UI test, lots of QA tests and lots of manual tests.
In this situation the QA department has created an automated test suite, but the development team has not. It will be very long running and flakey because the development team has not helped build the suite or architect the application in a way that makes it easy to test. It is broken by the devs very regularly and they are relying on the QA department to fix it.
There is probably both a manual test team and an automated test team. There is likely not enough trust in the test suite so regression is being performed twice, once manually and once automatically, because of this double work and because of the extra hand-offs between the testing teams this approach has made the company go slower.
Testing hourglass
In this situation there is a lot of Unit and UI tests but little service based tests.
This is better, there is no manual testing, the QA department has created a test suite that the company is relying on. The developers are writing a lot of unit tests, perhaps they are practicing TDD. However a lot of the logic that is being tested in the unit tests is also being tested in the user interface tests as well. Again causing double work. The QA team is almost solely relying on browser based tests for regression, instead of cheaper service based tests.
Both the hourglass and the ice cream cone is an indicator that there is a lack of collaboration and communication between the QA and Development departments, they are probably separated organizationally and even by location.
III.Unit test (Level Unit)
UNIT TESTING is a level of software testing where individual units/ components of a software are tested. The purpose is to validate that each unit of the software performs as designed. A unit is the smallest testable part of any software. It usually has one or a few inputs and usually a single output. In procedural programming, a unit may be an individual program, function, procedure, etc. In object-oriented programming, the smallest unit is a method, which may belong to a base/ super class, abstract class or derived/ child class. (Some treat a module of an application as a unit. This is to be discouraged as there will probably be many individual units within that module.) Unit testing frameworks, drivers, stubs, and mock/ fake objects are used to assist in unit testing.
When doing unit test, to avoid dependency, you should use mock tools. (Java is mockito, Python has mock function in unittest)
IV.Contract test (Level Unit)
You can refer to Martin Fowler page about contract test.
Often used between different teams perform microservices.
Contract test is done by developer.
One team is expecting the other to response certain types of result (could be string, integer, or regex).
We will define the test with mocking data.
Real world example:
I want to go to a new specialist tech shop during lunchtime but it’s along a route I have never walked before and I am worried that I won’t have enough time to get there and back before my 2pm meeting. I need information on how long this journey will take.
Given my current resources, there are 2 approaches I could take - Walk there, timing the walk and double the elapsed time to work out the journey total. Check an online route mapper for the estimates for the route.
One option is a lot faster than the other, has no dependencies and gets me fairly near the answers I want but omits lower level detail (windspeed, gradient) of the journey that I would learn by timing the walk myself.
This is exactly what contract testing does. It mocks or interrogates a response to allow you to get some of the information about functions you require.
V.API test
API is the acronym for Application Programming Interface, which is a software intermediary that allows two applications (services) to talk to each other.
Real world example for better imagination like:
In a restaurant, you are one customer in there.
In order to get the food you want, you would like to call the waiter/waitress.
He will get the information about the food what you want, and then go talk to other cooks in the kitchen.
When the food is ready, you will get the food from the waiter/waitress.
The waiter/waitress is the API I'm talking about.
API Testing is entirely different from GUI Testing and mainly concentrates on the business logic layer of the software architecture. This testing won't concentrate on the look and feel of an application.
Instead of using standard user inputs(keyboard) and outputs, in API Testing, you use software to send calls to the API, get output, and note down the system's response.
API Testing requires an application to interact with API. In order to test an API, you will need to
Use Testing Tool to drive the API
Write your own code to test the API
Some tools for implement API test : cURL, Postman, Jmeter, or writing your own test script call HTTP request to the service.
VI.UI test
We will verify functional of the application directly in GUI.
This tends to be closer to real usage of the customers. But it takes time and hard to verify all the test cases
VII.Manual test
Doing the verification manually.
Take a lot of time and resources to do but almost can do all kinds of testing in here.
Best practice for this is we should only do when the test can't be automated (doing test in search engine, 500 error..)
Top comments (0)