Disclaimer: I don't mean to convey any negative messages towards any individual or organisation. Treat this post as an opinions, and my opinion wouldn't change the world. There maybe some subjects or contents I have misunderstood due to language barrier so feel free to enlighten me.
Introduction
As a self learner with no mentorship I treat internet as my mentor, and I have preached numerous times that you don't need a course or pay for any books because that information is available on the internet in some form. However the lightning struck when I stepped into the confusing lair of google seo to teach me software testing.
I was not willing to learn it through courses or books so searched google for any resources. I found two blog series
- Wikipedia
- Browserstack guides on testing
There were few more, but they were completely off track from the core software testing topic so I didn't read them.
Testing type, a total mess
Browserstack and Wikipedia both confused me a lot, during learning I mostly had to piece out what I think was relevant and assume the rest of it as outdated of false.
The first I got confused reading about software testing was Browserstack functional testing guide.
The functional testing guide named as Functional Testing : A Detailed Guide teaches about functional testing where Unit testing is mentioned as a part of functional testing as mentioned in their diagram just below Types of Functional Testing.
As mentioned in the guide, Unit Testing is mentioned as a category of functional testing.
The link here references another guide named as Unit Testing: A Detailed Guide
When you reach to the Unit Testing Techniques section, you'll encounter that functional testing which links us back to the Functional Testing Guide is mentioned as a technique of Unit testing but we just read that Unit testing is a type of functional testing.
What adds more to the confusion is the TDD section under Unit Testing guide, the Read More section links an article named as Differences between Functional Testing and Unit Testing.
I was already trying to make a sense of this cyclic reference of Unit and Functional Test, and now there's an article that writes about the differences between those.
There were similar articles other than Browserstack's that had the same cyclic reference problem.
The Jargon Problem
It kills my soul when I see a veteran dev or a tester who happen to read about all the jargon ask the newbies to tell differences between tests like Smoke and Sanity tests and carries on a sly smirk.
While working in some orgs, I have realised terms like Sanity doesn't mean anything and testers and software developers love to throw around this word without any context or meaning.
Software testing terms don't provide any meaningful context from the name, some of them are not even native to Software Testing and hence lack the meaning.
- Smoke Testing
- Sanity Testing
- Regression Testing
- Structural Testing
- Big Bang Testing (what the hell does it even mean!)
The list goes on as you delve more into the types of tests.
Software Development in today's standard is also a responsibility of the developer and not just the tester, however the terms used in both disciplines are the same but the whole function and pipeline of the test differ.
Sanity Vs Smoke Tests
If you will search about sanity test and smoke test individually, you'll find their definition same. Wikipedia refers Sanity Tests as a alias of Smoke Tests in their wiki Smoke Testing.
Luckily after browsing a lot a common pattern appeared which was the difference on how these tests are done and when they are done.
Smoke tests can be done using automation and are performed before they were handed over to the QA, which I imply as a developer running their suite of tests before handing their app to QA. Whereas Sanity Tests are performed after the developer hands over their build to QA.
In other words, smoke tests are performed by developers after they complete their feature or before handing out the final build, and sanity tests are performed by tester after the developer hands out the build of the application.
Smoke tests and sanity tests by definition are the same, both are confidence tests but the key difference lies in the stage of development pipeline.
Conclusion
I would like to conclude this blog by saying Software Curriculum on web is a mess apart from the weird guides the theoretical content regarding Software Testing is highly outdated and severely lacks in abstracting its core concepts while completely ignoring the modern software development value stream and we need new and better structure to understand software testing.
Top comments (1)
"Smoke tests can be done using automation and are performed before they were handed over to the QA, which I imply as a developer running their suite of tests before handing their app to QA. Whereas Sanity Tests are performed after the developer hands over their build to QA.
In other words, smoke tests are performed by developers after they complete their feature or before handing out the final build, and sanity tests are performed by tester after the developer hands out the build of the application."
I didn't know that! Usually, I just write tests with my favorite automatic test runner and just pass them off to QA. Unfortunately, QA doesn't typically write code, so I have doubts as to whether they will understand what they're looking at sometimes. It's happened before. One time I was updating a simple UI date field in jQuery because a QA thought there was a bug in the form. The line in question was essentially:
<input value='DateTime.now()'and the issue was that hisDateTime.now()was different from myDateTime.now()... because of his browser!Why does this matter? Well, I think it relates to your article for two reasons: 1) weird shit happens and no amount of specialized terminologies or 'Agile best practices' will stop a bug and 2) QA won't always be helped by the terminologies you've outlined, which are already confusing as you've laid out. Hell, I don't know half of those and I've been in software for 7 years!
Opinion
There's also a 3rd reason, at least for me personally. I hardly ever write Unit Tests (or any kind of test, TDD or not, or whatever term we want). And when I do, I tend to write them as 'punchy'. They must solve or prove a specific scenario, like a combination of things we want to do, like "check for old orders, update orders with new info, then verify calculations are correct", not "GetOrderById". Passed (who would've thunk it?), "UpdateOrder". Pass. ZZZ... "CalculateOrderTest"... No, it has to be something complex enough to FAIL. Or maybe we should write tests so that they fail, or scenarios where we'd expect them to fail. Then maybe we won't need 5 or more different TDD terminologies to say we tested our software thoroughly.