DEV Community

loading...
Cover image for Software tests as a documentation tool

Software tests as a documentation tool

noriste profile image Stefano Magni ・4 min read

Why tests are perfect to tell a story of your code

I'm working on a big UI Testing Best Practices project on GitHub, I share this post to spread it and have direct feedback.


Documenting is generally hard, it requires precise and meticulous work and it requires that all the members of the team understand and value writing good documentation. Documenting is selfless work and for selfless, I mean that it is dedicated to other developers and the future we.

I have ever been convinced that testing methodologies are a great way not only to be sure that we code what the project needs, not only to grant we do not introduce regressions but to document the code and the user flows too (please, remember who is writing is a front-end developer). And recently my test-as-documentation idea received one more confirmation: I moved to a new company and I had very little time to tell a new developer what I did until now.

Handover the project to a new developer

First of all, take a look at:

  • the architecture of the back-office I was working on, developed with React and tested with both Cypress and Jest

The architecture of the [Conio](https://business.conio.com/) Backoffice.The architecture of the Conio Backoffice.

  • a video of some of the user flows

  • a screenshot of some of the Jest tests

Some integration tests of the [Conio](https://business.conio.com/) Backoffice. Please note that, at the time, I broke a golden testing rule: no tests should share the state.Some integration tests of the Conio Backoffice. Please note that, at the time, I broke a golden testing rule: no tests should share the state.

The steps of telling what –and how– the project does were:

  • telling some tech details about the project and the whole architecture

  • asking the new developer to explore the Storybook stories

  • asking the new developer to run and watch the Cypress tests

  • asking the new developer to run and read the output of the Jest tests

  • asking the new developer to read the code of the tests

  • measuring the new developer understanding of the project asking him to change and test some flows

The handover went so well that I decided to write this post 😊. The new developer (a quite smart one, I have to admit, hi Lorenzo 👋) showed me to know pretty well the user flows and to be able to change big parts of the project with confidence (and the related tests too). Nevertheless, he was able to implement new user flows and test them the same way I did with all the rest of the project.

The good and the bad parts

Well, even if I am pretty satisfied with this experience, I must admit that something was not perfect. I am not speaking about the process itself or the idea of using the tests as a documentation tool, but the way I wrote the tests. There were some details that misled the comprehension of the developer that should read my tests. Let’s examine a quick list of the pros and cons of the tests I wrote:

What was good with my tests:

  • Testing everything from the UI perspective with Cypress: for a front-end flow, the UI speaks more than everything

  • Having a well-compiled Storybook: writing stories in Storybook is already testing! It is visual testing that can be easily frozen with a plugin like Storyshots

  • Straightforward tests code: the code of the tests must be super simple. Simple to be read, condition-free, with a low-abstraction level, with a good level of logging, etc. Always remember that the tests must reduce the cognitive load of reading and understanding the code, hence their complexity should be an order of magnitude lower compared to the code to be understood

  • Sharing some step “ids” between the code and the tests: if a user flow is quite long, it could be useful to share some “steps” between the code and the code of the tests (mines were comments like “/** #1 /”, “/* #2 */”, etc.)

  • Having more low-level tests for parts of the code (like some sagas, as shown in the above screenshot) that could be hard to be understood (and hence hard to be updated or refactored to a simpler version)

What was bad with my tests:

  • Some test descriptions were not perfect: good storytelling skills are pretty important while writing the description of the tests

  • Not leveraging Gherkins to write the tests themselves: I was not so experienced at the beginning and I decide to not consider writing BDD-style tests with Gherkins. Take a look at it to understand the storytelling advantages

  • Sharing some fixtures between different tests: the fixtures are the static version of the responses of the server, the idea of recycling them when they are identical has nothing wrong, but I should have cared more about their names. Using a “registration-success.json” fixture for both the registration and the login flows (it is just an example, I did this mistake in more complex cases) leave some doubts to the new developer. This is the kind of things that are frozen in the memory of the developer who wrote the code (why you can use the same fixture for two different cases?), a really bad thing from the company perspective

In the end, writing tests allows you to:

  • have well-descriptive documentation: the description of the tests are always written from the user perspective, not from the developer one

  • have easy handover

  • avoid relying on the historical memory of some employees (you, for example)

  • document some choices that could sound “strange”, or simply complex, when reading the code, but perfectly reasonable from the user perspective

not to cite the obvious testing advantages like working regression-free, leveraging some automated and fast tools, etc. 😊

I am eager to learn your experiences on the same topic! Feel free to leave a comment about them 🤗

Discussion

pic
Editor guide
Collapse
evanplaice profile image
Evan Plaice

I agree 100%

My 3 general rules for digging into an unfamiliar codebase are:

  1. Documentation is the best documentation except when it's incomplete and/or out-of-date.

  2. The test suite is the next best thing. In some rare cases, a high quality test suite can be even better than the docs.

  3. When all else fails, start digging into the history and become a code archaeologist.

Collapse
wernercuristul profile image
Info Comment marked as low quality/non-constructive by the community. View code of conduct
werner-curistul

I'm sorry, but this post is a joke, right?

You're trying to define some UI Testing Best Practices just to promote that Cypress tool.

Here's why I wouldn't use Cypress:

  1. It claims to be Open Source, but that's just a honeypot trap for developers. A company who would use it would need to buy the Cypress Dashboard Service, which costs money. A real Open Source project is developed and supported by a community, not by some company who pours money into Marketing (mostly bad Marketing).
  2. You cannot test file uploads. No browser will allow you to write a local file path with JS.
  3. You have difficulties with iframes.
  4. You cannot control the order in which the test cases are executed.
  5. It only supports Chrome and Firefox partially. Real users are also using other browsers.
  6. You keep trashing Selenium, which is a good example of Open Source. Selenium has helped a lot of developers and companies build stable automated tests. Tests with Selenium are not brittle, as you and that awful Lisa Crispin person claim.

You are disgrace to the Open Source community.

You should be ashamed of yourself.

Some users will want to use open source Selenium, others will choose proprietary solutions such as Endtest or others.

I don't have anything against those solutions.

But at least those solutions are not trying to trick developers like you're trying.

Cypress should be banned from dev.to

Collapse
evanplaice profile image
Evan Plaice

Cypress is free for Open Source use, not commercial. Dual licensing for commercial use is 100% acceptable.

The Author's statements about Selenium are true. It was the best tool when it was the only tool for browser automation. Nowadays, better alternatives (ex Puppeteer, Cypress) exist.

If you don't like it, to each their own. That's no excuse to insult the Author or spread misinformation.

Source: Jaunary, 2020 marks 10 years I've been contributing to OSS.

Collapse
noriste profile image
Stefano Magni Author

Even the most negative comment can be useful, and so yours is. You made me think that I should be more clear with the goals of my "UI Testing Best Practices" repository. I try to clarify some points:

  • First of all: is Cypress involved in the "UI Testing Best Practices" project?
    No, trust me

  • Did Cypress push me somewhat to promote their product with my articles and my repository?
    Again: no

  • And so, why you do it?
    Because I'm an enthusiast Cypress user, I love UI Testing and when I deepened the topic the first time I realized there are a lot of blog posts and resources but it's not an easy road, that's why I started writing the "UI Testing Best Practices" project

  • What about the Cypress Ambassador program?
    After some talks promoting Cypress they contacted me, the program allows me and other enthusiast developers to be in contact with the Cypress team and offer better support to our own followers

  • Why you created a "UI Testing Best Practices" repo and you speak only about Cypress?
    Because it's the tool that I know better! But my goal is not to limit it to Cypress. Take a look at the draft of the contents to be written (github.com/NoriSte/ui-testing-best...), I have written that experts from other tools are welcomed! I do not know everything about every tool, learning is a goal of my repo, too!
    You can take a look at one of the chapters I wrote
    github.com/NoriSte/ui-testing-best...
    and the suggestion I received from a TestCafè user
    medium.com/@artjomlav/hi-stefano-m...

  • Cypress is not open source!
    Cypress is open source, MIT license, they've created an amazing tool that works perfectly without the Dashboard (I have not used the Dashboard until they allowed me to do so for free)

  • Cypress is a bad tool!
    I do not think so but no problem if I and you think different things. You have highlighted some negative aspects of Cypress (it is not perfect, I know that it has limitations) but you have not listed the pros and you have not listed the Selenium' cons. I think that Cypress makes your testing life easier soon and often. Creating solid tests with Selenium requires to be more experience. When I was testing with Puppeteer, for example, 70%of the best practices I used were to "workaround" the limitation of the tool. Most of them are useless with Cypress. And debugging the front-end app under test is not so easy with Selenium.

BTW: you made me realize that I must clarify and publish the reasons why I started writing the repo, it would be clear for every user. I hope to convince other non-Cypress developers to help me writing the book, instead of making them angry because of my inner positive bias for Cypress.

Collapse
hyftar profile image
Simon Landry

Pardon my inexperience, but where does it say a company would need to buy Cypress' Dashboard Service? The test runner is under MIT license and is fully open source, of course their service is interesting for a company, but it's no-where near required.

As for point #4, that's also true for a lot of testing engines, tests should not be dependant on the order they're ran in in order to work so I don't see why that matters.

Don't you think you're being harsh by saying he's a disgrace and should be ashamed of himself? Sure you disagree with him on a lot of things, but I don't think that's a reason to be ashamed nor one call someone awful or a disgrace.