DEV Community

Cover image for Whatever happened to the test pyramid?

Whatever happened to the test pyramid?

Daniel Irvine πŸ³οΈβ€πŸŒˆ on February 06, 2020

Mike Cohn’s wonderful book Succeeding with Agile talks about the Test Automation Pyramid, shown in the image above. This pyramid is beautifully ex...
Collapse
 
patryktech profile image
Patryk

I don't think there is a one-size-fits-all approach, when it comes to testing.

If you are writing utility functions, those should definitely be covered with unit tests.

If they are useful, you might want to make them a separate library, and not keep them with your web projects, however.

If you are writing a JS framework (Vue, React, Angular, etc.), you should definitely write unit tests to cover it.

If you are leveraging a framework, you shouldn't be writing tests that the framework does what it's supposed to do, so you'll probably end up writing more integration tests.

I highly recommend Kent C. Dodd's *Write tests. Not too many. Mostly integration. blog post / talk.

Modern front-end applications involve user workflow, receiving and sending data through network requests, handling session state and data validation. For applications that involve content creation, there’s plenty of business logic to contend with, too.

All of these lend themselves well to unit testing.

To me, that sounds more like integration tests than unit tests.

Whatever is back-end / API will be tested by unit tests in pytest (as I use Django for my back-ends - maybe I'll eventually take the plunge with something like Node or Express).

I guess you can mock network requests, and call it a unit test... lines get kind of blurry at that point :D

Collapse
 
d_ir profile image
Daniel Irvine πŸ³οΈβ€πŸŒˆ

Thanks for the reply. There’s a lot in this so perhaps I can just pick up a few things:

  1. The distinctions between unit & everything else is possibly more useful than other distinctions. I think the key distinction of β€œunit” is that they are isolated and so are protected from change / breakage. Non-unit tests are more brittle in the sense that because they execute a larger surface area, they are more likely to require change as the codebase changes. This is why we think of unit tests as cheaper.

  2. It’s the guidance of preferring integration tests that I am challenging in this post. I’m suggesting that this advice came about not because unit tests don’t make sense on the front-end, but because they often prove extremely difficult for people to get right. I.e. they end up mocking everything, or writing convoluted tests, or their tests get in the way of refactoring, and so on.

  3. On the point of frameworks -- I exist in the camp of people that believes that we should always ”minimize” frameworks as much as possible. In the case of React that means try to suck as much code out of React components as possible. That way you avoid lock-in and unit testing becomes simpler.

Collapse
 
patryktech profile image
Patryk

It’s the guidance of preferring integration tests that I am challenging in this post. I’m suggesting that this advice came about not because unit tests don’t make sense on the front-end, but because they often prove extremely difficult for people to get right.

They definitely are. To me, the key reason to write tests is because they add value. If I have code like:

<div v-for="foo in foos">{{ foo.bar }}</div>

writing a test with mocked data feels like I am testing that Vue.js isn't broken and that v-for works. That's Vue's responsibility to test, and I won't bother - it doesn't add value to me.

I may write a test that mocks an API call to make sure it renders everything correctly, but again, that, to me, is an integration test.

I'd be curious to see what you consider a good unit test for frameworks. (I'll be sure to look through your Svelte testing series, even though I am not currently interested in learning another framework - maybe I'll find an answer there.)

I exist in the camp of people that believes that we should always ”minimize” frameworks as much as possible.

I can respect that position. As I'm not good at design, and don't care to spend 2 months writing apps that work well on OSX, and iOS, and Safari, and IE11, etc., I personally like frameworks that offer a lot of abstractions, and let me focus on features. I love working with Quasar (based on Vue). But that's a personal choice, and I don't think there is one good answer - we've all got our own strengths, weaknesses, and preferences.

Collapse
 
cescquintero profile image
Francisco Quintero πŸ‡¨πŸ‡΄

I think Unit tests are not suited for modern frontend development. I'm a backend developer and doing Unit testing in RoR is simple and makes all the sense as we have classes and instance methods.

In modern frontend frameworks, it looks more obvious to do integration and snapshot testing and this has been a good experience my coworker has had in one of our projects.

I also think is important more frontend developers start/continue doing testing BUT the testing the helps them do their job and doesn't add trouble to their daily tasks.

Collapse
 
d_ir profile image
Daniel Irvine πŸ³οΈβ€πŸŒˆ

You may be aware of this talk between DHH, Martin Fowler and Kent Beck, which came about as a result of confusion about how to unit test Ruby on Rails.

Frameworks do make it harder to unit test, but it doesn’t make it impossible.

With Ruby on Rails a key example is Active Record. This pattern makes it difficult to write ”unit” tests because it’s hard to test an Active Record model without interacting with a real database resource. For some people this breaks a rule about what a unit test should be. They then replace Active Record with something like the repository pattern. For people like myself, we just shrug our shoulders and use the database, and still call it unit testing. Sure, my tests aren’t as fast as they might otherwise be, but I’m still gaining all the other benefits of unit tests over higher-level tests.

On the front-end... I think your use of the word simple is important. People don’t believe that unit testing UI code is simple. In my opinion, that’s why they don’t do it. And since I know that I’m having a lot of success with unit testing on the front-end, I’m trying to share that knowledge with others.

Collapse
 
cescquintero profile image
Francisco Quintero πŸ‡¨πŸ‡΄

I'm in the group of people who think doing unit testing in frontend isn't simple πŸ˜†

One of the reasons I think people feel that way is that it isn't obvious what to unit test in frontend world. Do we test styles? Do we test structure of HTML? Of course we should be testing JS functions but what about event handlers? Probably we've been doing it wrong.

And even though your article answers those questions, doing unit tests in the way you mentioned isn't as easy as frontend devs would like/hope.

since I know that I’m having a lot of success with unit testing on the front-end, I’m trying to share that knowledge with others

Then I hope to see your content about this topic πŸ˜ƒ

BTW, I wasn't aware of the DHH, Fowler, and Beck video. Thanks for sharing πŸ‘πŸ½

Collapse
 
georgecoldham profile image
George

This is why I have started to love React Testing Library, I am forced to test UI behavioural changes only when testing my React components and pages. It stops a lot of the complexity and lets you test.

Collapse
 
d_ir profile image
Daniel Irvine πŸ³οΈβ€πŸŒˆ

Good call. I’m curious: what were you using before that?

Collapse
 
kwstannard profile image
Kelly Stannard

In my experience the distinctions of the test pyramid are not great given the fractal nature of well designed systems. I find it much easier follow a general guide of writing very few expensive tests and very many cheap tests. That is also more understandable to business people.

Collapse
 
d_ir profile image
Daniel Irvine πŸ³οΈβ€πŸŒˆ

I generally agree with this, especially that the model doesn’t speak to business people.

Models like the test pyramid tend to lose their usefulness once we gain enough practical experience with the subject area. I still think the test pyramid is something we should all learn and understand, if only to get across the point that there are ways to judge the cost of a test before you’ve even written it.

Collapse
 
jackmellis profile image
Jack

Nice article! I think frontend testing has become quite blurred and more than other platforms, the different testing levels have much more specific uses.

There is no point unit testing a simple UI component, for example. A visual regression test (I don't mean a jest snapshot of the html, I'm talking actually capturing the css from the browser) is far more valuable.

For more complex components, I make a point of extracting any logic out and unit testing it in isolation. Then writing higher level integration tests to ensure the component as a whole works as expected.

Something that gets glossed over quite a lot is that react-testing-library is mostly built for integration testing. I don't think this is stated enough. This is why unit testing a component is quite awkward.