DEV Community

loading...
Cover image for BDD rather than TDD: Result-Oriented Testing

BDD rather than TDD: Result-Oriented Testing

yaser profile image Yaser Al-Najjar ・3 min read

TDD doesn't click, give me the solution

I've talked before about TDD (Test Driven Development), and how you end up testing your framework instead of the logic of your app.

In case you haven't read the previous topic, you can find it here: https://dev.to/0xrumple/when-tdd-doesnt-click-something-else-should-click-harder-2h9h

BDD (Behavior Driven Development) to the rescue

This time, I decided I will do testing no matter what, so I chose an approach that doesn't focus on mocks, fakes, interfaces, dependency injection... but it's all about checking the outcomes, and that's what BDD is for: BEHAVIOR.

Did I try it?

Yes, I didn't allow myself to write this post until we (me and my creative friend @alhakem ) finished this dormitory management system (where students can search/reserve rooms and managers can approve/reject reservations and manage their dorms):

demo

Here is the whole project, "features" folder contain the tests: https://github.com/coretabs/dorm-portal
Feel free to PR or create issues.

Fun facts

I worked on a bit complex filtering engine (where there are multiCheckbox/singleCheckbox/integer filtering criteria), and with BDD I finished it in about one week... all tests green, woohoo!

The whole project was done in about two months, and... all green 😉

How can I do BDD?

It's really simple:

1. Write an acceptance criteria (scenario)

The common way is using Gherkin language, now don't get scared cuz it's really intuitive, just look at this sample (should be self explanatory):

Feature: Reservation

    Scenario: As a student
              I want to make reservations
              So that I get a room to stay in

        Given we have 2 dormitories (and 1 room each available to reserve)
        Given two students who reserved nothing

        When create a reservation
        Then quota of the room should decrease

2. Implement the steps

Depending on your language/framework of choice this might differ; I'm using django (with python of course), and there is a great way of doing it using behave-django package.

Here are the above mentioned acceptance criteria implemented:

from behave import given, when, then
from api.engine.models import RoomCharacteristics
from features.steps.factory import (create_alfam_dovec_with_4_rooms,
                                    create_student,
                                    create_reservation)


@given('we have 2 dormitories (and 1 room each available to reserve)')
def arrange(context):
    create_alfam_dovec_with_4_rooms(context)


@given('two students who reserved nothing')
def arrange(context):
    context.user1 = create_student(context, 'Owen')
    context.user2 = create_student(context, 'Tia')


@when('create a reservation')
def act(context):
    context.previous_quota = context.room1.allowed_quota
    context.reservation1 = create_reservation(context.room1, context.user1)


@then('quota of the room should decrease')
def test(context):
    context.room1 = RoomCharacteristics.objects.get(pk=context.room1.id)
    assert context.room1.allowed_quota == context.previous_quota - 1

3. Make them green

Now comes the part of writing your actual code, and the goal is to pass that quota test.

from django.db import (models as django_models, transaction)
from .exceptions import NoEnoughQuotaException


class Dormitory(django_models.Model):
    name = django_models.CharField(max_length=60)


class RoomCharacteristics(django_models.Model):
    allowed_quota = django_models.PositiveIntegerField(default=0)

    dormitory = django_models.ForeignKey(
        Dormitory, related_name='room_characteristics', on_delete=django_models.CASCADE)

    def decrease_quota(self):
        if self.allowed_quota == 0:
            raise NoEnoughQuotaException()

        self.allowed_quota -= 1


class Reservation(django_models.Model):
    user = django_models.ForeignKey(
        User, related_name='reservations', on_delete=django_models.CASCADE)

    room_characteristics = django_models.ForeignKey(
        RoomCharacteristics, related_name='reservations', on_delete=django_models.CASCADE)

    @classmethod
    def create(cls, *args, **kwargs):
        room_characteristics = kwargs['room_characteristics']
        result = cls(*args, **kwargs)

        with transaction.atomic():
            room_characteristics.decrease_quota()
            result.save()
            room_characteristics.save()

        return result

Seriously?! That sounds tedious!

Nah! believe me, cuz I worked on other projects where adding features or changing one simple thing might break many other parts in your project, it literally becomes like:

drawing
image source: giphy

You will realize that once you add X feature and the customers come to you shouting: "registration doesn't work"... IT SUCKS TO CODE WITHOUT TESTING.

When to do BDD vs TDD?

doing-bdd-tdd

References

  1. Gherkin language: https://docs.cucumber.io/gherkin/reference/
  2. BDD course on Pluralsight: https://www.pluralsight.com/courses/pragmatic-bdd-dotnet
  3. Different types of testing explained: https://dev.to/thejessleigh/different-types-of-testing-explained-1ljo

Discussion (14)

pic
Editor guide
Collapse
yucer profile image
yucer • Edited

When to do BDD vs TDD?

I would better say: How BDD relates to TDD?. Normally vs is used to denote a comparison where both term are opposed and you should choose one.

The problem is that BDD is a kind of TDD. One simple definition is:

TDD : If developers write the tests first, and implement the feature later.

BDD : If developers write the tests first (specified in a language that describes the behaviour of the system as a whole) and implement the feature later.

Something curious is that when I started to use this kind of tests, some years ago, it had the same acronym but a wider concept. It was named Business driven development.

Ohh those were times where software development were more like a Science that an Art ! :-D

The software engineering, the architecture patterns, the methods and practice had a leading role. The object oriented paradigm was designed to divide the complexity inherent of software and make its components flow through the software teams. It was specially useful to develop software of industrial strength whose complexity was specified by requirements of the problem domain (how the business work).

But with the boom of the cloud, social networks, and software as a service did appear a myriad of new possibilities to drive the other complexity with lower costs. That is the one defined by non-functional requirements (storage, location, concurrent execution, data availability). Big companies did provide frameworks and services for them and millennial developers could done more with less.

From my point of view, the software development companies like to be more Agile to lower costs. And that means for many "do it as you can, but fast". Many team leaders are programmers with a big knowledge of the trending frameworks and without a deep understanding software architecture. Methodology is an scary word for many investors that want the software to be done fast and have less respect by the way the software is internally organized.

From old times we inherit the name business scenario. That time where BDD did mean Business Driven Development. Nowadays they name used is Behaviour Driven Development and the focus remain in the software system.

The problem is that the scope is more restricted. Let us say that the term Behaviour Driven Scenario is related to Business Driven Scenario in the same proportion that Software Arquitecture is related to Enterprise Arquitecture.

The software is just a part of the big system that is the enterprise. The software architecture just cover the Applications layer of the following diagram:

Enterprise

...while the Enterprise Architecture covers also the Business Layer, the Technology Layer, etc.

As the Pyramid implies, the changes at the upper layers imply changes of bigger complexity at lower layers.

The concepts at every layer have corresponding concepts at the lower layers. And my opinion is that the corner stone of all is the Business Scenario.

The Business Scenario is defined by the methodologies of Enterprise Architecture like TOGAF.

My opinion is that the Agile people have tried reduced the scope of BDD concept, but there will come a time where the scope will be widened again.

In general, I would say that:

BDD is related to TDD the same way as the Business is to the Software. A wider concept.

What you provide is a test system that describe, in a simple language that the stakeholder can understand, the main source of changes in the application.

So it is a very important contribution. Congratulations!

Collapse
yaser profile image
Yaser Al-Najjar Author

Thanks for the input, I really see things much clearer now.

And, I always felt that BDD is just a kind of TDD done such that:

  1. TDD: focuses more on software problems (which management/business folks don't care about)
  2. BDD: focuses on business problems more than software problems (which all the people care about starting from the end-user till the CEO).

BDD is related to TDD the same way as the Business is to the Software. A wider concept.

This says it all !

Collapse
imthedeveloper profile image
ImTheDeveloper

Interesting post thanks for this. I've naturally been more drawn to bdd but I always find there is a wealth of information for TDD along with testing frameworks to support with little information associated with the bdd counterpart.

Collapse
yucer profile image
yucer

That's because TDD is just an approach that can be implemented with almost all the existing test frameworks that the developers use.

While the software developer is more familiar with the application layer and Software Arquitecture in the business layer the stakeholders are the experts, they are normally Business Analysts and are familiarized with the Enterprise Architecture.

They might think that the automated tests is something out of their scope. At most they try to do manual tests, because programming tests in one of the existing frameworks requires knowledge of ... programming.

Have you heard about the impedance that exist between developers and stakeholders ? E.g.: between the programmers and the the future users of the systems ?

That impedance is given by the fact that one has lower expertise in the domain of the other. The developer normally knows about technologies that can be applied in many domains. The users are expert of the domain, but they usually don't know the details of how technology works, or the whole spectrum of advantages that it can provide. (check the book Object Oriented Analysis and Design with Applications from Grady Booch for more on this)

The same existence of the new software changes this correlation. Even from its prototype phase. The developer knows more about the problem domain and the users starts to realize of how much advantage the technology provide to their business.

Normally the stakeholder represents the communication channel between both. They negotiate the requirements with both parts and estimate the costs. They are normally experts to some limited degree of both fields. Usually both parts depend on them for the success of the project, and this provides big part of the economic benefits.

I think BDD provides more benefit here to the stakeholder, they can realize immediately when the implementation of one scenario conflicts with the implementation of the other. The sooner this is detected is better in order to optimize the use of the resources.

For everybody is better that the functional requirements are cleared stated sooner, but the stakeholder has most of the benefits because they have a possible artifact to validate their work...

... but they don't know normally how to program the tests. That's why you see the BDD tests less popular.

And also, because programmers are usually lazy to map all the concepts of the business notations to the BDD test language.

Collapse
yaser profile image
Yaser Al-Najjar Author • Edited

Programmers are also lazy to write scenarios for everything.

Actually, many developers I know prefer to write their unit tests as thin as possible (instead of seeding the db with data per scenario to cover all aspects in the scenario).

I believe their approach sucks eventually since they still cover things from the tech perspective more than the business side.

Collapse
yaser profile image
Yaser Al-Najjar Author • Edited

I agree with you.

TDD is so popular that every framework vendor tries to provide tooling to do it... though TDD might not be a good option for your project 🙄

Collapse
yucer profile image
yucer • Edited

With the time I have seen as a trend to use the first person to describe the interactions with the system (described by verbs).

So instead of:

When create a reservation

I use:

When I create a reservation

So all of them are described as my actions and the response of the system.

That makes the design of your language simpler.

Strictly speaking, in the Business Layer all the Business Functions are performed by Business Actors who play a Business Role. Look at this:

Business Layer

They would be Users and Groups in the Software Application that you are testing.

That's way I normally implement a login step:

Given I am logged in as an Student
 When I create a reservation

I try always to start all the scenarios with a login statement so that the Business Role can be easily identified.

Collapse
yaser profile image
Yaser Al-Najjar Author • Edited

Though I try to make my sentence as compact as possible since the result would affect the console display as in the picture below (some text gets cropped), I like the idea of adding the subject pronoun (would definitely make it more readable).

cmder-python-behave

Collapse
matthewbdaly profile image
Matthew Daly

My experience has been that learning BDD is the easiest way to get started writing tests:

  • It will work with any code base, even one that's too messy to be easily testable otherwise
  • It tests from an end user's perspective, making it similar to the sort of tests you'd expect to be made manually
  • Because in most cases it works by driving a web browser, it's easy to grasp the concept behind it, and even if you don't, once you see the browser being automated it usually clicks. That makes it an ideal way to get started writing tests.
  • Gherkin scenarios are simple enough that they can be understood by non-technical stakeholders without difficulty, making them a convenient way to check with them that the application does what they want before you've gone too far down the rabbit hole of building the wrong thing
  • They also eradicate repetition in tests - because Gherkin encourages the creation of reusable test steps, there's far less boilerplate to write compared to a traditional xUnit-style test

Personally, Cucumber was my introduction to BDD back in 2012 - at the time I was working on a CodeIgniter application, but I used Cucumber, with the steps written in Ruby. Later I used Behave on a couple of Django apps, but as I'm now predominantly a PHP dev I've migrated over to Behat, but in principle it's just a matter of the language the steps are implemented in. While most of the Gherkin scenarios I've written were driving a web browser, I've actually used Behat very successfully to test a REST API before with Laravel's Browserkit tests.

However, there are some downsides too, which I'll elaborate upon:

Speed

If you're using a web browser to run your scenarios, they will always be a bit on the slow side. Most of mine have tended to take around a minute or two, which isn't that much, but it's long enough for the developer's mind to wander. To practice TDD properly, I've heard it said that no test run should take longer than 10 seconds, and my own experience has borne that out.

Not suitable for testing libraries

Anything without a user interface, such as a library, can't really be tested easily using that sort of BDD approach. I've heard the idea before that BDD can be divided into two separate approaches:

  • StoryBDD - the approach described above for high-level testing from an end-user's perspective
  • SpecBDD - the approach taken by tools such as PHPSpec, which are more like traditional xUnit-style testing frameworks

For libraries, I find the SpecBDD approach works really well, but more on that later...

Brittle tests

I've always found that those kinds of automated acceptance tests can be brittle. It can be difficult to set up a test database of some kind, especially if you're working with a legacy application, and it may not always be practical to roll back the changes after the test run. That can make it difficult to ensure the tests work the same every time.

For those reasons, I never use a StoryBDD-style tool such as Behat or Behave as my sole method of testing a code base. I generally save those high-level acceptance tests to be run by my continuous integration server and rely on lower-level functional and unit tests for the most part. In particular, I need my test suite to run quickly enough that my mind doesn't start to wander.

The typical testing strategy I'll aim for (but not necessarily achieve) is the so-called Double-Loop TDD, whereby the high-level acceptance tests are used to keep me working on the particular feature I'm meant to be implementing, and the unit tests are used to design each class as I work on it.

I do find that traditional xUnit-style testing frameworks are rather tiresome in this regard and don't do very well in this role because it's often dull and repetitive having to set up expectations and mock out dependencies for each test. By contrast, SpecBDD tools tend to involve writing far less boilerplate, and make it extremely intuitive to mock out dependencies, making them a better prospect for writing low-level unit tests.

I found using PHPSpec made for a far, far better TDD experience than PHPUnit ever has, and my projects that use PHPSpec generally have better test coverage without me even having to think about it because to create the method I'm used to the idea of creating the spec for it and having the boilerplate be subsequently generated in the class. Also, I'm writing tests by describing what things it should do, and what attributes it should have, which feels more intuitive. With SpecBDD-style tools I generally have a much better experience writing unit tests than with xUnit-style tools.

Low-level unit tests can't be the only testing tool in your toolbox (I've seen too many cases where the unit tests all pass, but the application doesn't actually work because they don't fit together properly), but neither can higher-level tests. I've found the main value of unit tests is less about actually testing the implementation and more about driving a better implementation, and in that respect the SpecBDD approach beats the xUnit approach hands down, at least in my experience. A combination of StoryBDD to drive the overall direction of the project and ensure it's a cohesive whole, and SpecBDD to drive the development of each individual class, makes for quite a good double-loop TDD experience.

Collapse
yaser profile image
Yaser Al-Najjar Author • Edited

Thanks for sharing your take with BDD!

I totally agree with you about not relying on low-level or higher-level unit tests... what matters is that the tests check X feature is actually working.

Actually, I do some cheating in some parts that my BDD tests sometimes look like:

  1. Typical unit test, like checking the (200 OK, 403 Forbidden) responses from my endpoints as in here: github.com/coretabs/dorm-portal/bl...

  2. Low-level unit test. Since, sometimes I have to check how the function behaves under different scenarios.
    E.g: checking create_review function and checking the date (and freezing at a specific point of time):
    github.com/coretabs/dorm-portal/bl...

  3. Integrated test. Like what I did when I tested uploading photos via API:

github.com/coretabs/dorm-portal/bl...

I used a real photo file cuz I have to check if hitting that endpoint is gonna ACTUALLY upload it in my media folder.

Collapse
rembou1 profile image
remi bourgarel

Hi very good article, I also wrote something about it a while ago remibou.github.io/Moving-out-of-un...

Collapse
yaser profile image
Yaser Al-Najjar Author

Thank you.. your article is great too, have you tried nSpec?

Collapse
rembou1 profile image
remi bourgarel

No I have nit, I didn't know about it in fact. I'll check it out

Thread Thread
williampattrix profile image
williampattrix

TDD is so popular that every framework vendor tries to provide tooling to do it... though TDD might not be a good option for your project Result