I'm a professional PHP, Python and Javascript developer from the UK. I've worked with Django, Laravel, and React, among others. I also maintain a legacy Zend 1 application.
My experience has been that learning BDD is the easiest way to get started writing tests:
It will work with any code base, even one that's too messy to be easily testable otherwise
It tests from an end user's perspective, making it similar to the sort of tests you'd expect to be made manually
Because in most cases it works by driving a web browser, it's easy to grasp the concept behind it, and even if you don't, once you see the browser being automated it usually clicks. That makes it an ideal way to get started writing tests.
Gherkin scenarios are simple enough that they can be understood by non-technical stakeholders without difficulty, making them a convenient way to check with them that the application does what they want before you've gone too far down the rabbit hole of building the wrong thing
They also eradicate repetition in tests - because Gherkin encourages the creation of reusable test steps, there's far less boilerplate to write compared to a traditional xUnit-style test
Personally, Cucumber was my introduction to BDD back in 2012 - at the time I was working on a CodeIgniter application, but I used Cucumber, with the steps written in Ruby. Later I used Behave on a couple of Django apps, but as I'm now predominantly a PHP dev I've migrated over to Behat, but in principle it's just a matter of the language the steps are implemented in. While most of the Gherkin scenarios I've written were driving a web browser, I've actually used Behat very successfully to test a REST API before with Laravel's Browserkit tests.
However, there are some downsides too, which I'll elaborate upon:
Speed
If you're using a web browser to run your scenarios, they will always be a bit on the slow side. Most of mine have tended to take around a minute or two, which isn't that much, but it's long enough for the developer's mind to wander. To practice TDD properly, I've heard it said that no test run should take longer than 10 seconds, and my own experience has borne that out.
Not suitable for testing libraries
Anything without a user interface, such as a library, can't really be tested easily using that sort of BDD approach. I've heard the idea before that BDD can be divided into two separate approaches:
StoryBDD - the approach described above for high-level testing from an end-user's perspective
SpecBDD - the approach taken by tools such as PHPSpec, which are more like traditional xUnit-style testing frameworks
For libraries, I find the SpecBDD approach works really well, but more on that later...
Brittle tests
I've always found that those kinds of automated acceptance tests can be brittle. It can be difficult to set up a test database of some kind, especially if you're working with a legacy application, and it may not always be practical to roll back the changes after the test run. That can make it difficult to ensure the tests work the same every time.
For those reasons, I never use a StoryBDD-style tool such as Behat or Behave as my sole method of testing a code base. I generally save those high-level acceptance tests to be run by my continuous integration server and rely on lower-level functional and unit tests for the most part. In particular, I need my test suite to run quickly enough that my mind doesn't start to wander.
The typical testing strategy I'll aim for (but not necessarily achieve) is the so-called Double-Loop TDD, whereby the high-level acceptance tests are used to keep me working on the particular feature I'm meant to be implementing, and the unit tests are used to design each class as I work on it.
I do find that traditional xUnit-style testing frameworks are rather tiresome in this regard and don't do very well in this role because it's often dull and repetitive having to set up expectations and mock out dependencies for each test. By contrast, SpecBDD tools tend to involve writing far less boilerplate, and make it extremely intuitive to mock out dependencies, making them a better prospect for writing low-level unit tests.
I found using PHPSpec made for a far, far better TDD experience than PHPUnit ever has, and my projects that use PHPSpec generally have better test coverage without me even having to think about it because to create the method I'm used to the idea of creating the spec for it and having the boilerplate be subsequently generated in the class. Also, I'm writing tests by describing what things it should do, and what attributes it should have, which feels more intuitive. With SpecBDD-style tools I generally have a much better experience writing unit tests than with xUnit-style tools.
Low-level unit tests can't be the only testing tool in your toolbox (I've seen too many cases where the unit tests all pass, but the application doesn't actually work because they don't fit together properly), but neither can higher-level tests. I've found the main value of unit tests is less about actually testing the implementation and more about driving a better implementation, and in that respect the SpecBDD approach beats the xUnit approach hands down, at least in my experience. A combination of StoryBDD to drive the overall direction of the project and ensure it's a cohesive whole, and SpecBDD to drive the development of each individual class, makes for quite a good double-loop TDD experience.
I totally agree with you about not relying on low-level or higher-level unit tests... what matters is that the tests check X feature is actually working.
Actually, I do some cheating in some parts that my BDD tests sometimes look like:
Low-level unit test. Since, sometimes I have to check how the function behaves under different scenarios.
E.g: checking create_review function and checking the date (and freezing at a specific point of time): github.com/coretabs/dorm-portal/bl...
Integrated test. Like what I did when I tested uploading photos via API:
My experience has been that learning BDD is the easiest way to get started writing tests:
Personally, Cucumber was my introduction to BDD back in 2012 - at the time I was working on a CodeIgniter application, but I used Cucumber, with the steps written in Ruby. Later I used Behave on a couple of Django apps, but as I'm now predominantly a PHP dev I've migrated over to Behat, but in principle it's just a matter of the language the steps are implemented in. While most of the Gherkin scenarios I've written were driving a web browser, I've actually used Behat very successfully to test a REST API before with Laravel's Browserkit tests.
However, there are some downsides too, which I'll elaborate upon:
Speed
If you're using a web browser to run your scenarios, they will always be a bit on the slow side. Most of mine have tended to take around a minute or two, which isn't that much, but it's long enough for the developer's mind to wander. To practice TDD properly, I've heard it said that no test run should take longer than 10 seconds, and my own experience has borne that out.
Not suitable for testing libraries
Anything without a user interface, such as a library, can't really be tested easily using that sort of BDD approach. I've heard the idea before that BDD can be divided into two separate approaches:
For libraries, I find the SpecBDD approach works really well, but more on that later...
Brittle tests
I've always found that those kinds of automated acceptance tests can be brittle. It can be difficult to set up a test database of some kind, especially if you're working with a legacy application, and it may not always be practical to roll back the changes after the test run. That can make it difficult to ensure the tests work the same every time.
For those reasons, I never use a StoryBDD-style tool such as Behat or Behave as my sole method of testing a code base. I generally save those high-level acceptance tests to be run by my continuous integration server and rely on lower-level functional and unit tests for the most part. In particular, I need my test suite to run quickly enough that my mind doesn't start to wander.
The typical testing strategy I'll aim for (but not necessarily achieve) is the so-called Double-Loop TDD, whereby the high-level acceptance tests are used to keep me working on the particular feature I'm meant to be implementing, and the unit tests are used to design each class as I work on it.
I do find that traditional xUnit-style testing frameworks are rather tiresome in this regard and don't do very well in this role because it's often dull and repetitive having to set up expectations and mock out dependencies for each test. By contrast, SpecBDD tools tend to involve writing far less boilerplate, and make it extremely intuitive to mock out dependencies, making them a better prospect for writing low-level unit tests.
I found using PHPSpec made for a far, far better TDD experience than PHPUnit ever has, and my projects that use PHPSpec generally have better test coverage without me even having to think about it because to create the method I'm used to the idea of creating the spec for it and having the boilerplate be subsequently generated in the class. Also, I'm writing tests by describing what things it should do, and what attributes it should have, which feels more intuitive. With SpecBDD-style tools I generally have a much better experience writing unit tests than with xUnit-style tools.
Low-level unit tests can't be the only testing tool in your toolbox (I've seen too many cases where the unit tests all pass, but the application doesn't actually work because they don't fit together properly), but neither can higher-level tests. I've found the main value of unit tests is less about actually testing the implementation and more about driving a better implementation, and in that respect the SpecBDD approach beats the xUnit approach hands down, at least in my experience. A combination of StoryBDD to drive the overall direction of the project and ensure it's a cohesive whole, and SpecBDD to drive the development of each individual class, makes for quite a good double-loop TDD experience.
Thanks for sharing your take with BDD!
I totally agree with you about not relying on low-level or higher-level unit tests... what matters is that the tests check X feature is actually working.
Actually, I do some cheating in some parts that my BDD tests sometimes look like:
Typical unit test, like checking the (200 OK, 403 Forbidden) responses from my endpoints as in here: github.com/coretabs/dorm-portal/bl...
Low-level unit test. Since, sometimes I have to check how the function behaves under different scenarios.
E.g: checking create_review function and checking the date (and freezing at a specific point of time):
github.com/coretabs/dorm-portal/bl...
Integrated test. Like what I did when I tested uploading photos via API:
github.com/coretabs/dorm-portal/bl...
I used a real photo file cuz I have to check if hitting that endpoint is gonna ACTUALLY upload it in my media folder.