Experiences with BDD

Daniel Pope

Comments

Source

What is BDD?

Behaviour Driven Development (BDD) is a practice where developers collaborate with business stakeholders to develop executable specifications for pieces of development that they are about to start. Test Driven Development (TDD) says that tests should be written before development, but it doesn't say how tests should be written.

BDD builds on TDD by proposing that the first tests should be functional/acceptance tests written in business-oriented language. Using a business-oriented language rather than code allows stakeholders to be involved in verifying that a feature satisfy the business' requirements before work on that feature even commences. You might then do TDD at the unit test level around individual components as you develop.

Gherkin

The tools for BDD have generally come to revolve around Gherkin, a simple structure for natural language specifications.

My favourite description of Gherkin-based tools is given by the Ruby Cucumber website:

  1. Describe behaviour in plain text
  2. Write a step definition
  3. Run Cucumber and watch it fail
  4. Write code to make the step pass
  5. Run Cucumber again and see the step pass

To summarise, a feature might be described in syntax like:

Feature: Fight or flight
    In order to increase the ninja survival rate,
    As a ninja commander
    I want my ninjas to decide whether to take on an
    opponent based on their skill levels

    Scenario: Weaker opponent
        Given the ninja has a third level black-belt
        When attacked by a samurai
        Then the ninja should engage the opponent

You then write code that binds this specification language to a test implementation. Thus the natural language becomes a functional test.

This results in three tiers:

  1. The specification language
  2. The specification language code bindings
  3. The system(s) under test

Python Tools

In Python there are a few tools that implement Gherkin:

Of these I've only had experience with lettuce (we hacked up an internal fork of lettuce with HTML and modified xUnit test output), but outwardly they are similar.

Experiences of implementing BDD

A complaint I've heard a couple of times about BDD as a methodology is that it remains difficult to get the business stakeholders to collaborate in writing or reviewing BDD tests. This was my experience too, though there is a slightly weaker proposition of Specification by Example where the stakeholders are asked just to provide example cases for the developers to turn into tests. This doesn't imply the same bi-directionality and collaboration as BDD.

If you don't get collboration with your stakeholders there are still benefits to be had from BDD techniques if you put yourself in the shoes of the stakeholder and develop the BDD tests you would want to see. It gives you the ability later to step back and see the software at a higher level than as a collection of tested components. You may find this level is easier to reason at, especially for new starters and new team members.

Another complaint is that it seems like it's more work, with the two-step process - first write natural language, then work out how to implement those tests - but in fact, I found it makes it much easier to write tests in the first place. Where in TDD you have to start by thinking what the API looks like, in BDD you start with a simple definition of what you want to see happening. You soon build up a language that completely covers your application domain and the programming work required in creating new tests continues to drop.

Another positive observation is that the three tiers give your tests are protected from inadvertant change as the project developers. While your code might change, and the corresponding specification language code bindings might change, well-written Gherkin features will not need to change. Without using BDD I have encountered situations where functionality was broken because the tests that would have caught it were changed or removed at the same time that the implementation was changed. BDD protects against that.

The natural language syntax is helpful at ensuring that tests are written at a functional level. Writing tests in natural language makes it much more visible when you're getting too much into implementational detail, as you start to require weirdly detailed language and language that the business users would not understand.

Pitfalls

There are a couple of pitfalls that I encountered. One is just that the business stakeholders won't be good at writing tests, and so the challenge of collaborating to develop the BDD tests is hard to solve. Just writing something in natural language isn't enough, you need to get on the path of writing tests that take advantage of existing code bindings and that are eminently testable scenarios.

Another pitfall was that you need to ensure that the lines of natural language really are implemented in the code binding by a piece of code that does what it says. Occasionally I saw code that tested not the desired function, but some proxy: assuming that if x, then y, let's test x, because it's easier to test. You really really need to test y, or the BDD tests will erroneously pass when that assumption breaks.

Comments

Comments powered by Disqus