What are your experiences with "executable specifications" or "living documentation"?

Comments

[Meghan (she/her)]

I like that it promotes the continued development of the technology the specification is meant for, but I can't help thinking it makes implementing a lot harder if they can't discretely track changes and verify to a specific version.

[Bertil Muth]

Can you elaborate a bit or give an example what kind of executable specifications you are referring to?

[Meghan (she/her)]

Many open Web standards managed by some W3C working groups have shifted to the Living Standard model, see WHAT WG compared to a more easily trackable yearly release like the TC39 from ECMA.

[1] spec.whatwg.org/
[2] tc39.github.io/ecma262/
[3] ecma-international.org/publication...

[Bertil Muth]

Interesting. I think you have a different kind of "living documentation" in mind that I have.

I thought about it in terms of documentation generated from software code. I should have defined that.

Anyway: I will have a look at the links. I am working for a standards body as well (the OMG), and I was not aware of the "Living Standard" model. Thank you for providing the links.

[Bertil Muth]

To kickstart the conversation a little bit: one common form of living documentation is called "specification by example". A common language for this is Gherkin.

You write scenarios in natural language, following the template Given When Then. Here's the outline as explained on the Cucumber website:

Scenario: Some determinable business situation
Given some precondition
And some other precondition
When some action by the actor
And some other action
And yet another action
Then some testable outcome is achieved
And something else we can check happens too

So let's make a trivial example for a shopping cart:

Scenario: Add item to empty cart
Given the cart is empty
When the user puts a banana in the cart
Then the cart contains a banana

Scenario: Add item to cart that contains an item
Given the cart contains a melon
When the user puts a pear in the cart
Then the cart contains a melon
  And the cart contains a pear

Of course, these examples are not realistic. I just use them to explain how to use the templates.

Now, why would you write the scenarios? There may be two reasons.

Reason #1: To explore a domain and improve understanding

If you develop software in a domain that is hard to understand for you as a developer, you can sit down with a domain expert. You ask the domain expert for examples, and write them down using the template.
For example: based on example inputs a, b, and c, how is the discount calculated?
The benefits of discussing and writing down the examples is that examples are often easier to understand than abstract requirements.

Reason #2: Acceptance Test Driven Development (ATDD)

Tools like Cucumber or FitNesse allow you to take text files containing the scenarios as an input, process them in test case classes (e.g. via regular expressions), and then execute automated tests based on the scenarios.

What that means is: you define your tests in natural language first. The you write a failing automated test. Then, you implement production code until the test passes. You now know that the scenario works as specified.

That documentation, the text files, can be called "living documentation". Because the tools will tell you based on the scenarios, which tests worked and which not. So you get documentation that by definition is never outdated - in sharp contrast to office documents that decay in deeply nested folders.

That's my short summary.
Do you have any experience with this kind of specification/documentation?
What do you think of it?

[outsource.dev]

BDD is a key item for living documentation but normally BDD describes the behaviours or the system from the user perspective ie in the problem space.

Code is in the solution space, ie actual implementation to meet the needs defined int he problem space.

From a black box perpective the BDD spec typically describes what the black box should do, and the code is inside the box, and needs a white box definition. Hence it is hard to match the 2 together.

But .. that doesn't mean living documentation is a bad thing, but it isn't easy and there is no universal standard or accepted way of doing it. There are a lot of building blocks today but it requires a level of serious intervention to get them all aligned and working.

More detailed explanation of Living documentation at medium.com/geekculture/living-docu...