DEV Community: Fred Heath

Agile Myths Exploded

Fred Heath — Fri, 18 Sep 2020 09:51:06 +0000

One of the few good things about the current pandemic is that it forced people to rethink some of the most rigid 'agile' practices, such as:

Co-location

It turns out that development teams can function perfectly well in an agile environment without being physically located in the same space. We kind of suspected that due to the pre-pandemic success of many remote-working companies, such as Basecamp, Zapier, Moleskine and many others. But now most of us found out that -with a few workflow adjustments- any team can be productive when working remotely. Not only that, but the lack of commuting and workplace distractions means that many people can be more productive away from the office rather than in it. It also means that we can include people who would find it hard to commute to an office every day, such as disabled ppl or ppl with parenting/caring commitments.

Stand-ups

These were short meetings where attendants would be literally standing up. The idea was that this would encourage short and focused meetings. The end of co-location also meant the end of stand-ups. People discovered that they could have short, focused meetings on Zoom or Skype. Turns out, the duration and focus of meetings depends on the attitude and culture of the participants and organizers, not on everyone standing up together in a room. Who would have thought?

"No documentation needed", a.k.a "You just need to have a conversation"

This is one of the most abused notions in agile. One of the Agile principles is that

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

This is absolutely true. Sure, you may need to repeat the same thing to different people many times. And, yes, some information gets lost through verbal transmission, as the 'telephone / chinese whispers' game demonstrates. Also, some ppl are much more receptive to visual info, rather than verbal.

But, overall, talking to ppl is the best way to convey information. However, it should never be the only way. The Agile Manifesto doesn't state that we shouldn't be documenting things. Just that we prefer working software over comprehensive documentation.

You have to realise that at the time the manifesto was written, working code was the last thing a software team was meant to produce. First it needed to deliver documentation: SDDs, STDs, SRSs and other horrible acronyms, were some of the bureaucratic documents we had to deliver before even stating to think about coding (I know, I used to do this, I'm that old!). An analyst would write a requirements document and then hand it off to the development team and forget about it. The agile founders said: "This is stupid and counter-productive. Stop it!". For that, I am eternally grateful. But they never said to not document what we do.

The pandemic exposed the weaknesses on relying on conversations and omitting documentation. Before, if we needed some information we could just turn round and ask the person at the next desk. It was easy to just shout out something across the office so that everyone would hear. It is no longer that easy. The immediacy of response is lost. Now, putting a message on the team slack channel does not mean you'll get back an immediate response or even acknowledgement. Also, talking about something is much quicker than writing. If I have to write a response to the same question again and again, I may as well paste that response in a document and link to it.

People have to rely on documents and diagrams much more now, to ensure that information is conveyed reliably and accurately. This is how it should be. Documentation is persistent in way that verbal communication can never be. It is more resistant to corruption. It also helps clarify or enlighten in ways that verbal communication can't (try explaining complex/parallel workflows without sequence diagrams, for instance).

Conversations should form the core of information flow, but they must always be supplemented by proper documentation.

"UML is not agile/too elaborate/old-fashioned/...etc"

A side-effect of the "no documentation" fallacy was this myth. UML was a 'waterfall' thing. Only old ppl use UML. It's not needed in agile world.

The thing is, UML is just a notation. Yes, it's been used as the basis of rigid and complex methodologies. But that's not UML's fault. One of the pandemic consequences of ppl starting to use more diagrams in order to convey information (see previous section), was that the lack of common standards became exposed. What does that dotted line in your diagram mean? How is that rectangle different to that oval shape? Does that thing represent a class or a database table? If only we had a universal notation so that we can all express our designs in the same way. If only we had a ubiquitous visual 'language' understood by all.

....hold on. We do! It's called UML. It helps us communicate our thoughts visually and uniformly.Good communication is critical to an agile workflow, so UML is too, as many more people are beginning to realise.

Epilogue

Often, certain practices may not make sense to us but we are reluctant to call them out because of peer pressure and cultural acquisition. Sometimes it takes a universal and external event to expose the weaknesses or irrelevance of these practices. The pandemic is making us rethink much of the way we work and live our lives. But despite the many negative effects, it can also help us change some things for the better. Stay safe and keep productive everyone!

Managing Software Requirements the Agile Way

Fred Heath — Wed, 19 Aug 2020 10:39:51 +0000

My book, titled as above, has now been published. One of the reasons I'm writing about it here was that the dev.to community played a part in my book becoming a reality. Allow me to explain the how and why.

The problem

I've been working in software engineering for well over two decades. In the last 10 years or so, I've been working with agile processes and frameworks, such as Scrum, Scrumban and others. One of the things that always struck me with agile implementations was how lackadaisical their approach to capturing and managing requirements was. I went from a bureaucratic, documentation-laden, analysis-paralysis, waterfall cycle to a "let's write a user-story and have a chat about it", 'agile' approach. At first this seemed like a really cool, laid-back way of doing things. Over the years though, I started to realise how inefficient and ineffective this approach was.

User Stories as the foundation of a development and delivery cycle are dubious, for reasons too numerous to mention here (I describe them in the book). One of their many problems is that they don't help provide a Specification, that is a way of knowing how our software will realise the users' requirements. At best, a user story will have some acceptance criteria that the developers will use to verify their code output. However, the output is not the outcome. Just because our code produces an expected result does not mean we satisfied the user's need, nor that we helped them realise their goal. To achieve that we need to work the other way round: identify the stakeholders' goals, then the impact the stakeholders need to have on our system in pursuit of their goals and finally the system functionality which will realise that impact.

The light at the end of the tunnel (no it wasn't a train)

I found Behavior Driven Development (BDD) to be a great way to create specifications that can also be verified against the deployed software. BDD is centred around Features, so the big question became "How do I reliably create or derive valid Features based on the requirements?" This puzzled me for a long time until I came across Impact Mapping. Impact Mapping allowed me to answer the big question of What, Why, How and for Whom are we doing what we're doing (spoiler: the What part is our Features). It also provided a great way to bridge the gap between Requirements and Specifications.

After years of trial and error, I finally managed to refine a working methodology for capturing and modelling requirements, creating executable specifications for these requirements and delivering them within an agile process or method like Scrum or Kanban. So for the last couple of years I'd been thinking of putting all this knowledge down in a book but I never felt bold enough to go for it. Until last year.

The Post

In April 2019 I posted my thoughts on the user-story chaos in an article titled Enough with the User Stories already! It became my most read post here on dev.to. It generated many comments and I also received a few emails creating conversation on some of the points in the article. It triggered my decision to write what I know and practice in a book, and set it free out in the wild.

The Book

So here comes the self-advertising part :) My book contains an end-to-end methodology for building the right software. It shows techniques, methods and tips for:

Eliciting requirements from your Stakeholders
Modelling requirements with Impact Mapping
Defining behaviour in Features
Writing verification code for your Features
Creating a feature-based product backlog
Deliver the right software within Scrum or Kanban

Along some established techniques like BDD and Impact Mapping, I also demonstrate some home-grown ones like parsing requirements with D3 and software delivery with Feature-first development. The book won't show you how to create software correctly, but I'm confident it will help you create and deliver the correct software ;)

The book is out on Packt Pub and Amazon. Hope you find it useful and thanks to everyone who's contributed to the user-story and requirements discussion.

Career advice for junior developers

Fred Heath — Tue, 14 Apr 2020 10:37:40 +0000

I had a nice chat this morning with our new intern, who's due to start working with us this summer. One of the things he asked me was if I could give him any tips about things he should be looking to learn in order to advance his career in programming. In my mind, I started formulating a list of programming languages, web frameworks, libraries and technologies. I was then struck by the thought that most things on my list would be obsolete in 5-10 years. I considered whether knowing the intricacies of a specific language, or how a framework is put together or how the Model-View-Controller paradigm works, was a solid foundation for a successful career in Software Development. The clear answer to my own question was that no, all these things are transient. They are here today, gone tomorrow.
An old proverb sprang to mind: "You give someone a fish, they'll eat for a day. Teach them to fish and they'll be eating for a lifetime".

So this is what I told him: "Learn whatever catches your eye, excites you and you feel comfortable with. But whatever you do, keep in mind the following":

1. Enjoy what you do

You'll be spending one of the biggest parts of your life working (the other big part is sleeping but we can't help that). If the tools or people you're working with frustrate, impede or depress you, then change them. Sometimes, this may require changing jobs. That's fine, you have no obligation to suffer in exchange for money or out of some mis-placed sense of loyalty. By enjoying what you do at work, you're going to be better at it and more productive. Enjoying being round your colleagues will make you more collaborative and help you learn new things. Most importantly, by working in an enjoyable environment you will also be safeguarding your mental health, this most precious of goods.

2. Coding isn't enough

To be a good developer you need to be able to code. To be a great developer you need to be able to:

Communicate your assumptions and expectations to others.
Empathise with others and understand their point of view.
Plan ahead with forethought and discretion.
Manage other people's expectations of you.

Unfortunately, there aren't any boot camps for these skills. These are skills you need to hone and cultivate yourself. They take time and effort and not everyone manages to accomplish all of them. But mastering these skills can make all the difference between yet another coder and an effective software developer.

3. You read more code than you write

You may not think so, but, trust me, you do. So what does this mean? It means that when you write code you must keep in mind that someone else (or even worse, you 12 months later) will have to read it. So make sure your code is readable, well-documented and trimmed down to what is needed to deliver the intended piece of functionality. Ensure your codebase is not bloated with clever code hacks or complex design patterns and architectures which only exist to inflate people's egos or adhere to some dogmatic pattern or methodology. Keep your code simple, modular and readable. Your colleagues and future self will thank you for it.

4. Separate the things that change from the things that stay the same

Everything changes over time, but there are things that change at predictable occasions or at regular frequency. Make sure these things are separated from stuff that doesn't often change. So keep your UI code well away from your business-logic code. Keep your data-reading code away from your data-formatting code. If you are dealing with external APIs, hide them behind an adaptor or facade. The APIs will be changing a lot more frequently than your internal usage of them, i.e. your facade. Apply this principle throughout your code and designs.

5. Develop iteratively and incrementally

Never try to build a complete and perfect system in one go. Focus on the most needed and clear features first and provide the simplest working solution to them. Then build-up your project over many iterations, each iteration refining existing features and delivering new ones.

If you're not sure what an iterative and incremental approach is, then this should help clarify it.

6. Premature optimisation kills projects

Do not try to anticipate future needs without solid evidence that these needs are likely to arise. If your web-site ever needs to handle 10 zillion requests per minute then don't worry, at that stage you'll be making enough money to comfortably retire at your own private island, so focus on that instead ;) .The point is, do not optimise pre-maturely, 'just in case'. Do not work on the basis of 'what-ifs'. If you do, you'll only add unnecessary complexity in your code, increase the likelihood of bugs and reduce its readability. On that note, prefer languages and frameworks that promote flexibility and ease-of-change.

7. No Silver Bullet

To quote Mr Brooks

There is no single development, in either technology or management technique, which by itself promises even one order of magnitude [tenfold] improvement within a decade in productivity, in reliability, in simplicity.

Over your career you'll see many 'hotnesses' come and go. You'll hear people raving on about the fastest web framework, the coolest design paradigm, the best programming language, and so on. All these things may be true within the narrow slice of reality of the people clamouring for them, but they are not necessarily true in your reality. Complex problems, like the ones we face in software development, are never solved by simple solutions, like using a single programming language or web framework. Never be fooled by thinking you found that one thing which will solve all your programming problems.

8. Generalise, don't specialise

In nature, the species of animals more likely to survive are the generalist species, that is the species which can exploit different environments and food sources. In contrast, the species more likely to go extinct are the specialists, the ones that rely on a single food source or can only thrive in one environment. The same applies in software development. Do not put all your eggs in one basket, as the saying goes. The ability to do many things well, will see you farther than the ability to one thing brilliantly.

Epilogue

Here you go, I'm sure there are a couple of more items I forgot to add, but I think these capture the fundamental principles I've learned over many years as a software developer. I wish someone told me these things when I was first starting out. Hopefully they may help someone else, so thanks for reading and feel free to add your own advice in the comments.

Smart, nice and gets things done. What to look for when hiring devs!

Fred Heath — Tue, 08 Oct 2019 21:48:07 +0000

Joel Spolsky famously coined the phrase Smart and Gets Things Done back in the naughties, when summarising the qualities he was looking for when recruiting new hires. This phrase quickly became the mantra of many a start-up and recruitment firm as it seemed to fully describe the kind of people one would want to work with.

Smart, or 'clever' in UK English, is a much desirable trait. I mean who doesn't want to work with clever people, right? People who can grasp difficult concepts quickly and easily, people who can find innovative solutions to difficult problems, optimise an existing design to the max, see challenges where others see dead-ends, etc. Having smart people in your team gives you a great competitive advantage for sure.

People who 'get things done' is another favoured trait. Efficiency is highly-valued in our industry. I mean, you can employ the smartest, brainiest, genius in the world but if they can't (or aren't willing) to get their hands dirty and implement the solution they dreamed up, or deploy their code to CI, or document their design, then they aren't much use are they?

So yes, people who are smart and get things done is an ideal combination. Or is it.....?

Many years of working in software teams have taught me that there is another trait that we should value as much as any other: Niceness. Now, niceness is a very broad term and can be used to describe a variety of characteristics. So let me narrow it down. A nice person is someone who has the following,

Empathy
Courtesy
Honesty
Generosity

Allow me to elaborate:

Empathy

Is the ability to understand and be aware of the feelings and experiences of another person. An empathetic person is someone who offers to help you with your workload when you are struggling, instead of shrugging and saying "it's not my problem". An empathetic boss is someone who -when they see you down with a bad cold, or plagued by a family or personal problem- say "Take a couple of days off to recover" instead of "Are you going to meet the deadline on Friday?". Lack of empathy is a strong indicator of sociopathic behaviour or psychopathy. Empathetic people, on the other hand, tend to make a team function more productively and much less stressfully.

Courtesy

A fundamental level of politeness to be extended to other people. You don't have to know someone in order to be courteous to them. Heck, you don't even have to like them. Being polite to someone not only makes them feel better, it also makes you more likely to be approached by others, invited to their social gatherings and -very importantly in a work environment- to get listened to.

Don't get me wrong, I would never advocate being courteous to someone who harasses or bullies you, such people take courtesy as a weakness. But, under normal circumstances, being polite and pleasant even to the most annoying colleagues will make you both feel like better persons.

Honesty

Telling the truth is not always a black and white process. It usually works best combined with Empathy. Would you tell a colleague that they have spinach stuck to their teeth? Maybe not, as it would embarrass them. But what if they were about to walk into a very important meeting? It's better that they embarrass themselves to you than in front of the executive board or a very important client. Weighing the consequence of telling the truth is not easy but is something we always need to do instead of just choosing the easy way. The easy way, for instance, would be not to tell your team that the solution you developed fails under some extreme scenario. After all, it works 99% of the time so who needs to know about the 1%, right? Well, they don't until one of your colleagues does an important demo for a client and they hit that 1% scenario and the system crashes and burns. Being honest may be painful in the short term but always pays in the long term.

Generosity

Generosity isn't just about material giving. Yeah, we all love the person who comes in the office with a big bag of doughnuts every week or who buys coffee for everyone when going to the cafe next door. But it's generosity of spirit that makes the difference. A person of generous spirit takes complete responsibility for their lives and work. They do not blame others or circumstances for their problems. They own their mistakes and admit them. They are not afraid to give their time in order to help someone else. They don't boast about their victories and they are happy to share their successes with others.

I've worked in software in many different areas, in various roles and with a variety of people. I've learned to value the above qualities in my colleagues as much -or even more- than I value their intelligence and effectiveness. Our industry has devised many ways to detect and measure people who are 'smart' and 'get things done' but sadly no way to find people whose character qualities can improve a team so very much. So yeah, go ahead and hire smart people who get things done but also make sure they are nice people. You won't regret it!

So, do you know how to build a software system for your client? (part 3 - modelling Requirements with Impact Maps)

Fred Heath — Mon, 23 Sep 2019 08:33:36 +0000

In the previous article of this series we explored the Requirements domain entities, i.e. the things we need to know about if we want to capture and manage our Requirements correctly and efficiently. These entities were:

Stakeholders and Actors
Business Goals
Capabilities
Features

In this article, we'll learn how these entities are related and how we can model them in a visual and intuitive manner.

Impact Mapping

Back in 2012, Gojko Adjiz defined the concept of Impact Maps, a technique that he evolved from UX-based effect-mapping methods, to improve communication, collaboration and interaction in teams and organisations. Impact Mapping allows us to explore, analyse the discover answers to the major questions we have when starting to build a system:

Why are we doing it?
Who will benefit from it?
What do we need to do?
How are we going to do it?

The answer to these questions help us discover the entities involved in our Requirements and Specification model

Business Goals (Why)
Stakeholders (Who)
Capabilities (What)
Features (How)

We can additionally discover and define Tasks, that is the units of work we need to do in order to build our system.

Example Impact Map

So, how do we get to create a well-defined Impact Map armed with only our client's wishes and desires (i.e. their raw requirements)? Read on to find out.

The D3 method

As System Builders we have requirements thrown at us from many directions: client representatives, project managers, product owners, business analysts and fellow developers, they all want something from our system. These requirements come in many shapes and forms: formal statements, conversational statements, user stories, flowcharts and other diagrams and many other forms. Our job is to analyse these requirements and model the relevant entities in an Impact Map. Let's see how we would go about this task:

Let's say our client has told us they want their system to notify users when a certain product they have bought previously goes on sale. They might have expressed this in a formal statement ("The system shall notify user who have previously bought laptops..."), as a user-story ("As a past buyer of a laptop, I want to be notified...") or simply in an over-the-phone verbal exchange. To analyse and model this, we shall use a simple method I call D3: Decomposition-Derivation-Discovery.

Decomposition

Decomposition is the breaking down of the requirement to determine how many and what kind of entities it represents (Business Goals, Capabilities, Features or Tasks). A good heuristic for doing this is looking at the verbs of the main clause i.e. "The system shall notify..", "I want to be notified...", etc.

By examining our sample requirement in this light, it becomes apparent that there is only one entity involved and it has to do with notifications. It could have been that there were more than one verbs in the main clause, in which case we'd be dealing with two entities, but luckily that's not the case in our example.

The next step is to decide which type of entity it is. Obviously the whole 'notifications' thing isn't a Business Goal or a simple Task, so it's either a Feature or a Capability. A Capability is a high-level, coarsely-grained ability the actors can leverage in order to achieve their Goals. It answers the question "What can the system do for the actor?" A Feature is a medium-to-low-level, fine-grained piece of functionality that the system provides to it's actors. It answers the question "How can the actor realise this Capability?"

Given that what we know about the requirements is an abstract wish to notify users under certain conditions, then it would be reasonable to assume that we are dealing with a system Capability. Let's give it an appropriate name and add it to our model:

Defining a Capability

Derivation

Derivation is the creation of other Requirement entities that our original entity depends on, or that are dependent on our entity. So far we have identified a Notification Capability for our system. As every Capability must exist to realise a Business Goal, we must ask ourselves what the Business Goal is in our case. It's best not to make assumptions at this point and converse with our client in order to understand what they want to achieve by offering this Capability. Remember that we can and must validate the Business Goal, according to the standards described here. Let's say that in our case our client wants to use this Capability to entice their customers to buy more stuff, thereby increasing the client's revenue. That's a perfectly valid Goal, so let's add it to our model.

Defining a Goal

Now let's look at the right side of our Impact Map. We haven't yet defined any Features for our Capability. Let's do so now. We simply need to answer the question "How can we Notify users of product sales?", or else "How can we deliver the Notifications Capability?" Two possible answers would be: 1) Email notifications and 2) In-app notifications. As usual, we need to check with our client that these Features are acceptable to them. Assuming that they are, let's go ahead and add them to our Impact Map.

Defining Features

Finally, let's get down to the nitty-gritty of it and create some Tasks for each of our Features. Tasks are pieces of work that need to be done in order to implement a Feature. Most often they will involve coding, but they may also be about writing documentation, setting up a CI/CD pipeline or anything else that needs to get done to get our Feature up and running.

Defining Tasks

And that concludes the Derivation phase. We started with a single requirement and now we have a fully-populated Impact Map. But we're not done yet.

Discovery

Often the most overlooked phase, Discovery is about discovering new Capabilities that our system may need. The Business Goal is a good starting point. Are there any other Capabilities our system could offer to help attain that Goal? In our example our client wants to entice users to buy more so we're offering the 'Notification of sales' Capability. Another Capability that would help towards this would perhaps be a 'Discount for bulk buying' Capability. This is something we could (and should) discuss with our client. After all, we are more than just code monkeys, we are System Builders who want to help our clients achieve their Goals.

Discovering other Capabilities

Naturally, once we add new system Capabilities we should go back to the Derivation phase and make sure we create some Features and Tasks for our new Capabilities.

Epilogue

So there you have it, we started with our client's raw requirements and used a structured approach to create a well-defined model of our Requirements and Specifications. There is one more step we need to go through before we can start coding away and this is to detail our Features. We'll do that by writing some Executable Specifications. More of that in the next article :)

So, do you know how to build a software system for your client? (part 2 - the Requirements Domain)

Fred Heath — Thu, 11 Jul 2019 19:32:15 +0000

The Requirements Domain

In every knowledge domain, be it medicine, manufacturing or software development, we need to have a mental model of the domain entities involved and their interactions, before we can start thinking about providing solutions in that area. A Domain Model is a representation of all the important entities, activities and their relationships in a specific Domain.

So, for instance, in the Pizza-making Domain (one of my favourites) we would expect to find entities such as Dough, Toppings, Restaurant, Delivery Person, etc. Similarly, in the Requirements Domain we'd expect to find entities and relationships that help us conceptualise requirements.

The entities of the Requirements Domain are:

Stakeholders
Business Goals
Capabilities
Features

Stakeholders and Actors

A Stakeholder is someone who derives value, benefits from or influences our system. An Actor is a Stakeholder who interacts with our system, either directly or indirectly. All Actors are Stakeholders, but a Stakeholder is not necessarily an Actor

There is a small, but important sub-set of Stakeholders which is the set difference between Stakeholders and Actors. We call these 'Non-acting Stakeholders'. These would usually be people like Directors, Business Owners, Enterprise Architects or Senior Managers. They will have a stake in the system and they will likely make decisions that influence the system, but don't expect to see them sitting down in front of a screen and using the system any time soon.

Actors may be divided into two categories:

Primary Actors interact with system directly through a UI or API in order to achieve a specific goal.
Secondary Actors are actors that the system needs assistance from in order to achieve the Primary Actor’s goal.

Business Goals

Goals that benefit the Stakeholders. Business Goals are usually defined by Non-acting Stakeholders, such as Directors, Senior Managers, etc. The Business Goal's incentive and ultimate outcome should ultimately fall under one of these categories:

Increasing revenue
Reducing costs
Protecting revenue
Avoiding future costs

A useful technique for determining a Business Goal's incentive is the 5 Whys technique. If after the 5 questions the answer isn't one of the above, then you should seriously question the value this Goal adds to the Business.

Example: Reward loyal customers
Why? So that returning customers can feel valued and also save some money
Why? So that customers keep returning more often
Why? So that they spend more money on our products (increase revenue)

Tip: Good Business Goals add value to Stakeholders by specifying goals tangential or extrinsic to the system's inherent abilities and functionality. For instance, 'I want customers to buy books' is not a valid goal for an online book-store. 'I want customers to buy more books from us than competitor X' is better, though it still lacks a strategic aspect. Speaking of which...

Strategic goals are the best goals

The best Business Goals are the ones that aren't too abstract, nor too specific, the ones that outline strategy but not tactics. For instance, the goal of "Rewarding loyal customers" can be framed as "Increase sales by rewarding loyal customers". "Increasing sales" is the end-goal and "rewarding loyal customers" is the strategy. If the goal was simply "Increase sales" or "reduce costs" then it would ultimately fall to whoever happened to implement that goal, a business analyst or -shock, horror- a software developer to determine what the best strategy would be. The best people to define the strategy behind the goals are the Non-Acting Stakeholders, the directors, business owners and senior managers.

On the flip-side, should the Goal become something like "Increase sales by rewarding loyal customers by giving them free gift-bags if they spend over $500 in a single transaction" then we are mixing strategy with tactics. This can be a Bad Thing, as creating tactics requires a level of domain, system and architectural knowledge that is usually lacking in Non-Acting Stakeholders (and I'm saying this in the best possible way). For example, the director or sales manager specifying the 'free gift-bag' ploy may not be aware of any logistics or distribution problems in delivering that many gift-bags. Tactics are better developed by other Stakeholder participation and should be captured separately as system Capabilities.

Capabilities

A Capability is a system ability that enables the Stakeholders to achieve a Business Goal. Capabilities define what the system does but not how it does it. They are high-level, coarse-grained descriptions of activities which enable Stakeholders to derive benefit from the system. A Capability can only be seen within the context of a Business Goal and its associated Stakeholders.

Examples

The HR staff are able to use the system for the on-boarding of new employees
The Seller can use the system to see real-time stock levels
End-users are able to log onto the system using their social media accounts

Our ultimate job as System Builders is to deliver Capabilities to our Stakeholders.

Features

A Feature describes a system behaviour. It's closely associated with a Capability and answers the question "How are we going to deliver this Capability?"

At a minimum, a Feature consists of a feature Title and a Scenario. A Scenario is a look at the Feature's functionality from a specific perspective. So if our Feature is titled 'Withdraw cash from cash-point machine', for instance, we could have Scenarios like 'Cash demand exceeds account balance' or 'Bank note denominations don't match requested amount'.

However, having just a Title and a scenario doesn't help us describe the functionality much. A fully-featured Feature (excuse the pun) will include:

Feature Title: a brief description, e.g. Cash Withdrawal

User Story: we use the following template

As an [Actor]
I want [specific System Behavior]
So as to achieve [a goal contributing to a Capability]

Impact: A link to the impact map this Feature relates to (more on these in the next post)
Notes: any other text that helps the reader better understand the Feature
Background: (if applicable): a common prerequisite

Scenarios: are written in a structured manner

Given [a Condition]
When [an Action is invoked]
Then [an expected Outcome occurs]

A valid Feature must have at least one defined Scenario.

To derive Features we have to look at our system Capabilities and answer the question: 'What functionality must we implement to deliver this Capability?'.

Let's look at the: "The Seller can use the system to see real-time stock levels" Capability. How can we deliver this Capability? We could start by implementing these Features:

Feature 1: Integrate Warehouse API to our system
Feature 2: Create notification mechanism for warehouse stock changes
Feature 3: Build UI that shows current stock levels

Obviously there will be more Features than these in a real-life system, but hopefully you get the gist: Features are about implementing functionality, Capabilities are about Stakeholder ability to achieve a goal. We'll be discussing these more in later posts in this series.

Summary

Stakeholders, Business Goals, Capabilities and Features are fundamental entities in the Requirements and Specification domain. In the next post, we'll explore using these entities to model our system Requirements and derive Specifications with a technique called Impact Mapping.

So, do you know how to build a software system for your client? (part 1)

Fred Heath — Sun, 23 Jun 2019 21:56:36 +0000

You're a good developer. May be even a great one. You know your programming language(s) inside out. You're a wizard with databases and key-value stores. You're at ease with functional design as well as OO one. You can Dockerize anything and deploy it to AWS in a jiffy.

...but can you build a software system for your client??

The reason I'm asking is because even though you may have all the skills mentioned above, that doesn't mean you can build a software system for your client. Sure, you can build a software system, but by 'software system for your client' I mean a system that does, at a minimum, what your client needs and, ideally, what they want.

"Sure", you say, "my client tells me what they want and I build it!". Really?! How do you know that you're building what they want? Yes, they tell you what they want but how do you know that this is what you're delivering? How do you know you even understand what they're telling you? More importantly, how does your client know that you're building what they want? The thing is, building software is more than just coding. Yes, coding is a crucial part of building a system but it's not the only part and -arguably- not even the most important.

There have been many studies conducted on the causes of software project failure. They all agree on one thing: The main cause of failure is a combination of requirements ambiguity, poor communication and unrealistic expectations. Let me summarise this in a sentence:

If your project fails, chances are it will fail because you didn't properly manage its requirements.

Managing requirements involves the elicitation, modelling, analysing and communicating of your client's requirements. That's right, you can be be the best coder in the world, head of a team of super coders but that will not make a big dent in the odds of your project failing. So I'll ask again:

Do you know how to build a software system for your client?

By now you should have caught on to the fact that unless you know how to handle your client's requirements, your answer to the above question should be a resounding NO. Because you may be a great Developer but unless you can manage requirements you're no System Builder!

So, where do you start? You start at the beginning: By understanding what Requirements and Specifications are.

Requirements vs Specifications

Even after more than two decades in software development I still get surprised by how many people, even experienced professionals, don't know the difference. They often use the terms interchangeably. This is the road to abject project failure.

Simply put, a Requirement is a stakeholder's expression of a need, wish or desire about the system being built. Requirements come in different shapes, forms and sizes and from various sources. Requirements can be provided as:

formal statements (“The system shall provide a document searching facility”)
rules (”accounts with monthly deposits larger than $1000 receive a 10% discount”)
examples (“Joe didn’t have to pay for his coffee because it was his 11th coffee in that store”)
User Stories
Business Processes
Flow-charts, activity charts, or other types of diagram
some other obscure way that I can’t even imagine at this point

A Specification, on the other hand, is a description of the system behaviour required in order to fulfil or realise a Requirement. So for our “The system shall provide a document searching facility” above, a specification could be something along the lines of specifying how to add some search strings on a search bar. The Specification is just a way of defining how we’re going to realise the Requirement. Nothing more, nothing less.

The Requirements gap

So let's get this absolutely straight:

Requirements tell us what our client wants or needs. Ultimately, it is our client's responsibility to provide us with the requirements. What we can do is help them identify, refine and validate them. There are a number of techniques to help us achieve that and they will be covered in the next posts of this series.
Specifications tell our client (and our team) how we will meet these requirements in the delivered system. Specifications are essential to building a successful system. They drive the whole development and testing process. To quote Joel Spolsky:

"Failing to write a spec is the single biggest unnecessary risk you can take in a software project. It’s as stupid as setting off to cross the Mojave desert with just the clothes on your back, hoping to 'wing it'."
Our job as System Builders is to elicit our client's Requirements and translate them to Specifications.
Our client's job is to confirm that the Specifications reflect their Requirements.

This series aims to outline a number of agile methods and techniques that will help you:

Elicit requirements
Model them
Create executable specifications

The gap between Requirements and Specifications is a deep and dangerous one. The next article will show you how to start building a bridge over it, so stay tuned.

PS: the articles in this series are related to, and a continuation of, my earlier post on User Stories

Enough with the User Stories already!

Fred Heath — Tue, 02 Apr 2019 11:07:13 +0000

The fog of Agile

Agile, at its core, is a set of guiding principles that are interpreted by the creators and practitioners of various processes, methods and frameworks. Like a language with many dialects, this results in the same word meaning slightly different things in each dialect. Asking the question "what's a user story?" will give you a dozen different answers. A user story can be a requirement, a feature, a description, an end-goal, a high-level abstraction, a small piece of business value and many other things, depending on who you ask - and when you ask them ;)

I tend to agree with Mike Cohn's definition: A user story is a description of a feature. Of course, this only moves the question to "what is a feature?". A quick Googling shows us that definitions abound: a feature is a functionality, a capability, a requirement, a client-valued function expressed in the form , a part of an Epic, etc. And so it goes on and on.

Here's a common pattern I've come across many, many times and I'm willing to bet that you have too: A stakeholder asks to look at the 'user stories'. The stakeholder later returns complaining that the user stories are too technical or too abstract or that the wrong actors are involved

Here's an example:

As an end-user 
I want to log-in with my social media credentials
so I don't have to remember extra usernames and passwords

That's a user story, right? You can give this to your stakeholder but they'll probably complain that this is too generic to be of any value. And they'll be right. You then give them this story:

As an end-user
When I log-in I want to be re-directed to the login page of my selected Identity Provider
So that, upon entering my credentials, the IP can redirect me back to our system with a valid SAML token

The stakeholder looks at you blankly. You don't understand what the problem is. They wanted User Stories, you gave them User Stories, so WTF? The thing is, the stakeholders want something very specific but they don't have the right words to express it because User Stories is a loaded term. Let's hold that thought for a minute.

A Requirements Domain Model

In his BDD in Action book, John Ferguson Smart elaborates on the usage of Impact Maps, a technique introduced by Gojko Adzic that is useful for answering the big questions of the Requirements and Specifications model:

Why are we building the system?
Who benefits from it?
What do we need to build?
How will we build it?

These can be directly translated to Requirements and Specifications Domain Entities:

Business Goal (Why): The intended benefit of our system
Stakeholder (Who): Someone who is affected by our system
Capability (What): A system ability that enables the Stakeholders to achieve a business goal
Feature (How): A functionality that helps deliver a Capability

And, just like that, we have well-defined Domain Entities. Our requirements are represented by Capabilities in the context of Stakeholders and Business Goals. Our specifications are the Features, which describe the system functionality we will implement in order to deliver the system's Capabilities. In true BDD fashion, a Feature is described by a User Story followed by a number of scenarios/acceptance criteria. A Feature is not a User Story! We can use User Stories to describe Capabilities, Features and Business Goals too.

Our system Requirements are the Capabilities in the context of the beneficiary Stakeholders and valid Business Goals
We can describe Capabilities and Business Goals with User Stories. This doesn't mean that the stories are these things.
Our Specifications are descriptions of the system behaviour required in order to deliver a Capability, i.e. the Features

User Stories are just descriptive devices for Requirements Domain Entities.

So let's jump back to our confused stakeholder. When they asked you for User Stories, what they were asking for was the Features! They wanted a description of the system functionality that you'll provide in order to enable them to achieve their goals. Nothing more, nothing less. User Stories are not Domain Entities, they are just descriptors. We need to start communicating using Domain Entities, not descriptors.

So we provide the following to our stakeholder:

Feature: Login with Twitter
As a end-user with a Twitter account
I want to log-in with my Twitter credentials
so I don't have to know extra usernames and passwords

  Scenario: First-time login with Twitter, already signed-on
    Given the user visits our system's login page
    And the user hasn't signed-on our system with Twitter before
    And the user is not already signed onto Twitter
    When the user chooses to sign-on our system with Twitter credentials
    Then the user is presented with a Twitter login page

  Scenario: First-time login with Twitter, not currently signed-on
    Given the user visits our system's login page
    And the user hasn't signed-on our system with Twitter before
    And the user is already signed onto Twitter
    When the user chooses to sign-on our system with Twitter credentials
    Then the user is presented with a Twitter permissions and confirmation page

... (more scenarios)

Note how we present a User Story beneath our Feature title in order to best describe and illustrate our Feature. Also note that our Feature would still be valid and usable even if we didn't use a User Story. The User Story is not the Feature! It just gives us a nice narrative so it's useful to use it in our Feature. But that is all. We're not creating User Stories, we're defining Features.

As Developers, Product Owners or Business Analysts we get flooded with our clients' wishes, needs and desires. Our first question should be: "What should the stakeholders be able to do with our system in order to fulfil their wish or need?". Once we've answered that, our next question should be "How are we going to deliver that capability?". The result of this thought process will give us a set of Specifications (Features) that realise the stakeholders' requirements towards achieving some Business Goals. So let's clear the fog and start talking more about Capabilities and Features, instead of their generic and variant descriptors. Enough with User Stories already!

Parallelising ETL workflows with the Jongleur gem

Fred Heath — Mon, 14 Jan 2019 09:39:14 +0000

Our assignment

Our company has just gone on a huge recruitment spree and has just hired 100,000 new employees. The HR department has sent us a spreadsheet with the employee details, asking us to save them in the company staff database. After having a look at the data, we notice a few actions we need to take before inserting the details into our database and we come up with the following algorithm:

Read data from the spreadsheet
Break employees' full name into first name and last name fields
Assign each employee a unique company number
Insert employee into the database as long as their first and last name is less than 255 characters long

A single-process ETL pipeline

We then proceed to write a crude but effective ETL pipeline

require 'csv'
require 'pg'
require 'ostruct'
require 'securerandom'

@users = []
@conn = PG.connect( dbname: 'testdb' )
DATA_FILE = "data/users.csv"

def extract(data_file)
  CSV.foreach(data_file) do |row|
    user = OpenStruct.new
    user.name = row[0]
    user.email = row[1]
    @users << user
  end
end

def transform
  @users.map! do |usr|
    usr.first_name = usr.name.split[0]
    usr.last_name = usr.name.split[1]
    usr.staff_no = SecureRandom.uuid
    usr
  end
end

def load(usr)
  if( usr.first_name.length < 255 &&  usr.last_name.length < 255)
    @conn.exec_params(
      "INSERT INTO STAFF (FIRST_NAME, LAST_NAME, EMAIL, STAFF_NO) VALUES ($1, $2, $3, $4)",
      [usr.first_name, usr.last_name, usr.email, usr.staff_no]
    )
  else puts "Validation failed for #{usr.first_name} #{usr.last_name}"
  end
end

extract(DATA_FILE)
transform
@users.each {|usr| load(usr)}

Our extract method reads the data from file and stores it in memory. The transform method then splits the full name field into first and last name and assigns a UUID to the employee. Finally, our load method ensures the name fields don't exceed the expected length, before inserting the data into the Staff table.

We test our script and it works fine, however all this data crunching takes a long time and, us being the perfectionist that we are, would like to make it faster.

The need for speed

Enter Jongleur a Ruby gem that allows us to create tasks that are executed as separate OS processes and that can be ran with a certain precedence.

We'll use Jongleur to split our ETL pipeline into two separate and parallel pipelines, each dealing with one half of the input data. We want our task graph to look like this:

Each Extract, Transform and Load task will do exactly what we described in our initial algorithm above, but with only half of the spreadsheet records. The Cleanup task will kick-in last, to sweep away any undue remnants after our pipeline has been ran.

Representing the above graph in Jongleur is a simple matter of creating a Ruby Hash:

etl_graph = {
  Extract1: [:Transform1],
  Transform1: [:Load1],
  Extract2: [:Transform2],
  Transform2: [:Load2],
  Load1: [:Cleanup],
  Load2: [:Cleanup]
}

We can then just tell Jongleur to use our task graph

Jongleur::API.add_task_graph etl_graph

Design considerations

One thing we need to consider before creating our parallel tasks is how to share data between tasks which are ran as separate processes. Shared Memory could be used but has some pitfalls which is why we'll be using the wonderful in-memory data store which is Redis. Redis is (mostly) single-threaded, which means concurrent requests are executed sequentially and safely (and blazingly quickly). Furthermore, we'll take advantage of one of Jongleur's great features whereby each task can access the process ids (pids) of its predecessors. This means that we can use a task's pid as a key for its data. So our Extract1 task can save data to Redis with its own pid and when Transform1 task starts it will know exactly which data was created by Extract1, its predecessor!

Common attributes

Every Jongleur task class must inherit from the base class Jongleur::WorkerTask. This means that we can use this class to define data that we want accessible from any and all WorkerTask instances. In our case we want any Extract, Transform and Cleanup tasks to be able to access our Redis data store. To achieve that we simply define the Redis connection as a class variable in the Jongleur::WorkerTask hierarchy:

class Jongleur::WorkerTask
  @@redis = Redis.new(host: "localhost", port: 6379, db: 15)
end

Extracting

Both our Extract1 and Extract2 tasks will need to have some common functionality, namely to parse records from the csv file and save them to Redis:

class Extract < Jongleur::WorkerTask

  DATA_FILE = "data/users.csv"

  def process_row(row, rowno)
    user = {}
    user[:num] = rowno
    user[:name] = row[0]
    user[:email] = row[1]
    user

    @@redis.hset(Process.pid.to_s, "user:#{rowno}", user.to_json)
  end
end

Now the only divergence between Extract1 and Extract2 comes when reading the input data from the csv file. Extract1 will only read and store the first 50,000 records, while Extract2 will read and store the remaining 50,000 records.

class Extract1 < Extract
  def execute
    CSV.foreach(DATA_FILE).with_index(1) do |row, rowno|
      break if rowno >= 50000
      process_row(row, rowno)
    end
  end
end

class Extract2 < Extract
  def execute
    CSV.foreach(DATA_FILE).with_index(1) do |row, rowno|
      next if rowno < 50000
      process_row(row, rowno)
    end
  end
end

Transforming

Each Tranform task will need to use its Extracting predecessor's pid in order to load the extracted records it needs to transform. It will then proceed to split each record's name, assign it a UUID and save it back to Redis:

class Transform < Jongleur::WorkerTask
  @desc = 'Transforming'
  def execute
    loader_id = Jongleur::Implementation.get_process_id(predecessors.first)
    users = @@redis.hgetall(loader_id)
    users.keys.each do |usr_key|
      transformed_user = {}
      data = JSON.parse users[usr_key]
      transformed_user[:first_name] = data['name'].split[0]
      transformed_user[:last_name] = data['name'].split[1]
      transformed_user[:staff_no] = SecureRandom.uuid
      transformed_user[:num] = data['num']
      transformed_user[:email] = data['email']
      @@redis.hset(Process.pid.to_s, "user:#{data['num']}", transformed_user.to_json)
    end
  end
end

class Transform1 < Transform; end
class Transform2 < Transform; end

Loading

Load1 and Load2 will pick up the records stored by their respective predecessor Transform tasks, will validate the name length and will insert the records in the Staff table of our HR's Postgres database:

class Load < Jongleur::WorkerTask
  @desc = 'Loading'

  def execute
    loader_id = Jongleur::Implementation.get_process_id(predecessors.first)
    users = @@redis.hgetall(loader_id)
    conn = PG.connect( dbname: 'testdb' )
    users.keys.each do |usr_key|
      data = JSON.parse users[usr_key]
      if( data['first_name'].length < 255 &&  data['last_name'].length < 255 )
        conn.exec_params(
          "INSERT INTO STAFF (FIRST_NAME, LAST_NAME, EMAIL, STAFF_NO) VALUES ($1, $2, $3, $4)",
          [ data['first_name'], data['last_name'], data['email'], data['staff_no'] ]
        )
      else puts "Validation failed for #{usr.first_name} #{usr.last_name}"
      end
    end

  end
end

class Load1 < Load; end
class Load2 < Load; end

Cleanup

All our Cleanup task has to do is to erase the Redis data created by the previously-ran tasks

class Cleanup < Jongleur::WorkerTask
  def execute
    @@redis.flushdb
    puts "...Cleanup"
  end
end

Running our pipelines

We now have

Defined our Task Graph and configured Jongleur with it
Implemented an #execute method for each of the Tasks in our Task Graph

We are now ready to invoke Jongleur and run our parallel pipelines:

API.run do |on|
  on.completed do |task_matrix|
    puts "Jongleur run is complete \n"
    puts task_matrix
    puts "oh-oh" if API.failed_tasks(task_matrix).length > 0
  end
end

The output is:

Starting workflow...
starting task Extract1
starting task Extract2
finished task: Extract1, process: 22722, exit_status: 0, success: true
starting task Transform1
finished task: Extract2, process: 22723, exit_status: 0, success: true
starting task Transform2
finished task: Transform1, process: 22726, exit_status: 0, success: true
starting task Load1
finished task: Transform2, process: 22727, exit_status: 0, success: true
starting task Load2
finished task: Load1, process: 22729, exit_status: 0, success: true
finished task: Load2, process: 22732, exit_status: 0, success: true
starting task Cleanup
...Cleanup
finished task: Cleanup, process: 22745, exit_status: 0, success: true
Workflow finished
Jongleur run is complete
#<struct Jongleur::Task name=:Extract1, pid=22722, running=false, exit_status=0, finish_time=1546800512.6342049, success_status=true>
#<struct Jongleur::Task name=:Transform1, pid=22726, running=false, exit_status=0, finish_time=1546800518.355283, success_status=true>
#<struct Jongleur::Task name=:Extract2, pid=22723, running=false, exit_status=0, finish_time=1546800512.967079, success_status=true>
#<struct Jongleur::Task name=:Transform2, pid=22727, running=false, exit_status=0, finish_time=1546800519.273847, success_status=true>
#<struct Jongleur::Task name=:Load1, pid=22729, running=false, exit_status=0, finish_time=1546800533.147315, success_status=true>
#<struct Jongleur::Task name=:Load2, pid=22732, running=false, exit_status=0, finish_time=1546800534.1630201, success_status=true>
#<struct Jongleur::Task name=:Cleanup, pid=22745, running=false, exit_status=0, finish_time=1546800534.899818, success_status=true>

Jongleur gives us a thorough account of each Task's details and success status. Notice how both initial Tasks (Extract1, Extract2) start at the same time and each new Task starts only once its predecessor on the Task Graph has finished. If a Task failed, for whatever reason, Jongleur wouldn't execute its dependent tasks but it would continue running any other tasks it could. The status of all Tasks would still be visible in the Task Matrix structure yielded by the completed callback.

Performance

Running the simple single-process ETL pipeline at the beginning of this article produced the following timings:

9.55s user 1.40s system 32% cpu 33.494 total

Running the parallel Jongleur pipelines produced:

19.99s user 5.23s system 89% cpu 27.210 total

User time is increased with Jongleur (I'd guess due to using Redis) and so is kernel time, obviously due to forking and managing many processes, but overall time is approx. 18% faster using Jongleur with two pipelines. Also consider that in this example we only used two parallel lines, i.e. we split the data load into 2 parts. It is a simple matter to split it into 4, 8 or even 16 parallel lines (dependent on our CPU cores) and thus gain a massive performance gain, using the same logic and principles demonstrated here and simply adding more Tasks.

Jongleur advantages

Performance, through parallelisation.
Implicit precedence handling. Jongleur will start running a task only once its predecessor tasks have successfully finished running. If a task fails, its dependent tasks wil not be ran.
Code modularisation. If a task fails, for whatever reason, it will be marked as such by Jongleur and its dependent tasks will not be run. However, that won't stop any other tasks from being run, which means that partial failure does not necessitate complete failure, in use cases where this is desirable.

The Complete Code

including data file for this demo is freely available

The Future

Feel free to suggest improvements and new features for Jongleur or to talk to me about trying it out for your own Use Cases :)

Originally published at http://bootstrap.me.uk

The Three Ghosts of FizzBuzz: A Christmas Story

Fred Heath — Fri, 21 Dec 2018 15:07:15 +0000

FizzBuzz is a simple kids game, often used as a test at programming interviews. It goes like this:

"Write a program that prints the numbers from 1 to 100. But for multiples of 3 print “Fizz” instead of the number and for multiples of 5 print “Buzz”. For numbers which are multiples of both 3 and 5 print “FizzBuzz”.

I fell asleep yesterday while thinking about all the different ways we can solve FizzBuzz in Ruby. Then three ghosts appeared in my dream. The first one was the Ghost of Imperative Programming (GImP, for short). It showed me how to do FizzBuzz in an imperative way:

def imperative(n)
  if (n % 3 == 0) && (n % 5 != 0)
    'Fizz'
  elsif (n % 3 != 0) && (n % 5 == 0)
    'Buzz'
  elsif (n % 3 == 0) && (n % 5 == 0)
    'FizzBuzz'
  else n.to_s
  end
end

"Simple conditionals are easy to read", it said. "They're also relatively fast. They're a good, basic way to implement your solution"

Next, the Ghost of Antecedent Declarative Inference appeared (GhAnDI). "Don't listen to GImP" it said. "Be more clever. Look here, the conditions that are evaluated to produce each FizzBuzz value are mutually exclusive, if you represent them as Hash values then only a maximum of one of our Hash keys will have the value of 'true' for any given number. Use this to your advantage."

def declarative(n)
  h ||= {
    'Fizz' => (n % 3 == 0) && (n % 5 != 0),
    'Buzz' => (n % 3 != 0) && (n % 5 == 0),
    'FizzBuzz' => (n % 3 == 0) && (n % 5 == 0),
    n.to_s => (n % 3 != 0) && (n % 5 != 0)
  }
  h.key(true) || n
end

"You then make sure you return this 'true' key (h.key(true)) from the method. Of course if all the keys have the 'false' value (i.e. the passed number is neither divisible by 3, nor by 5) then h.key(true) will evaluate to nil, so use the logical OR operator to ensure that -in this case- you return the actual passed number."

"That's pretty smart" I said. "We are not specifying a sequence of steps leading to the solution, but rather the conditions required and we leverage the language features to work out the correct one. Cool!"

As GhAnDI departed, I sensed a new presence arriving. It was wearing trendy clothes , had a well-oiled beard and there was a large and vocal group of developers following it around. It was... the Ghost of Functional Erudition (GoFEr).

"The other two are the past", GoFEr said. "I will show you the future"

def functional(n)
  [[15, 'FizzBuzz'], [5, 'Buzz'], [3, 'Fizz']].select { |x| (n % x[0] == 0)}.map{|a| a[1] }.first || n.to_s
end

"We specify both conditions and outcome together" it said. "Ruby doesn't have tuples, but we can get a similar effect with an Array of Arrays. We then select the 'tuple' which satisfies the condition of zero-mod division with the tuple's divisor (x[0]). If more than one tuples match, we only select the first one, since we cleverly ordered our Array that way. And if there are no matches then we return the original number input. Simples."

And with that GoFEr shimmied away to the sounds of 'California dreaming' and I woke up. I learned some useful techniques from this encounter, but mostly I came to appreciate just how powerful and flexible the Ruby language really is.

Merry Christmas everyone and happy Ruby-ing!

PS: As a Christmas present, the Ghosts put all their code along with some bench-marking on https://gitlab.com/snippets/35596

Truth tables as Ruby Hashes

Fred Heath — Mon, 17 Dec 2018 08:20:46 +0000

Truth tables are a common way of defining and testing code behaviour. A truth table treats a component or function of our system as a black box, with well-defined inputs and outputs. At the same time, Ruby Hashes are flexible enough that they can be used as decision objects, receiving an input (key) and determining its value. Hopefully you can see where I'm going with this ;) Let's look at an example. The problem in question is this:

"Write a method that receives the month and the year and outputs how many days there are in that month".

Sounds easy, doesn't it? We all know which months have 30 and which 31 days in them. Apart from February, that is. February usually has 28 days, except that in leap years it has 29. How do we know which years are leap years? There are certain rules that allow us to determine leap years:

The year is evenly divisible by 4
If the year can be evenly divided by 100, it is NOT a leap year, unless
the year is also evenly divisible by 400. Then it is a leap year

To sum it up, a year is a leap year when

it is evenly divided by 4 but NOT evenly divided by 100.
It is evenly divided by 4, 100 AND 400.

The truth table for this problem would be:

Month	Year	No of Days
jan, mar, may, jul, aug, oct, dec	any	31
apr, jun, sep, nov	any	30
feb	year % 4 != 0	28
feb	(year % 4 == 0) && (year % 100 != 0)	29
feb	(year % 4 == 0) && (year % 100 == 0) && (year % 400 != 0)	28
feb	(year % 4 == 0) && (year % 100 == 0) && (year % 400 == 0)	29

Now we could use a multi-branch conditional or maybe a Case statement to implement this truth table. But there's another way. We can leverage two powerful Ruby features to model our truth table as a Hash:

Everything is an expression in Ruby. I mean everything, and that includes Hash keys and values. Every statement gets evaluated to an object and that's a beautiful thing.
Ruby is great for List Comprehensions. It offers some great ways of making lists out of lists, either in an iterative or a functional manner.

Knowing all this, we can write our method as follows:

def month_days(year, month)
    h = {
        %w(jan mar may jul aug oct dec) => 31,
        %w(apr jun sep nov) => 30,
        %w(feb) => ((year % 4 == 0) && (year % 400 == 0)) ||
                    ((year % 4 == 0) && (year % 100 != 0)) ?
                    29 : 28
    }
    # find the Hash key the includes the required month, return its value 
    h.select {|k, v| k.include? month}.values
end

We use Arrays for the Hash keys and we use the ternary operator as a value for the february key. Our returning object is the value of a Hash key that is generated by filtering the original Hash's keys (Arrays) based on the desired month. Let's run it:

$> puts month_days 1900, 'feb'
=> 28
$> puts month_days 2000, 'feb'
=> 29
$> puts month_days 1900, 'sep'
=> 30

Beautiful. This is much cleaner and elegant than a big If or Case statement. Moreover, a Hash can be easily memoized so that any intricate calculations become just a simple lookup and performance is boosted. Of course, Ruby being Ruby, there'll be a different or better way, so if you know of any feel free to share it with me by commenting below.

Originally published at Bootstrapped Thoughts

Python Slices vs Ruby blocks

Fred Heath — Tue, 11 Sep 2018 08:40:18 +0000

A couple of my Python colleagues tried to impress me today with Python's named slices feature. The way it works is like that:

 s = list('helloworld!')
  => ['h', 'e', 'l', 'l', 'o', 'w', 'o', 'r', 'l', 'd', '!']
WORLD = slice(5, 10)
s[WORLD]
 => ['w', 'o', 'r', 'l', 'd']

So you can have your own customised slicing mechanism that you can apply to any list. Which is kind of cool. That prompted me to demonstrate how we can do the same thing with Ruby. Luckily, in Ruby world we have blocks, procs and lambdas, the ultimate play-dough that allows us to stretch and flex in every direction.

s = ['h', 'e', 'l', 'l', 'o', 'w', 'o', 'r', 'l', 'd', '!']
  => ['h', 'e', 'l', 'l', 'o', 'w', 'o', 'r', 'l', 'd', '!']
world = proc {|x | x.slice(5, 10)}
world[s]
 => ["w", "o", "r", "l", "d", "!"]

so we can do things like:

first = proc { |x| x.slice(0) }
first[s]
=> "h"

or even things that we can't do with Python's named slicing, since it doesn't allow us to pass the receiver as an argument to the block (x in the example below)

last = proc {|x| x.slice x.length-1, x.length}
last[%w(dog rabbit fox cat)]
=> ["cat"]

median = proc {|x| x.slice(x.length / 2) }
median[%w(dog rabbit cat)]
=> "rabbit"

and of course we're not just restricted to slicing arrays,

domain_extractor = proc { |x| x.gsub(/.+@([^.]+).+/, '\1') }
domain_extractor["fred@mydomain.co.uk"]
=> "mydomain"

and since a block is just an anonymous Proc object, we can use it with any method that accepts Proc parameters

email_list = ["fred@mydomain.com", "john@gmail.com", "mary@yahoo.co.uk"]
=> ["fred@mydomain.com", "john@gmail.com", "mary@yahoo.co.uk"]
email_list.map(&domain_extractor)
=> ["mydomain", "gmail", "yahoo"]

Blocks and Procs (a.k.a lambdas) are ideal for some quick, reusable functionality or for when defining a full-blown method would be too much, but also for more serious uses such as callbacks and deferred execution. IMHO, they are a fundamental part of what makes Ruby such a flexible and powerful language. Learn them, use them, enjoy them :)