DEV Community: Vitaly Sharovatov

QA myth busting: Quality can be measured

Vitaly Sharovatov — Thu, 11 Jul 2024 10:14:50 +0000

Let’s bust some QA myths.

First myth: Quality can be measured.

Everyone wants to measure quality, but the idea that quality can be definitively measured is a myth.

Imagine trying to measure the quality of a family road trip. What would make the road trip an indisputable success for the entire family? You’ll have a hard time finding metrics that could give you a solid answer.

Of course, that doesn’t mean that we shouldn’t use the tools and metrics at our disposal to analyze and improve quality. It just means we need to have a broader understanding of what quality measurements tell us and what to do with that information.

Definitions of quality give us a general idea to work towards

Without a clear, shared understanding of “quality,” there is too much ambiguity in what the end goal is. This is particularly important for such a complicated topic as quality and quality assurance.

ISO 9000 defines quality as “a degree to which a set of inherent characteristics fulfills requirements”. This implies that quality of a product is how it satisfies clients’ needs and desires, expressed in the form of requirements.

ISO 25000 offers a more specific definition for software quality: “capability of software product to satisfy stated and implied needs when used under specified conditions”. This focuses on the software’s ability to meet both explicit and implicit client needs within the client’s context.

Combining these perspectives, we can formulate a unified definition:

Quality is a match between what’s desired and what’s produced

Quality is subjective

Quality can only be achieved if we, the IT folks, first understand what the client wants and desires, and then are able to produce what we understand. If the clients use our software and are happy with it, this should mean we succeeded in understanding them and producing what they wanted.

Science has struggled with measuring and quantifying quality for this particular reason: quality is very subjective. People will only consider a product or service to be high quality if they feel that the product or service pleases them or helps them solve their problems.

Accepting the fact that quality is very subjective, is it even possible to fully measure it?

Let’s go back to the family road trip. You, the driver, have a destination (project completion) and planned stops (project milestones). Let’s say you made all the planned stops and made it to your destination. That would add up to a high quality road trip, right? No, because quality is not measured by getting to the destination, but rather the experience along the way. Let’s say one passenger got food poisoning, you spent several hours stranded with a flat tire, and a hotel at one of the planned stops lost your reservation so you had to sleep in the car one night. Would you still perceive that as a quality road trip?

Quality is heavily influenced by people’s perceptions

The overall quality of a product is ultimately determined by how people perceive it and perceptions can’t be fully measured. You can’t get into the heads of the customers and see what they think of your product.

However there are various measures for checking if clients are happy:

Employing dogfooding practice, where the employees become the first clients of the product, giving them a much better understanding of the customer;
Organizing customer surveys and focus-groups, getting direct feedback on the product;
Coming up with proxy metrics for quality, such as Net Promoter Score (NPS), Customer Satisfaction Score (CSAT), and Customer Effort Score (CES).

A proxy metric is used when it is expensive, very difficult, or impossible to measure something directly. Proxy metrics only correlate with the intended state rather than measuring it directly.

If the NPS metric is going down, it might be that the quality has not changed, but the NPS is now measured in a market with a different culture. Similarly, CSAT and CES only correlate with quality because people's responses are quite often significantly influenced by various response biases (when people simply don’t share the truth in the surveys). One can say that usually when quality goes down, NPS, CSAT and CES go down, but it’s not granted.

While the quality from the clients perspective can’t be fully measured, there is still some value in the proxy metrics: they serve as a signal for further analysis.

Continuing with the family road trip example, let’s say your destination is San Francisco, California. The quality of the journey depends on your family’s perception of the trip — and there is no definitive way to measure their perceptions. Before the trip, you may have agreed to hit specific sightseeing spots along the way and not to drive faster than 90 miles per hour, which serve as proxy metrics for the quality of the trip.

If you rely only on the proxy metrics, you’d think driving at a reasonable speed and spending 5 minutes at each stop would be enough. But if your family is frustrated by being rushed through planned stops, you play music everyone hates, and ignore all their requests and complaints throughout the trip, your family is unlikely to perceive the trip as a high quality experience.

Now let’s imagine gathering feedback from your family during the trip. At certain points, you ask your family to rate the quality of the road trip experience on a scale of 1-5. As you continue the journey, the score is gradually decreasing, even though you are staying below 90 mph and stopping at all the planned sightseeing spots. Two proxy metrics (speed and planned stops) tell you that you are achieving quality while another (feedback score) tells you that you are not.

The value in the proxy metrics is not to definitively determine quality, but rather to serve as a signal for further analysis:

If you are driving under 90 mph but you notice some family members are looking nauseous, you should analyze the situation. Are you swerving too aggressively? Braking too suddenly? Is under 90 mph still too fast?
If you see that the feedback score is dropping during the drive, you should look into the cause. Is your family unhappy with the time spent at each sightseeing spot? Do they need more snacks? Could the music be improved?

Internal quality metrics and employee perceptions

So far we’ve talked about the quality from the client point of view or “external quality.” There’s also “internal quality” of the product, or as Martin Fowler explores it: “how easy it is to keep modifying the code.”

Internal quality is all about the sustainability of development. Every engineer has had to deal with “messy code” or “bad architecture,” when it was ridiculously hard to make any changes to the code. There are a multitude of reasons why an engineer can perceive the code as “bad” or “low quality,” but this perception is also very subjective.

There is no universal metric showing how “good” or “bad” the code is, as all engineers, along with their skills and knowledge of the codebase, are different and have different opinions. Of course, there are general signs of “bad” code, such as high cyclomatic complexity of a module or a function. However, there are valid cases where it is not possible to reduce cyclomatic complexity, for example, when building finite state machines. In any case, all the signs of “bad” code are still subject to interpretation of the development team, and any human interpretation is inherently subjective.

For example, one might think that the percentage of test coverage would be a good objective metric of code quality, but there are cases where the code coverage percentage might be decreasing, while the internal quality is improving. This might happen, for instance, when the low-value extremely flaky tests are removed from the codebase, or when a trustworthy third-party library replaces custom-built modules.

Like external quality, all internal quality metrics are proxy metrics, only correlating with internal quality. And, once again, the value in the proxy metrics is mainly to serve as a signal for further analysis.

Here we need to talk about the road trip in terms of the driver, passenger, and car experience. As the driver, you have to watch the speedometer, monitor the map, and follow all the passenger requests, even if they are unreasonable. You become so focused on external quality (your family’s perception of the trip) that you ignore other signs of quality. Let’s say two of your family members insist you drive off road through a desert. You know that your car is not built for such tough driving conditions, but your passengers (the customers) want to see the desert, so you drive off the paved road and ignore signs of tire wear and strange sounds coming from the engine. You might even push through extreme fatigue to keep driving your passengers where they want to go.

Maybe your family is happy and perceive the road trip as high quality for a time, but while focusing on the external quality (your family’s happiness), you neglect the internal quality (the car's capabilities and your happiness), which eventually leads to the car (and you) breaking down.

Clearly, neglecting internal quality in the desire to satisfy customers isn’t an ideal approach either. And just like external quality, there is no way to fully measure internal quality.

Metrics are signals, not finite measurements nor goals

We’ve stated that both internal and external quality can not be fully measured, but there are certain correlating proxy metrics.

As the correlation does not necessarily mean that the change of the metric value always indicates the change of quality, the main value for the proxy metrics is to serve as a signal for analysis.

If we see that the test coverage percentage is dropping, we should analyze why. Is it that we removed redundant tests which weren’t yielding much value, or have we started rushing and deploying new features without any tests?

If metrics change, we should never aim to improve the metrics instead of analyzing the underlying reasons for their change. Only the analysis will show if any actions are needed, and if so, which ones and where.

A friend once worked for a startup, building a multimedia player app for Android tablets. They started with their own custom database engine, and even managed to cover 40% of it with tests. However, the clients kept complaining about data being sometimes lost or corrupted. The team discovered that most of these problems were due to the issues in the database engine. Upon extensive research and multiple test integrations, they replaced the custom database with SQLite. Clients stopped reporting issues.

Switching to SQLite significantly improved the overall quality of the product, even though the test coverage dropped significantly.

What if instead of improving the quality they set the goal to get 100% test coverage? They would then have two options:

Spend years writing tests for the database while stopping or slowing down other development, essentially risking the whole business
Pay SQLite hundreds of thousands of dollars to get access to TH3 (extensive test harness for SQLite) for little reason — SQLite always tests their releases with the same TH3 harness.

Both options would lead to unintended harmful consequences of setting the metric value as the goal. When metrics are set as goals, Goodhart’s law kicks in and unintended consequences arise.

Back to the road trip. You learned from last time and decided to set some metrics for measuring the quality of the road trip. You create a clear plan for what parts of San Francisco you’ll be visiting, assign someone to be in charge of music for the entire ride, and set a goal to hit at least 15 roadside attractions and spend at least 20 minutes at each stop. You’ve improved all measurable metrics, so your road trip is indisputably “high quality” right?

Unfortunately, no. By planning for more events in San Francisco, you reduced the food budget for the entire trip so now your passengers are unhappy with the meal choices. The frequent and lengthier stops at roadside attractions slow you down and interrupt the flow of the music that another passenger painstakingly planned out. And this time around, your partner is pregnant and much more prone to motion sickness and sensitive to food smells in the car — their perception of quality changed, just like customers’ perception of quality does over time.

Quality can’t be fully measured, but there is value in information

Different metrics correlate with various aspects of the quality of the product and can hint at where and what to analyze.

The broader the metric, the harder the analysis:

If your NPS is decreasing, you will need to analyze pretty much everything in your product: from the market changes to your product UX and design, from performance to defects in code
If you see the number of requests to customer support growing, you should discuss with CS, QA, and Dev to determine what’s going on
If you see the number of flaky tests growing, you need to meet with platform engineers, QA, and Dev to investigate the cause
If you see the number of returns from QA to Dev growing, you need to meet with QA and Dev to figure out the reason
If you see the code review average time growing, you need to figure out the reasons with the Dev team

The reasons for why each metric changed could vary. The measurement alone doesn’t tell you anything, it might only hint at where to start the analysis. Like in the road trip example — everyone liked the food the first time so you keep it the same, but the satisfaction score for food goes down on the second trip. The decrease in food satisfaction doesn’t necessarily mean the food is the problem, it just signals that you should look into it. Further investigation reveals that your partner’s pregnancy makes them particularly sensitive to greasy food, and their complaints about everyone else’s food smells in the car make the rest of the family have a less enjoyable time.

How to use metrics and measurements

Take the following steps:

1. Choose the metric, start measuring and visualize for monitoring

With internal software quality, there’s plenty of metrics to select for monitoring. In their book Software Metrics: A Rigorous and Practical Approach, Norman Fenton and James Bieman list several. Here are some to consider:

Cyclomatic complexity measures the number of linearly independent paths through a program’s source code.
Code coverage measures the percentage of code that is covered by automated tests.
Defect density measures the number of defects per unit of code.
Defect resolution time is the average time taken to fix reported defects
Cycle time is the time taken from the start of a development task to its completion.
Lead time is the total time from the initial request to the delivery of the feature.
The number of returns from QA to Dev is how many times tickets are passed back to development after QA finds bugs.

For more detailed guidance, check our article on defect management.

Let’s take “the number of returns from QA to Dev” as an example. Say we have an old-school process where developers work on features, and each feature has a ticket in Jira. When a developer completes the work, they pass the Jira ticket from “in development” to “ready for testing” status, so that testers can pick it up for testing. If testers find defects, they log all the necessary information and pass the ticket back to the “in development” status so that the developers can fix found issues.

Then, we decide to monitor how many times tasks are “returned” from QA to Dev because we know that any defect found slows down the development and we really want developers to assure quality before passing the feature on to testers. We decide that there’s some correlation between “how many tickets are passed back from QA to Dev” and quality. We think that when the testers start passing more tickets back to developers, it’s a good signal for us to analyze the situation.

To start monitoring the metric, we need to display it. Jira allows us to do so via JQL, custom fields, and automation. Essentially, you will see how many times tickets go from QA back to Dev in a certain period of time, such as one week.

2. Apply Shewhart’s control charts to the measurements

Shewhart control charts are tools used in statistical process control to monitor how a process changes over time. Shewhart control charts help you distinguish between normal variations (random fluctuations) and actual issues (signals that something is wrong). With Shewhart control charts, it’s easier to understand if the process is in a state of control or if there are variations that need attention. The reason why Shewhart control charts are beneficial is because every work item is different, and the metric value might fluctuate a little naturally. When natural fluctuations occur, there’s no need for analysis or any other action.

Applying Shewhart’s control chart to the metric of tracking how many times QA returns tasks to Dev is quite straightforward:

Get the number of times QA returns each Jira ticket to Dev each week
Calculate the statistics:
- Mean (average): the average number of returns per week
- Range: the difference between the highest and lowest number of returns in the dataset
- Standard deviation: the amount of variation or dispersion from the average
Create the control chart:
- X-Axis: weeks
- Y-Axis: number of returns from QA to Dev
- Center Line (CL): the average number of returns
- Control Limits: calculate and plot the Upper Control Limit (UCL) and Lower Control Limit (LCL). Typically, these are set at three standard deviations above and below the mean.
Plot the data: for each week, plot the number of returns on the chart
Analyze the chart and look for patterns or trends that indicate a problem:
- Points outside control limits indicate a potential issue that needs further analysis
- Run of points above or below the center line suggest a shift in the process that also might need further analysis

Shewhart’s control charts help us filter out statistically insignificant variations in the metric value, thus giving us a better understanding if the monitored metric is signaling us for analysis of the potential issues. So based on the above graph, we’d want to investigate why there was a sudden drop in tasks returned to Dev in week 6 and a spike in weeks 7-8 before returning to average. At first glance you might think that week 6 was a great week. But the reality might be that half of the QA team was on vacation in week 6 so less tasks were returned to dev during that week and the numbers spiked in weeks 7 and 8 because the team was catching up.

3. When detecting a surge or a drop, analyze the reasons

If we observe a statistically significant surge or drop, we need to analyze the reasons.

Reasons are always different as processes, people, product, organizational structure, and company culture form a unique context.

The number of returns from QA to Dev growing could be attributed to various reasons such as:

A new skilled tester is hired and they start finding more bugs. Nothing needs to change, as the developers see it as fair and start paying more attention to quality and testing. Even with the metric going up, quality eventually improves.
KPIs for QAs focusing on finding bugs are introduced by management, leading to even minor issues being treated as bugs and tickets being sent back to development. The metric goes up while the quality goes down alongside relations between developers and QA. The solution would be to remove the KPIs.
Layoffs hit the development team hard: the most experienced and therefore well-paid developers are fired, and the quality diminishes. The analysis shows that QAs did start finding more true defects in the code. There is no solution except for waiting until the remaining developers learn or for the company to realize their mistake and hire more experienced developers.
A new manager starts imposing deadlines on the development team, causing them to cut corners and rush to push the tickets to QA. The analysis shows that QAs did start finding real defects, but the only solution is to inform the new manager of the damage he is causing to the product.
The company is purchased by a larger one and is forced to change its approach to work. Previously, QAs would simply talk to developers and fix bugs together right away, so tickets were rarely passed back from QA to development. Now, QAs are forced to do their work after development is finished, resulting in more tickets being passed back. The quality remains the same, but the delays increase.

4. Reassess the metrics regularly

As we agreed, different metrics only correlate with various aspects of the product quality, and can only hint at where and what to analyze. There might be cases when the correlation disappears.

For example, a bank might decide to gradually replace the old Cobol codebase which had 80% test coverage with a new system built in Java. The team agreed to have only the core functionality fully covered with autotests, meaning that the whole refactoring project will constantly show an overall drop in the test coverage metric. In this scenario, there is no need to constantly analyze what’s behind the test coverage dropping trend. It would make perfect sense to ignore this metric until the refactoring is complete.

The rule of thumb for reassessing the metric is simple: if upon a few instances of analysis you see that there’s no correlation between the metric change and the product quality, consider pausing monitoring for this metric or replacing it with a different one.

Stop measuring quality metrics and start using them as signals instead

Remember that quality is not definitively measurable. You can plan the perfect road trip with your family, hit every milestone, and reach the intended destination and still end up with a car full of unhappy passengers.

If all your metrics are showing “good” performance but your employees are fleeing the company (passengers bailing on the road trip), you’re doing something really wrong. If all your metrics are showing “good” performance but your customers are choosing your competitor’s product (family chooses alternate vacation options), you’re also doing something wrong.

Metrics can only serve as a signal to analyze the reasons and improve the processes upon analysis. Embrace the inherent subjectivity of the quality, use metrics wisely, and employ TQM.

Designing a team that would produce software of good quality: overcome mental sets

Vitaly Sharovatov — Thu, 22 Dec 2022 08:50:27 +0000

Building quality product strongly depends on the team’s ability to solve clients’ problems effectively.

The previous article detailed the procedural mechanism of problem-solving. This article is devoted to overcoming a psychological effect that negatively influences professionals’ ability to rationally reason about the constraints and goals: ‘mental sets’:

a frame of mind involving a model that represents a problem, a problem context, or a procedure for problem solving. When problem solvers have an entrenched mental set, they fixate on a strategy that normally works well but does not provide an effective solution to the particular problem at hand. A person can become so used to doing things in a certain way that, when the approach stops working, it is difficult for him to switch to a more effective way of doing things.

Abraham Maslow, in his book The Psychology of Science: A Reconnaissance, coined this as:

it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail

Every engineer can easily recall a case when some technology or approach is chosen simply because ‘ah it solved the problem before’, even when the current context differs a lot from the one when the original choice was optimal.

This video shows how the Papuan New Guinea tribe welcomed the explorers and happily traded with them.

Western goods were slowly commoditising via air cargo delivery.

A curious thing happened next: a ritual emerged. In an irrational attempt to ensure delivery continuity, members of the tribe started building airplanes, airstrips, and airport watch towers. They constructed all these objects using available materials: wood, straw, grass, and stones. Even headphones were carved from wood.

None of these objects worked; they were created as part of the ritual. The people believed that the goods they received arrived because of the airplanes and that by mimicking the landing process, they could continue to receive them.

This set of beliefs is now known as Cargo cult.

Cargo cult falls into post hoc ergo propter hoc category of beliefs:

The argument offers an explanation, based on a temporal ordering of the events, that confuses co-occurrence with causality: A happened just before B, so A caused B.

‘Post hoc ergo propter hoc’ logical fallacy is deeply rooted in how homo sapiens think.

Evolutionarily, it is beneficial to make causal connections between events that happen one after the other.

For example, if a child touches a heated iron and feels pain, they will likely avoid doing it again. If a tribe experiences an earthquake followed by a volcano eruption in which many members die, those who survive will likely flee whenever they feel another quake. Similarly, if a person makes a joke and sees people frown, they will probably not repeat it.

The human brain is particularly good at memorizing the result of a sequence of events, especially when they are dangerous: it is better to flee immediately when a carnivore is spotted, even if the sighting is inaccurate or mistaken.

The mechanism behind this is called implicit learning:

an inductive process whereby knowledge of a complex environment is acquired and used largely independently of awareness of either the process of acquisition or the nature of that which has been learned

Implicit learning requires minimal conscious effort: a child learns their first language or how to ride a bicycle easily.

In a few years, this child may be able to speak a language fluently without being able to explain its grammar rules. They feel what sounds right and what doesn’t.

Similarly, a child might be unable to write a textbook teaching others how to ride a bicycle.

In these cases, knowledge and skills are acquired implicitly, while the development of science is based solely on explicit learning:

a conscious operation where the individual makes and tests hypotheses in a search for structure. Explicit learning is equivalent with intentional learning of information.

People study, learn, process, adapt, hypothesize, and validate the information contained in textbooks through conscious, explicit knowledge acquisition, which requires a lot of mental effort.

However, with time and practice, the amount of effort needed to solve problems decreases, leading to mastery of the subject. This mastery is sometimes referred to as "unconscious competence" or intuition, where performing a skill becomes almost natural to the person.

"Unconscious competence" can lead to the "mental set effect", where a person's focus is narrowed to only considering solutions that have worked well in the past: the brain applies the same short neural circuitry seen in implicit learning and tries to save energy.

"Mental set effect" may reduce professional's ability to solve problems and lead to the use of "cargo-cult" solutions.

Examples:

‘Jira worked for us well in other company, let’s install it here too’
‘Only mongoDB will do here, installing’

Mental sets effects are hard to notice; people think they are doing a great job and consciously solve problems. They will provide a seemingly rational explanation of their thought process if asked. Psychology calls this rationalisation:

explicit reasoning we offer to ourselves and to others is often rationalization, that we act instead on instincts, inclinations, stereotypes, emotions, neurobiology, habits, reactions, evolutionary pressures, unexamined principles, or justifications other than the ones we think we’re acting on, then we tell a post hoc story to justify our actions

In the previous article I stated that efficient problem-solving is a process of multi-fold synthesis.

To avoid mental sets’ hindrance to the synthesis process, certain conditions must be met:

Collective interactive problem-solving is to be employed;
Psychological safety needs to be ensured to enable people to express ideas freely;
The team should be diverse so that the number of hypotheses is higher and different heuristics mechanisms are present;
People should be genuinely interested in working together.

Collective problem solving

A famous Wason Selection Task shows how irrationally and illogically people approach problem-solving sometimes.

Another example of a mental sets effect is well-illustrated by "The Candle Problem":

Studies prove that switching from individual to team collective reasoning improves the results significantly:

Our results show that teams trade closer to the rational level, learn the solution faster, and achieve this with weaker, less specific performance feedback than individuals

Similar results are found with mental sets:

Engineering interactive problem-solving environments involving manipulable artefacts that embed reasoners in a distributed cognitive system is not simply an exercise in making the task more concrete: It engages a broad range of skills that are more representative of thinking as observed outside the laboratory.

There is a strong scientific consensus that mental sets almost always disappear when the whole team collaborates on solving a problem, and everyone comes up with ideas and thoughts.

NB: There is even a scientific theory postulating that the human mind is ‘social’ and that reasoning works solely through interaction.

The whole team should participate in the problem-solving activity.

Psychological safety

Pieces of advice from the knowledge overlap article are paramount to follow: people need not be afraid to develop ideas.

Team diversity

Many studies suggest that a team comprised of people with different functional and heuristics diversity would solve problems better:

An ideal group would contain high-ability problem solvers who are diverse

...

our result suggests that diversity in perspective and heuristic space should be encouraged. We should do more than just exploit our existing diversity. We may want to encourage even greater functional diversity, given its advantages.

Every team member's experience, heuristics, and way of thinking are important to the process.

The problem-solving process is improved by a greater variety of synthesised triples (constraints, goals, and transitions): the more proposals brought to the discussion, the better.

Designing the hiring process so that ‘different’ people are filtered out during the ‘cultural fit’ stage means that mental sets will just get reinforced.

Interest

Problem-solving is a creative game of synthesis.

While it is possible to force people to solve problems, the way their minds work cannot be controlled. People will naturally come up with more straightforward solutions. Mental sets will have an even more substantial impact.

Finding ways to provoke interest in problem-solving is a better choice of action.

Also, a good mood helps:

Participants experiencing positive affect were more likely to detect the simple solution than participants experiencing negative affect. These findings reveal that affect modulates how much we are constrained by current mindsets.

How to start

The simplest way to organise collective problem solving is through an activity resembling a brainstorming session:

Brainstorming is a method of generating ideas and sharing knowledge to solve a particular commercial or technical problem, in which participants are encouraged to think without interruption. Brainstorming is a group activity where each participant shares their ideas as soon as they come to mind. At the conclusion of the session, ideas are categorised and ranked for follow-on action.

TQM talks on brainstorming quite a lot too:

it is a systematic procedure of weighing pros and cons, and looks at all the forces for and against a decision. Since creativity is the main key for this tool, brainstorming is used during the procedure

There’s a good book on brainstorming.

P.S. Thinkertoys: A Handbook of Creative-Thinking Techniques discusses various methods that can help enhance an individual's creative thinking skills, but collaborating with others tends to be more effective.

References:

Designing a team that would produce software of good quality: set up dogfooding

Vitaly Sharovatov — Wed, 07 Dec 2022 13:54:37 +0000

Many successful companies employ Dogfooding practice:

Eating your own dog food or "dogfooding" is the practice of using one's own products or services.

In 1980 Apple wanted to dominate the computer market. To achieve that, among other measures they introduced dogfooding:

"EFFECTIVE IMMEDIATELY!! NO MORE TYPEWRITERS ARE TO BE PURCHASED, LEASED, etc., etc. Apple is an innovative company. We must believe and lead in all areas. If word processing is so neat, then let's all use it! Goal: by 1-1-81, NO typewriters at Apple... We believe the typewriter is obsolete. Let's prove it inside before we try and convince our customers."

Michael Scott and Steve Jobs wanted all their employees to become the first users of Apple products: the dogfooding practice emerged.

Any practice or process activity should have a rational reason.

This article explores the rationality of Dogfooding practice. The way how problems are generally solved needs to be reviewed first.

Solving problems

A customer comes to the shop at 11 am and purchases a burger.

A customer then comes to the shop at 8 pm and cannot purchase a burger as the shop is closed.

Is there a problem?

Did the customer stay hungry?
Did the shop lose money?
Both?

How can the shop solve the problem?

What does it even take to solve a problem?

Wikipedia says:

Problem solving is the process of achieving a goal by overcoming obstacles

The mathematical representation of this thesis would be: y=f(x)

Where:

y is the goal,
x is the context containing chosen obstacles/constraints,
f is the solution: the way we operate with the context to achieve the goal.

All the variables in the equation need to be defined:

constraints need to be chosen
goal has to be picked
a way to operate within these constraints to achieve the chosen goal should be found

NB: Those familiar with BPM will notice interesting similarities:

Picking the goal is very similar to describing a to-be model
Choosing x the context looks like building an as-is model
Designing f process is the main area studied: the synthesis of a transition from one state to another.

Choosing the constraints

The context always contains an infinite amount of constraints. The practicality requires picking those that are important.

x transforms into a set of variables: x = {x1, x2, x3, ... xn }.

Choosing the essential constraints demands a good knowledge of the current state of affairs.

For example:

x1 should be the information source: how do we know of the problem and how many customers are affected
x2 might be shop economics: how profitable it is, how much money can be invested
x3 can be laws and regulations: from labour law to licensing
x4 may be shop position in the market: how good its reputation is, how well known it is

Ignoring (or not knowing) some of the constraints might make achieving a particular goal not possible or optimal, e.g.:

If x1 is a guess, then any investment in solving the problem is high-risk,
If x3 is ignored, and the 24 hour license cannot be obtained, changing the schedule might not be an option,
If x4 is ignored, and there’s a McDonald’s opening at a stone’s throw soon; achieving certain goals might not be optimal.

Picking the goal

Picking the goal is deciding what the new system should be like.

Most managers call this their ‘vision’: a guesswork or intuition (so-called implicit knowledge).

Coming up with a goal is pure synthesis.

There are even methods designed to help with goals’ synthesis, e.g., TRIZ.

Different goals might be chosen for the shop example:

y1 — clients don’t come at night
y2 — clients are served 24/7
y3 — clients cook at home when the shop is closed

Defining the transition for each goal

Goal y1 — Clients don’t come at night

What are the ways to achieve this goal within the chosen constraints?

The simplest way would be to inform the clients of opening times, f1.

y1 = f1 (x1, x2, x3, x4)

This might reduce clients’ frustration (affect x4) as they won’t try visiting the shop at night.

The transition will not require any changes to the way the shop operates ( x2 and x3 won’t change). Most likely additional revenue will not be generated.

Goal y2 — Clients are served 24/7

What are the ways to achieve this goal within the chosen constraints?

One way would be to start working 24/7, f2.

y2 = f2 (x1, x2, x3, x4)

This will require quite a significant change to the way shop operates, and x2 and x3 constraints should allow these changes, while x4 may influence generating more revenue:

laws should allow purchasing 24/7 license, and its price should be affordable,
hiring more staff is required to introduce night shifts,
shop reputation and marketing expenses need to attract enough clients at night.

Goal y3 — Clients cook at home when the shop is closed

What are the ways to achieve this goal within the chosen constraints?

One way would be to start selling ingredients for clients to cook their favorite recipes at home, f3.

y3 = f3 (x1, x2, x3, x4)

This way will require an insignificant change to the way the shop operates. x2 might afford this easier than in f2 case. This transition might also positively affect the shop’s reputation x4 and generate more revenue x2.

Comparing the solutions

The solution to the problem is the goal and the transition.

Some goals might be picturing a brighter future than others, while the corresponding transitions might be riskier or more expensive. Some goals might be modest, while the corresponding transitions are cheap and low-risk.

A solution should be weighed with the constraints.

Naturally x2 (the economics) constraint’ weight is taken into consideration, but in most cases, people tend to turn almost a blind eye on x1:

x1 should be the information source: how do we know of the problem, and how many customers are affected?

What are the possible ways to see the problem or find out about it?

Lean manufacturing has the practice of Gemba walks, where management comes to the production facility and works in the process:

if you want to improve your business, you need to learn more about your processes, your people, your customers, you need to go and see for yourself.

Managers and business leaders today are often so separated from the actual work by corporate structures. They have not seen the process. They have not spoken to any customers. And they don’t even talk to the people who do the work daily.

Everyone, from CEO downwards, has the responsibility to spend time on the shop floor watching how to produce the product or service

Gemba pays special attention to the production process inside the company.

Gemba doesn’t help much with providing context on what’s beyond the walls. Gemba suggests the employees talk to the customers, yielding just one link in the information flow chain.

There’s an inherent information loss in every act of communication, and the fewer links in the information flow chain, the better.

It’s also possible to have zero links in the information flow chain: the employees might become the customers themselves. The dogfooding practice emerges.

Dogfooding

Dogfooding in the shop example context discussed above would potentially imply:

an engineer living in that particular area where the most shop customer reside
an engineer becoming one of many customers of the shop, following their usual shopping and behavioural patterns.

The engineer becomes an average customer, knowing all the peculiarities and fully understanding the customer’s context.

It is much easier to organise dogfooding in software development. As soon as the engineer becomes the first customer of the product or service, they begin to see the domain area's problems. The engineer becomes the trustworthy source of x1.

Good examples of companies implementing dogfooding practices are 37signals, Apple, Microsoft, and Gitlab.

Gitlab states dogfooding as the company value:

We use our own product. Our development organization uses GitLab.com to manage the DevOps lifecycle of GitLab itself.

Our entire company uses GitLab to collaborate on this handbook. We also capture other content and processes in Git repos and manage them with GitLab.

When something breaks, doesn't work well, or needs improvement, we are more likely to notice it internally and address it before it impacts our larger community.

37signals went even further, they build products for themselves first:

we built Basecamp out of desperate necessity. We needed it bad. Without it, we were embarrassing ourselves.

Dogfooding significantly reduces the information loss in defining x1 constraint, building the base for problem solving. When the presupposition of x1 constraint is wrong, any investment in problem solving is high-risk.

Engineering is problem-solving:

Engineering is all about solving problems using math, science, and technical knowledge. Engineers have solved a lot of problems in the world by designing and building various technologies. We have everything from machines that can breathe for you in hospitals to suspension bridges to cross rivers to computers we use every day. All of these things were once designed by engineers using the engineering design process.

Solution quality depends on the synthesis of the optimal goal and transition based on the chosen constraints.

Dogfooding culture helps reduce information loss in choosing the most important constraints.

Additionally, the dogfooding practice reduces ‘work alienation’.

This study discovers the seriousness of the work alienation problem:

work alienation is defined as a disengaged, negative and even painful outlook on one's job, or simply as “estrangement or disconnect from work”. Unsurprisingly, this state is empirically linked with poor performance, low commitment, career dissatisfaction, substance abuse and turnover intentions, and should therefore be avoided at any cost.

NB: Negative effects of high turnover are covered here.

Another study suggests that if the employee identifies with the role, they are not alienated:

We believe that people who are extensively committed to work not only identify with the work role, they are also engaged in the world of work

Dogfooding reduces alienation of work as the engineer sees and feels how their efforts resolve real problems, the daily problems they witness.

Dogfooding limits of applicability

Any practice has limits of applicability: conditions where the practice is suboptimal or even negatively affects the system.

Public tender contracts often disallow dogfooding; extensive requirements specification is provided instead.

Certain domain areas make dogfooding almost impossible or very expensive to practice, e.g.,

airplanes are not operated by engineers who design and build them
doctors use medical equipment on patients

Management needs to check if the dogfooding practice is viable to adapt.

How to start

If you decide that dogfooding practice might benefit your company processes and culture, there are two approaches: ‘revolution’ and ‘evolution’.

Apple used the ‘revolution’ approach in 1981: managers replaced the typewriters in the office with Apple II computers. The employees had no choice but to use the product.

Enforcing such a change might yield markedly undesired results if company culture resists the change.

A good example would be the culture of strict specialisation, where people are discouraged from partaking in activities outside their predefined role.

Introducing dogfooding will require slowly changing this culture. Managers must promote dogfooding and provoke employees’ interest in using the product.

Good leaders are role models; showing the employees how the leaders use the product daily might be a good idea.

The ‘evolution’ approach demands a slow but steady pace. Taking employees’ feedback — and acting upon it — is a must. As soon as people see their voices ignored, dogfooding is doomed.

References:

Designing a team that would produce software of good quality: get rid of individual performance reviews

Vitaly Sharovatov — Wed, 02 Nov 2022 13:14:04 +0000

Most companies practice individual performance reviews (also called performance appraisals).

This practice is quite well-established even though 40 years management science concluded that this practice caused great damage to the teams in multiple areas.

I can only guess the reasons why the practice still exists: lack of knowledge and/or sunk cost fallacy effect.

W. Edwards Deming condemned performance evaluations as a deadly disease afflicting American management. He argued that performance evaluations nourish fear, encourage short-term thinking, stifle teamwork, and are no better than lotteries.

This article is mostly focused on performance reviews’ detrimental effects on quality.

Teams have emergent properties, only teams can produce value for the client.

There’s almost no value in producing something, companies prosper by producing quality product.

Quality demands effective information flow, the following team characteristics are required too:

Low turnover
Constant learning from peers
Common goal and common interest in collaborative problem solving
Focus on teamwork

All these properties are severely damaged by the individual performance review practice.

Low turnover

Individual performance review practice increases the turnover.

The usual performance assessment is unavoidably subjective:

the assessee in self-presenting their results for a certain period of time
the assessor in assessing the presented results

The assessee compiles their self-assessment based on their subjective understanding of the achieved results.

Assessee’s judgement is clouded even more by a spectrum of biases, from Dunning-Krueger effect to Impostor Syndrome.

NB: Rationalwiki.org lists ~90 well-studied cognitive biases which affect human behaviour, cognition, thinking and decision-making.

NB: People are also almost unaware of the strength of the biases’ effect.

The assessor’s reads the compiled self-assessment and rates the results subjectively too.

Assessor’s judgement is clouded even more by a great number of biases, from Halo effect to Central tendency bias.

The longer the assessment period, the more clouded, subjective and biased both the assessee’s and the assessor’s judgement is.

Human memory is full of wonders, events are not stored and retrieved precisely and accurately, but lived through and changed every time a person recalls something.

There’s a separate list of memory biases, effects which not only cloud the recalled event, but sometimes change it quite significantly:

Rosy retrospection bias. We tend to remember the past as having been better than it really was, which leads to judging the past disproportionately more positively than we judge the present. As the Romans said: memoria praeteritorum bonorum, or “the past is always well remembered.”

Consistency bias. We incorrectly remember our past attitudes and behaviour as resembling our present attitudes and behaviour, so we feel like acting in accordance with our general self-image.

Mood-congruent memory bias. We better recall memories that are consistent with our current mood. For instance, feeling relaxed may bring back relaxing memories; feeling stressed may bring back stressful memories.

Hindsight bias. We have an inclination to consider past events as being predictable—also called the knew-it-all-along bias.

Egocentric bias. We recall the past in a self-serving manner, such as remembering our exam grades as being better than they really were, or remembering a caught fish as bigger than it was.

Availability bias. We often think that memories that come readily to mind are more representative than is actually the case. This is why people tend to overestimate the likelihood of attacks by sharks or the number of lottery winners.

Recency effect. We best remember the most recently presented information. At a trial, evidence presented last may be the clearest in the juror’s memory.

Choice-supportive bias. We remember chosen options as having been better than rejected options.

Fading affect bias. Our emotions associated with unpleasant memories fades more quickly than our emotions associated with pleasant memories.

Confirmation bias. Our tendency to seek and interpret memories in a way that confirms our prior hypotheses or personal beliefs.

To sum it up:

the assessee only believes they recalled something right (the longer period, the worse)
this belief is clouded even more by the assessee’s self-esteem-related and other personality biases
the assessor perceives the result and tries matching it with their clouded recollection of the events (beliefs, essentially)
the matching process is clouded even more by the assessor’s biases

It’s almost as if an unknown amount of noise is applied to a signal a few times, where the signal is the event in the assessment, and noise is all the recollection/reformulation/comparison process in the assessee’s and assessor’s brains.

Does it make any sense to conclude anything based on a result with completely unknown signal-noise ratio?

Studies prove this subjectivism often triggers the feeling of unfairness: the assessee thinks the assessor’s judgement is unfair:

They [assessments] are in many cases perceived as being procedurally unfair in terms of the Folger and Bies (1989) criteria. Inadequate consideration is given to employee views; there is evidence of bias, inconsistency, poorly justified feedback and a lack of honesty in the process.

Our research confirms findings that the appraisal reinforces power relationships and is often perceived as being open to abuse

Dissatisfaction with processes and the resulting distributive justice had a negative impact on employee perceptions of fairness and could act as a barrier to organizational effectiveness

Another study once again proves that people are all different, their memory and judgement is affected by biases differently, and that there’s almost no way the assessee’s judgement of their work and effort will match the assessor’s judgement.

Judgements about fairness can only ever be based on an individual’s expectations

Adding to this, people irrationally compare results from the current assessment with the previous ones, as this study shows:

Expectations about performance marks, and the comparison of those marks with the previous year, had a very strong influence over levels of satisfaction.

Multiple studies explore the detrimental effects unfairness causes on the productivity, such as this cross-cultural one:

Our results revealed that both unfairness approaches (OJ and ERI) were positively related to job-burnout and turnover intention

It makes perfect sense: as performance reviews make people feel that their efforts and results are judged unfairly, the motivation drops and they might consider leaving.

The situation gets much worse if pay rise is tied to performance reviews.

It’s a well-known fact that in most companies newjoiners negotiate higher salaries compared to existing employees.

Perceived judgement unfairness in this situation is even stronger — employees might even feel that the whole practice is established to persuade them in their knowledge and skills inferiority and postpone pay rise.

Loss of interest and motivation, similarly to increased turnover, have detrimental effects on quality.

Focus on teamwork

In case the employee is somehow still motivated to stay and work in the company, performance reviews’ focus on individual contribution will shift employee’s focus from collaborative team work to achieving what’s exactly measured.

Good bye teamwork, hello Goodhart’s law:

When a measure becomes a target, it ceases to be a good measure

Every measure which becomes a target becomes a bad measure – is inexorably, if ruefully, becoming recognized as one of the overriding laws of our times. Ruefully, for this law of the unintended consequence seems so inescapable. But it does so, I suggest, because it is the inevitable corollary of that invention of modernity: accountability

As soon as the employee is focused on improving the individual contribution measured in the assessment, quality decreases because:

the employee loses the motivation to help others, to teach and learn
the employee focuses only on what’s measured

While, as Dr. Deming says:

the most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.

All the effort managers and employees spend on executing the practice is waste, it simply could be spent on improving the system.

There’s a beautiful thought experiment illustrating Dr. Deming’s ‘95/5 rule’:

“95% of variation in the performance of a system is caused by the system itself; only 5% is caused by the people.”

The original goal of the individual performance reviews was to boost the teams productivity and yet the practice:

focuses on individuals, reducing the teams to mere groups of competing individuals
generates stress for everyone
negatively impacts motivation and interest
increases turnover
brings the feeling of unfairness
focuses people on what’s measured, distorting the system
simply wastes money

Samantha Evans and Dennis Tourish beautifully coined this in their study ‘Agency theory and performance appraisal: How bad theory damages learning and contributes to bad management practice’:

conventional appraisals prioritise hierarchy over intrinsic motivation, distrust over trust, and the importance of individual effort over that of building sustainable, co-operative systems.

References:

Designing a team that would produce software of good quality: turnover effect

Vitaly Sharovatov — Tue, 18 Oct 2022 14:49:26 +0000

Turnover:

turnover is the act of replacing an employee with a new employee

Since 1983 detrimental effects that companies suffer from high turnover have been studied quite a lot.

First and foremost, high turnover cost companies a fortune.

Popular sources like Gallup publish alarming articles like this one:

A trillion dollars.

That's what U.S. businesses are losing every year due to voluntary turnover. And the most astounding part is that most of this damage is self-inflicted.

The American think tank ‘The Center for American Progress’ published a more thorough meta-study where they reviewed 11 relevant research papers on the cost of employee turnover. Here’s the quote:

Very highly paid jobs ... tend to have disproportionately high turnover costs as a percentage of salary (up to 213 percent)

NB: ‘very highly paid’ they consider jobs yielding up to $200 000 per annum

So for every employee who’s gone, the overall price the company pays might be as high as twice his yearly salary.

It’s pretty easy to calculate the direct turnover costs, I suggest you do this too. Just factor in the following expenditures (taken from the study mentioned above):

‘separation’ costs: exit interviews, severance pay
temporary cover costs
replacement costs: advertising, agency fees, hiring (screening / interview / selection / bg-checks / signup bonus / relocation costs)
training costs
costs incurred as the new employee learns their job, including reduced quality, errors, and waste

You will find the last cost (associated with the quality loss) quite hard to measure. Maybe you can count the number of defects increased as folks did in this study:

Despite the manufacturer’s extensive quality control efforts, including stringent testing, each percentage point increase in the weekly rate of workers quitting from an assembly line (its weekly worker turnover) is found to increase field failures by 0.74%–0.79%.

But stats like this can be collected only for cases when there’s a clear separability between the production and consumption of manufactured goods. This separability allows insulating the productive operations from many environmental disturbances.

So I would say that the detrimental effect that high turnover causes on quality should be considered an indirect turnover cost.

Moreover, software development is an industry with a high proportion of intellectual labour, and software product quality highly depends on employees’ cognitive skills and emotions.

High turnover detrimental effects on knowledge, motivation and emotions are indirect turnover costs. This study categorises these indirect turnover costs like so:

Lost productivity for the departing employee
Lost productivity due to the need to hire temporary employees
Coping with a vacancy or giving additional work to other employees
Costs incurred as the new employee learns their job, including reduced quality, errors, and waste
Damaged employer brand
Reduced morale
Lost clients and lost institutional knowledge

I’ve written articles on how information flow efficiency, emotions, knowledge, feedback style and interest influence quality. To me, it’s quite evident that with every person leaving the team, all these areas suffer, rendering the quality worse.

What makes things worse — there’s no way to measure precisely how much worse the quality gets, all we know: it gets worse.

So what exactly happens to a team when its member leaves?

team communication graph is shattered, new connections are to be established and normalised
team morale is shattered, and social dynamics of fear/uncertainty emerge

When an employee leaves the team, the team is destroyed and the new one is to be assembled.

In some cases, the cost of reassembling the team is low (say, when the team knew and welcomed the employee’s departure); sometimes, it’s ridiculously high (say, the informal leader of the team got sacked) and can even cause others to leave or consider leaving.

In any case, as the quality depends much on information flow efficiency, it seems obvious that the quality-focused strategy should imply serious efforts invested into retaining the team.

And as Dr. Deming said:

the most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.

References:

Designing a team that would produce software of good quality: emotions and interest

Vitaly Sharovatov — Tue, 11 Oct 2022 09:15:19 +0000

As I’ve said in the first article, quality strongly depends on the efficiency of the information flow in the system — the more effective the information flow is, the less loss occurs, and the higher the quality.

Efficient information flow implies that parties perform a cognitive function well:

Cognitive function is a broad term that refers to mental processes involved in the acquisition of knowledge, manipulation of information, and reasoning. Cognitive functions include the domains of perception, memory, learning, attention, decision making, and language abilities.

People need to be able to communicate, learn, make decisions, etc.

The current scientific body of knowledge states that emotions affect cognitive functions, and I’m devoting this article to the topic of emotions.

As one study says:

Emotion has a substantial influence on the cognitive processes in humans, including perception, attention, learning, memory, reasoning, and problem solving. Emotion has a particularly strong influence on attention, especially modulating the selectivity of attention as well as motivating action and behavior. This attentional and executive control is intimately linked to learning processes, as intrinsically limited attentional capacities are better focused on relevant information. Emotion also facilitates encoding and helps retrieval of information efficiently.

As another study shows a strong correlation of the level of emotion positivity and memory performance:

In two experiments, context memory performance was disrupted when words were presented during negatively valenced film clips, whereas it was enhanced when words were presented during arousing film clips. Free recall of
words presented during the negatively valenced films was also disrupted. These findings suggest multiple influences of emotion on memory performance.

And another one:

Positive affect systematically influences performance on many cognitive tasks.

And another one:

positive emotional states could reduce switch costs and enhance both types of creative performances

And another one:

This article reviews evidence indicating that, in most circumstances, positive affect enhances problem solving and decision making, leading to cognitive processing that is not only flexible, innovative, and creative, but also thorough and efficient

And another one:

Creativity is enhanced most by positive mood states that are activating and associated with an approach motivation and promotion focus (e.g., happiness)

As you see, a growing body of scientific knowledge proves that positive emotions improve cognition, creativity, memory efficiency, etc. (while negative emotions and stress have a detrimental effect on cognitive function and memory).

Most companies and HR have long since started investing in employee well-being:

the state of feeling healthy and happy

The 2021 HR Sentiment Survey found that 68% of senior HR leaders rated employee well-being and mental health as top priorities. Companies now seem to have started understanding the importance of investing in positive emotions.

While I fully support the management initiatives focused at providing decent work condition, health support and perks of all sorts, I do believe that the cornerstone is missing — there’s no proper attention to a certain feeling which is very positive, crucial to the software development and inherently important to homo sapiens striving — interest:

a feeling or emotion that causes attention to focus on an object, event, or process

Designing the team culture and processes so that the interest is provoked seems quite a good idea for the company.

You often see the promise of ‘interesting tasks’ and ‘hard problems’ in the job description companies post, but rarely do they fulfill the promise.

At this moment, I usually hear, ‘yeah, but it’s impossible to make work interesting for everyone’.

And yet we get paid to solve users’ problems. There is no need to invent complicated ‘interesting’ tasks; we need to allow people a certain amount of exploratory freedom.

Engineering interest is solving problems, not doing tasks: if I assign you a task ‘code me this form’ or ‘paint the button red’, chances are you will consider these assignments boring as soon as you have finished a few hundred.

If instead, you gave me the user’s original problem or, even better, facilitated our dialogue, the user would tell me the problem, and suddenly solving the problem would be much more interesting.

I think you all must remember the pure joy of discovery amusement — where you suddenly realise how a certain problem can be solved.

Imagine your working day mainly consisted of problem-solving and discoveries, but not of tedious tasks.

And what’s even more peculiar:

The motivational function of interest extends to activities that are not inherently interesting or appealing. Interest can bolster motivation to complete tasks that are boring and tedious

If the team is interested in solving a problem, ‘boring’ tasks are not demotivating anymore.

In addition, there’s a great benefit in having a diverse team — a team of people of different levels of expertise and different specialisation.

Chances are some team members never tried solving a particular type of problem: what can be boring to a senior developer might as well be of great novelty to the junior one. If properly done, learning as well as teaching is a thrill.

There’s also a great benefit in organisational agility regarding roles: if a backend developer wants to learn some frontend to help the team solve the user’s problem, welcome it, and don’t force specialisation, knowledge overlap helps the team build things better.

There’s a great book — Exploring the psychology of interest, and here’s a few quotes:

Izard and Ackerman (2000) suggest a motivational function—“interest motivates exploration and learning, and guarantees the person's engagement in the environment

Diverse areas of research demonstrate beneficial motivational effects of interest. Studies of successful adolescents indicate that “undivided interest” promotes the growth of expertise (Rathunde, 1996, 1998, 2001)

research, reviewed in chapter 3, shows that people process texts more deeply and remember the material more accurately when the texts are interesting (Hidi, 2000; Schiefele, 1999)

interest is adaptive because it motivates people to develop diverse experiences that can be helpful when unforeseen events occur (Fredrickson, 1998).

The broaden-and-build model of positive emotions (Fredrickson, 1998, 2001) proposes that interest, like other positive emotions, lacks short-term functions associated with survival. Instead, interest serves long-term developmental goals: curiosity about the new and the possible broadens experiences and attracts people to new possibilities. The broaden-and-build model suggests that interest cultivates diverse experience by orienting people to new and unusual events and facilitates the growth of competence by motivating sustained activity in a specific area.

To sum it up:

interest is of essential value to the team morale, motivation, and performance
interest bolsters cognitive functions, creativity, learning, and growth
interest is provoked by novelty and problem solving, so don’t give tasks; help the team solve problems

References:

Designing a team that would produce software of good quality: management complexity

Vitaly Sharovatov — Mon, 03 Oct 2022 18:03:01 +0000

Every manager does their own bit of people management at work, even though many don’t realise it.

Some call it leadership; some call it just management.

By definition, people management is:

the practice of recruiting, training, engaging and retaining employees to optimize their talent and maximize their productivity

Essentially you as an EM assemble, manage and maintain a productive team.

You will undoubtedly recall thinking, ‘will Julie be capable of doing this task, or will she need mentorship?’ or ‘do we need another frontend developer?’ or ‘what happens with our testing when Mark goes on vacation?’.

All these questions are a day-to-day routine of an EM.

And the more people you have, the more managerial complexity you get.

Managerial complexity is essentially the mental effort price you pay for managing the team. You have to:

plan your team capacity and decide on hiring if needed
interviewing, hiring, and onboarding
providing training and supporting employees with mentorship
building, assessing, and adapting processes
providing transparency on goals and mission
maintaining the engineering culture
creating and managing a supportive atmosphere and dealing with conflicts

And it seems evident that the more people you have, the more effort you need to put in.

It’s quite a popular question among the fresh EMs — ‘how do I cope with so many 1-1s?’. Some people even try to automate what should never be automated — humane and honest communication — by creating ‘1-1 scripts’.

The popularity of this question confirms what I’ve stated in the previous article — it’s tough to sustain the managerial complexity of a team of more than 6-7 developers working on tasks individually.

I postulate that this limit is caused by ‘synchronisation costs’. Whenever you rely on individuals interpreting things, you have to bear in mind that there’s a high cost of synchronisation:

Every single person has their perception and understanding of every single task. I covered this in my article on information loss
Whenever people synchronise upon the significant work bit ‘finished’ (significant effort applied by the individual based on their understanding and perception), clash and frustration are inevitable: we always associate ourselves personally with the results of our work. And there are no measures to diminish the frustration fully: once it appears, it never can be erased. I covered this psychological issue in my article on code review.

Most work conflicts results from the clash or tension derived from this.

Try recalling examples of people changing attitudes to each other due to the synchronisation failure: ‘your code is bad’.

There are so many articles offering you a way to provide non-toxic / non-offensive negative feedback, so many strategies (sandwich technique, critique the work not the person) to provide palliative treatment to the issue.

But, as Harold F. Dodge said:

You can not inspect quality into a product.

Providing any feedback after the synchronisation failure is late.

Quality is assured before the defect appears.

Synchronisation issues are better prevented, and not dealt with.

And the only way to prevent synchronisation issues — surprisingly the same way which allows having more people on the team while retaining the quality and speed — pair programming:

Pair programming essentially means that two people write code together on one machine. It is a very collaborative way of working and involves a lot of communication. While a pair of developers work on a task together, they do not only write code, they also plan and discuss their work. They clarify ideas on the way, discuss approaches and come to better solutions.

If you employ this approach, you get the following benefits:

synchronisation costs are twice less
team performance increases
the number of defects decreases significantly
better psychological safety in the team

As two (or more) people work simultaneously on one task, they employ a constant feedback paradigm meaning that there’s no ‘significant effort’ wasted (sunk cost fallacy doesn’t kick in). There’s no reason for the conflict inside the pair.

And when the pair produces something which is not accepted by the whole team, they stand together, and therefore the individuals are not that hurt (‘there’s always someone on your side’).

Surprisingly, pair programming also means higher team productivity, quicker and easier learning and knowledge sharing, low-cost onboarding of new members, almost the best interview efficiency, easy sharing of the goals and the mission, and spreading and maintaining good engineering culture.

A few studies back my thesis up:

The Costs and Benefits of Pair Programming:

Using interviews and controlled experiments, the authors investigated the costs and benefits of pair programming. They found that for a development-time cost of about 15%, pair programming improves design quality, reduces defects, reduces staffing risk, enhances technical skills, improves team communications and is considered more enjoyable at statistically significant levels.

The Case for Collaborative Programming:

A field experiment was conducted using experienced programmers who worked on a challenging problem important to their organization, in their own environments, and with their own equipment. Findings revealed that all the teams outperformed the individual programmers, enjoyed the problem-solving process more, and had greater confidence in their solutions.

Evaluating Effectiveness of Pair Programming as a Teaching Tool in Programming Courses:

Findings suggest that the more students are involved in this activity, the more they enjoy it and the more they learn by collaborating with their partners. When comparing different effectiveness measures, their perceived learning, quality of work, and enjoyment during than increased productivity outcome.

To sum it up, the studies I mentioned in the previous article proved that the optimal number of people on the team working on individual tasks is 6-7.

Here I hypothesise that with pair/mob programming you can almost double the number of people on the team, let them do things in rotating pairs or a mob, and with that:

reduce the synchronisation costs
double the speed of the team
improve the quality of the product you’re building significantly

Essentially, with pair/mob you are getting your team to become a good team, not a group of individuals, so no wonder why you can get so many benefits. These all are called ‘emergent qualities of the team’.

Try it out for yourself. I suggest employing pair programming in your team, and only then think about increasing the number of people.

P.S. What’s even more interesting is that there’s a huge body of thought on an ‘interactionist account of reasoning'. Scientists are seriously considering that the human brain is not well-suited for individual reasoning and that humans can rationalise much better together.

There’s a great example: the so-called Wason Selection task — a seemingly simple task that reveals how irrational most human beings are individually. And there’s a great study showing how exposure to interaction with others improves reasoning.

We are social creatures, so let’s use this to our benefit!

References:

Designing a team that would produce software of good quality: team size

Vitaly Sharovatov — Mon, 26 Sep 2022 10:38:27 +0000

As discussed in the parent article, team size should be as small as possible to provide value to the client.

To note:

Studies prove that the smaller the team is, the higher the quality.

There’s another study by QSM which says:

Teams of 3 or fewer people were the most likely to achieve balanced schedule and cost/effort performance.

Teams of 2 or fewer achieved the best cost/effort performance.

Teams of 2 to 4 delivered the best schedule performance.

Teams of more than 4 people resulted in dramatically worse cost/effort and schedule performance.

The stated reasons are as follows:

information is lost in every act of communication (hence communication path should be as short as possible)
there’s a limit to productivity increase when work is parallelised (due to sync costs)
management complexity grows with every new team member

The first reason is well defined in the parent article.

I want to go through the second in more detail here and the third in the following article.

Limit to productivity increase with work parallelisation

The longer the team performs a task, the higher the chance that the task is waste — the world changes quickly, and clients’ needs change even quicker. The Agile movement started to motivate the companies to perform in smaller iterations much faster, checking in with the client to see if the delivered product satisfies their needs (and this is exactly what quality is!).

Naturally, the team is motivated to deliver each iteration as soon as possible — to check if the quality is at the desired level.

So speed is crucial to quality.

And we started this article by saying that the fewer people, the better the quality.

These two principles seem to contradict, don’t they?

Not really.

Amdahl’s law states that there’s a limit to a speed increase when the work is parallelised.

Any parallel execution requires coordination (synchronisation). Synchronisation by default is synchronous. People and tasks aren’t equal, so some people and tasks will have to wait for the sync to happen. When there’s a wait time, delivery time decreases.

Please read this article for more details, but here’s the essential quote:

Assuming 12% of the workload needs to be synchronized to facilitate healthy collaboration within the team, and if we want a reasonable minimum efficiency of 0.3, then the answer is a size of 6-7

The concept we get from Amdahl’s law is simple: reduce the amount of parallel work to the bare minimum (not more than 6-7 items in parallel) and make sure this work is performed as fast as possible.

Interestingly, Kanban method has a concept called WIP limit:

WIP is the number of task items that a team is currently working on. (in parallel — added by me)

WIP limits are considered an important prerequisite for delivering value to customers as fast as possible An empirical study of WIP in kanban teams

Kanban talks a lot about figuring out how much work to parallelise and what effects to expect.

There’s a study proving that the use of Kanban’s concept of limiting WIP does result in improving the team delivery speed.

So if we unite Kanban’s principles and Amdahl’s law, we see that the optimal number of people working in parallel on tasks shouldn’t exceed 6-7.

References:

Designing a team that would produce software of good quality: knowledge overlap

Vitaly Sharovatov — Mon, 19 Sep 2022 08:15:03 +0000

In one of the previous articles, I stated that team knowledge should overlap.

How can one achieve that?

Knowledge comes from learning and practice.

Learning can’t be enforced.

People learn best when team design, culture, and work conditions encourage them.

As our goal is to make people’s knowledge overlap, the most obvious way is to allow them to work together and explain to each other what and why is going on.

There are a few things that will certainly block these social dynamics:

EM (Engineering Manager) focus on employee’s individual performance
EM focus on employee’s time utilisation

EM focus on employee’s individual performance

Suppose an employee sees that the only (or the principal) thing which matters to their manager is individual performance in delivering tasks assigned to this employee. In that case, it’s unlikely that this employee would be motivated to help others or explain anything.

If you as an EM have individual performance metrics, check Goodhart’s law (possibly read Deming’s or Akoff’s books on systems) and get rid of these metrics.

EM focus on employee’s time utilisation

Physiology of learning

The neuroscientific research about learning has revealed the negative impact of stress and anxiety and the qualitative improvement of the brain circuitry involved in memory and executive function that accompanies positive motivation and engagement.

As learning can’t be done under stress, and even more — learning is an ‘exploration’ kind of activity, one needs free, or, possibly, specially dedicated time for learning.

However, learning and exploration take significant mental effort. If out of 8 hours you propose 2 hours for learning, expect much less performance in the remaining 6 hours.

A good piece of advice to initiate proper learning is to have a dedicated day once in a while (a week is good) where both the student and the mentor have no workload or pressing matters (releases, unfinished tasks, deadlines).

Psychology of learning

Learning requires people to show their vulnerability; hence trustworthy and safe environment with mutual respect is a prerequisite.

Vulnerability:

While learning, one moves from ‘not knowing’ to ‘knowing’, and the first step is to reveal that there’s a lack of knowledge, and this is psychologically challenging, and even more so if the person is very senior in one area and they are just starting in another.

The learning person needs to ‘abandon’ their seniority status and still be accepted and respected in the team.

The first step in achieving this is to make sure everyone does this — everyone learns something and shows vulnerability, which must start with the EM.

You, the EM, must lead people by your example: show your vulnerability, strip off your ‘status’, admit you don’t know some tech, ask the specialist to teach you, make the process transparent, and show how cool it is to your team. Make mistakes in public.

Safe environment:

Make sure no one judges the learning progress; as I’ve said, learning is exploration, and every explorer has their speed.

The strategic goal of getting knowledge overlap in the team can be easily undermined by an appealing urge to make people learn faster and compete.

Please do not copy the school grading system: it encourages competition and is currently severely criticized and revised by modern pedagogy scientists.

When we consider the practically universal use in all educational institutions of a system of marks, we can but be astonished at the blind faith that has been felt in the reliability of the marking systems. (Finkelstein, 1913)

Isn’t it hypocritical to preach about the importance of innovation in education while simultaneously clinging to a system of grading which is almost as archaic as it is useless. (Ferriter, 2015)

Do not introduce any tests or examinations to assess the level of ‘learning progress’ — learning never stops, and you don’t need to add stress bound to arbitrary dates.

Flexibility:

It’s tempting to set a ‘growth plan’ for each person in stone. However, your goal is not to make people equal; your goal is to have everyone learn from each other.

You don’t need abstract knowledge with grades in each area; you need knowledge overlap to reduce information loss in communication between people.

Essentially, you create a community of people who naturally want to learn and share (teach). Everyone is different; you as an EM can’t and shouldn’t make them all the same, but instead encourage them to support and teach each other.

Glory:

Everyone wants ‘a moment to shine’ i.e., some positive feedback on their newly acquired skills. Why don’t you provide an opportunity for people to show off what they learned via something like public talks?

Occasional volunteer Friday talks seem to be quite a good idea. Don’t push people, but encourage and help them prepare.

To sum everything up:

allow proper free time for learning, maybe a half a day per week, like Friday afternoon?
lead the process by example, learn from your team, and do it publicly
give people space and a good atmosphere: don’t grade, don’t ‘track progress’
allow people to show off what they’ve learned

References:

Designing a team that would produce software of good quality: no time pressure

Vitaly Sharovatov — Thu, 08 Sep 2022 10:53:47 +0000

In the previous post, I suggested considering getting rid of the whole concept of time pressure imposed on developers.

Here I will review the most common scenarios when this concept emerges and think with you if the concept can be avoided.

It’s well-established knowledge that the stress induced by time constraints causes loss of quality, unhappiness, loss of creativity, and even toggles serious health issues:

Time pressure comes in two forms: "let’s just do it faster" and "we need to meet our estimates".

So it’s obvious we need to get rid of the time pressure and somehow work without it, but how do we do that?

Getting rid of the first form (‘let’s do it faster’) is simple — just don’t rush the team.

But getting rid of the second form (‘estimates and deadlines’) is harder.

Let’s go through the most common use cases for estimations and deadlines and see how we can avoid them.

First, let’s define what the estimate is.

Cambridge dictionary:

a guess of what the size, value, amount, cost, etc. of something might be

A guess, a prognosis.

We live and work in non-deterministic systems and operate in highly volatile market. It is impossible to guarantee that our guess will come true.

However, most managers still use estimations; the top reasons I have witnessed as to why are:

The public tender contract requires estimation
The Manager “needs to know” when a feature is to be done
The Manager “needs to understand” how much a feature will ‘cost’
The Manager “needs to be able to choose between features based on cost”

There's proper math behind it, but the simple rule of thumb is: the more resources (time in most cases) we spend on estimation, the more accurate the estimate is.

Weather forecasting uses priceless supercomputers to increase prognosis accuracy.

Why spending so much?

Well, better (more accurate) weather forecasting yields multifold investment returns: logistics can be optimised, and human lives can be saved.

It seems a good idea to align each reason with the main goal of any investment — a goal of providing value.

So, what value does estimations provide for each reason listed?

Public tender has ‘time’ and ‘budget’ as a condition

The time and budget constraints are external to the company so if the company needs this contract it has to comply.

The company has to decide if producing the required system within required budget and time is economically beneficial.

➕ investing in estimations is rationalised

Manager needs to know ‘when it will be done’

This request is very abstract.

Most common reasons for this request that I’ve heard are:

marketing campaign starts on the X date and we need to understand if the team manages to build and launch a feature
an external event with set date emerged (for example, FIFA world cup) and managers need to understand if certain features can be launched on time

Marketing campaign

Marketing team works alongside IT team for the same company.

The whole company goal is to make client happier.

So why can’t these teams collaborate in a high-trust environment?

Can marketing predict exact number of sales they will generate for a certain budget (essentially, the outcome)?

If not, why does the IT team need to predict the outcome of their work?

Compare these two scenarios:

IT team produces some value for the clients and marketing team starts advertising this new value to potential and existing clients
Marketing team dictates the IT team ‘the advertising campaign will be launched on X date, you must be on time’.

To me it looks that the first scenario doesn’t provoke quality decrease (due to rushing) and also doesn’t require spending resources on estimation.

➖ investing in estimations is not rationalised

External event occurred

There might occur an external event which makes the company require estimation from the IT team.

For example, the country wins an opportunity to hold UEFA championship. Then the company needs to invest time in estimating a certain set of features to make sure it’s worthy to invest money into building those features so that they are delivered before the UEFA championshop.

If the result of estimation shows that even the minimum minimorum of features can’t be built on time, the company will decide now to even start building these features.

There’s a good question though — how often do events like this occur?

In the example of UEFA championship there are two things to weigh and compare:

amount of resources required to estimate + amount of resources required for ensuring the project is done on time
the probability of country winning the UEFA championship

If the probability (2) is high, it might be beneficial to bid on it and prepare the software in advance and just turn it on when needed.

➕/➖ not sure if estimation is rationalised

manager needs to understand how much a certain feature costs to develop

There usually are two reasons for this particular request:

manager wants to compare features based on their cost
manager wants to understand if a certain feature ‘is worth it’

These reasons are deeply intervened.

First, before any development estimation is done, product ‘estimation’ needs to be reviewed.

If a manager ‘just tells to build the feature’ with no prediction on how the feature will benefit the clients and the company, there’s absolutely no need to do any ‘estimation’ of development.

In this scenario manager just throws unsupported guesses to the development team – pure leap of faith.

So in this scenario manager doesn’t get any estimations — manager first need to explain why the company needs the feature, i.e. why the manager is going to spend development team time and effort.

If our manager does provide economical rationalisation (i.e. product value estimation) for the feature, next step is to check how accurate their estimations were.

If manager’s previous estimation accuracy is low, there’s no need to do development estimation, it is a ‘leap of faith’ anyway.

Here’s an illustration:

Let’s say that a Product Manager comes up with three features: Square, Circle and Triangle.

Product Manager provides the following profit estimation:

+-----------+-------------+
|  feature  |  estimate   |
+-----------+-------------+
|  Square   | $50k-$90k   |
|  Circle   | $150k-$200k |
|  Triangle | $300k-$400k |
+-------------------------+

Dev team gives the following estimations for time the team will spend on the delivering features:

+-----------+-----------+
|  feature  |  estimate |
+-----------+-----------+
|  Square   |  10-40d   |
|  Circle   |  15-30d   |
|  Triangle |  20-24d   |
+-----------+-----------+

Built together with team time converted to money (1d to $1k for simplicity):

+-----------+---------------+------------+
|  feature  |  revenue est. |  cost est. |
+-----------+---------------+------------+
|  Square   |  $50k-$90k    |  $10k-$40k |
|  Circle   |  $150k-$200k  |  $15k-$30k |
|  Triangle |  $300k-$400k  |  $20k-$24k |
+-----------+---------------+------------+

It seems that all these three features were well-worth to do, according to our estimations:

Square will yield at least $10k in the worst case scenario
Circle will yield at least $120k in the worst case scenario
Triangle will yield at least $276k in the worst case scenario

Now let's see what happened when the features Square, Circle and Triangle got developed and delivered to the clients:

+-----------+----------------------------+-------------------------+
|  feature  |  revenue result            |  cost result            |
+-----------+----------------------------+-------------------------+
|  Square   |  $90k   (from $50k-$90k)   |  $13k  (from $10k-$40k) |
|  Circle   |  $200k  (from $150k-$200k) |  $30k  (from $15k-$30k) |
|  Triangle |  $30k   (from $300k-$400k) |  $40k  (from $20k-$24k) |
+-----------+----------------------------+-------------------------+

As we can see, a few estimations were wrong.

Square yielded $57k.
Circle yielded $170k.
Triangle yielded loss of $10k.

Well, real life is tough, estimations fail lots of the times.

But here's what's important: it makes more sense optimising the quality of estimation on the left, on the product side.

There's another conclusion: there's no point in doing estimations in IT if PM doesn't have a decent estimation quality (and a proven record of providing estimates of decent quality).

It’s also quite obvious that comparing any features on their cost only makes pretty much no sense as well — it makes sense only in the case when PM’s revenue estimation quality of prognosis is ridiculously high :)

➖ investing in estimations is not rationalised

Considering all the reasons and knowing the negative impact of time pressure, decide rationally if you still need some prognosis mechanism.

If you do decide that the estimation is required, check statistical forecasting in Kanban. The approach is simple yet provides enough accuracy for most cases and doesn’t impose any time pressure on the team.

There’s also a good book #NoEstimates which explains how to meet goals without imposing time pressure on the developers.

So the advice from the previous post stands: consider running the development process without the following concepts: estimations, story points, deadlines, sprints (or any other practice imposing time pressure on engineers).

Designing a team that would produce software of good quality: team processess issues

Vitaly Sharovatov — Thu, 01 Sep 2022 15:24:15 +0000

In the first chapter, I focused on team topology aspects that affect the information flow in principle.

Here I will focus on the aspects of team design that influence people's behaviour (or motivation).

I’m sure you must have seen ‘tired and unhappy’ receptionists at some hotel, try to recall: were they keen on satisfying your needs?

Why were they behaving this way? The information could flow; nothing was technically stopping it; they could get all the required details from you, the client.

They might have had all the skills required to do the job.

Merriam-Webster defines skill as:

the ability to use one's knowledge effectively and readily in execution or performance

So, granted, we created a good team topology and hired skillful employees; what is the next prerequisite for the successful information flow?

It's tempting to say that the receptionist in our example simply lacked motivation (or desire) to set the information flowing and get all the details from us.

But things are much more complicated. Along with the information flow, motivation depends severely on the system design too.

The main concept here is 'unhappiness'. When your employee is not 'happy', they won't even want to make the client happy; they won't even want to try to understand the client.

Happiness is a very broad concept, but surely work influences employee’s happiness.

Many studies ( 1, 2 ) show how strongly productivity and quality are related.

There’s also a great overview study from Daniel Graziotin (Universität Stuttgart) and Fabian Fagerholm at Aalto University. The study tries to answer the big question — are happy employees more productive? And the answer is: yes, they are more productive when happy.

So it seems pretty reasonable to postulate that unhappy engineers yield worse results in quantity and quality.

Let’s see what the study shows as reasons for engineers’ unhappiness.

1. There should be no time pressure

Being stuck in problem-solving and time pressure are the two most frequent causes of unhappiness ... developers feel bad when they are stuck and under pressure, and several detrimental consequences do happen

Engineers are pressed for time when the software development process employs concepts like estimates, deadlines, and sprints.

Very rarely, there’s a rational reason to have deadlines in software development, and even more, estimations at all. I suggest you watch my talk on the topic.

There’s also a good book #NoEstimates which explains how to meet goals without imposing time pressure on the developers.

Advice: consider running the development process without the following concepts: estimations, story points, deadlines, sprints (or any other practice imposing time pressure on engineers).

2. Professional ethics should not be broken

The third most frequent cause of unhappiness is to work with bad code and, more specifically, with bad code practices. Developers are unhappy when they produce bad code, but they suffer tremendously when they meet bad code that could have been avoided in the first place. As our participants stated, bad code can be a result of management decisions aiming to save time and effort in the short term.

Martin Fowler has an excellent article explaining how ‘cutting costs’ on tests makes development more expensive.

This obligation to do the work properly is called professional ethics.

When a patient has surgery, no one can demand a surgeon to finish the operation sooner. Everyone respects the surgeon’s work. Surgeons’ professional ethics rely solely on the surgeon’s professional judgment of how the procedure is to be done for the best of the patient.

In software development, it’s the team's professional judgment that is to be used to determine how the code should be written.

The advice here is simple: don't break the team's professional ethics by telling them to skip something to achieve short-term savings. As Fowler says, you'll lose more in the long term. And the team happiness will decrease too.

3. Process activities should not toggle inadequacy of self

Similar negative effects were mentioned regarding third persons (such as colleagues, team leaders, or customers) who make developers feel inadequate with their work, forced repetitive mundane tasks, and imposed limitations on development

Some examples of activities that might toggle the feeling of inadequacy of self:

testing after development
code review

3.1 Testing shouldn’t be done only after the development

Testing after development (i.e. quality control) is the verification process if the delivered software matches the requirement. The process is not that bad, but it shouldn’t be used exclusively.

If it’s used exclusively, there are two bad outcomes:

if the quality is not good, a significant rework is required, and this has a detrimental effect on the team's happiness (and is a huge waste of resources, too) — no one will be happy when the result of their work is thrown away
a person who performed the QC procedure might be perceived as someone diminishing the effort the engineer spends on producing the software

Shift left testing approach appears to solve these issues. Read it, employ it.

3.2 Code reviews should potentially go away

Like the previous factor, checking at the ‘done’ stage is too late and ineffective.

Here’s an article to read:

Every human being associates her work results with themselves. Valuing things we spend effort on doing is innate to our nature.

Whenever a developer receives feedback like “this has to be redone”, they will inevitably associate this feedback with themselves.

The more effort a person spent on doing something, the more vulnerable she is to the critics. And while she hadn’t been taught properly how the problem should have been solved, she considers negative feedback even worse.

Internet’s full of posts and articles on how to “solve” this “negativity problem”. Most of these posts talk about how to provide more “positive” feedback.

Microsoft even hired a fully dedicated specialist working on this issue.

However, this all seems palliative treatment, none of these approaches offer to solve the problem properly.

Solving the problem properly would be introducing the teaching or planning phase before the effort on doing the task is spent and a feeling of achievement emerges.

Advice: think about what behaviors your process activities encourage in developers. Are they encouraged to collaborate (as in pair/ensemble programming or shift-left testing), or are they inspired to frown upon the results of each other work (as in code review or testing after production)?

Employ pair programming, or at least move away from code reviews when you see that there are no more significant issues found during the code reviews.

4. Boredom should be avoided

Similar negative effects were mentioned regarding third persons (such as colleagues, team leaders, or customers) who make developers feel inadequate with their work, forced repetitive mundane tasks, and imposed limitations on development

The key points here are: repetitive and mundane.

Software development (as pretty much any other type of intellectual labour) is all about solving problems. World changes often, new problems arise all the time, and there’s pretty much always something new to solve.

Yet, sometimes engineers do need to do something ‘not new’, something they know for sure how to do.

The problem which toggles unhappiness is when the amount of repetitive tasks is getting high.

Some examples:

manual verification of code quality (testing)
writing boilerplate code
solving the same task over and over again (after code review rejection, for example)

Advice:

check the amount of rework (waste), think about minimising it (possibly get rid of code review in favour of pair/ensemble programming or at least pair/ensemble technical design)
consider having repetitive work automated (shifting from manual regression testing to automated testing or even TDD)

Another fundamental concept here is ‘mundane’ work. Mundane generally implies a lack of interest. But what is this lack of interest? How does interest emerge in developers?

If there’s no understanding of why the work is needed and what value it brings to the team or the client, ‘interest’ will unlikely emerge.

So it seems to be very important to have full transparency on why the work that’s requested is important and what value it brings.

To summarise the advice here:

Don’t impose non-rational time pressure on your employees; use either a statistical approach to estimations or no estimations; prioritise projected value, not cost.
Don’t break the professional ethics: you hired professionals to do their job; intruding the process to cut the costs negatively impacts the total cost of ownership of the system and the team motivation.
Design the process to minimise waste: employ pair programming, TDD, shift-left testing.
Make sure employees fully understand a clear vision of the purpose of work.

References:

Designing a team that would produce software of good quality: information loss problem

Vitaly Sharovatov — Thu, 25 Aug 2022 11:38:00 +0000

As Harold F. Dodge said:

You can not inspect quality into a product.

He meant that the system should be designed to produce a quality product.

This article is about team design: principles of team construction that will allow the team to produce a quality product.

We know what ‘product’ is: an information system allowing the client to do something.

Are you aware of the team & process design aspects which influence the quality of the product your team builds?

Let’s dig in and start with the definitions.

What is quality?

Two standards are defining the quality:

General one (for products) — ISO 9000:

quality degree to which a set of inherent characteristics fulfils requirements

And the one talking specifically about the software quality — ISO 25000:

software quality — capability of software product to satisfy stated and implied needs when used under specified conditions

Both talk about a "match" between what the client desired and what's present (produced).

We want to produce exactly what's needed or wanted, so the match is "maximised", i.e., the quality is at its highest.

A few areas of problems related to quality might occur when you create a team.

Information loss in conversion

The idea or desire can never be fully expressed: idea-to-language transformation yields information loss. Consider it similar to digital -> analogue transformation.

Then the idea (transformed to language) during the perceiving process will have even more information lost; consider this analogue -> digital transformation.

The reason for the loss is simple:

the language itself is ambiguous
the language can only provide us with a symbolic representation of the meaning
the symbol is perceived differently, so the meaning is not constant but temporally, contextually, and individually bound

The picture above illustrates the ambiguity of the symbolic system caused by the interpeters’ different perspectives (viewpoints).

There are two areas of science studying this particular set of problems: Semiotics and Semantics.

Very simply put, "chess" can be describing "the board of chess" or "the game of chess", "green" can't be explained to a blind person, "salty" can be perceived differently not only by different people but by the same person at different times.

Broken telephone experiment illustrates how the information loss is increased: the more links in the information flow chain we have, the more times the information undergoes perceiving/verbalising transformation, the more information loss we have.

1. The less links in the information processing chain, higher the quality.

This principle has four implications:

teams should be sitting closer to the client
teams should be as small as possible
teams should be crossfunctional (no silos)
teams members’ knowledge should overlap

1.1 Team should be sitting closer to the client

The client is the source of information, the closer the team sits with the client, the better the quality is: simply less information loss. Ideally, the client representative should sit right with the team.

Also, if the client representative sits with the team, the team can show the interim results more often and get more feedback.

This is very important simply because the client might not even fully understand the need, or the first way their desire is implemented might not be satisfactory for the client. The shorter the feedback cycle, the less unnecessary work is performed.

1.2 Team should be as small as possible

If just one engineer is helping the client to achieve what they need, the information loss is minimal — there are just two links in the chain: client -> engineer.

If the information follows the client->business analyst->architect->system analyst->engineer->QA flow, there are six links, six times the conversion is done, the information is lost significantly.

There are only two ways to overcome the inherent information loss: specify everything in more detail or reduce the number of links in the chain.

Specifying everything in more detail requires time, which is of essential value (the world changes too quickly). The whole Agile movement started with the goal of delivering clients what they wanted but faster and better.

Studies prove that the smaller the team is, the higher the quality.

One should also note that the more developers the team comprise, the higher the chance of ‘ownership blurring’ where individuals see less obvious results from their efforts.

This might result in people losing motivation or having ‘this can be done by someone else’ attitude.

Managers should ask themselves: what is the bare minimum of people required to produce what the client wants?

Is it possible to produce the software product with just 1 employee talking directly to the client (or their representative)? If not, can we get away with 2? Or 3?

1.3 Team should be cross-functional (no silos)

Social psychology has multiple studies on a concept called In-group favoritism:

the tendency to respond more positively to people from our ingroups than we do to people from outgroups.

Minimal group paradigm describes:

Experiments using this approach have revealed that even arbitrary distinctions between groups, such as preferences for certain paintings, or the color of their shirts, can trigger a tendency to favor one's own group at the expense of others, even when it means sacrificing in-group gain

Basically, people easily form groups and instantly start prejudice against the outgroup members or even discriminate against them.

As shown, the reason for the separation might be as insignificant as the colour of the shirt.The reason for this, as studies say, is:

Because our ancestors lived in small social groups that were frequently in conflict with other groups, it was evolutionarily functional for them to view members of other groups as different and potentially dangerous. Differentiating between “us” and “them” probably helped keep us safe and free from disease, and as a result, the human brain became very efficient in making these distinctions

Knowing this particular social dynamic that people have in groups, it seems evident that there’s no place for separating teams based on specialisation.

That’s exactly why the concept of an interdisciplinary (or cross-functional) team was created.

The cross-functional team comprises a number of people of all specialisations required to deliver value to the client.

Do not separate team members working on a product into sub-teams based on specialisation, and you won’t get negative social dynamics of intergroup discrimination or prejudice.

1.4 Team members’ knowledge should overlap

When your team comprises people specalising only in a very narrow field of expertise, each act of communication will require additional ‘bridging’ efforts to overcome the information loss.

Generally speaking, if your iOS developer understands backend development, their communication will be easier.

This is one of the reasons for such concepts as T-shape, Devops and TestOps emerged.

To summarise the outcomes of the ‘less links’ principle:

get a client (or their representative) right in the team
figure out what minimal amount of people with what specialisations your team needs
don’t separate specialists into sub-teams, have them focused on delivering what the client needs
make sure their knowledge overlaps

This will reduce the information loss and therefore reduce ‘waste’:

Waste is any activity that produces no value for the customer or user. This depends on the product you’re building.

In the following article, I will cover the concept of quality loss due to system design peculiarities.