An Incomplete Guide to Product Development

#scrum #agile #management #architecture

During the last years, I have seen a lot of companies struggling to setup their processes that allow them to efficiently develop software products and services they can sell to their customers. In this post, we will try to identify the main concerns and responsibilities that need to be adressed for successful product development and provide practical guides on how a possible process structure could look like.

But there is SAFe?

You might argue that we do not need to do this exercise as we already have SAFe (or LeSS, Spotify, OKRs ... you name it). Those frameworks work very well if you control the whole problem space and you can either develop internal products or have other companies developing them for you. But as a company having a dozen or even hundreds of customers, your customers will bring their own frameworks with them, expecting you to be integrated in their processes.

This creates a world where we have one or multiple product contexts as well as a potentially large number of customer contexts. However you will structure your processes and teams, the different, often conflicting needs the existence of those contexts implies will be there - if you address them or not.

So - What are the Needs?

To get a clearer picture of what we actually would like to solve, let us have a look at the different stakeholders in the overall process:

Stakeholder	Needs and Expectations
Customers	They want a solution to their problem, and they want it cheap and right now. They expect you to understand their internal wording and to align with their processes. They do not care whether you are developing a product or a customized solution for them - in fact, they would prefer a custom solution, as long as it is cheap.
Your Management	They want happy customers that are willing to pay a lot more than the costs you produce. They want an illusion of stability, so please provide a roadmap for the next ten years. Also they will sabotage this roadmap and the identity of your product on first opportunity to get a new customer.
Product Owner	The poor soul that has the responsibility to make customers, the management, the developers, and basically everyone happy. Either they accept that they cannot be perfect or they will quickly burn out.
The Team	They are mainly expected to give a high throughput of well-designed and high quality features. They are often more interested in their software product than the problems the product should solve. They will annoy everyone in the company about technical debt until their throughout falls below a certain threshold were customers become increasingly unhappy, causing management to dictate a whole rewrite that will block your company for the next three years.

Of course there are additional roles such as architects, project managers, product managers, business analysts, scrum masters, testers, devops and security specialists, UX designers and so on, but for our processual solution we will focus on the very core - be assured that they will find their place in our framework as well.

Separating Concerns

Now that we know the needs, we can start to shape our requirements for our product development process, already bringing us closer to a possible solution. Let us have a look at what we want and how we could address those issues.

The Customer Space

We want someone who understands what the customers wants, how they work and what is really important for them (meaning what they are willing to pay for, and if possible, how much). We are willing to make this experience as pleasant as possible for the customer, so they get the feeling they can come to us with new ideas and opportunities. If they are using SaFE, we are happy to send someone to day-long meetings.

There are a lot of different ways to organize this, usually a combination of responsible sales guys, key accounts and project managers. As you will not have enough resources to flatter all customers in this manner, you will need to categorize and set priorities. This organization is out of scope for this post.

For our process the main considerations are

Provide a space where you can organize your collaboration with the customer the best way possible
Do not expect nor force the customer to understand your company, processes or tooling, it just wastes their time
Do not allow this problem space to infiltrate your product spaces (for example the wording, but never something like release cycles), but ensure to take away the important requirements, time schedules and constraints (we will see how to do this later)

Practical examples would be:

You may want to have a separate Jira project for them, do not allow their input to directly create user stories in your product backlog
Do not force them to join your reviews (mind you, they do not care about your product)
If they prefer phone calls, get on the phone, if they want a status meeting, schedule one
Have your key accounts or project managers join your reviews and they will eagerly tell their customers in a language they understand

The Management Space

This space allows us to align the requirements of our customers with our product strategy and the roadmaps for each product. We want every larger topic to be visible in a unified list, allowing us to prioritize the value the company should provide next. If you have product managers, this is their home base - if you do not, hire some.

As with the customer space, our considerations for the management space are:

Provide a clear view on the next topics that are important for your organization
Specify use and cost for each topic (or even better: a business case)
Keep the topics based on use cases, so do not assign them directly to a product to avoid Conway's Law on this level. Use cases may involve multiple products at once. This also means that there can only be one management space.
Force decision makers to prioritize this list. Avoid priorities outside of this list as much as possible. Embrace conflict instead of avoiding it.
Avoid generating too much overhead to describe and estimate the topics. We do not need detailed concepts for things that will never be implemented (or in two years).
Add some kind of input process that ensures quality of the items as well as a clear communication strategy

This space is crucial for your process to work. Practical considerations:

Management will love this process until there is the first conflict where they cannot get everything they want as soon as they want. Defend this process at all costs and do not allow it to be replaced once a year with a new one that will suffer from the same underlying problem.
Separate the preparation of a topic from the decision. Product managers can prepare cases which then can be decided by management
Define iteration lengths and schedule regular meeting for prioritization. Anything from one to three months is fine.
Break down large topics and use cases into smaller ones, allowing to deprioritize or remove parts without meaningful value to your company. Do not allow topics larger than three months, or people will get impatient and the feeling of low performance arises. If needed, introduce a parent-child relation (such as "Epic" and "Feature" borrowed from SAFe).
Ensure everything that requires large chunks of resources of your development teams to be visible here. Both developers as well as your IT or DevOps platform will try to undermine this process for "technical tasks". Make those topics visible, which requires a clear value to be assigned to the topic. Otherwise management will lose trust in this process, and rightly so.
Small improvements should be able to by-pass this process. Think about defining some criteria (such as size) to determine when a topic needs to be added.
Keep the process simple. The more Jira fields you need to maintain, the less accepted this process will be.
If this process works for you, you can extend it with retros, time tracking, progress indicators etc.

The Product Space

Now that we have customer and management spaces, the life of your poor product owner got way easier. They will still contribute to and communicate with product and project managers, but their team simply can pull the next topics from the unified list, from which the product owner can also forge a product roadmap.

Besides this, there are not many considerations for this space, as we can apply regular Scrum here. We might question how we map solutions with products and technical services (Conway's Law once again), but this is out of the scope of this article.

Bringing Everything Together

To summarize the considerations above, we can sketch basic diagram to visualize the overall dependencies. This is not a process diagram but rather the way we separated our concerns.