loading...
Cover image for Anarchitecture

Anarchitecture

risavkarna profile image Risav ・9 min read

Development in Post-Agile, Managerless Teams

Software teams often need a personnel as a bridge to the world of businesses and customers. However do they really need these bridging professionals who have likely never developed any non-trivial software to also manage enterprise software teams? Can a team, especially a software team, self-organize?

Managers of Bullshit

Anthropologist David Graeber argues in his book aptly named Bullshit Jobs about the existence and social harm of meaningless jobs. Among the five types of entirely pointless jobs, non-technical professionals managing software teams would fall into the 'box tickers' and 'taskmasters' categories. These managers use paperwork and shallow documentations as a proxy for actually needed actions. They may even create extra work, unnecessary burden, avoidable tight deadlines and management strategies unsuited for a creative and technical team. I have known and worked with brilliant project managers over the years who definitely do not belong to these categories but I have also seen first hand the enormity of harm from these managers of bullshit and their similarly bullshit management techniques.

'Uncle Bob' Martin, one of the authors of the original Agile Manifesto, in his talk cum rant 'The Future of Programming' explains precisely why most software development teams are facing this manager invasion problem. He draws from multiple decades of living what we now learn as history of software development and pin-points that software development professionals have in the modern days come to be workers without a self managing and risk handling discipline. You can see in this writeup how his talk leads to the post-agile sentiment I am bringing forth here:

Businesses created this mess

"...business understands discipline... so business really liked scrum. They like scrum because of the discipline. They like scrum because it was simple and obvious - you could do these little sprints and you could do all these meetings and you could do these plannings and it made sense to the business and the business thought, "Yes, this certification thing is a great idea. We need to certify people."

... because they do not understand our craft and disciplines

"But the business does not understand us - doesn't understand programmers and in particular the business does not understand our disciplines. Business understands scrum disciplines. Business does not understand pair programming, ... test driven development, ... refactoring, ... simple design. These technical disciplines are not within the expertise of business and they shouldn't be. They belong to us; they are a part of our expertise. So business cannot approve or endorse them... or evaluate the risks. The risk frankly is ours to take... so we have to take the risk as part of our normal professional operations, as professionals. And that's what a professional does."

... and because they made agile movement a managerial mess

" ... [but then] there was the invasion of the project managers caused by certification. Certification turned into this siren song and attracted, frankly, all the wrong people. Agile was developed by technical people. It was a creation of the software industry. Programmers sat in that room and created the agile manifesto and agile principles. And then came certification and with certification, hordes and hordes of project managers started to get certified. They would sit for two days in a class and get a piece of paper and feel that that was somehow significant. And they literally took over the agile movement. The agile movement shifted dramatically towards the project management side."

... which paved the way for the post-agile world

"When you think now about the stuff going on in agile, you are thinking about kanban and, you know, lean this and, you know, lean startup that. Where's the technical discipline? Where's [sic] the programmers? Frankly, the programmers have all fled. If you go to an agile conference now, you don't see an awful lot of programmers there and you don't see lots of technical tracks there. Although they try...

Kent Beck said at Snowbird,

"The goal for Agile was to heal the divide between business and programming"

and I believe that the [agile movement] has completely failed... Somebody has to take the lead... Civilization depends on us in ways it doesn't yet understand. In ways we don't yet understand."

Take the lead?

So who should be the somebody that takes the lead? Would it be the technical XPers and craftsmen or would it be the businesses with their newfound love in a bastardized agility? Could it be both?

Martin Fowler explains in his article on FlaccidScrum about how scrum, albeit not a technical framework, can accommodate the craftsmanship best practices and discipline of, say, XP or TDD.

"In defense of Scrum, it's important to point out that just because it doesn't include technical activities within its scope should not lead anyone to conclude that it doesn't think they are important. Whenever I've listened to prominent Scrummers they've always emphasized that you must have good technical practices to succeed with a Scrum project. They don't mandate what those technical practices should be, but you do need them...

I always like to point out that it isn't methodologies that succeed or fail, it's teams that succeed or fail... Many people are looking to Lean as the Next Big Agile Thing. But the more popular lean becomes the more it will run into the same kind of issues as Scrum is facing now. That doesn't make Lean (or Scrum) worthless,

it just reminds us Individuals and Interactions are more valuable than Processes and Tools."

The Manifesto, XP and Failures

The Agile Manifesto reads:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

XP values feedback, simplicity, communication, courage and respect. Agile's original best practices include standups, stories, retrospectives, iterations, estimates, tests, refactors and continuous deployments.

Most teams fail because of lack of XP values and/or the managerial world view that prioritizes shipping faster - under the guise of pragmatism- over shipping with agile discipline - under the disparaging label of ideology. Some failures are softer and, in my experience, are caused by managers who make their developers so disconnected from business problems over time that they are no different from machine or telephone operators from the days of past. Then there are those managers who promote the idea that QA tools and tests are not part of getting the job done but rather a luxury.

However the biggest and most cliched failure source is viewing programmers as a species that

  • does not know how to talk
  • does not have 'social skills'
  • cannot understand clients' complex requirements

Hence businesses create the need to bring forth a magical middleman with their own non-technical management strategies and their greedy algorithm based product shipment strategies. These would also be the managers who will never agree to epics and tasks centered around code refactoring let alone complete overhauls and rewrites. Effectively, these are also the managers that cannot make rational risk management decisions owing to their lack of in-depth technical knowledge.

Developer Driven Development and *DD

Post-agile movement is primarily about removal of these aforementioned categories of managers. We do not need business managers leading software teams certified after a few weeks of training about software development management principles. We instead need an inversion whereby software developers learn management principles suited for their domain. And yes, this applies for even the junior most developers in the team since domain specific management principles are useful to know for everybody in the domain.

Another profound realization that led to Fred George's vision of the post-agile programmer anarchy is the fact that partnerships between the customer and the development team to drive the software development project is inherently flawed and cripplingly slow. Partnership with the customer should be about defining their problems and desirable features of the solutions, nothing more. Otherwise we would be in the business of building the fabled horse carriages of customer's dreams when we could be building them cars or even star ships that the customers do not yet know about.

This radical departure from the classical and agile status quo is possible because development teams understand business metrics, customer oriented development and follow the supremely powerful continuous feedback-continuous deployment cycle.

On a more fundamental level, it is the realization that

if you are going for a drive, you need to put the driver in the driver's seat.

Secondly if the driver is an experienced professional whom you have told where to take you,

the backseat drivers are not needed.

Over the decades we have identified that we need Test, Feature, Behavior, Acceptance Test, Data, Domain, Design, Model or, if you will, * Driven Development. Perhaps one or more of these apply to your team's needs but none of it will go into practice if the developers themselves cannot drive the development practices. As both of the Martins above would say, the teams fail not the tools and this responsibility of applying the correct technical tools is a technical team's responsibility not a business domain concern.

Developer Anarchy, Self-Organization and Anarchitecture

As separation of concerns is a dearly held value and a common meta-design pattern among developers, developers need to use this same principle for the architecture of their software and the development team's communication structures. These two are known to be inseparable and mutually influencing according to Conway's law.

Anarchy: Seeds and Needs

Businesses will always need to be lead by business managers. Software development will always need to be lead by software developers and architects. Businesses need not be concerned with the solutions and therefore should invert the control of solution design back to the developers to maintain orthogonality. Businesses can still partner with software teams in an agile way to properly define the problems and agree upon the desired features for any possible solution that may emerge in the solution space after enough iterations.

This is obviously only going to work if your development team has enough senior developers with industry experience and if there is enough communication among all its members. Working as a professional is very different from actually becoming a professional. But once you are lucky to have a team of professionals and younglings eager enough to become professionals, every developer can rightfully be an architect for their own region of the system they are building. You do not always have to go for a micro-service architecture for this to be possible. Simply let the most experienced developers create a functional core and everyone else develop their own parts of the imperative shell around it as efficient and well-integrated plugins. More trusted developers would get the responsibility of pairing up with the developers of the more important plugins.

Self-Organizing Superorganisms

This so-called 'post-agile' technique is still very much compliant with the original agile manifesto. However there are no managers of programmers whose wrongly placed formality and misplaced rules may constrain technical innovation and creativity. As these developers will like to fail as much as possible in the early iterations, the classical managers will have no reliable metrics of progress anyways. The ideal anarchic development team has team members jumping up to take responsibilities that no outsider manager could even have possibly foreseen. Such a team could democratically divide work based on the particular sub-specializations, interests and expertise of the team member instead of forcing everyone to see through a story or task's completion from top to bottom of the tech-stack owing to the assigned 'user stories'. These developers would work directly with the product owner or the customer and build more trust than what waterfall or impure agile derivatives can provide.

It follows that one big change in the development team's architecture would be the fact that development can no longer be story driven. As mentioned earlier, customers should produce formal, comprehensive problem descriptions and perhaps the desired features of the solutions but never directly participate in solution designs and technical specs. Anarchic teams cannot allow for customer side stories that disconnect developers from the actual problems the customers face. Neither customers nor managers know enough to tell the developers what or how the solution should be. If they did, the programming part would already have been automated or made into a business manager friendly automation DSL or UI. Developers who understand the problem clearly can empathize better with the end users and come up with better specs and deadlines for the solutions. They might also be able to squeeze in enough room for early failures and experimentation. Overall, such teams will behave as self-organizing superorganisms.

Anarchy to Anarchitecture

Lastly, neither Fred George nor the two Martins are advocating radical anarchy or complete departure from the lessons learned from scrum and other agile frameworks, be it managerial or technical. Even the anarchists are supposed to converse and ask each other. Anarchic teams have emergent leaderships and self-organization strategies rather than pre-ordained leadership or lack of well-suited organizational strategies.

How would the architecture of a system produced by an anarchic team look like? Surely the anarchy would leak into the architecture itself. Perhaps there will be an explosion of plugin and microservice architectures. Perhaps not. But I am sure that software architecture itself will be radically affected. What kind of architecture allows for flexible micro-architectures? The best kind. Probably. Could we still call it an architecture? Once again, the solution emerges from the problem definition itself.

We could call it anarchitecture.

Posted on by:

risavkarna profile

Risav

@risavkarna

Developer by need, theoretical CS guy by passion. My original posts reside at https://risav.dev. Please help us fight COVID using tech: https://bit.ly/covidsimteam

Discussion

pic
Editor guide
 

Complements on the enthusiasm and time taken to put into good text! You feel strongly about this, so the energy is there, and this is the most important to drive a community.

Nevertheless...

Where is your evidence? Besides convenient quotes, I see an article with confirmation bias scoring through the roof.

The whole article is centered on the developer and their experience from conducting business. This is very convenient to make your life good, to be listened to on your opinions, and to feel good about yourself in successes (I helped win) and failures (somebody else messed up).

Have you created a business where you have put bread on the table for 3-15 employees?

Have you surveyed successful business on their models in writing your article? With statistical significance of the surveys?

I suspect not.

In my opinion, the described model can only really work for large open source projects (even those are dictatorship to succeed with massive value), or niche consultancy and freelance. One may see this model also work, if a business has made quite some money that provide substantial insulation from market pressure, and wants to experiment with an ecosystem, but doubting this can work under any real market pressure.

The piece that in my opinion has most value from this article is, that it addresses the pressure on developers that stifles their creativity. Flow and Creativity are the most important commodities for a developer. Any improperly done pressure for dates, for compromises, and so on, will stifle creativity. It is a bit of an art to fix that, but has nothing to do with anarchy.

 

Programmer anarchy is an idea promoted by Fred George, an industry veteran who has had successes with his approach. You can see him claim here that, "managerless processes are not only feasible, but nearly indispensable" albeit just for "solving certain classes of problems (Complex problems in the Cynefin model) and for achieving rapid, continuous delivery (exploiting MicroServices)".

I would also encourage you to look up his case studies if you are genuinely interested. However these still do not make a large enough sample for statistical evidence. It is hard to have non-anecdotal evidence for something that has not been tried out widely enough.

As I mention in the earlier comment, I will be reporting back on how it plays out for our new team, which is comprised of smaller autonomous teams in 3 continents and developers often crossing time zones to meet partners and clients. However this is not my hypothesis and therefore on me to prove actually. I am merely summarizing the project management journey of the industry and how the founders of agile movement view the current state.

You can still make your own judgment of the situation by looking at companies like SonarQube, or the ones I have worked at as a developer/architect (you can google or find my LinkedIn/Xing linked in dev.to profile, since you asked) where the managers themselves are experienced developers and communicate with both customers and developers with near to zero semantic gaps.

Lastly, I insist that anarchy is the essential fix for teams with enough senior developers who can take the lead as and when needed. "Anarchic teams have emergent leaderships and self-organization strategies rather than pre-ordained leadership or lack of well-suited organizational strategies." That's why this is merely a lengthy opinion piece piggybacking on historical summary.

 

Thanks for the response, I am looking forward to your reports on success of the projects and business as a whole. Thanks for the useful pointers too, will checked them out.

A few notes.

Bad management is bad in general, not just to developers or in the context of software projects. The goal of a good manager is to "remove" the need of themselves through training independent, productive and powerful team members.

Emerging managers from a tactical background have one big challenge - how to delegate, really. A domain-expert manager under pressure for delivery often ends up "robbing" (subconsciously) his team mates from sharing opinions (most experienced person in the room), failing on their own, etc. As such, an manager that is expert can actually fail to build a powerful team of independent empowered experts.

On the other end, a pure manager over a technical team can never reach the depth needed for full velocity, or fail to gain the respect of the team members. It is a bit ironic really.

Therefore, a practical thing may be to actually compose a "handbook for the emerging tech lead" how they can let their team mates grow.

The goal of a good manager is to "remove" the need of themselves through training independent, productive and powerful team members.

Amen to that. Even a good developer should aim to make oneself obsolete although you may only get asymptotically closer to that goal currently.

I often cite the example of the movie Armageddon where they chose to train miners into becoming astronauts rather than choosing to train astronauts to learn the parts of mining necessary for their asteroid problem. I would love to see the industry shift towards training developers to be managers instead of certifying managers for technical project management.

I am still chuckling as my friend just shared his piece of misery to me after reading this article:

Developer: "This is out of the scope for this sprint. It will take extra resources. We cannot manage it now."

Project Manager: "I just feel like I should learn how to program and just show you; you can do it over night."

This was overheard at a prestigious company that my former roommate quit after 6 months so as to not let his PhD go to waste.

Good move, that guy, he apparently knows where his career needs to go, commendable on not taking shit too long!

What is funny, is that both roles can be right, from my personal experience on both sides of the trenches.

The project manager comes off as disrespectful in the end.

The developer is probably OK.

Having mentioned academic background, there is one note I want to make.

It is very demotivating to live as a developer knowing that you always have to make compromises with designs to make money and meet project timelines.

There is a positive way to rephrase this that is very very powerful:

  • "Find the right problem definition!"

A good developer with an academic background knows what is the BEST possible theoretical solution 90% of the time. This is what they are solving. Subconsciously, everything they solve is measured by that, and hence the satisfaction level is correlated by the size of compromise.

This is not good way to live.

What is important to understand when in business, is that the realistic problem rarely is academic. You can be in business with a half baked awkward and expensive to run legacy monster service for 20 yrs. But it is beautifully reigning in money, until the competition comes.

Therefore, the developer requires a lot of respect, but also training in pragmatic problem description. A dev has to wire up their internal reward system to always giving 100% to what is necessary for the problem at hand.

Identifying the right problem to be 100% no compromise on the spot is the art form.

Some call it scope, but there is psychology behind it too which if not aligned, is massively demotivating.

Well said regarding problem definition. I happen to have a similar but a bit of an extreme opinion on this, pieced together here:

"Businesses, [also clients or managers] can still partner with software teams in an agile way to properly define the problems and agree upon the desired features for any possible solution that may emerge in the solution space after enough iterations [but nothing more...]
Otherwise we would be in the business of building the fabled horse carriages of customer's dreams when we could be building them cars or even star ships that the customers do not yet know about."

Indeed! Building a team that draws motivation and satisfaction of delivering iterativelly is the challenge. It is the essence of realizing what developer profession means.

 

This is a really good idea on how to get rid of the bad parts of a lot of agile processes and give the developers the possibility to self-organize again.

However, "anarchitacture" is nothing that will always work out of the box once you remove the useless managers.

For a self-organizig team to be truly great, you need to have diversity in your team. And by diversity I do not mean having both men and women.

You need experienced developers as well as new developers.
You need people that like to explore and try new things and people that favor the proven ways of doing things.
You need developers with a broad view on the software and the industry as well as developers with a deep knowledge in important fields.

When your team is lacking much of this diversity than in can be difficult for your team to establish clear roles within the team.
But when your team has lots of widely diverse people the self-organization works just like magic.

One of the worst parts of bad agile processes is that the managers value the processes more than the people and not vice versa as it is described in the agile manifesto. This is definitely a point that would improve when trying out "anarchitecture".

Thanks a lot for this innovative idea :-)

 

That's an interesting point about diversity. I came close to touching upon it but still not quite close enough when I mentioned having enough senior developers in the team and "younglings eager enough to become professionals".

I will be mindful of the effect of technical diversity of our team and include your point when I report back on how anarchitecture plays out for our current projects.