Adyen Tech Radar: engineer’s autonomy to the next level

Startups have the strategic advantage of being flexible and to react to changes in far faster ways than larger organizations. This is possible because the amount of people involved is so small that communication and ideas flow easily among everyone. To imitate this advantage, agile methodologies suggest that larger companies aim to keep teams small while simultaneously making them owners of specific parts of the source code. This increases the autonomy of people at a team level, but not across the whole organization. Ideally, it feels like a cluster of many small organizations within a bigger one.

Taking autonomy within Adyen to the next level

How can one individual propose a new technology to be used across the entire company? How can this individual know if there's anybody else already trying it out, and join forces instead of reinventing the wheel? How could these ideas be heard, get attention and become relevant? How much freedom do we expect from our engineers to make these decisions?

These questions were raised when we started to discuss and define ways to push autonomy to the next level at Adyen. Together with these questions, we also identified the following problems:

1. There was no centralized (single source of truth) place to visualize the stack of technologies being used. This information was spread around and outdated most of the time.
2. Several factors were hard to identify: which technologies needed to be improved or deprecated? which new technologies were being evaluated? the benefits and impact of current and newly adopted technologies.
3. People (especially new joiners) had little or no context at all on the "why" technologies were chosen and “how” they evolved internally.
4. It was unclear who to talk to about a specific technology or a new proposal.
5. It was difficult to keep everyone informed and up-to-date.

If we look at these problems carefully, we see they're all related to communication and knowledge sharing of the whats and whys of our payment platform.

A Tech Radar fixes part of these problems

The good old Tech Radar fixes part of the problems mentioned above. It shows relevant tech stack information in a nice and interactive way. It segregates topics by quadrants representing different areas and rings (Assess, Trial, Adopt and Hold) to indicate the adoption level. A topic is introduced in Assess ring, moving up to Trial and Adopt for further evaluation, or moving down to Hold as a way to be no longer considered. It's really easy to understand the information and what it means. Thought Works had this great insight a decade ago when they needed a way to identify emerging and evolving trends in technology.

Alone, the Tech Radar is not enough

So with the Tech Radar we found the way to communicate, but we didn't find it as the way to boost our engineers' autonomy. Originally, it's made yearly by a group of experts after some weeks of constant interactions to re-evaluate the previous Tech Radar topics alongside new emerging trends to be included in the next release. Eventually they generate and publish a report to the whole community with all the insights they believe are relevant to share.

A bottom-up approach

We took a very different approach to determine who, when and what to include and share in our Tech Radar. Essentially anyone can include a topic in the Assess ring at any time. It's a lot of freedom that comes together with a lot of responsibility of becoming the owner of the proposed topic. We worked hard to create a lean framework that enables people to share, discuss and evolve their ideas. Based on iterations and feedback loops, it aims to bring together people to discuss, share their knowledge and thoughts. There's convergence and different opinions, but what we definitely see is discussion enrichment. Every topic must clarify three points:

1. Which of Adyen’s problems does it solve?
2. Why is it in the current ring?
3. Provide a link with a clear definition.

The steps

From Assess to Trial ring, the owner needs to ensure that:

1. Team members and anyone impacted by the topic is aware of the new proposed changes and provides feedback.
2. Someone experienced in the company should also be interested in the proposal.
3. And because we process personal and financial data, the security team is deeply involved and shares its views on the topic, including suggestions for changes (if needed).

If there's enough people interested in these discussions, we can move it to Trial ring where the owner needs to ensure that:

1. A hands-on POC (Proof of Concept) is created and the results evaluated.
2. A low-risk project is delivered to look at different perspectives.
3. It's shared to a broader audience in our Architecture meeting.
4. The topic receives another round of feedback.

Finally it is time for the big decision: Should the topic be moved to Adopt ring? If so, it means that the topic is the recommended way to solve that specific problem. At this point, after the POC, the low-risk project and the feedback, enough people deeply understand what are the pros and cons of the proposal, and they should be able to reason whether to adopt it or not. In Adopt ring, the owner ensures that:

• More people are involved and educated about the changes.
• If needed, a migration plan should be defined.

A topic can be moved to Hold from any other ring, meaning that we tried, but it didn't satisfy our expectations, so it should be parked for now.
We use ADRs (Architecture Decision Record) to document a decision when moving a topic between rings and keep track of the changes.

We are still rigorous about our tech stack

We enabled anyone to be able to propose and guide changes with big impact on the platform in production, but by no means we'll ever ignore security or allow hurtful ideas to move on. We realized that this process is really powerful to make everyone understand other perspectives and improve collaboration. There are two extreme situations that are avoided here:

1. There are only a few people interested in a brand new shiny trendy topic, and it has big consequences for the current platform.
2. The technology is too old, maybe it has been used for a very long time. There are better alternatives, but a small group of people still supports it and they don't want to change.

They are in essence the innovators and laggards. While innovation is fully supported, there's strict control on what goes into production (i.e. something not mature enough or without a really good reason) although anyone can set up an isolated sandbox to explore new cutting-edge technology.

The Maturity vs Complexity chart mentioned some years ago by Pinterest and referenced before in our blog continues to be used as a way to help people to make reasonable decisions. The adoption depends on how much complexity a topic introduces and how mature it is. We don't want to adopt a new tool when it's going to make a big impact, but it's still experimental. In the same way, we don't want to get stuck with old technology without an established and active community.

We also consider the Impact vs Urgency decision chart to reason about the decision to be made.

As our Tech Radar topics are intended to be a long term guide, the majority of them fit in the consensus or democratic areas.

From the perspective of a new joiner

Starters bring their own experiences when joining Adyen. Their experiences can't be ignored but should be leveraged by everyone else. A starter can compare different approaches and technologies from previous jobs to propose improvements. The problem is that this person usually has little or no proper context to understand the impact of such changes. The feedback loops in the process provide ways to solve this challenge by bringing close peers and more experienced people together to evaluate the idea at the initial stages. When the proposal is valid, it receives the support from others, who in turn also share the responsibilities and merits of moving it forward.

One point that we push hard is the message of not stopping at the first obstacle -some people might not be interested, but others might see value in the proposal. We also notice the Tech Radar being used as an index for new joiners to find links to clear topic definitions and topic owners (to be able to get in contact if they have a question).

From the perspective of a long-term team member

Those already working at Adyen found their autonomy and responsibility increased. The process enabled engineers to get the support needed to experiment more. For instance, if access rights or resources are needed to accomplish a POC, the topic introduced in the Tech Radar puts a weight on the request, because for a topic to be at this stage, it means there's at least some good support behind it. Because a clear and lean procedure to experiment is established, anyone feels more confident that there's a good evaluation before considering any adoption.

Conclusion

The Tech Radar is the tool that allows us to keep all information together and, at the same time, to kick off an iterative feedback process.

Because there's a clear, full picture of what's actually being used, people are inspired to keep reviewing the tech stack. Another good effect is a topic to be updated more often because there's someone responsible for it.

This process has evolved a lot since the first release, and we keep improving it. We are very proud of its positive impact at Adyen: Anyone can share an idea and make a difference, there is no room for criticism without putting forward an alternative solution. If someone doesn't like the way something is being handled, the door is open for improvements.

If the proposal solves a real problem with numbers to support it and people interested to move it forward, then why not?

Some Inspiring articles

https://www.thoughtworks.com/en-ca/insights/blog/how-we-create-technology-radar
https://www.thoughtworks.com/insights/blog/build-your-own-technology-radar
https://www.adyen.com/blog/from-0-100-billion-scaling-infrastructure-and-workflow-at-adyen
https://blog.pragmaticengineer.com/software-architecture-is-overrated/
https://medium.com/pinterest-engineering/learn-to-stop-using-shiny-new-things-and-love-mysql-3e1613c2ce14
https://martinfowler.com/articles/scaling-architecture-conversationally.html