Tech stories: make me a microservice architecture! But what's the product?

"Make me a microservice architecture!"
"OK, but what's the project?"
"Use node.js! And run!"
"Sir...? Sir...?"

Try: createNewProduct({ tech: 'node.js' })

I started my career with the .NET stack: C# over the annoyingly misconfigurable service: the IIS.

After spending about 4 years on it, I gained confidence, my code got better, cleaner, leaner. Obviously I wanted to deepen my knowledge and stay with it, now that I had a sense of success with it.

One day managers came and told us, that the Big Boss over the ocean decided we are going to make a new greenfield project, an Internet of Things product.

Many of us got transferred to the new team. In fact around 60 people in two countries.

After working on a legacy application to make sure it can finally be unit-testable, this sounded rather exciting!

The first red flags 🚩

So, naturally we asked: "what now?". The product manager said: "I don't know it, guys, hold on, let's return to this on Thursday." My lead decided to pass the time with setting up a modern C# project, getting some infrastructure ready, and let's start with already having a simple unit test.

So we did those. Obviously, we did not have anything to do next day. So we waited. Came Thursday: "Today nothing has happened yet, sorry, next Monday!"

This went on for a month. I took a big vacation and came back in January.

We were summoned again: "Guys, thanks for coming. I know all of you are .NET pros but the _Big Boss decided we land this project in node.js using JavaScript."_ Many of us took it very bad. After all, we had no prior training, and we were quite invested in C#.

I am not proud about but I was loud about my discontent. _"Look, András, I understand you are upset, but this was a 100% business decision. They say node.js is a hot topic now for the investors. It's an instant financial boost, basically." For correctness I need to tell you that my closest bosses were very patient; they gave me a second chance after my explosion.

I felt betrayed that I have to abandon the stack I liked, so I did not like the assignment one bit. I also disliked having to sign that I had 1 year of JS experience. They were correct, though, and knew we desperarely needed training. So they organized about 2 weeks of node.js training.

Chaos ensues!

However, when we started working, there was an absolute chaos:

• We had about 6 teams working on a product without a vision longer than a sentence. 6 teams guessing separately.
• None of was familiar with microservices.
• Most of us only had that 2 weeks of JavaScript training.
• Nobody wanted to look like not doing anything. Everyone was working crazy at something.
• Security team were juniors: every week they broke something.
• Some people asked to go back to their previous teams, others silently resigned and left.
• Some people noted the abscence of leadership and were planning to fill in the void even though it might have been the Titanic, just to have the title "Captain" (team lead) for a while, so their CV would be beefier for the next gig.
• Two teams in the States went with CofeeScript and they were also not really taking themselves too seriously (I still remember my colleague telling me that he found an error named ShiaLeboufError in their codebase). Also a bit later somebody had the beautiful task of fixing bugs in a codebase transpiled from CoffeeScript to JS. He was not happy about it.

It felt like an entire construction team from the architect to craftspeople working on the roof were summoned and were told to work immediately.

The people responsible for the roof started looking for roof tiles, though they were not sure if the roof would be a flat concrete surface or not. The artitect was debating with the client what exactly they want. The rest of the builders discussed it and decided to build a modular, but working toilet, because in any case, you will need one and you have to try out the new materials we have to work with anyways on something.

One team had created an extremely tightly coupled unit test system: it was impossible to change any controller without breaking the tests for all the other untouched controllers. There was also a meeting with my team, I remember, where they were arguing that there should be a queue, but there was a heated debate if it would be polled every 5 seconds or should it be finally a proper websocket.

None of this mattered.

By summer my motivation (and likely other people's as well) dipped. To "fix things" they hired a guy to be the tech lead of my team, slighting my ambitious colleague, who was eyeing the position.

The new tech lead was also ambitious but he did not get the concept of motivation. He thought of it as a thing solely dependant on the employee, therefore it was my fault I felt depressed and it was only my perception that management was not up to the task.

They almost fired me during this time. (Later they fired my ex-boss when he made remarks about someone's hand shaking from stress he caused.)

We (or mostly I) also had serious doubts about the product: we were working on a UI for support agents that might be used in case somebody decided to buy or mobile SDK. But most products were experimental with people toying around. They did not know yet if they would sell or not.

But we wanted to be already there when the inevitable support needs arise. After all, even a fraction of that future 1 trillion $ IoT market is going to be a huge sum! But, clearly, no one needed it yet. I confided with the project manager about my worries about the product and he reassured me, "Don't worry, it's gonna be great! People expect this."

A little later he moved on to another company.

By autumn many of the teams were reorganized into other projects. The project we worked on got scrapped and they started from scratch.

I got another chance in another team where I could learn JS well finally and where they adopted TypeScript.

Catch: (fail)- How it should have been done?

It was an unbelievably tough period of my life which stayed with me for a long time. I was searching unboxing my hunches for years afterwards.

I think the following things should have been done differently:

New product? Start with a small, seasoned team!

They should have put together a small team with motivated senior developers and find somebody with JavaScript expertise . It was the early days of node.js - there was a funny moment when recruitment was scolded for putting 5 years of node.js experience in the job requirements when node was just 3 years old.

Starting we that small, efficient team you can iterate over prototypes and technology without the inertia of many teams.

Not to mention it is way cheaper to pay for 3-4 engineers than to pay about 40.

Do not convert seniors into juniors

As I wrote in my Anatomy of an engineer's knowledge article, seniors are best performing over stacks AND platform they have gained years of experience on.

By telling 40 people to start a coding frenzy over a platform they haven't seen, with a language they only read about, to realise a vague product you are making sure the codebase will be rampant with anti-patterns.

Those occur naturally as a mismatch between a language and mental models forced upon it - we get rid of them as we gain experience with the language and platform we deal with.

In other words we were doing noob mistakes, even though as we were experienced, we were quicker to "google" and solve the error messages.

Do not start with microservices

Unless you have scaling issues and a clear need for a microservice, DO NOT start with one. In our case, teams owned their microservices, code was built without collaboration, so things broke constantly without notification. This was a reflection of their product manager having different vision than ours as well. We added features to the microservices based on who got the task and not because it made sense to be there.

In my opinion, creation of a microservice is a carefully planned refactoring step.

You are going to have performance drops when formerly in-memory operations now have to travel cross-service.

Demotivation is usually a result of bad management

At this point it should be clear that working with unknown tech on an unknown platform to land a product without real customer need or vision is not motivating.

In my opinion what really motivates developers is when they can understand a customer problem (be it a narrow-scale B2B project or a worldwide, open service) and how their code could solve it step-by-step.

My next project was a rewrite of a legacy, horrible looking product with no UX design behind it. The new design to be done was slick, clean, modern. We had a UX designer interviewing random people in bars. We knew customers were complaining about the old app.

We had purpose and clear goals.

Tech choices should be a tech team decision

I know that in reality the owners and C-levels can decide anything. But without honest debates and discussions motivation will drop. They could have asked the question "Who is interested in trying out this emerging tech?", compared to, say, summoning 40 bakers and tell them that they are now chefs in a rotisserie. And please wait a bit until they finish building the rotisserie.

Decent discussions, ideas shared early will prepare people and they will feel included.

Finally

To this day I feel that those three quarters was a total waste. Waste of finances and waste of human capital. But I gained insights to how software products can be effectively managed and I got the chance to learn new language and new tech.

I also understood the importance of reading great books on software engineering; in my case You don't know JS by @getify 👏 basically cured my depression and drop of self-esteem.

Thanks man!