DEV Community: Ivan Krnic

Microfrontend, what is it good for, absolutely everything!

Ivan Krnic — Tue, 14 Apr 2020 16:07:22 +0000

Organizations providing complex user experience should start considering the microfrontend approach as their default frontend implementation strategy.

Traditional monolithic and tightly coupled architectures proved cumbersome and clumsy. The main issues with such architectures come from the size and interdependencies of source code. Such systems are difficult to fully understand. It is hard to identify all dependencies. It is hard to implement new features with confidence that new defects haven’t been introduced. It is hard to efficiently scale such a system. All these issues have a negative influence on lead time and quality of service.

At the same time, all these issues can be remediated with different architectural principles, i.e. introducing modularity and loosely coupled components. Microservice architecture relies on these principles and we as an industry have been fairly successful at recognizing the need and implementing these principles when developing backend systems.

However, we still equally suck in recognizing the need and applying these principles in the frontend!

As modular and independent our backend services are, we still tend to build large monolithic frontends.

Typical problems that we encounter are:

large and messy codebase, large deployments, big batches, low speed of delivery, more difficult regression testing hence more errors reaching the customers
when upgrading the underlying frontend framework, we are reduced to the Big-bang approach: we cannot iteratively upgrade parts of our frontend thus we cannot deliver value to the customer sooner and reduce implementation risk by shrinking the blast radius.

The solution is for organizations to start applying proven principles of modularity and decoupling also to the frontend development.

Nowadays, modern frontend solutions are built as Single Page Applications (SPA) using popular frameworks such as Angular, React, Vue, ExtJS or similar. Recognizing the need for modularity and decoupling, modern frontend architectures leverage the concept of modules (or components).

Just as the backend system is built using independent services that tied together provide full system functionality, the frontend system is built using independent components that combined provide a full user experience. This approach is called microfrontend. Components are designed, developed, tested and deployed independently. Moreover, since these components are independent, each one can be implemented using different frontend framework that best suits a particular need. That is not to say that a great variety of different frameworks is preferred or even suggested. But it is a nice feature.

Much bigger benefit stems from the fact that these components are independent and have their own lifecycle. Therefore, each component can be modified and upgraded independently. This is particularly convenient when dealing with complex frontend systems. To upgrade such a system to a newer version of say Angular, an organization should upgrade the whole system before releasing everything to production.

With microfrontend approach, an organization can upgrade each component at its own pace, releasing upgrades iteratively, continuously providing customer value and controlling blast radius in case of errors.

Infrastructure as Code vs. GitOps

Ivan Krnic — Wed, 19 Feb 2020 18:59:25 +0000

A couple of years ago we were in the middle of a huge project that required hefty processing capacity. We were relying on Red Hat OpenShift as a container platform (we still do :), and as capacity demand grew, we had to periodically add new worker nodes.

Adding new worker nodes was a breeze but more often than not our applications failed miserably on these new nodes for various reasons which all boiled down to something stupid like missing some configuration properties. Very soon it became clear to us that we needed a mechanism for establishing the Desired state. And that’s where the concept of Infrastructure as Code fits perfectly.

Infrastructure as Code is a mechanism for creating and maintaining runtime infrastructure based on a definition usually stored in a version control system.

The main idea here is that runtime infrastructure can be defined via machine-readable files that can be stored and versioned, much like regular source code. If you can build a specific application version based on the source code in a version control system, why wouldn’t you be able to do the same with your infrastructure – provision specific infrastructure based on the definition files in your version control system.

How we hit rock bottom...

Before Infrastructure as Code, our nodes (although meant to be identical) were rather unique because configuring nodes was a human task. With configuration documentation scattered over several Wikis and Word files, and humans not being shell scripts (therefore forgetting things), every now and then somebody dropped the ball and forgot to set some properties. In effect, our nodes ended up being snowflakes – each one very unique and with subtle configuration differences that were not visible to the naked eye, but nevertheless were causing our applications to fail on specific nodes… for no particular reason at all… at least it seemed so.

Infrastructure as Code takes care that infrastructure configuration is clearly defined in the version control system and consistently applied to infrastructure. Changes to infrastructure are done exclusively through pull requests in the version control system. Need additional CPU for your application? No worries, just make that change in the configuration file and make a pull request. When somebody approves a pull request, the change will be merged to trunk and the process will be initiated to provision new infrastructure configuration – new Desired state.

Cool IaC features

In order for Infrastructure as Code to work there has to be a way to fully automate tasks. This is where various tools come handy. Every public cloud has its own flavor: AWS CloudFormation, Azure Resource Manager and Google Cloud Deployment Manager, to name some. Other popular tools include Terraform, Ansible, and so on.

Additionally, automated tasks need to be idempotent which means that they can be executed multiple times with the same effect as they were executed only once. This ensures that complete configuration can be applied over and over again without fear of breaking something. Idempotent operations dramatically simplify implementation since we don’t need to take care of state (i.e. recording which changes/scripts were already executed and should be omitted from future execution).

Infrastructure as Code is such a powerful concept that enables us to add new cluster nodes that are identical to others as well as to create whole new environments that are the spitting image of the production. And when something goes terribly wrong, clusters can be automatically recreated with identical configuration as the old ones. Remember that analogy with pets and cattle? A proper restore-from-backup operation takes a lot of time. It’s usually easier and much faster to just drop the whole cluster and rebuild it from scratch. This is, to be honest, what we would do with full-fledged application servers and other heavy-weight runtime environments if we had a fast and reliable rebuild process.

And then GitOps came along...

Historically, Infrastructure as Code pattern dealt only with managing infrastructure in data centers, like servers and network parameters. But, two things happened in the meanwhile: Git became prevailing version control system, and second, with the rise of the cloud, an additional level of abstraction surfaced in the form of container orchestrator, most notably Kubernetes.

Since IaC was such a useful pattern, wouldn’t it be cool to extend it in a way that it uses Git as a version control system and manages not only underlying servers but also Kubernetes instance? And, voila, GitOps emerged.

Much like IaC is a pattern that manages infrastructure, GitOps is a pattern that manages Kubernetes instance and handles tasks such as setting namespaces, limits, and quotas, controlling the number of container instances and so on. As the name says, GitOps uses Git as a single source of truth for Kubernetes instance management. Being container- and cloud-native-aware, GitOps is ideal for modern architectures and modern technology stack. Using GitOps you can recreate your Kubernetes cluster, set up all namespaces, limits, and quotas, and redeploy all containers. On a push of the button, you can tear everything down and recreate an identical state as before.

IaC vs GitOps mess... #”$%$

People often ask what is the difference between IaC and GitOps. It’s not an easy question. IaC is version control system agnostic. GitOps, on the other hand, uses only Git. IaC deals only with the underlying infrastructure. GitOps focuses on managing Kubernetes instance above. Does that mean that GitOps shouldn’t also manage infrastructure? No, why wouldn’t GitOps also manage infrastructure?

We can look at GitOps as “doing Ops” while using Git for storing configurations. Notice here how “doing Ops” is not limited only to managing servers or only to managing Kubernetes instance. We can look at it holistically as managing whatever it takes. In that sense, we can look at GitOps as a more specific implementation of IaC pattern.

Lately, we’ve been using Argo CD for implementing GitOps and are thrilled with how smoothly it works. Denis wrote a nice article about Argo CD and I warmly suggest you read it.

Why you should seriously consider IaC and GitOps?

Infrastructure as Code and GitOps work best together! Benefits are huge:

Speed, since installing and configuring Kubernetes instance can be automated.
Consistency, since all instances are always built and configured using the same procedure and same input parameters.
Audit, since all changes are done through configuration stored in a version control system it is easy to find out who did particular change.
Traceability, again since all changes are done through configuration stored in a version control system it is easy to figure out how the system ended up in the current state.
Self-service and developer-centric, since changes to infrastructure can be done even by developers… as opposed to only by Operations. On the other hand, we don’t want anarchy… that’s why we use pull requests that somebody needs to approve.
Developers can develop (and later troubleshoot) in an environment identical to production. Developers love this.
Feedback loops are shorter since QA can be done in an environment identical to production. Problems are caught before production.
Performance and stress tests also make sense only when done in an environment identical to the production

If you’re interested in receiving interviews with thought leaders and a digest of exciting ideas from the world of DevOps straight to your inbox, subscribe to our 0800-DEVOPS newsletter!

Mind the Hump - how the market influences your delivery approach

Ivan Krnic — Sat, 14 Dec 2019 16:27:30 +0000

If you’re interested in receiving interviews with thought leaders and a digest of interesting ideas from the world of DevOps straight to your inbox subscribe to our 0800-DEVOPS newsletter!

Last week we had an opportunity to host our friend Andrea Tomasini and discuss ORGANIC agility. One of the key things Andrea mentioned was that an organization should behave differently depending on the maturity of the market in which it operates. He explained this using the famous curve introduced by Geoffrey Moore in Crossing the Chasm.

While the organization is still on the left-hand side of the hump and building a product targeted for Innovators and Early adopters, it needs to be nimble. The organization is still looking for the right product, so it needs to move fast in order to learn fast. At this point, effectiveness is more important than efficiency. Speed is crucial. If having more people working on a solution will yield the solution earlier than that is what an organization needs to do. The organization is in the “sell-to-make” mode – it first has to sell the product (the idea!) to the customers in order to come in the position to actually produce it! Occasional slips are perfectly fine – it is far more important to ship something quickly and solicit feedback than to analyze every single edge case before shipping it out. Remember what Mark Zuckerberg used to say when Facebook was in this phase… “Move fast and break things”.

Organizations must move quickly in this period because this is when the market share is acquired. Once the market becomes saturated and the Late majority kicks in, it is hard to increase market share.

Once the organization moves over the hump, different mechanics apply. It is now clear what is the product that customers are looking for. The market is in a saturated state. The only way to compete with others is through small tweaks and price optimizations. Hence, the focus is now on efficiency over effectiveness. Speed is no longer crucial. Efficiency is. So is the quality. Slips are no longer tolerated because that is not how products in saturated/mature markets operate. It is expected that the product works flawlessly, and each error is scrutinized. On the right-hand side of the hump, the organization is in the “make-to-sell” mode – the product is known, there is nothing new to discover, all it needs to do is build the product as efficiently as possible and sell it with the best margin it can get.

Every organization will sooner or later end up on either side of the curve. The important thing is to recognize in which mode it should operate and adjust its behavior accordingly.

Recently we had an opportunity to participate in one Proof of Concept with the goal of designing and piloting a new solution for a widely used consumer-facing web application. The PoC was timeboxed and defined only with a handful of enabling constraints set by the Customer. It was our job to build the best possible solution with these constraints in mind. We were obviously in a sell-to-make mode, so we acted accordingly.

Creativity and speed were crucial. We have put some smart heads together and gave them full autonomy to self-organize and generate ideas. By iterating rapidly over a large number of possible ideas we were able to distil those worth pursuing further. During this process, nobody ever asked for any estimates or whether all these people were truly necessary during all the discussions. All that mattered was moving fast. Even if that meant not implementing every single edge case but documenting it for later.

Team engagement was spectacular. Working in this industry for almost 15 years now, rarely have I seen projects that were so owned by the team. It was clearly their product, their masterpiece. Again, connecting to the work of Daniel Pink, we have given them autonomy to self-organize, an opportunity to stretch their mastery and a compelling vision of changing how consumers use Customer’s services.

Recognizing on which side of the hump you are and acting accordingly plays a major role in the success of your efforts. In our example, understanding that we were addressing Innovators in this PoC helped us adjust our approach and deliver precisely what the Customer needed. This might seem intuitive, but I have personally seen a number of cases where this was not the case.

So, keep your eyes open and mind the Hump.

If you’re interested in all things DevOps subscribe to our 0800-DEVOPS newsletter!

You will receive in your inbox interviews with thought leaders and a digest of interesting ideas from the world of DevOps, technical practices and increased productivity!

We will never spam you, just share things we discuss internally in our CROZ DevOps Community of Practice. Join us!

Making systems more resilient (1) - Circuit Breaker

Ivan Krnic — Sat, 07 Sep 2019 17:47:00 +0000

Things break. Usually in production. In this article series we will present various mechanisms for making your systems more resilient so your users will never experience real outage in spite of possible problems under your hood.

Circuit Breaker has an interesting name that comes from the analogy found in the physical electrical world. Every decent electrician recognizes that playing with electricity is dangerous and that she must install security mechanisms to protect the rest of the electrical network if one device breaks down. What do electricians do? They install fuses. If a device (say toaster!) breaks down and produces short circuit, strong electric current will flow through the network resulting in damage, overheating and possibly fire. Unless there is a fuse in the network! If there is strong electric current flowing, the fuse will melt down or otherwise break the electrical circuit, stop the current and save the rest of the network.

Circuit Breaker mechanisms in software are very much like electrical fuses in the real world. Suppose one of your services is calling another:

If Service B stops responding, Service A will patiently wait until timeout happens and then will either throw an exception or return questionable results. None of this is acceptable. We need a better solution that would prevent the whole system from going down. That’s where the Circuit Breakers come into play.

Circuit Breaker is positioned between two services and service call passes through it. If everything is fine, Circuit Breaker just passes calls through. What happens when Service B doesn’t respond?

Circuit Breaker will detect that and return an error or alternative response. In electrical terms, Circuit Breaker will "open the circuit". What’s more, Circuit Breaker will remember that Service B is down and whoever calls Service B will immediately get an error without Circuit Breaker even passing the request to Service B. Circuit Breaker will periodically pass one of the requests through to Service B, just to check if Service B is still down. If that is so, Circuit Breaker will continue to return an error without even passing requests to Service B.

If, on the other hand, Service B recovers and returns a valid result, Circuit Breaker will detect that, return the result to the caller (Service A) and "close the circuit" again, i.e. continue passing all requests through to Service B.

When Circuit Breaker detects that called service is down, it can do various things. It could return a meaningful error to the caller but it could also return a “good enough” result that would provide some value for the user.

Let’s take an example of a banking application that shows account balance fetched via separate service. If that service goes down, Circuit Breaker could return the last known balance from the cache. Yesterday’s balance might not be valid today but accompanied by a proper message it’s better than nothing.

"Dear customer, we’re experiencing some trouble getting your balance at the moment, but your balance 3 hours ago was €234.55" beats "Your balance is €null" anytime. Of course, what Circuit Breaker returns needs to be carefully thought through from a business perspective.

Circuit Breakers are really cool. In Part 2 of this series, we will lift the hood and show concrete Circuit Breaker implementation.

— Photo by rawpixel.com from Pexels.

Accelerate State of DevOps Report 2019: Communities of Practices are so dope!

Ivan Krnic — Sun, 25 Aug 2019 14:54:00 +0000

New Accelerate State of DevOps Report 2019 has been published!

In case you’re wondering what’s all the fuss about, it’s the largest and longest-running research of its kind revealing what truly successful software delivery companies do differently than their less successful counterparts. Wow, who wouldn’t want to take a look at that?! The research is led by Dr. Nicole Forsgren and here is her take on the Report.

There is a deeply rooted belief that the speed of software delivery process comes at the expense of the stability and vice versa. If you’re delivering something fast, you’re bound to break things in production. Therefore, if you want a reliable system, you cannot go fast. And if you need to go fast, rest assured your system will not be reliable.

What data say?

Year after year, this report, backed by data and scientific methods, proves that speed and stability are not working against each other. Rather, speed and stability enable each other.

Elite performers massively outperform low performers. Numbers are staggering: 208 times more frequent code deployments and 106 times faster lead time from commit to deploy.

Is this possible? It sure is, when using proper organizational and technical practices.

How to move the needle?

One of the recognized organizational practices is establishing Communities of Practices (CoP) as opposed to Training Centers or Centers of Excellence (CoE). As confirmed by the Report, CoE represents a bottleneck for the relevant expertise in the organization. CoE also scales badly across the organization. Furthermore, the Report recognizes that CoE "establishes an exclusive group of "experts" in the organization, in contrast to an inclusive group of peers who can continue to learn and grow together."

The Community of Practice is precisely the latter, an informal group of peers gathered around a common topic that is of interest to everyone. Group members share knowledge, experiences, and skills, and work together toward common goal. Do you feel like your organization could benefit from a better testing process? Find your like-minded peers and recruit them into Community of Practice for Testing. Work together on improving testing process. Define your working agreement. Everything is acceptable: you can come together every morning over coffee, or later during the day, or after work over beer. The only thing that is important is that participation is voluntary and members are genuinely interested and motivated.

Our experiences with Communities of Practices

Time and again, this approach proved its value at CROZ. Striving to build a healthy and inclusive organizational culture, we have zero tolerance to Ivory Towers where experience and knowledge are hidden from the rest of the organization. Some of our most active communities are CoP for Agile, CoP for DevOps, and CoP for Product Management.

Each community is fully open and everyone can participate and contribute. Members meet regularly and share their experiences as well as useful books, blogs, tools, insights, and ideas. Equally important: all communities have full autonomy in defining their goals. Members work together to define steps and experiments which will get each community closer to its goal. No management is involved here. Community members autonomously collaborate around tasks and navigate toward defined goal.

The social aspects of CoPs are spectacular! People are getting together. I’ve seen new friendships hit off. I’ve seen numerous old hatchets buried. I myself got really close to some people. Today I would do whatever it takes for them and I’m sure they would do the same for me. That’s what CoPs do to your organizations. Try them.