DEV Community: SamKnowsCoding

What is Infrastructure as Code?

SamKnowsCoding — Sat, 14 May 2022 14:10:25 +0000

Infrastructure as code(IaC)

What is Infrastructure as code?
TL;DR,

Infrastructure as code describes the infrastructure in executable text format.
The short-lived infrastructure can be used and then discarded. Servers are built on demand, through automation.
Rather than patching a running container, immutable delivery modifies the container image and then redeploys a new container.

Infrastructure-as-code is the practice of describing infrastructure in text format. This is not about documentation, I'm talking about an executable text format, which is also called code. You want to be able to configure your infrastructure in a textual description that you can give to a tool to execute.

Configuration Management Systems

So, the configuration, AKA code, will be stored in somewhere and executed later when in need. You want to be able to configure your infrastructure with textual descriptions that you can give to a tool to perform. The tools that accomplish this are called configuration management systems. These tools, such as Ansible, Puppet, and Chef, allow you to describe your infrastructure in code, and then they create that infrastructure and maintain its state.

You never want to manually make system changes to the software configuration. This is not reproducible and is extremely error-prone. You want to use templates and scripts that describe how to install and automate configuration elements, such as systems, devices, software, and users. You can then take this text code and store it in your git so that you have a history of all changes. That way, everyone knows which version is the latest and what the infrastructure should look like. This way everyone knows which version is the latest version and how the infrastructure should look. Docker, Vagrant, Terraform, and even Kubernetes, also allow you to describe your infrastructure as code, and this code should be checked into version control.

Ephemeral Immutable Infrastructure

The reason this is so important is that server drift is a major cause of failure. Over time, servers are updated for a variety of reasons, and not always by the same people. This causes them to drift from their initial configuration. Sometimes the accumulation of these changes can cause failures in unpredictable ways. Worse, there are servers that should be the same, but one of them keeps failing due to a misconfiguration somewhere. I think that the servers should be considered as cattle, instead of pets. You see, cattle there are thousands of them, it is waste of time to name and care each one of them. But the pet on the other hand? When they get sick, they are lovingly pampered and cared to keep healthy. The message here is to not lovingly handcraft servers or spend too much time debugging them when they don't work. You want to be able to replace them with an identical server that works properly. This means you have to think of your infrastructure as something ephemeral or transient. It only exists for as long as you need it, and then you remove it when it is not in use.

Immutable Delivery via Container

Application are packaged in containers

Docker is a packaging technology that allows us to bring things up and down in a consistent way in an isolated environment called a container.

Same container that running in production can run on your laptop

Docker supports infrastructure-as-code, allowing you to specify how to build images from code called Dockerfiles. These Dockerfiles build the same images the same way every time. Docker then creates a container from that image in the same way each time it is deployed. This means that the same container running in production can be run on the developer's laptop. This is the ultimate development-production parity.

Immediate rollback and rolling update

You're not installing the application, seeing if it works, and uninstalling it if it doesn't. You're just starting it with a new version of the Docker container. If it starts having problems, you stop it, shut it down, and take out the previous version that's already installed in its own container. It's literally a matter of seconds. You can use the same approach for containers that start misbehaving. You delete this container and create another one to replace it. The new container will be exactly the same as the old one was on the first day it appeared. The reason for this is simple: if you patch a container, it dies, a new container is brought up to replace it, and the new container is unpatched.

So, IaC is an important part of the DevOps process that describes service facility in text and then executes that code to create environment. In practical development needs, by combining technologies such as webhook, through automated testing, and then CI/CD pipeline to release code, developers can release code faster and more efficiently.

Thanks for reading!

Design for failure

SamKnowsCoding — Sat, 14 May 2022 08:41:08 +0000

Once you design your application as a collection of stateless microservices, there are a lot of moving parts, which means there is a lot of potential for things to go wrong.
Services can occasionally be unresponsive or even break, so you can't always rely on them being available when you need them. Hopefully these events are very transient, but you don't want your application to fail just because some dependent service is running slow or has a lot of network latency on a given day. That's why you need to design for failure at the application level. Since failure is inevitable, you must build your software to resist failure and to scale horizontally.

Embrace the failure

Failure will happen. So that is why we must design for failure.
Failure is the only constant. We must change our thinking, from moving from how to avoid failure to how to identify failure when it happens, and what to do to recover from it. This is one of the reasons why we moved DevOps measurements from “mean time to failure” to “mean time to recovery.” It’s not about trying not to fail. It’s about making sure that when failure happens, and it will, you can recover quickly.

Plan to be throttled and retry, degrade gracefully

Plan to be throttled. You pay a certain level of quality of service for your backup service in the cloud, and they hold you to that agreement. Let's say you choose a plan that allows 20 database reads per second. When you exceed that limit, the service will throttle you. You will get a 429_TOO_MANY_REQUESTS error instead of 200_OK, and you will need to deal with this problem.
In this case, you would retry. This logic needs to be in your application code. When you retry, you want to back off exponentially on failure. The idea is to degrade gracefully.
Also, if you can, cache where appropriate so you don't always have to make remote calls to these services if the result won't change.

Retry Pattern

This allows the application to handle transient failures by transparently retrying and failing operations when trying to connect to a service or network resource. I have heard developers say, you have to deploy the database before starting my service because it expects the database to be there at startup. This is a fragile design that is not suitable for cloud-native applications. If the database is not there, your application should wait patiently and then retry again. You must be able to connect, and reconnect, as well as fail to connect and connect again. This is how you design robust cloud-native microservices. The key here is a retry mode that backs up exponentially, with longer delays between each attempt. Rather than retrying 10 times in a row and overwhelming the service, you retry and let it fail. You wait a second, and you retry again. Then you wait 2 seconds, then you wait 4 seconds, then you wait 8 seconds. Each time you retry, you increase the wait time a bit until all the retries have been used up, and then you return an error condition. This gives the back-end service time to recover from the factors that caused the failure. It could just be a temporary network delay.

Circuit Breaker Pattern

The circuit breaker pattern is similar to that of your home's electric circuit breaker. You may have experienced a tripped circuit breaker in your home. You may have done something that exceeded the circuit power limit and caused the lights to go out. That's when you took a flashlight down to the basement and reset the breaker to get the lights back on. This breaker pattern works in the same way. It is used to identify a problem and then do something to avoid a cascading failure. A cascading failure is when one service is unavailable and causes a cascading failure of other services. With breaker pattern, you can avoid this by tripping the breaker and having an alternate path back to something useful until the original service is restored and the breaker is off again. It works in such a way that everything flows normally as long as the circuit breaker is closed. The circuit breaker is monitored for failure up to a certain limit. Once this limit threshold is reached, this particular threshold, the circuit breaker trips and all further invocations of the circuit breaker return an error, not even an invocation of the protected service. Then after the timeout, it enters this semi-open state and tries to communicate with the service again. If it fails, it goes back to the closed state. If it succeeds, it goes back to the fully open state.

Bulkhead Pattern

Bulkhead Pattern can be used to isolate a failed service to limit the scope of the failure. This is a pattern where the use of a separate thread pool can help recover from a failed database connection by directing traffic to another thread pool that is still active. It gets its name from the design of bulkheads on ships. The compartments below the waterline are separated by walls called "bulkheads". If the ship is damaged, only one compartment will fill with water. Bulkheads prevent water from affecting other compartments and sinking the ship. Using bulkhead pattern isolates consumers from services that fail as a cascade, allowing them to retain some functionality in the event of a service failure. Other services and functions of the application continue to work.

In general, failure is inevitable, so we design for failure, rather than trying to avoid it. Developers need to build resilience so they can recover quickly. Retry pattern works by retrying failed operations. Circuit breaker pattern is designed to avoid cascading failures. Bulkhead pattern can be used to isolate failed services.

Thanks for reading.