DEV Community: Michael Cade

#90DaysOfDevOps - Introduction - Day 1

Michael Cade — Sun, 17 Apr 2022 10:12:40 +0000

Introduction - Day 1

Day 1 of our 90 days and adventure to learn a good foundational understanding of DevOps and tools that help with a DevOps mindset.

This learning journey started for me a few years back, but my focus then was around virtualisation platforms and cloud-based technologies, I was looking mostly into Infrastructure as Code and Application configuration management with Terraform and Chef.

Fast forward to March 2021, I was given an amazing opportunity to concentrate my efforts around the Cloud Native strategy at Kasten by Veeam. Which was going to be a massive focus on Kubernetes and DevOps and the community surrounding these technologies. I started my learning journey and quickly realised there was a very wide world aside from just learning the fundamentals of Kubernetes and Containerisation and it was then when I started speaking to the community and learning more and more about the DevOps culture, tooling and processes so I started documenting some of the areas I wanted to learn in public.

So you want to learn DevOps?

Let the journey begin

If you read the above blog, you will see this is a high-level contents for my learning journey and I will say at this point I am nowhere near an expert in any of these sections but what I wanted to do was share some resources both FREE and some paid for but an option for both as we all have different circumstances.

Over the next 90 days, I want to document these resources and cover those foundational areas. I would love for the community to also get involved. Share your journey and resources so we can learn in public and help each other.

You will see from the opening readme in the project repository that I have split things into sections and it is 12 weeks plus 6 days. For the first 6 days, we will explore the fundamentals of DevOps in general before diving into some of the specific areas. By no way is this list exhaustive and again, I would love for the community to assist in making this a useful resource.

Another resource I will share at this point and that I think everyone should have a good look at, maybe create your mind map for yourself and your interest and position, is the following:

DevOps Roadmap

I found this a great resource when I was creating my initial list and blog post on this topic. You can also see other areas go into a lot more detail outside of the 12 topics I have listed here in this repository.

First Steps - What is DevOps?

There are so many blog articles and YouTube videos to list here, but as we start the 90-day challenge and we focus on spending around an hour a day learning something new or about DevOps, I thought it was good to get some of the high level of "what DevOps is" down to begin.

Firstly, DevOps is not a tool. You cannot buy it, it is not a software SKU or an open source GitHub repository you can download. It is also not a programming language, it is also not some dark art magic either.

DevOps is a way to do smarter things in Software Development. - Hold up... But if you are not a software developer should you turn away right now and not dive into this project??? No. Not at all. Stay... Because DevOps brings together a combination of software development and operations. I mentioned earlier that I was more on the VM side and that would generally fall under the Operations side of the house, but within the community, there are people with all different backgrounds where DevOps is 100% going to benefit the individual, Developers, Operations and QA Engineers all can equally learn these best practices by having a better understanding of DevOps.

DevOps is a set of practices that help to reach the goal of this movement: reducing the time between the ideation phase of a product and its release in production to the end-user or whomever it could be an internal team or customer.

Another area we will dive into in this first week is around The Agile Methodology. DevOps and Agile are widely adopted together to achieve continuous delivery of your Application.

The high-level takeaway is that a DevOps mindset or culture is about shrinking the long, drawn out software release process from potentially years to being able to drop smaller releases more frequently. The other key fundamental point to understand here is the responsibility of a DevOps engineer to break down silos between the teams I previously mentioned: Developers, Operations and QA.

From a DevOps perspective, Development, Testing and Deployment all land with the DevOps team.

The final point I will make is to make this as effective and efficient as possible we must leverage Automation.

Resources

I am always open to adding additional resources to these readme files as it is here as a learning tool.

My advice is to watch all of the below and hopefully you have also picked something up from the text and explanations above.

If you made it this far, then you will know if this is where you want to be or not. See you on Day 2.

#90DaysOfDevOps - Day 7

Michael Cade — Fri, 07 Jan 2022 09:38:12 +0000

This post originates from https://github.com/MichaelCade/90DaysOfDevOps come and get involved, star and fork the repo and start your own journey. You don't need to follow what I am doing but the sections might help.

The Big Picture - DevOps & Learning a Programming Language

I think it is fair to say to be successful in the long term as a DevOps engineer you've got to know at least one programming language at a foundational level. I want to take this first session of this section to explore why this is such a critical skill to have, and hopefully, by the end of this week or section, you are going to have a better understanding of the why, how and what to do to progress with your learning journey.

I think if I was to ask out on social do you need to have programming skills for DevOps related roles, the answer will be most likely a hard yes? Let me know if you think otherwise? Ok but then a bigger question and this is where you won't get such a clear answer is which programming language? The most common answer I have seen here has been Python or increasingly more often, we're seeing Golang or Go should be the language that you learn.

To be successful in DevOps you have to have a good knowledge of programming skills is my takeaway from that at least. But we have to under why we need it to choose the right path.

Understand why you need to learn a programming language.

The reason that Python and Go are recommended so often for DevOps engineers is that a lot of the DevOps tooling is written in either Python or Go, which makes sense if you are going be build DevOps tools. Now this is important as this will determine really what you should learn and that would likely be the most beneficial. If you are going to be building DevOps tools or you are joining a team that does then it would make sense to learn that same language, if you are going to be heavily involved in Kubernetes or Containers then it's more than likely that you would want to choose Go as your programming language. For me, the company I work for (Kasten by Veeam) is in the Cloud-Native ecosystem focused on data management for Kubernetes and everything is written in Go.

But then you might not have clear cut reasoning like that to choose you might be a student or transitioning careers with no real decision made for you. I think in this situation then you should choose the one that seems to resonate and fit with the applications you are looking to work with.

Remember I am not looking to become a software developer here I just want to understand a little more about the programming language so that I can read and understand what those tools are doing and then that leads to possibly how we can help improve things.

I would also it is also important to know how you interact with those DevOps tools which could be Kasten K10 or it could be Terraform and HCL. These are what we will call config files and this is how you interact with those DevOps tools to make things happen, commonly these are going to be YAML. (We may use the last day of this section to dive a little into YAML)

Did I just talk myself out of learning a programming language?

Most of the time or depending on the role, you will be helping engineering teams implement DevOps into their workflow, a lot of testing around the application and making sure that the workflow that is built aligns to those DevOps principles we mentioned over the first few days. But in reality, this is going to be a lot of the time troubleshooting an application performance issue or something along those lines. This comes back to my original point and reasoning, the programming language I need to know is the one that the code is written in? If their application is written in NodeJS it won’t help much if you have a Go or Python badge.

Why Go

Why Golang is the next programming language for DevOps, Go has become a very popular programming language in recent years. According to the StackOverflow Survey for 2021 Go came in fourth for the most wanted Programming, scripting and markup languages with Python being top but hear me out. StackOverflow 2021 Developer Survey – Most Wanted Link

As I have also mentioned some of the most known DevOps tools and platforms are written in Go such as Kubernetes, Docker, Grafana and Prometheus.

What are some of the characteristics of Go that make it great for DevOps?

Build and Deployment of Go Programs

An advantage of using a language like Python that is interpreted in a DevOps role is that you don’t need to compile a python program before running it. Especially for smaller automation tasks, you don’t want to be slowed down by a build process that requires compilation even though, even though Go is a compiled programming language, Go compiles directly into machine code. Go is known also for fast compilation times.

Go vs Python for DevOps

Go Programs are statically linked, this means that when you compile a go program everything is included in a single binary executable, no external dependencies will be required that would need to be installed on the remote machine, this makes the deployment of go programs easy, compared to python program that uses external libraries you have to make sure that all those libraries are installed on the remote machine that you wish to run on.

Go is a platform-independent language, which means you can produce binary executables for *all the operating systems, Linux, Windows, macOS etc and very easy to do so. With Python, it is not as easy to create these binary executables for particular operating systems.

Go is a very performant language, it has fast compilation and fast run time with lower resource usage like CPU and memory especially compared to python, numerous optimisations have been implemented in the Go language that makes it so performant. (Resources below)

Unlike Python which often requires the use of third party libraries to implement a particular python program, go includes a standard library that has the majority of functionality that you would need for DevOps built directly into it. This includes functionality file processing, HTTP web services, JSON processing, native support for concurrency and parallelism as well as built-in testing.

This is by no way throwing Python under the bus I am just giving my reasons for choosing Go but they are not the above Go vs Python it's generally because it makes sense as the company I work for develops software in Go so that is why.

I will say that once you have or at least I am told as I am not many pages into this chapter right now, is that once you learn your first programming language it becomes easier to take on other languages. You're probably never going to have a single job in any company anywhere where you don't have to deal with manage, architect, orchestrating, debug JavaScript and Node JS applications.

Resources

Now for the next 6 days of this topic my intention is to work through some of the resources listed above and document my notes for each day. You will notice that they are generally around 3 hours as a full course, I wanted to share my complete list so that if you have time you should move ahead and work through each one if time permits, I will be sticking to my learning hour each day.

See you on Day 8.

#90DaysOfDevOps - Day 6

Michael Cade — Thu, 06 Jan 2022 14:59:45 +0000

DevOps - The real stories

DevOps to begin with was seen to be out of reach for a lot of us as we didn't have an environment or requirement anything like a Netflix or fortune 500 but think now that is beginning to sway into the normal when adopting a DevOps practice within any type of business.

You will see from the second link below in references there are a lot of different industries and verticals using DevOps and having a hugely positive effect on their business objectives.

Obviously the overarching benefit here is DevOps if done correctly should help your Business improve the speed and quality of software development.

I wanted to take this Day to look at succesful companies that have adopted a DevOps practice and share some resources around this, This will be another great one for the community to also dive in and help here. Have you adopted a DevOps culture in your business? Has it been successful?

I mentioned Netflix above and will touch on them again as it is a very good model and advanced to what we generally see today even still but will also mention some other big name brands that are succeeding it seems.

Amazon

In 2010 Amazon moved their physical server footprint to Amazon Web Services (AWS) cloud this allowed them to save resources by scaling capacity up and down in very small increments. We also know that this AWS cloud would go on and make a huge amount of revenue itself whilst still running the Amazon retail branch of the company.

Amazon adopted in 2011 (According to the resource below) a continued deployment process where their developers could deploy code whenever they want and to whatever servers they needed. This enabled Amazon to achieve deploying new software to production servers on average every 11.6 seconds!

Netflix

Who doesn't use Netflix? obviously a huge quality streaming service with by all accounts at least personally a great user experience.

Why is that user experience so great? Well the ability to deliver a service with no recollected memory for me at least of glitches requires speed, flexibility, and attention to quality.

NetFlix developers can automatically build pieces of code into deployable web images without relying on IT operations. As the images are updated, they are integrated into Netflix’s infrastructure using a custom-built, web-based platform.

Continous Monitoring is in place so that if the deployment of the images fails, the new images are rolled back and traffic rerouted to the previous version.

There is a great talk listed below that goes into more about the DOs and DONTs that Netflix live and die by within their teams.

Etsy

As with many of us and many companies there was a real struggle around slow and painful deployments. In the same vein we might have also experienced working in companies that have lots of siloes and teams that are not really working well together.

From what I can make out at least from reading about Amazon and Netflix, Etsy might have adopted the letting developers deploy their own code around the end of 2009 which might have been before the other two mentioned. (interesting!)

An interesting take away I read here was that they realised that when developers feel responsibility for deployment they also would take responsibility for application performance, uptime and other goals.

A learning culture is a key part to DevOps, even failure can be a success if lessons are learned. (not sure where this quote actually came from but it kind of makes sense!)

I have added some other stories where DevOps has changed the game within some of these massively successful companies.

Resources

Recap of our first few days looking at DevOps

DevOps is a combo of Development and Operations that allows a single team to manage the whole application development lifecycle that consists of Development, Testing, Deployment, Operations.
The main focus and aim of DevOps is to shorten the development lifecycle while delivering features, fixes and functionality frequently in close alignment with business objectives.
DevOps is a software development approach through which software can be delivered and developed reliably and quickly. You may also see this referenced as Continuous Development, Testing, Deployment, Monitoring

If you made it this far then you will know if this is where you want to be or not. See you on Day 7.

Day 7 will be us diving into a programming language, I am not aiming to be a developer but I want to be able to understand what the developers are doing.

Can we achieve that in a week? Probably not but if we spend 7 days or 7 hours learning something we are going to know more than when we started.

#90DaysOfDevOps - Day 5

Michael Cade — Wed, 05 Jan 2022 09:19:49 +0000

Plan > Code > Build > Testing > Release > Deploy > Operate > Monitor >

Today we are going to focus on the individual steps from start to finish and the continous cycle of an Application in a DevOps world.

Plan

It all starts with the planning process this is where the development team gets together and figure out what types of features and bug fixes that they're going to roll out in their next sprint. This is an opportunity as a DevOps Engineer for you to get involved with that and learn what kinds of things are going to be coming your way that you need to be involved with and also influence their decisions or their path and kind of help them work with the infrastructure that you've built or steer them towards something that's going to work better for them in case they're not on that path and so one key thing to point out here is the developers or software engineering team is your customer as a DevOps engineer so this is your opportunity to work with your customer before they go down a bad path.

Code

Now once that planning session's done they're going to go start writing the code you may or may not be involved a whole lot with this one of the places you may get involved with it, is whenever they're writing code you can help them better understand the infrastructure so if they know what services are available and how to best talk with those services so they're going to do that and then once they're done they'll merge that code into the repository

Build

This is where we'll kick off the first of our automation processes because we're going to take their code and we're going to build it depending on what language they're using it may be transpiling it or compiling it or it might be creating a docker image from that code either way we're going to go through that process using our ci cd pipeline

Testing

Once we've built it we're going to run some tests on it now the development team usually writes the test you may have some input in what tests get written but we need to run those tests and the testing is a way for us to try and minimise introducing problems out into production, it doesn't guarantee that but we want to get as close to a guarantee as we can that were one not introducing new bugs and two not breaking things that used to work

Release

Once those tests pass we're going to do the release process and depending again on what type of application you're working on this may be a non-step. You know the code may just live in the GitHub repo or the git repository or wherever it lives but it may be the process of taking your compiled code or the docker image that you've built and putting it into a registry or a repository where it's accessible by your production servers for the deployment process

Deploy

which is the thing that we do next because deployment is like the end game of this whole thing because deployments when we put the code into production and it's not until we do that that our business actually realizes the value from all the time effort and hard work that you and the software engineering team have put into this product up to this point.

Operate

Once it's deployed we are going to operate it and operate it may involve something like you start getting calls from your customers that they're all annoyed that the site's running slow or their application is running slow right so you need to figure out why that is and then possibly build auto-scaling you know to handle increase the number of servers available during peak periods and decrease the number of servers during off-peak periods either way that's all operational type metrics, another operational thing that you do is include like a feedback loop from production back to your ops team letting you know about key events that happened in production such as a deployment back one step on the deployment thing this may or may not get automated depending on your environment the goal is to always automate it when possible there are some environments where you possibly need to do a few steps before you're ready to do that but ideally you want to deploy automatically as part of your automation process but if you're doing that it might be a good idea to include in your operational steps some type of notification so that your ops team knows that a deployment has happened

Monitor

All of the above parts lead to the final step because you need to have monitoring, especially around operational issues auto-scaling troubleshooting like you don't know
there's a problem if you don't have monitoring in place to tell you that there's a problem so some of the things you might build monitoring for are memory utilization CPU utilization disk space, api endpoint, response time, how quickly that endpoint is responding and a big part of that as well is logs. Logs give developers the ability to see what is happening without having to access production systems.

Rince & Repeat

Once that's in place you go right back to the beginning to the planning stage and go through the whole thing again

Continuous

Many tools help us achieve the above continuous process, all this code and the ultimate goal of being completely automated, cloud infrastructure or any environment is often described as Continuous Integration/ Continuous Delivery/Continous Deployment or “CI/CD” for short. We will spend a whole week on CI/CD later on in the 90 Days with some examples and walkthroughs to grasp the fundamentals.

Continuous Delivery

Continuous Delivery = Plan > Code > Build > Test

Continuous Integration

This is effectively the outcome of the Continuous Delivery phases above plus the outcome of the Release phase. This is the case for both failure and success but this is fed back into continuous delivery or moved to Continuous Deployment.

Continuous Integration = Plan > Code > Build > Test > Release

Continuous Deployment

If you have a successful release from your continuous integration then move to Continuous Deployment which brings in the following phases

CI Release is Success = Continuous Deployment = Deploy > Operate > Monitor

You can see these three Continuous notions above as the simple collection of phases of the DevOps Lifecycle.

This last bit was a bit of a recap for me on Day 3 but think this actually makes things clearer for me.

Resources

If you made it this far then you will know if this is where you want to be or not.

See you on Day 6.

#90DaysOfDevOps - Day 4

Michael Cade — Tue, 04 Jan 2022 12:16:59 +0000

DevOps & Agile

I think a good place to start is understanding a little more about a common angle I have seen in learning this area and that is DevOps vs Agile even though they have similar goals and processes. In this section, I am going to summarise this hopefully.

The biggest difference I can see here between the two though is the automation that comes with DevOps, tools are very similar but automation is where the main difference. But there are some more differences that we will cover now.

Do you know the difference between DevOps and Agile they were formed as standalone concepts? But now the two terms are getting fused in this post we will examine the crucial differences between agile and DevOps and find out why the two are connected so tightly let's start with definitions

Agile Development

Agile is an approach that focuses on delivering small results faster rather than releasing one big interaction of the product software is developed in iterations the team releases a new version every week or month with non-incremental updates the final goal of agile is to deliver an optimal experience to the end-users

DevOps

We have been covering this for the past few days with a few different ways of describing the end goals of DevOps, DevOps usually describes software development
and delivery practices based on cooperation between software developers and operations specialists the main DevOps benefits are delivering a simplified development process and minimising miscommunication.

What is the difference between Agile and DevOps

Different participants

Agile focuses on optimising communication between end-users and developers while DevOps targets developers and operation team members. We could say that agile is outward-oriented towards customers whereas DevOps is a set of internal practices.

Team

Agile usually applies to software developers and project managers the competencies of DevOps engineers lie in the intersection of development, QA and operations they are involved in all stages of the product cycle.

Applied Frameworks

Agile has a lot of management frameworks to achieve flexibility and transparency. Scrum > Kanban > Lean > Extreme > Crystal > Dynamic > Feature-Driven >. DevOps focuses on the development approach in collaboration but doesn't offer specific methodologies

Feedback

In Agile the main source of feedback is the end user in DevOps the feedback from stakeholders and the team itself has a higher priority on them.

Target areas

Agile focuses on software development more than on deployment and maintenance DevOps focuses on software development as well but its values and tools also cover deployment and post-release stages

Documentation

agile prioritises flexibility and tasks at hand over documentation and monitor DevOps on the other hand regards project documentation as one of the essential project components seven risks agile risks derived from the flexibility of the methodology agile projects are difficult to predict or evaluate as priorities and requirements are continually changing DevOps risks derive from a misunderstanding of the term and the lack of suitable tools some people see DevOps as a collection of software for the deployment and continuous integration failing to change the underlying structure of the development process

The Tools Used

Agile tools are focused on management communication collaboration, metrics and feedback processing the most popular agile tools include JIRA, Trello, Slack, zoom SurveyMonkey and others. DevOps uses tools for team communication software development deployment and integration like Jenkins, GitHub Actions, BitBucket, even though agile and DevOps have slightly different focuses and scopes the key values are almost identical therefore you can combine the two

Bring it all together… good idea or not? Discuss?

The combination of Agile and DevOps brings the following benefits you get

Flexible management and powerful technology
Agile practices help DevOps teams to communicate their priorities more efficiently
It leads to strengthening the team adopting agile practices will improve collaboration increase the team's motivation and decrease employee turnover rates
As a result, you get better product quality

Agile allows coming back to previous product development stages to fix errors and prevent the accumulation of technical debt how to adopt agile and DevOps
simultaneously just follow 7 steps

Unite the development and operation teams
Create build and run teams all development and operational concerns are discussed by the entire DevOps team
Change your approach to sprints assign priority ratings to offer tasks by the same scale that you use to great development projects encourage development and operations teams to exchange their opinion on other teams workflow and possible issues
Include QA in all development stages
Choose the right tools
Automate everything you can
Measure and control by using tangible numeric deliverables

What do you think? Do you have different views? I want to hear from Developers, Operations, QA anyone that has a better understanding of Agile and DevOps that can pass comments and feedback on this?

Resources

#90DaysOfDevOps - Day 3

Michael Cade — Mon, 03 Jan 2022 10:21:40 +0000

This post originates from https://github.com/MichaelCade/90DaysOfDevOps

DevOps Lifecycle - Application Focused

As we continue through these next few weeks we are 100% going to come across these titles (Continuous Development, Testing, Deployment, Monitor) over and over again, If you are heading towards the DevOps Engineer role then repeatability will be something you will get used to but constantly enhancing each time is another thing that keeps things interesting.

In this hour we are going to take a look at the high level view of the application from start to finish and then back round again like a constant loop.

Development

Let's take a brand new example of an Application, to start with we have nothing created, maybe as a developer you have to discuss with your client or end user on the requirements and come up with some sort of plan or requirements for your Application. We then need to create from the requirements our brand new application.

In regards to tooling at this stage there is no real requirement here other than choosing your IDE and the programming language you wish to use to write your application.

As a DevOps engineer, remember you are probably not the one creating this plan or coding the application for the end user, this will be a skilled developer.

But it also would not hurt for you to be able to read some of the code so that you can make the best infrastructure decisions moving forward for your application.

We previously mentioned that this application can be written in any language. Importantly this should be maintained using a version control system, this is something we will cover also in detail later on and in particular we will dive into Git.

It is also likely that it will not be one developer working on this project although this could be the case but even so best practices would require a code repository to store and collaborate on the code, this could be private or public and could be hosted or privately deployed generally speaking you would hear the likes of GitHub or GitLab being used as a code repository. Again we will cover these as part of our section on Git later on.

Testing

At this stage we have our requirements and we have our application being developed. But we need to make sure we are testing our code in all the various different environments that we have available to us or specifically maybe to the programming language chosen.

This phase enables QA to test for bugs, more frequently we see containers being used for simulating the test environment which overall can improve on cost overheads of physical or cloud infrastructure.

This phase is also likely going to be automated as part of the next area which is Continuous Integration.

The ability to automate this testing vs 10s,100s or even 1000s of QA engineers having to do this manually speaks for itself, these engineers can focus on something else within the stack to ensure you are moving faster and developing more functionality vs testing bugs and software which tends to be the hold up on most traditional software releases that use a waterfall methodology.

Integration

Quite importantly Integration is at the middle of the DevOps lifecycle. It is the practice of in which developers require to commit changes to the source code more frequently. This could be on a daily or weekly basis.

With every commit your application can go through the automated testing phases and this allows for early detection of issues or bugs before the next phase.

Now you might at this stage be saying "but we don't create applications, we buy it off the shelf from a software vendor" Don't worry many companies do this and will continue to do this and it will be the software vendor that is concentrating on the above 3 phases but you might want to still adopt the final phase as this will enable for faster and more efficient deployments of your off the shelf deployments.

I would also suggest just having this above knowledge is very important as you might buy off the shelf software today, but what about tomorrow or down the line... next job maybe?

Deployment

Ok so we have our application built and tested against the requirements of our end user and we now need to go ahead and deploy this application into production for our end users to consume.

This is the stage where the code is deployed to the production servers, now this is where things get extremely interesting and it is where the rest of our 86 days dives deeper into these areas. Because different applications require different possibly hardware or configurations. This is where Application Configuration Management and Infrastructure as Code could play a key part in your DevOps lifecycle. It might be that your application is Containerised but also available to run on a virtual machine. Which then also leads us onto platforms like Kubernetes which would be orchestrating those containers and making sure you have the desired state available to your end users.

All of these bold topics we will go into more detail over the next few weeks to get a better foundational knowledge of what they are and when to use them.

Monitoring

Things are moving fast here and we have our Application that we are continuously updating with new features and functionality and we have our testing making sure no gremlins are being found. We have the application running in our environment that can be continually keeping the required configuration and performance.

But now we need to be sure that our end users are getting the experience they require. Here we need to make sure that our Application Performance is continuously being monitored, this phase is going to allow your developers to make better decisions about enhancements to the application in future releases to better serve the end users.

This section is also where we are going to capture that feedback wheel about the features that have been implemented and how the end users would like to make these better for them.

Reliability is a key factor here as well, at the end of the day we want our Application to be available all the time it is required. This then lends to other observability, security and data management areas that should be continuously monitored and feedback can always be used to better enhance, update and release the application continuously.

Resources

I am always open to adding additional resources to these readme files as it is here as a learning tool.

My advice is to watch all of the below and hopefully you also picked something up from the text and explanations above.

Continuous Development I will also add that this is focused on manufacturing but the lean culture can be closely followed with DevOps.
Continuous Testing - IBM YouTube
Continuous Integration - IBM YouTube
Continuous Monitoring
NOT FREE The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win

If you made it this far then you will know if this is where you want to be or not. See you on Day 3.

#90DaysOfDevOps - Day 2

Michael Cade — Sun, 02 Jan 2022 10:21:44 +0000

This post originates from https://github.com/MichaelCade/90DaysOfDevOps

Responsibilities of a DevOps Engineer

Hopefully you are coming into this off the back of going through the resources and post on Day1 of #90DaysOfDevOps

It was briefly touched on in the first post but now we must get deeper into this concept and understand that there are two main parts when creating an application. We have the Development part where software developers program the application and test it. Then we have the Operations part where the application is deployed and maintained on a server.

DevOps is the link between the two

To get to grips with DevOps or the tasks in which a DevOps engineer would be carrying out we need to understand the tools or the process and overview of those and how they come together.

Everything starts with the application! You will see so much throughout that it is all about the application when it comes to DevOps.

Developers will create an application, this can be done with many different technology stacks and lets leave that to the imagination for now as we get into this later. This can also involve many different programming languages, build tools, code repository etc.

As a DevOps engineer you won't be programming the application but having a good understanding of the concepts of how a developer works and the systems, tools and processes they are using is key to success.

At a very high level you are going to need to know how the application is configured to talk to all of its required services or data services and then also sprinkle a requirement of how this can or should be tested.

The application will need to be deployed somewhere, lets keep it generally simple here and make this a server, doesn't matter where but a server. This is then expected to be accessed by the customer or end user depending on the application that has been created.

This server needs to run somewhere, on-premises, in a public cloud, serverless (Ok I have gone too far, we won't be covering serverless but its an option and more and more enterprises are heading this way) Someone needs to create and configure these servers and get them ready for the application to run. Now this element might land to you as a DevOps engineer to deploy and configure these servers.

These servers will have to run an Operating System and generally speaking this is going to be Linux but we have a whole section or week where we cover some of the foundational knowledge you should gain here.

It is also likely that we need to communicate with other services in our network or environment, so we also need to have that level of knowledge around networking and configuring that, this might to some degree also land at the feet of the DevOps engineer. Again we will cover this in more detail in a dedicated section talking all things DNS, DHCP, Load Balancing etc.

Jack of all trades, Master of none

I will say at this point though, you don't need to be a Network or Infrastructure specialist you need a foundational knowledge of how to get things up and running and talking to each other, much the same as maybe having a foundational knowledge of a programming language but you don't need to be a developer. However you might be coming into this as a specialist in an area and that is a great footing to adapt to other areas.

You will also most likely not take over the management of these servers or the application on a daily basis.

We have been talking about servers but the liklihood is that your application will be developed to run as containers, Which still runs on a server for the most part but you will also need an understanding of not only virtualisation, Cloud Infrastructure as a Service (IaaS) but also containerisation as well, The focus in these 90 days will be more catered towards containers.

High Level Overview

On one side we have our developers creating new features and functionality (as well as bug fixes) for the application on one side.

On the other side we have some sort of environment, infrastructure or servers which are configured and managed to run this application and communitcate with all its required services.

The big question is how do we get those features and bug fixes into our production and make it available to those end users?

How do we release the new application version? This is one of the main tasks for a DevOps engineer, and the important thing here is not to just figure out how to do this once but we need to do this continously and in an automated, efficient way which also needs to include testing!

This is where we are going to end this day of learning, hopefully this was useful. Over the next few days we are going to dive a little deeper into some more areas of DevOps and then we will get into the sections that dive deeper into the tooling and processes and the benefits of these.

Resources

I am always open to adding additional resources to these readme files as it is here as a learning tool.

My advice is to watch all of the below and hopefully you also picked something up from the text and explanations above.

If you made it this far then you will know if this is where you want to be or not. See you on Day 3.

#90DaysOfDevOps - Day 1

Michael Cade — Sat, 01 Jan 2022 22:46:18 +0000

This post originates from https://github.com/MichaelCade/90DaysOfDevOps

The progress has been continued but not logged here. I need to automate how the days are linked to dev.to via GitHub Actions. I have seen a few examples of this.

Introduction - Day 1

Day 1 of our 90 days and adventure to learn a good foundational understanding of DevOps and tools that help with a DevOps mindset.

This learning journey started for me a few years back but my focus then was around virtualisation platforms and cloud based technologies, I was looking mostly into Infrastructure as Code and Application configuration management with Terraform and Chef.

Fast forward to March 2021, I was given an amazing oppurtunity to concentrate my efforts around the Cloud Native strategy at Kasten by Veeam. Which was going to be a massive focus on Kubernetes and DevOps and the community surrounding these technologies. I started my learning journey and quickly realised there was a very wide world aside from just learning the fundamentals of Kubernetes and Containerisation and it was then when I started speaking to the community and learning more and more about the DevOps culture, tooling and processes so I started documenting some of the areas I wanted to learn in public.

So you want to learn DevOps?

Let the journey begin

If you read the above blog you will see this is a high level contents for my learning journey and I will say at this point I am no where near an expert in any of these sections but what I wanted to do was share some resources both FREE and some paid for but an option for both as we all have different circumstances.

Over the next 90 days I want to document these resources and cover those foundational areas, I would love for the community to also get involved share your journey and resources so we can learn in public and help each other.

You will see from the opening readme in the project repository that I have split things into sections and it is basically 12 weeks plus 6 days. The first 6 days we will explore the fundamentals of DevOps in general before diving into some of the specific areas, by no way is this list exhaustive and again would love for the community to assist in making this a useful resource.

Another resource I will share at this point that I think everyone should have a good look at and maybe create your own mind map for yourself and your interest and position is the following:

DevOps Roadmap

I found this a great resource when I was creating my initial list and blog post on this topic. You can also see there are other areas that go into a lot more detail outside of the 12 topics I have listed here in this repository.

First Steps - What is DevOps?

There are so many blog articles and YouTube videos to list here, but as we start the 90 day challenge and we focus on spending around an hour a day learning something new or about DevOps I thought it was good to get some of the high level of "what DevOps is" down to begin.

Firstly, DevOps is not a tool. You cannot buy it, it is not a software sku or an open source github repository you can download. It is also not a programming language, it is also not some dark art magic either.

DevOps is a way to do smarter things in Software Development. - Hold up... But if you are not a software developer should you turn away right now and not dive into this project??? No Not at all, Stay... Because DevOps brings together a combination of software development and operations. I mentioned earlier that I was more on the VM side and that would generally fall under the Operations side of the house, but within the community there are people with all different backgrounds where DevOps is 100% going to benefit the individual, Developers, Operations and QA Engineers all can equally learn these best practices by having a better understanding of DevOps.

Another area we will dive into in this first week is around The Agile Methodology DevOps and Agile are widely adopted together to achieve continuous delivery of your Application

The high level take away is with a DevOps mindset or culture its about taking a way the long drawn out software release process from potentially years to being able to drop smaller releases more frequently. The other key fundamental to take away here is it's about breaking down silos between the teams I previously mentioned, Developers, Operations and QA.

From a DevOps perspective, Development, Testing and Deployment all land with the DevOps team.

The final point I will make is to make this as effective and effcient as possible we must leverage Automation

Resources

I am always open to adding additional resources to these readme files as it is here as a learning tool.

My advice is to watch all of the below and hopefully you also picked something up from the text and explanations above.

DevOps in 5 Minutes
What is DevOps? Easy Way
DevOps roadmap 2022 | Success Roadmap 2022

If you made it this far then you will know if this is where you want to be or not. See you on Day 2.

#Project_Pace – Kasten K10 Demo environment for everyone

Michael Cade — Mon, 01 Nov 2021 14:15:33 +0000

My goal here was to make a super easy way to deploy Kasten K10 and be able to demonstrate the value of K10 in a simple and easy way without having to deploy public cloud managed Kubernetes environments, or having to rely on virtual environments. Most people have access to your own laptop or desktop machine be it Windows, MacOS or Linux.

I wanted to make something that would be easy to spin up and down but give us the capability of showing the benefits that K10 brings to your Kubernetes Data Management in the real world. But then I got thinking a lot of developers use local Kubernetes environments on their own systems.

These local Kubernetes clusters have an equally overwhelming list of options when it comes to choosing and I have been playing mostly with K3D locally but also KinD (Kubernetes in Docker) and more recently MiniKube.

The overall goal as I mentioned was to create an easy way for anybody and everybody to roll out a Kubernetes cluster locally and have the Kasten K10 functionality to hand, be it a proof of concept, learning or using the Kasten K10 free edition to protect local development environment.

MiniKube

I chose minikube for this simply because I found that the Addons available make things so easy, yes, we have helm charts for the addons or at least for the most part but minikube having them built in makes life easy. To begin with regardless of your workstation OS, you can run minikube. First, head over to the project page here. The first option you have is choosing your installation method. I did not use this method, but you might choose to vs my way.

My way of installing minikube

I have been using arkade for some time now to get all those Kubernetes tools and CLIs, you can see the installation steps on this github repository for getting started with Arkade. I have also mentioned this in other blog posts where I needed something installing. The simplicity of just hitting arkade get and then seeing if your tool or cli is available is handy.

As part of the long list of available apps within arkade minikube is one of them so with a simple “arkade get minikube” command we are now downloading the binary and we are good to go.

We will also need kubectl as part of our tooling so you can also get this via arkade or I believe that the minikube documentation brings this down as part of the curl commands mentioned above.

Getting a Kubernetes cluster up and running

For this particular section I want to cover the options available to us when it comes to getting a Kubernetes cluster up and running on your local machine. We could simply run the following command and it would spin up a cluster for you to use.

So the above is great but we want to be able to do more with our cluster particularly as we want to run some data services in our environment so we need to take advantage of some of the minikube addons that are available.

If you run the following command you can see some of these addons,

For our project we are using volumesnapshots and csi-hostpath-driver.

I am also defining in our project some additional configuration, apiserver is set to 6433 instead of a random API port, I define the container runtime also to containerd however docker is default and CRI-O is also available. I am also setting the Kubernetes version so that I know my Kasten K10 deployment is at a supported version.

This will give us our Kubernetes cluster.

Deploy Kasten K10

There are two options when it comes to deploying Kasten K10 into our new cluster, we can use the helm chart and walk through the deployment this way or we can also use arkade which I have mentioned a little above.

Deploy a Data Service (MySQL)

In order to start testing Kasten K10 we need a data service, we have chosen MySQL as the first option but in the future, I can see options for many more being added as different people have different data services so might want to protect them, maybe we also include a wordpress deployment which has a database backend that might need to be protected. My goal is to add additional data services here to show the power of protection.

K10 Overview

I am not going to go through a complete K10 overview here but just know that from here we can protect our application using the deployed CSI-Hostpath and volumesnapshot, now we know a snapshot is not going to protect you from anything locally but it gives you the look and feel and a very fast recovery However I am also considering or exploring how we can add minio into our minikube configuration so that we have an export option as well, now again this would also land on the same infrastructure as our Kubernetes cluster which would not protect us against local failure but this is a demo environment and a learning journey. We could also add in external object storage options if you wish to protect some data here.

Cloud-Native Data Management Day - Live Stream Event from LA

Michael Cade — Tue, 12 Oct 2021 20:47:18 +0000

hey everyone, I am watching remote this event on YouTube and I wanted to share this with everyone here - https://www.youtube.com/watch?v=NalJmdmSa-U

Cloud-Native Data Management Day 2021 NA is Next week!

Michael Cade — Thu, 07 Oct 2021 14:02:42 +0000

You might have seen a few posts and tweets about Cloud-Native Data Management Day from me and Kasten over the last few weeks and maybe months.

Firstly, don’t worry there is no registration for the virtual event but if you are heading to LA then don’t miss out on the Grammy museum, also be sure that the in-person entrance fee will be fully reimbursed with adequate SWAG!

If you haven’t seen it yet, the agenda for our event was posted this week, and it’s full of fantastic panels discussions! Promotion for the live in-person sessions is happening now.

https://cndmday.com/events/cndm-day-2021-na/

Then we have the virtual experience for you as well! If you are unable to attend the in-person event like many including me, we have you covered!

You can catch the events live stream here.

CNDM Day NA 2021 – LIVE from the GRAMMY® Museum L.A. Live in Los Angeles, CA

https://youtu.be/NalJmdmSa-U

Then we also have over 30 sessions that have been pre-recorded and sent in from the community. These will form part of the OnDemand content, don’t worry we are not expecting you to get through over 30+ hours of content in one day… no having to turn sessions to x4 to get through them.

Community Track – CNDM Day NA 2021

https://youtube.com/playlist?list=PL8Nlv_BickZ2L52XRIfAvcdCBxwDCmIys

Technical Track – CNDM Day NA 2021

https://youtube.com/playlist?list=PL8Nlv_BickZ28VENAfZeouKvhVzPw_4dB

Industry Track – CNDM Day NA 2021

https://youtube.com/playlist?list=PL8Nlv_BickZ39MZgg92yfN6Jh6YK0brDE

Finally, Community is the key to this event, and we are growing!

Please consider subscribing to the

YouTube channel

Twitter

Slack

We are also leaving the Call for Presentations open as we have seen some great sessions submitted that we will look to include in our Live Stream sessions over the next few weeks following KubeCon and CNDM Day NA.

Submit a session

Any questions please ask

My new “Budget” Custom Mechanical Keyboard

Michael Cade — Thu, 16 Sep 2021 09:38:56 +0000

Like many during however long this pandemic has been going on for we have found other interests and hobbies to either drain our money away or at least entertain us and sometimes both whilst we are not or were not allowed out and about. My lockdown started with a tech refresh which grew into creating a home studio to create content and generally make life a little easier whilst working from home for extended periods, then I found mechanical keyboards!

The first cut is the deepest

As I was refreshing the systems at home, I purchased a new Logitech G915 mechanical keyboard, pretty much an off the shelf keyboard that looks sleek and smart (don’t hate on me I was coming from the Apple ecosystem) The noise thing makes is amazing and I still use this keyboard daily, but this would start something, someone call it an obsession.

The Journey

Ok, you start with one and then naturally you go hunting for another, right? You start to join subreddits and follow some out there YouTube content creators that review only mechanical keyboards and build them. (we were in lockdown for a long time) You learn that you have different size of keyboards, full keyboard > tenkeyless (TKL) > 65% > 60% > 40%. Then you have the different switches and keycap sets. It is a whole world and if you are reading this and you have not taken the blue pill then pull back because you will end up with more keyboards than you have systems to use them. (But it is fun)

There becomes a time though that you just want to build something yourself, and by build, I mean buy the parts and put them together, I mean some people build their complete keyboard and there are some crazy steampunk looking stuff out there. I quoted the budget term in the title because I am massively aware that not everyone has the luxury of being able to do this and I am extremely lucky to be able to, I also used it because budget is a great term to use with the better halve when justifying your spend on something like this.

My First Custom

I was dubious about shopping on AliExpress as I had never used the service before, but it doesn’t seem the same as something like Amazon, but it was really the only place I could gather up the parts at a reasonable price vs spending almost double on places like Etsy and eBay. Apart from the switches I was able to source everything on AliExpress which did take a few weeks to arrive but overall everything was very good. The switches I got from Amazon on a next day once I decided the right ones for the build.

The case & PCB

The choice of The Case, PCB, positioning board was the TM680 or sometimes known as the TOM680. You can find out more information about what you get in the box here as well, the link is also where I purchased it from and everything went well, packaging good etc and it was timely considering it was coming from China. I had seen a few mentions of this kit on a few YouTube channels about this being a budget keyboard kit, so it was the right starting point on this custom build journey.

You can get the unit in various colours and not just the ones I found below now, but I chose black because of the colour scheme I was working towards.

Switches

Now I am not going to get into why I chose switches because everyone is going to have a different opinion which is fine but I went with the Gateron Yellow Linear switches from here. My advice here is watch some YouTube videos and find out which one you would like, and if you are serious then maybe a switch tester is in your future where you could get a tester with all variants of switches in.

Keycaps

Customising your keyboard is the fun part, it’s no longer just a black or a beige keyboard we can get very creative when it comes to keycap sets but you can also spend a lot of money, so I went and found a clone copy of some keycaps in the colour scheme that I wanted. AKKO NEON Keycaps Set Cherry/ASA Profile Double-shot PBT 108 Full Keycaps Set, with 49 Extra Novelty Keycaps from here.

I also went a little silly on some custom ideas for the space bar some of the options are below.

I will save the one I went with till the end.

Coiled Cable

Although I went with the Bluetooth, wireless and cabled option I also wanted to complete the setup with a nice, coiled cable that matches the colour scheme. Again, Aliexpress had an affordable selection again I was dubious, but all came through good. And I went with this one.

The finished article

With that we have our new custom keyboard that joins the collection along with all the other boards that don’t have a device but it’s the thought that counts. In this house we will never have a shortage of keyboards that’s for sure.