DEV Community: Skyler

DevOps Basics: CI/CD/CD

Skyler — Tue, 13 Sep 2022 16:42:49 +0000

Today, I'm inaugurating a new category on the blog: DevOps basics! and we will start with CICDCD.

It is a question usually asked in interviews: what is CI/CD, what do you think of Continuous Deployment?
Let's see first what a deployment is, then establish what are those CI, CD, CD about.

Deployment

A deployment is an act of installing/activating the software/binaries. During deployment, if an existing version is there, then uninstalling/deactivating the previous version takes place. A release is the culmination of all the activities to get changes safely into production.

Continuous Integration

CI is the automation of builds. The code change needs to be merged, committed into the source code repo, and checked against it.
Continuous Integration solves 3 main problems: having the builds be:

repeatable,
consistent,
available.

The build pipeline has to be fast and reliable, as it will be run multiple times after being triggered.

Continuous Deployment

Continuous Deployment focuses – just like the name implies – forcuses on the deployment; the actual installation and distribution of the bits.
As Continuous Integration provides a deployable artifact, Continuous Deployment can take that artifact forward into additional environments.

Continuous Delivery

Continuous Delivery is the automation of steps to safely get changes into production.
It's important to make clear that every change that passes all stages of your production pipeline is released to your customers. There's no human intervention, and only a failed test will prevent a new change to be deployed to production.
As you can imagine, testing needs to be more than solid before considering this strategy.

Opinions

Interviewers usually ask about Continuous Delivery as it gives a good insights into someone thoughts and experience.
For example, in some companies, it's totally accepted and recommended to have releases automatically going into production. Very little manual testing is done - if at all.
In a lot of companies, it is unthinkable. Every change would go through a monthly/or-more release, heavy manual testing and business signing and testing on top.

Continuous Integration and Continuous delivery is usually already present in every company (except some start-ups). They're usually set up with Jenkins, Team City, Azure DevOps, ...

That was it for today :). Hope this was useful, if you have any questions, don't hesitate!

Interviews: questions to ask and how to present yourself

Skyler — Wed, 07 Sep 2022 15:40:29 +0000

Hello! It's been a while on this blog. I just came back from Toronto, I feel very inspired and wanted to put down a few advices for you guys.

I just came back from Toronto, Canada!

I took some nice pictures and I feel a lot more “refreshed” now! Looking forward to keep that energy going ☺️ pic.twitter.com/FbobmCbGyp
— Skyler (@skylerdevops) September 7, 2022

Today we talk interviews. How to present yourself, a few questions to ask. I will keep this brief so that you can easily pin this somewhere and refer to the main points.

Let's go!

How to prepare:

What are your main strengths? This is a mix of 1) looking at the job description and what they explicitly mention (refer to those qualities - if you have them!), 2) personal introspection. For example: I'm great at communicating with the Team. This helps me to establish a good report with everyone I work with.
Main accomplishments? What can you bring to the company? Again, try to refer to the job description. For example: I set up a Kubernetes cluster at my first job. I'm able to find solutions by myself, although I appreciate team work!
Finance: If you have any metric that is related to money, companies love that. For example: I created a script that would turn off the VM at night. Therefore, our bill went down by x %.

Here, it's really important to understand that you are selling yourself. It's about what you can bring to them, what value your addition to the team will make.

Questions

At the end of an interview, you should have a time allocated for questions for them. This is now **your **time to see if they are a good fit for you. You might need to change ASAP because of financial reasons, but if you can take your time, please do. Finding a company that ticks most, if not all, your boxes will go a lot further than a "quick, I just want to change ships".
Anyway, a few questions that I always ask:

What tech do they work with?
Is there an On-Call Rota?
Do they work with multiple timezones? (this mean you might need to stay later/come earlier)
Hybrid? If yes, how many days a week/month in the office?
Team/department structure/composition? This is very important if, for example, you are a woman and want to make sure you won't be the only one in the department...!
Are there any socials organised? Does the team participate?
Career progression,
What would make someone successful in this role,
How would you describe the culture of the company?

I understand this seems like a lot - but it isn't. You want to have this information before joining.
A few more questions I like, that I still think you should ask, although they seem more "nosey":

Why are they hiring? Why did the last person leave?
How do they deal with someone that under performs?
What is the main challenge for someone in this role in this company?
How do the manager of the team deal with unreasonable timeline or pushback? Again, those are situations you might find yourself in, or see a colleague in. You will want to know how your manager react before having signed a contract!

This is it for this article, I hope this helps you out in your next search! :)

Why you shouldn't keep a**holes in your team

Skyler — Sat, 12 Feb 2022 12:59:37 +0000

Hi! I recently made a Twitter post about a**holes at work, and I thought I would expand a bit more in a blog post, so here we go!

First of all, there are factual proofs, research and articles that has been done around "why you don't want a toxic person in a team". It's not just me spouting nonsense. Let's dig into the subject a bit more.

Risk

What is in someone's head stays in their own head if they do not know how to communicate. The person ends up hoarding the information and does not know/want to talk around it. This is a risk to the company long term because you end up with one person really skilled, surrounded by others who aren't really sure what's actually behind x process or Y release, as if there was a "mist" around some subjects. This is a** one man** dependency.

Burn out

If you have ever worked with one of those specimen, you would have noticed something very quickly: they tend to love work, and they don't get tired of it!

However, people surrounding them feel the opposite. Therefore, you end up with a higher turnover surrounding that person. A few reasons:

It gets tiring to discuss a ticket when you know the other person will not back down, will not acknowledge they might be wrong and that, for hours.
Feeling of demotivation,
" I haven't progressed as much as I thought in this role",
...

Attitude

There is a lot to say about this one, but I will keep it short. It's obvious a**holes don't have a good attitude - that's why they are what they are. A few examples of behaviours:

Monitor and doubt others,
Set unrealistic expectations,
Likes to point out others 'faults in a non-constructive way,
Reject situations where they do not feel in control,

Reputation

That one is usually forgotten in the articles I've read. If a team/department contains a schmuck, I can promise you that they have a reputation. Because, honestly, who wants to include a team who contains someone like that? I have seen situations where a department totally ignored a team, as unprofessional as it might sound.

Conclusion

In a few words, why you do not want a prick in your team:

turnover of people goes higher,
productivity can actually go down as people don’t feel appreciated/respected,
if the company has not taken measures against an asshole just because he does tickets… it says a lot.

Have you ever had an a*hole in your team? **Next article, I will share a few tricks on how to deal with the situation :)!*

How to structure your Terraform project

Skyler — Mon, 03 Jan 2022 17:07:01 +0000

Hi! This article is "Terraform beginners" friendly and for anyone curious about TF best practices. If you've learned anything, it would be awesome to share this article on Twitter or to react in the comments here :)

This time, we are going to talk about Terraform, more specifically, how to you organise your project/repository. What I'm going to share is "best practices" and how most companies I've seen use. If you do differently, please share!

.
|
├── modules                    
│   ├── database               # first module
│   ├── networking             # second module
│   └── ...
└── main.tf
└── version.tf
└── variables.tf
└── outputs.tf

Let's look in details:

modules: This is where you put re-usable code. Let's say you need to call multiple times a database. Stick the configuration in /modules/database, call it from main.tf (or whatever name) and that's it, you've got code that's a bit more DRY!
main.tf: The best practice is to have a main.tf with either your whole configuration (if small), or the core of it and others .tf files (if long).
version.tf: This is the file where you put the required_providers and required_version versions. More information in this blog post.
variables.tf: This file does differ from companies to companies. Some companies use a variables file specific for each environment, some use this file only for the main generic one and use another directory which I will mention below.
outputs.tf: You don't always have outputs from your code. Outputs are what you want to access from somewhere else. Usually used for modules, so that you can access ID's, names and others from the resources you create in there. Example with an EKS cluster: You might want to access the endpoint. You can find outputs by looking at the section called "Attributes Reference".

Slightly advanced: add a config folder (the name does not matter).

.
|
├── config                  
│   ├── cli         
│   ├── dev         
│   └── ...
...

We will use this example of a cli file:

storage_account:
   tier: Standard

We'll make this works in a few steps:
1) Create a locals block:

locals {
   config_content = file("./config/${var.environemnt}.yaml")
   config = yamldecode(local.config_content)
}

2) Then, in the code you would access by calling local.config.storage_account.tier.

I hope this made sense! I want to talk more about Terraform, as it's one of my favourite tech, feel free to give me questions or suggestions :).

Powershell-inception: how to create a background process that spawns another process to run a script

Skyler — Thu, 02 Sep 2021 14:45:05 +0000

Hello! Today, we're back for a technical article.

If you survived reading the title, then you might be wondering why the heck you would want to do that. And you know what? You're right. This is one of these situations where I really think you should ask yourself "do I really need to do this".

It is possible to do "powershell inception", however, I have been made aware it might not be the best tool to do it.

Let me explain why this happened, then I will show the code (I know that's what you want).

I was requested to do this at work. We wanted to use the agentless feature in TeamCity. The great thing is that, once your build is doing a call to external service, TC will give your build agent to another build configuration. that's cool. What is less cool is that you need the "external service" to send back a PUT saying "yes, I'm finished". We wanted to use it for Azure services, and well, Azure doesn't send this confirmation back.

So here we are spawning another process that won't get killed when the current process does.

Let's just get started.

We will cut this in 3 scripts:
1) Detached.ps1: Spawning a separate process that is not the child of the process running the script,
2) RunDetached.ps1: that script spawns another process that will itself run a script,
3) Script.ps1: The script that actually does the work you wanted.

Detached.ps1:

param (
    [string]
    $file,

    [parameter(Mandatory=$true, ValueFromRemainingArguments =$true)]
    $arguments
)

$process = Invoke-CimMethod -Class Win32_Process -MethodName Create -Arguments @{ CurrentDirectory = $location.Path ; CommandLine = "powershell -NonInteractive -ExecutionPolicy Bypass -File $file $arguments "}

if ($null -eq $process.ProcessId)
{
    Write-Warning "Background process did not work"
    Exit 1
}
else
{
    Write-Host "Started background process"
}

Okay, a lot of things happen here. Let's look a bit closer.

ValueFromRemainingArguments is my latest discovery in Powershell and it is one of the best feature in my opinion. Basically, it will pass the remaining arguments to the next script. Let's say you call ./Detached.ps1 -script something.ps1 -whatever stuff -more value, it will understand that -more value is related to the script you're calling! I encourage you to have a look online, it is a bit tricky to explain.

You don't have to write CurrentDirectory = $location.Path ;, it just means you will spawn up the process in the current directory you are at, not in C:\Windows\System32 which is the default.

We then checked if the creation was successful or not.

RunDetached.ps1

param (
    [string]
    $anotherScript,

    [parameter(Mandatory=$true, ValueFromRemainingArguments = $true)]
)
)

function GetMessages {
    ## this will be used to get the messages from the script you will call, otherwise, you won't see anything.

    Get-Event -SourceIdentifier ProcOutput -ErrorAction SilentlyContinue | %{
        if ($_.SourceEventArgs.Data) {
            ##do whatever you want, write-Host or send to file, ...
        }
        Remove-Event -EventIdentifier $_.EventIdentifier
    }
    Get-Event -SourceIdentifier ProcError -ErrorAction SilentlyContinue | %{
        if ($_.SourceEventArgs.Data) {
            ##do whatever you want, write-Host or send to file, ...
        }
        Remove-Event -EventIdentifier $_.EventIdentifier
    }
}

try {
    #process information
    $pinfo = New-Object System.Diagnostics.ProcessStartInfo
    $pinfo.FileName = "powershell.exe"
    $pinfo.RedirectStandardError = $true
    $pinfo.RedirectStandardOutput = $true
    $pinfo.UseShellExecute = $false 
    $pinfo.Arguments = "-ExecutionPolicy Bypass -NonInteractive -File `"$anotherScript`" $arguments" #using `" `" to escape the double quotes

    #Create process object
    $proc = New-Object System.Diagnostics.Process
    $proc.StartInfo = $pinfo

    #Register event handler, allows us to read the streams asynchronously. We need to do that or we might deadlock the child script
    Register-ObjectEvent -InputObject $proc -EventName OutputDataReceived -SourceIdentifier ProcOutput
    Register-ObjectEvent -InputObject $proc -EventName ErrorDataReceived -SourceIdentifier ProcError

    #starting process
    $proc.Start() | Out-Null
    $proc.BeginErrorReadLine()
    $proc.BeginOutputReadLine()

    while (!$proc.WaitForExit(100)) {
        GetMessages
    }
    GetMessages #final one

    if ($proc.ExitCode -ne 0) {
        Exit $proc.ExitCode
    }

}
catch {
    #do the catch however you want
}

Let me explain a bit more what happens here:
In GetMessages, you streamline the output and the errors to come up so you can see them (by default, you don't, as it is another process).
In the try, we spawn up a new process.

Then in $anotherScript you just put the Script.ps1 that does the work you actually wanted.

I hope this helps anyone at some point, this definitely took some time to figure out for me :).

See you next time!

DevOps Mentoring

Skyler — Fri, 09 Apr 2021 15:58:01 +0000

I recently have been looking at mentoring and what I can bring to the table.

I would really appreciate it if you could tell me your opinions and maybe share this blog post with people in your network.

Are you interested in learning DevOps?
Do you have any notions of DevOps?
How do you see DevOps?
What subjects are you the most interested in?

Thank you for your answer!

Lessons learned from changing jobs in a difficult time

Skyler — Wed, 16 Dec 2020 18:18:32 +0000

Hello!

Glad to see you again in a new post - or welcome if it's your first time reading me! I thought I would write this column about my new job, how it went and how it is going.

Disclaimer: I'm still a DevOps Engineer, I just changed the company I worked for. The previous company was in Media, I now work in FinTech (tech for the financial industry). I have been working in this field for two years now.

#1: Remote

As the situation imposes it, the company is fully remote. In my own opinion, and just mine, this made joining a new company harder. I know many of you will want to go fully remote in the future and it is not my case - at least for now. A mix of office and WFH when "things go back to normal" (whatever that means) would suits me best.

Joining a company, learning new things, it can already be a difficult moment. Add on top of that, the situation, the tiredness of December, the "English is not my first language" and sprinkle some Imposter Syndrome and you have a great mix.

#2: I don't know sh*t

As I've mentioned in another post (Kubernetes : how to deploy as a beginner and by yourself), I had a lot of responsibilities when I was just a junior/graduate. I felt "good" on the moment about it - you need something with Kubernetes? Call SkylerDevops. You need Terraform? Call SkylerDevops. (Yes, this is a shameless promo).

Looking back on it, I'm not sure this was right.

I joined this new team, with people who all have more experience than me. And boy.

Oh boy.

It’s only Monday and I’m already tired of struggling at work

Help pic.twitter.com/xVQI1OVj8R
— Skyler (@skylerdevops) November 9, 2020

So I started my new job last week

And boy
Oh boy pic.twitter.com/rAd2pQ5Meh
— Skyler (@skylerdevops) November 5, 2020

This is me. A lot.
I'm currently learning how to use my stress in a good way and not let me turn into mush.

#3: Doubts

Related to the previous point, I found myself in a team that was better than at, well, everything. Initially, I struggled with this idea a lot.

"What am I supposed to add", "They must be so annoyed with me".

Well, let me tell you something. If you arrive in a company, with 2 years experience, and you know more than them, you need to ask some questions (I know, there are exceptions).

The idea when you start somewhere, especially at the beginning of your career is to learn. Let me repeat.

Learn.

It is not right to impose you to "figure it out" by yourself, with little to no mentoring. Believe me, I have been there. I used to think "I can do this, you see. I'm GOOD". I encourage you to read jesuisundev article about developer's ego, which retraces the story of many new devs with too many responsibilities. I will write my own story one of these days.

Within the first month of working at the company, I have written the best Powershell scripts I have ever written. I have been asked "why" and having to form a coherent answer, not just "she's devops, she knows".

You should learn to depend on your colleague and to not hesitate to send them "hey, I'm not quite sure about this, what do you think?".

I'm going to wrap up this blog post here, but I hope you got a few things from this. I would be curious to know other people's experience during these unprecedented times.

See you next time!

Juniors and overtime

Skyler — Thu, 15 Oct 2020 14:33:54 +0000

Junior and overtime seem to go together these days. I see a lot of people encouraging "staying in the office" (well, staying to your desk at home, more likely) because that's how you will get noticed, right?

I realise many people tell me "they don't have a choice". I wanted to write an article about my experience on the subject, being a young person in the field for now 2 years. I would be curious to learn more about people’s point of view before doing so.

What is your opinion on juniors and overtime? Did you do it for your career? Did it help?

But mainly... Do you regret doing it?

Disclaimer: This is a safe space. I am not here to make fun of your experience or belittle you. I am genuinely curious as to what push people to accept repeated overtimes. :)

Monitoring for Kubernetes: introduction

Skyler — Sun, 11 Oct 2020 08:01:46 +0000

After the last blog post, you should now have a working Kubernetes cluster!

First of all, congrats!

I'm sure your next question is regarding monitoring, and you would be right. We have a brand new playground, but if something goes wrong, we won't really know why.

This article will be an introduction to all the tools I would recommend, you won't see code in here (the article would be way too long). The tutorial part will come later.

Application logs

The first type of monitoring would be the application logs. Are requests successful? Who is making that request?

I personally use Azure App insights (as it's not very pricey plus you have a free tier), but feel free to look at others.

You install a small SDK in your app, and bam, you're sending the telemetry data to Azure.

You can instrument not only the web service application, but also any background components, and the JavaScript in the web pages themselves. The application and its components can run anywhere - it doesn't have to be hosted in Azure.

I know AWS has some logs for apps as well, but I personally haven't used them.

Infrastructure

This part will be seen in more details in future articles.
The usual combination for Kubernetes infra is Prometheus + Grafana + service mesh.

Prometheus + Grafana

Prometheus is a data collector, and you can add alertmanager on top to send you alerts when things are not going exactly the way they should. Prometheus will pull the metrics from your system, which will not add load to it.

You can integrate AWS cloud watch metrics to be collected by Prometheus as well, as you will surely be using other AWS services (for example, RDS).

Grafana is the nice UI you usually see with Prometheus tutorials.

Service mesh

We personally use Linkerd as it was so easy to install (2h top), and we didn't need any of the 'fancy' ISTIO was promising.

By default, Linkerd stores metrics internally in prometheus style, on an internal Prometheus database. It is easy to add your own centralised Prometheus server, which is something you might want to consider as Linkerd only keeps metrics for 6hours (more here ).

You can as well add a standalone grafana to it.

If you need to change the default configuration, it is possible, just a tad more work (I wanted to mention it, although I wouldn't expect first-time beginners to do this!).

I will as well add that it is a quite new product (the first PR was in December 2017) so there are some things missing - or that could be done better. The team is quite helpful and quick to answer, so don't hesitate to ping them if needed.

With all those, you should have all the keys to identify problems related to your cluster. Obviously, if you haven't worked with Kubernetes before, you will find yourself looking at logs for a long time - and this is fine!

I realise this is a very short article - and to be honest, my goal is not to re-explain documentation that can be found online (for example, Linkerd has a whole page explaining how they work).

I intend to write a couple of article regarding monitoring (alertmanager, for example). Is there anything that you would like more details on? 😊

See you in the next article!

Kubernetes : how to deploy as a beginner and by yourself

Skyler — Tue, 29 Sep 2020 19:48:01 +0000

Kubernetes is one of those things that is very easy if simplified correctly but can be a tad hard once you start looking at the details. Do not worry about details right now, try to get something going.

Let's get to it.

Kubernetes is one of those things that is very easy if simplified correctly but can be a tad hard once you start looking at the details. Do not worry about details right now, try to get something going, and believe me, with time and experience, everything will make sense. :)

A couple of basics "reminders" first. I will not bore you with the definition, I assume you know what it is.
Kubernetes stands on multiple "core" resources. You can add others (as StatefulSets, LimitRange, …) but with those, you will have something working :

Services. It is basically what defines access to pods,
Deployments: you declare the pods and the replicas you want (how many pods you want),
Ingress: manage how to access your services in a cluster,
Ingress Controller: I highly recommend Ingress Nginx Controller,
Namespaces: Environments.

On top of that, you would add DNS names (route53 …) and certificates.

To deploy, you would follow those "general" steps:

Create the cluster,
Create the Ingress controller and load balancer, link the DNS/certificates,
Write the resources needed (deployments, ingress, …), those "needs" to be cut by environments and applications.

In this series, we will be focusing on EKS.

Create the cluster:

You can create a cluster in various ways, with the console, CloudFormation and Terraform. I have tried those 3 different ways. I started with the Cloudformation, then went on manual then Terraform. I am someone that needs to understand how something works, therefore the need to do manually before going to TF.

I followed the guide of AWS https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html and can only recommend it. I do know I struggled at the beginning, here are a few things I remembered:

You need to install everything in the same VPC.
At the time, totally private clusters were not possible. I think they are working on this, but something will always be open. I reckon it was because the instances need to register they are Kubernetes. But that was in v1.10. The situation might have changed.
If something does not work, try the "let me open too many ports, check it works, then restrict gradually".

The best thing I can recommend is to try to do it a couple of times. If you struggle, twitter has a big kubernetes community. Feel free to contact me as well (@SkylerDevops or in the comments here).

Create the Ingress Controller and Load balancer

To be honest, this guide (https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes [you don't need the cert-manager]) is really straight forward and easy to follow.

An Ingress Controller basically gets the request someone places and send it to the correct services/deployment. I would suggest creating the nginx ingress controller, then the load balancer. The LB can take some time to appear in AWS.
Here is the NLB version:

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: #{Certificate}
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "ssl"
spec:
  externalTrafficPolicy: Cluster
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: https
      port: 443
      targetPort: https

Write the resources needed

I will give you the template I am currently using. Feel free to tweak it.
With time, you might look into moving the environment variables to a ConfigMap. Envs are not the "safest" way to do, but they do the job good enough.
You don't need to use the same ports than me. However, you will need to expose those ports in your Dockerfile.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: #{Name}
  namespace: #{Namespace}
spec:
  replicas: 2
  selector:
    matchLabels:
      app: #{Name}
  template:
    metadata:
      labels:
        app: #{Name}
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - #{Name}
      containers:
        - name: #{Name}
          image: #{AWSAccount}.dkr.ecr.#{Region}.amazonaws.com/#{Name}:#{Number}
          env:
            - name: "something"
              value: "something"
          ports:
            - containerPort: 80
              name: #{ContainerPort}


---
apiVersion: v1
kind: Service
metadata:
  name: #{Name}-svc
  namespace: #{Namespace}
spec:
  ports:
    - port: 8080
      targetPort: 80
  selector:
    app: #{Name}
  type: NodePort

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: #{Name}
  namespace: #{Namespace}
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
    - host: #{Host}
      http:
        paths:
          - path: #{Path}(/|$)(.*)
            backend:
              serviceName: #{Name}-svc
              servicePort: 8080

And that's basically it for a beginner-friendly EKS cluster!

I would as well suggest installing the UI dashboard (https://docs.aws.amazon.com/eks/latest/userguide/dashboard-tutorial.html).

Have fun!