DEV Community: Raphael Habereder

My first real frontend app in 13 years, and it's actually just a joke

Raphael Habereder — Sat, 22 Aug 2020 18:11:58 +0000

The problem of having a too big ego

So today I had a discussion with a girl friend of mine.
She watched a Netflix show wherein there was a little "mood webapp".

The boyfriend would go on this website and would instantly see the mood of his girlfriend.

This is the reference-material I had:

My friend told me about it and asked with huge adorable puppy eyes "can you make something like that too?". Looking at the screenshot and miserably failing to keep my big ego in check, I, obviously, said "Of course I can!".

The Problem

As you can see in the screenshot, in the show the girlfriend programmed this app in Rust and Webassembly, but screw that!

My secret

Now, here's the deal. Just between you and me, I might be confident enough to proudly call myself a DevOps, but the Dev part of that never had anything to do with Frontends. I'm more the service-guy/automation dude.
Looking at my portfolio of languages I mastered, there is nothing really web-centric, so I would be starting at 0, regardless of what I could have chosen.

This is what I am comfortable programming in:

Java
Go
C#
A tad of Python (ansible)
A little bit of Ruby (Chef/Puppet)

The realization

I suck at Javascript and CSS. Full-Stop. My knowledge is brutally outdated, I slacked off in the Frontend department.
The last time I had anything to do with JavaScript and CSS was in the beginning of JQuery about 2007 or so.
That's a very long time for anything IT and especially the ever faster evolving JS-World.

Onwards to (hacky) glory

I thought, "to hell with it, I'll try one of these fancy new frameworks" and settled on Vue.js. No idea why, maybe because the name is written funny and I remembered it because of that. I even learned that it's pronounced "view", not "wu", so grant me bonus points for that please. I will need them to balance my hacky stuff later.

The Stuff

So what is it I used?

Vue.js with Vuetify
Express
Tons of dependencies, cause I have no idea what I'm doing and I'm going in the deep end

What is missing:

Some persistent storage (maybe a cassandra or ignite for some real nice overkill)
Docker
Kubernetes
Some kind of easy to use admin-interface for my friend Laura

And yes, I absolutely will shoehorn Docker and Kubernetes in somehow, I actually need to use something I feel comfortable with.
Including this post, this project took me 6 hours, which feels horribly slow to be honest.

Demo Time!

So how does my version look?

If they feel like it, they can also add a Banner-Message that jumps into your face, which is only rendered when set

If this doesn't get your immediate attention, I don't know what will.

What is that abomination??

To get this done as close to the reference material as possible, I needed some kind of ready to use component that included a bar and something to move along that bar. Since I had no idea how to do it myself, I thought "Hey! I could wrestle a slider into submission for this!"
So I used a vuetify v-slider and just styled the hell out of it so it looks how I wanted it.
What you can't really see, is that the picture is actually animated to move indefinitely up and down along the bar. I was told it did that in the show too, so I had to copy it.

Behold the abomination I made of the slider:

>>>.v-slider {
    height: 200px;
    background-image: linear-gradient(to right, rgb(255, 100, 100), rgb(255, 255, 89), lightgreen);
    border-style: solid;
    -webkit-box-shadow: 0 0 40px white;
    box-shadow: 0 0 40px white;
}
>>>.v-slider__thumb {
  height: 300px;
  width: 350px;
  -webkit-animation: mover 2.5s infinite  alternate;
  animation: mover 2.5s infinite  alternate;
  content: url('../assets/laui.png');
  color: transparent;

  display: block;
  margin-left: auto;
  margin-right: auto;
  margin-top: 1px;
}

>>>.v-slider--horizontal .v-slider__track-container {
  display: none;
}

@-webkit-keyframes mover {
    0% { transform: translateY(0); }
    100% { transform: translateY(-190px); }
}
@keyframes mover {
    0% { transform: translateY(0); }
    100% { transform: translateY(-190px); }
}

Since I don't really know what I am doing, this might be very very bad. But it works, so I am fine with it for the moment.
Another side-effect is, that the actual slider bubble thing is enormous with 300x350px. This somehow results in the picture going way off the bar at the min-value 0 and max-value of 100. I need to tweak that a little more. So 10-90 as values have to suffice until I find out why the bloody thing does that.

As of now, the whole "page" consists of one Vue-Component, a Lauimeter, which was named so by my friend.

The component gets all it's data, including labels and texts, from an express-service with the following endpoints:

get("/") to just get everything stored
post("/moodUpdate") to just update the mood value
post("/bannerUpdate") to update the bannermessage
post('/config') to configure the service for testing

To configure the app, a simple curl is enough:

curl localhost:3000/config \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{ "mood": 90, "minMessage": "Not today", "maxMessage":"Fantastic!", "updateText":"Update", "meterName":"Moodmeter", "bannerMessage": "" }'

Now I have to figure out how to get a "easy to use config-page" in there, so my friend can actually use the bloody thing without having to learn cURL. And a persistent storage would be nice.

Maybe this stupid little fun project will find it's way to github too one day. But that is something for another day. As the first javascript experiment in 13 years, I am absolutely okay with how hacky this turned out.

It was actually a lot of fun, I can see myself doing more of this. But hopefully better in the future.

Feel free to leave feedback. All this is very new to me, so any criticism and/or guidance is most welcome. Maybe old dogs can learn new tricks.

When and why do you use go?

Raphael Habereder — Wed, 05 Aug 2020 07:31:12 +0000

I really like go, as a replacement to python for system automation and admin tasks.

In the past I tried building client-software using fyne-io/fyne, which was fun, but it was not as feature-rich as I hoped.

I haven't gone beyond the simple rest-service, or automation for quite a while. That's where I always look back at python.

What is your opinion on go and it's use-cases?

A rant about discussion culture on the internet

Raphael Habereder — Sat, 01 Aug 2020 10:51:25 +0000

Today was the first time I found a comment of mine flagged as "not constructive by the community".
This irked me a little bit, since I see myself as rather inclusive, yet strong-minded and fair.

Don't worry, this post will go a little bit deeper into the topic, just bear with me for a second.

Now I'm quite sure a user has no way to "negatively impact" someone else on here, except for submitting a hard report.

So here is the question, where does this come from?
It says it right there on the tin, "by the community", even if there is no voting system on comments.

So it must have been a report, right? Let's follow this chain a little deeper.

If you want to submit a report, you'll get a handful of categories to describe the issue:

Rude or vulgar
Harassment or hate speech
Spam or copyright issue
Inappropriate listings message/category
Other

This seems like a solid selection, except for "rude or vulgar" and the obvious "other".
Rude is an attribute that is solely subjective to the receiver.
Speaking from experience, being a typical franconian german, being direct without any sugarcoating is more often than not perceived by people as being rude.

Personally, I think it's quite flattering if someone tells me straight how it is. If you don't like something, say it. Beating around the bush gives leeway to misinterpretation. Misinterpretation again escalates situations even further.

So this, in my humble opinion, is a problem.

The Other category seems to be just another catchall for We are not sure where it fits, but want to reserve judgement, which is perfectly fine.

That out of the way, let's get into the topic of the title.

I've been on DEV for a bit over 3 years now, and most of the time I've been quiet.
I read a lot of controversial topics on here, especially when the US launches yet another political outrage party. Let's be honest, something that happens in the US is found here or on other social media platforms shortly after. The US is just that much of a powerhouse in global communication.

What I have noticed is, the discussions here, and the rest of the internet, are becoming more and more of an echo-chamber, rather than actual discussions.
The level on DEV is still far above other platforms, let's be real for a second, but the discussion culture of DEV is slowly getting there.

I read the code of conduct, and with most parts I agree. It's solidly written, even though I am quite sure back in 2017 it was more compact than it is today.
The trouble I have is, I can't find a way to "engage someone" in a debate with what the COC deems as "safe".

Two points strike me as particularly deadly for discussion culture:

Examples of unacceptable behavior by participants include:

Dismissing or attacking inclusion-oriented requests

A discussion is just that. You discuss points and if you argue against a differing view point, it is, by design, an attack on someones opinion. This, in no way, means you should actually insult someone. But an objective, solid argument, or let's go with the times and pardon my language, you call someones bullshit in question, that is an attack.

I want to be clear, I am absolutely in favor of including people that have been excluded for way too long. But not everything that is marketed as "inclusive" is done for the excluded. PR and virtue-signalling are a thing, they have been for a long time, and they work.

The other point that strikes me as terribad for discussions is this one:

We pledge to prioritize marginalized people’s safety over privileged people’s comfort. We will not act on complaints regarding:

Someone’s refusal to explain or debate social justice concepts

This is a prime example of echo-chambers. You say something that is currently controversial, either because you beliefve in it, or because you want to spark a flame-war (lets be honest, who doesn't love a good flamewar?)
Someone engages your argument, you submit a report and classify it as harassment because you "refuse to explain or debate social justice concepts".

And the COC supports that. Why?
If you don't want to discuss something, don't post it on the internet. There is always someone that disagrees, the typical "am I the only one?" question is almost always answered by a simple "thats a 1 in seven dot eight billion chance, so probably not".

It seems to me, that a lot of people jump on bandwagons here, freely repeating along half-truths they read a few posts back, further strengthening the echo. Some actually debate these things because they believe in them, which is exactly what should happen.

Others on the other hand, just repeat controversial things to farm some heart-emojis.
While this is one of the more typical behaviors of a social-platform with any kind of rating-mechanism, what DEV and many other platforms are missing is the counter weight.
A solid mechanism to downvote things and equalize echos.

Of course, you may say
this would lead to brigading, vote-flooding, or vote-bombing, which should be held in check!
I agree, but you don't achieve this by nuking everything you don't agree with, that itself is going against the very thing you want to protect:

fostering an open and welcoming environment

Let's go down the rabbit hole further. By stripping out the feature to express your disagreement safely without being punished, you get the exact same rapid momentum in the opposite direction.
There can be no equilibrium, if a rating/voting mechanism goes only one way.

But you can write comments to express your differing view point!, is what you might think now. Absolutely, I agree, but that loops us back to the beginning, being "marked as not constructive by the community".

What I learned is, that "inclusion" is a paradox.
That if you want to be inclusive, and heavily enforce this, you almost always exclude a huge chunk of people. Thereby inclusion is inherently exclusive.

So where does this post lead us? I don't know, it's just the rant of a dude that doesn't understand the discussion culture of the internet. A guy that is direct, and not afraid to step on your foot if you spout half-truths. A guy that gets marked as not-constructive and follows up with writing a gigantic tirade :D

Do I have a solution? No, not really. I would be rich if I had one on hand. All I want is a "dislike" button, but that's a thing I won't get in today's internet. I'll just leave comments and hope I'll get more cool discussions out of it :)

Feel free to chime in, and please, don't just echo me, hit me with the good stuff. I actually want discussions to embiggen the "apparently not yet big picture" I see.

Rock on and have a great weekend, I hope your weather is just as lovely as in my part of this beautiful rock!

Continuous Test-Reporting with Allure

Raphael Habereder — Fri, 26 Jun 2020 19:35:57 +0000

The disconnect between Tech and Client

I had a talk with one of my clients which went like this:

Tekton and Kubernetes are nice and all, our devs seem to be quite happy, but where do I see Test results? Jenkins has a feature like this, but now all developers use this tekton stuff and all these fancy console commands.
I just want to look at my reports and get a glimpse of how the softwaretests went. Those terminals don't really do that for me.
Can you get me something like that?

That's a spot-on observation by my client. A lot of Kubernetes tools are heavily focused on CLI-Usage, or in the "early stages of development", when features are higher up on the priority-list than shiny UIs.

One of these examples would be the tekton ecosystem. There are rarely any UIs, and if there are, like the Tekton Dashboard for example, they are very barebones.

Since I am a very CLI-Centric guy and by habit don't really think about using UIs that often, the request of my client ran me over like a truck.

So we dug around a bit and found a very promising tool.
The Allure Framework.

What is allure and why should I use it?

The allure framework does a few things:

Generating Results in json-Format (thank god no more xml)
Report creation
History-Data Collection

All of this is then slapped together in a pretty static-html report, ready to be served on your favorite server.
If you don't have one on hand, the optionally downloadable cli can also spin up a Webserver via the allure serve pathToReportDir command and host the reports itself. Magical!

To give you a little tour of how a report looks, have a gif!

I tried to get a gif to work, but 80% of the time I get errors from dev, and the ones that render turn into potato-quality.
No idea what this cloudinary thing does, but it's not great..

So if this is too small/ugly for you, which I can probably only blame on myself, don't worry! I'll show you how to spin up a demo yourself :)

How does it work?

First off, Allure has tons of connectors for different frameworks and languages, which are documented here.

But let me warn you: The documentation seems to be very minimal right now. There is a lot of trial and error involved to get what you want
To be fair, it seems to be a very powerful tool with many supported languages and Frameworks. Documenting everything to the smallest detail would probably be an enormous task.

My client was especially interested in the Cucumber and JUnit5 support. As of now we roughly finished implementing a continous integration pipeline with Report generation for JUnit5 Testsuites. Cucumber is probably following as soon as I find the time.

Let's test stuff!

To get allure to generate a report for you, all you need is the following additions to your pom.xml:



<properties>
  <surefire-plugin.version>2.22.2</surefire-plugin.version>
  <allure.version>2.13.3</allure.version>
  <aspectj.version>1.9.5</aspectj.version>
</properties>

<dependency>
  <groupId>io.qameta.allure</groupId>
  <artifactId>allure-junit5</artifactId>
  <version>${allure.version}</version>
  <scope>test</scope>
</dependency>

<build>
  <plugins>
    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>${surefire-plugin.version}</version>
      <configuration>
        <testFailureIgnore>true</testFailureIgnore>
        <argLine>
          -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
        </argLine>
        <systemPropertyVariables>
          <allure.results.directory>${project.build.directory}/allure-results</allure.results.directory
          <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
        </systemPropertyVariables>
      </configuration>
      <dependencies>
        <dependency>
          <groupId>org.aspectj</groupId>
          <artifactId>aspectjweaver</artifactId>
          <version>${aspectj.version}</version>
        </dependency>
      </dependencies>
    </plugin>
    <plugin>
      <groupId>io.qameta.allure</groupId>
      <artifactId>allure-maven</artifactId>
      <version>2.10.0</version>
      <executions>
        <execution>
          <phase>package</phase>
          <goals>
            <goal>report</goal>
          </goals>
          <configuration>
            <reportVersion>${allure.version}</reportVersion>
            <allureDownloadUrl>https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/${allure.version}/allure-commandline-${allure.version}.zip</allureDownloadUrl>
          </configuration>    
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Allure also brings its own set of annotations for Testcases.
Here are a few I like to use:



@Feature("Some Name")
@Epic("Some Name")
@Story("Some Name")
@Severity(SeverityLevel.CRITICAL)
@Description("non-dev readable description")
@Link("link to JIRA-Tickets or some cool resources)

Want to give it a go?
Here is a simple Dockerfile I slapped together, that needs minimal work from your side.
Just create a simple maven project, configure your pom.xml like I've shown above, add some tests, annotate them and give it a run for it's money with a mvn package!



FROM maven:3.6.3-jdk-11 as builder

WORKDIR /opt/service

# This is for caching purposes in local builds
# That way you don't have to download the whole internet every time your source changes
COPY pom.xml . 
RUN mvn dependency:go-offline

COPY src/ src/

# Run the Build and generate a report
RUN mvn clean install && mvn allure:report


FROM nginx:1.19.0-alpine as server

# Copy the report and serve it via nginx
COPY --from=builder /opt/service/target/site/allure-maven-plugin/ /usr/share/nginx/html/

The challenge of CIs

Now we have a nice tool, but what now?
I quickly noticed that those reports may be nice, but if I am going to deploy these reports in a container, I need a little bit more than what I showed you earlier.
Allures history-feature can't do anything this way. Allure can't create a history if there is no data of previous runs to be found.

So let's create a solution for this!

I thought I had a nifty idea by designing the report generation process like this:

As you can see, I introduced a new component. MinIO as S3 Storage Provider. Why minIO? Because it's commandline client has a great --newer-than/--older-than Feature. So minio could do the sorting work for me, I don't have to do that myself!

If I wanted to create a report with Data up to 7 Days old, it would be as easy as



mc cp --recursive --newer-than 7d minio/allure/results/ results/
allure generate results/ -o report/
mc cp --recursive report/ minio/allure/reports-history/$(date +%y-%m-%d-%H%M)/

And I'd get a report with Data for 7 Days. It would get stored in minio and would be ready to be put into a container.
Honestly, I loved that idea and was kind of proud of that solution. If you have a better idea, I am all ears!

Let's containerize it!

So how would this look inside a container?



FROM maven:3.6.3-jdk-11 as builder

WORKDIR /opt/service

# This is for caching purposes in local builds
# That way you don't have to download the whole internet every time your source changes
COPY pom.xml . 
RUN mvn dependency:go-offline

COPY src/ src/
RUN mvn clean install && mvn allure:report

FROM openjdk:14-alpine as generator

WORKDIR /tmp
COPY --from=builder /opt/service/target/allure-results/* results/
COPY --from=builder /opt/service/target/site/allure-maven-plugin/* reports/

RUN apk add curl tar gzip \
    && curl -L https://bintray.com/qameta/maven/download_file?file_path=io%2Fqameta%2Fallure%2Fallure-commandline%2F2.13.4%2Fallure-commandline-2.13.4.tgz | tar xvz \
    && curl -LO https://dl.min.io/client/mc/release/linux-amd64/mc \
    && chmod +x mc \
    && ln -s $PWD/mc /usr/bin/mc \
    && mc config host add minio http://ipofminio:9000 minioadmin minioadmin \
    && mc mb -p minio/allure \
    && mc cp --recursive results/* minio/allure/results/ \
    && mc cp --recursive reports/* minio/allure/reports/$(date +'%Y-%m-%d-%H%M')/ \
    && mc cp --recursive minio/allure/results/ results/  \
    && allure-2.13.4/bin/allure generate results/ -o reports-with-history-data/ \
    && mc cp --recursive reports-with-history-data/ minio/allure/reports-history/$(date +'%Y-%m-%d-%H%M')/

FROM nginx:1.19.0-alpine as server

COPY --from=generator /tmp/reports-with-history-data/ /usr/share/nginx/html/

What we did is, we introduced another stage with FROM openjdk:14-alpine as generator
in which we do all the upload/fetching/sorting stuff for our history-reports.
All you would need to build this is a minio instance, which is, for local testing, easily deployed via:



docker run -p 9000:9000 --name minio --rm minio/minio server /data

But what about tekton and kubernetes?

You may ask this, rightfully, since I teased it in the beginning.
Making this run in tekton was pretty easy, even though it needed a little bit more prep-work.

First off would be to create an Image, that would be buildable outside of Kubernetes, and still has all the connections to minio we need.
The Dockerfile up there would probably crash the build, since a minio running in kubernetes wouldn't be reachable by your CI, which normally is running outside of kubernetes. So let's modify our Container a little bit to work with those requirements.

What do we have to do?
First off, I want to split up my container into two.

a reporting container that builds the reports and pushes them to minio
a nginx container that pulls the report and serves it

Why do I want this?
Having to build an image that contains your report every time the report changes, doesn't seem viable to me. It takes unnecessary space in your docker registry and doesn't bring that much benefit. IMO it would be better to have a dynamic container that can be parameterized and does the whole result-pulling, generating and archiving in it's entrypoint. So let's do just that!

Allure-Reporter Container

Dockerfile.allure



FROM openjdk:14-alpine

WORKDIR /tmp

RUN apk add curl tar gzip \
    && curl -L https://bintray.com/qameta/maven/download_file?file_path=io%2Fqameta%2Fallure%2Fallure-commandline%2F2.13.4%2Fallure-commandline-2.13.4.tgz | tar xvz \
    && curl -LO https://dl.min.io/client/mc/release/linux-amd64/mc \
    && chmod +x mc \
    && ln -s $PWD/mc /usr/bin/mc \
    && ln -s $PWD/allure-2.13.4/bin/allure /usr/bin/allure \
    && allure --version \
    && mc --version

WORKDIR /opt/myproject
VOLUME /opt/myproject/results

COPY entrypoint-allure.sh /usr/local/bin/entrypoint
RUN chmod +x /usr/local/bin/entrypoint 

USER nobody
ENTRYPOINT bash -c entrypoint

entrypoint-allure.sh



#!/bin/bash

set -e

# Capture the name of the project, so we can differentiate results in minio
CURRENT_DATE=$(date '+%Y%m%d%H%M') 

# Gather all results you can find
# Should be Emulated via Volume Mount locally
mkdir results
for resultDir in $(find . -type d -iname allure-results);do
  cp $resultDir/* results
done
# Generate the actual allure report of this testrun and publish it
allure generate results/ -o allure-report/

# Configure Minio and make sure the bucket allure exists
mc config host add minio $MINIO_ENDPOINT $MINIO_ACCESS_KEY $MINIO_SECRET_KEY 
mc mb -p minio/allure 

# Publish the results
mc cp --recursive results/ minio/allure/$PROJECTNAME/allure-results/
# Publish the report
mc cp --recursive allure-report/ minio/allure/$PROJECTNAME/allure-report-independent/$CURRENT_DATE/

# Get all results, including those from past runs, and generate a history report to publish
mc cp --recursive minio/allure/$PROJECTNAME/allure-results/ results/ 
allure generate results -o allure-report-history/ 
mc cp --recursive allure-report-history/ minio/allure/$PROJECTNAME/allure-report-history/$CURRENT_DATE/

Allure-Server Container

Dockerfile.nginx:



FROM nginx:1.19.0-alpine as server

ARG MINIO_ENDPOINT=http://minio.default.svc.cluster.local:9000
ARG MINIO_SECRET_KEY=minioadmin
ARG MINIO_SECRET_PASS=minioadmin
ARG PROJECTNAME=myproject

RUN curl -LO https://dl.min.io/client/mc/release/linux-amd64/mc \
    && chmod +x mc \
    && ln -s $PWD/mc /usr/bin/mc \
    && ln -s $PWD/allure-2.13.4/bin/allure /usr/bin/allure \
    && mc --version

COPY entrypoint-nginx.sh /usr/local/bin/entrypoint
RUN chmod +x /usr/local/bin/entrypoint 

USER nobody
ENTRYPOINT bash -c entrypoint

entrypoint-nginx.sh:



#!/bin/bash

set -e

# Capture the name of the project, so we can differentiate results in minio
CURRENT_DATE=$(date '+%Y%m%d%H%M') 

# Configure Minio and make sure the bucket allure exists
mc config host add minio $MINIO_ENDPOINT $MINIO_ACCESS_KEY $MINIO_SECRET_KEY
mc mb -p minio/allure 

NEWEST_REPORT=$(mc find minio/allure/$PROJECTNAME/allure-report-history/ --maxdepth 1 --newer-than 1d | cut -d'/' -f5- | sort -r | head -n1)
NEWEST_REPORT=${NEWEST_REPORT%/}
mc cp --recursive minio/allure/$PROJECTNAME/allure-report-history/$NEWEST_REPORT report
rm /opt/nginx/html/index.html
cp -r report/$NEWEST_REPORT/* /opt/nginx/html

nginx

With these two containers and four files, you should be pretty much set to go.
We split our all-in-one-container into two separate containers, that are buildable without a connection to minio.
Our reporter and it's nginx-server.

Let's go k8s

I am assuming, you have a running kubernetes setup, if not, you can take a look at my "lazy k8s solution" to running kubernetes, which can take care of that for you.

As final part of the tekton integration, you can use this task to run the report-generator (The first Container up there) and deploy the nginx-server via the file that is referenced in the kubectl step.
allure-reporter-task.yaml:



apiVersion: tekton.dev/v1beta1
kind: Task
metadata: 
  name: allure-reporter-task
  namespace: tekton
spec:
  workspaces: 
    - name: git
  params: 
   - name: minioEndpoint
     default: http://minio.default.svc.cluster.local:9000
   - name: minioAccessKey
     default: minioadmin
   - name: minioSecretKey
     default: minioadmin
   - name: pathToDeploymentYaml
     default: k8s/allure-server.yaml
  steps:
    - name: generate-and-publish-allure-report
      image: allure-report-generator:1.0.0
      command: 
        - /bin/bash
      args:
        - -c
        - |
          cd /workspace/git/
          entrypoint
      env:
        - name: MINIO_ENDPOINT
          value: $(params.minioEndpoint)
        - name: MINIO_ACCESS_KEY
          value: $(params.minioAccessKey)
        - name: MINIO_SECRET_KEY
          value: $(params.minioSecretKey)
    - name: serve-allure-report
      image: lachlanevenson/k8s-kubectl
      command: ["kubectl"]
      args:
        - "apply"
        - "-f"
        - "$(workspaces.git.path)/$(params.pathToDeploymentYaml)"

To get your allure-server deployed, the following file should be in your apps git-repository, so tekton can find it during a taskrun

Example of k8s/allure-server.yaml:



---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: allure-server
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: allure-server
  template:
    metadata:
      labels:
        app: allure-server
    spec:
      containers:
        - name: allure-server
          image: your-registry/allure-server:1.0.0
          imagePullPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: allure-server
  namespace: default
spec:
  selector:
    app: allure-server
  type: ClusterIP 
  ports:
    - port: 80
      targetPort: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: allure-server-ingress-route
  namespace: default
spec:
  entryPoints:
    - http
  routes:
  - match: Host(`localhost`) && PathPrefix(`/allure`)
    kind: Rule
    services:
    - name: allure-server
      port: 80
    middlewares:
    - name: allure-stripprefix
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: allure-stripprefix
  namespace: default
spec:
  stripPrefix:
    prefixes:
      - /allure

The difference in this approach, in comparison to the Dockerfile I demonstrated up there, I put all the logic of the Dockerfile into entrypoint scripts.
You know, all the copying, report generation, sorting, everything.

That way I don't have to push the same image over and over into my registry, just because the report was updated. I can just have one reporter image that dynamically pulls the data it needs from storage.

Also, the images you need can be built by a CI, which normally shouldn't have any access to storage solutions like minio. So a build of the Dockerfiles with a minio-connection would break your build. That's exactly why we put all the networking stuff into the entrypoints, as they will not be executed by your CI.

The future

I am already working on a practical demo of allure and tekton with zero-to-k8s. So you don't have to do all the copying and pasting from here.

End of story

But for now, I leave you with this.
A random assortment of stuff, just to get some nice shiny UI for your testresults :D

If you have questions, suggestions, or other feedback, feel free to hammer away at your keyboard :)

How to set up Gitlab to trigger Jenkins on push

Raphael Habereder — Tue, 16 Jun 2020 09:41:27 +0000

Last time we built a simple Jenkins CI Pipeline with polling, so our images are always up to date when we push changes to our codebase.

I don't like using polling for this, and you shouldn't either. The Polling approach is either unreliable if the timing is too slow and incluces too many commits in a single image, or produces unnecessary load if you let it poll every second.

Imagine your Boss going "hey, is the feature done? How about now? It's been 5 seconds, anything new?".
A) it's annoying and B) you won't get anything done that way, right?

Rather you would prefer to inform your Boss "hey, the feature is done, take a look!" and go on to do something new, right?

So let's implement this and notify our CI-Server whenever a commit is pushed.
Don't feel left out if you don't use Gitlab or Jenkins in your projects, most git/ci toolsets offer a feature of this kind.

Setup

This is our to-do-list:

Make DNS-Names for container-communication work, because looking up IPs of containers is annoying
Boot a Jenkins Instance and create a Job we can trigger via Webhooks
Boot a Gitlab Instance and create a repo with a demo Dockerfile and Webhook

So let's get crackin!

DNS-Resolution

I could go on a tangent and tell you the grand story, but in short, the default bridge network does no dns resolution because of reasons. If you want dns-resolution for containers by their name, you need to create a custom bridge network. Which thankfully, is very easy to do!



$ docker network create dns-bridge
$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
9508cfea746d        bridge              bridge              local
67aa52615190        dns-bridge          bridge              local
ae9791d28429        host                host                local
d2c62d84bc06        none                null                local

That's it, now you have a user-defined bridge that automatically does dns resolution. Let's test it:



docker run -d --name busybox1 --network dns-bridge busybox:1.28 sleep 3600
docker run -d --name busybox2 --network dns-bridge busybox:1.28 sleep 3600

$ docker exec -ti busybox1 ping busybox2
PING busybox2 (172.20.1.2): 56 data bytes
64 bytes from 172.20.1.2: seq=0 ttl=64 time=0.071 ms
64 bytes from 172.20.1.2: seq=1 ttl=64 time=0.057 ms

And the reverse:
$ docker exec -ti busybox1 ping busybox2
PING busybox2 (172.20.1.2): 56 data bytes
64 bytes from 172.20.1.2: seq=0 ttl=64 time=0.071 ms
64 bytes from 172.20.1.2: seq=1 ttl=64 time=0.057 ms

That's looking pretty good!

If you want to connect/disconnect an already running container to a network:



$ docker run -d --name busybox-ext busybox:1.28 sleep 3600
2efe297352a6ff1e9876a46c853b786a415480235d05df9451672689d580ad6e

$ docker network connect dns-bridge busybox-ext
$ docker network inspect dns-bridge -f "{{range .Containers}}{{println .Name}}{{end}}"
busybox-ext
busybox2
busybox1
$ docker network disconnect dns-bridge busybox-ext
$ docker network inspect dns-bridge -f "{{range .Containers}}{{println .Name}}{{end}}"
busybox2
busybox1

Alright, let's go on to setup our Jenkins instance

Jenkins

For this post I built a custom Docker-In-Docker Jenkins based on Alpine, which I will tell you, was a horrible experience.
I'm not happy about this dockerfile, so don't crucify me please.
While I could put it up on github or something, I don't want you to clone a random repo and just run it without having at least taken a look at it. So I am going to be the one bad teacher nobody likes and let you copy the two files you need from here :)



FROM alpine

USER root
RUN apk add --no-cache \
  bash \
  coreutils \
  curl \
  git \
  git-lfs \
  openssh-client \
  tini \
  ttf-dejavu \
  tzdata \
  unzip \
  openjdk11-jdk \
  shadow \ 
  docker 

# Install Gosu
ENV GOSU_VERSION 1.12
RUN set -eux; \
    \
    apk add --no-cache --virtual .gosu-deps \
        ca-certificates \
        dpkg \
        gnupg \
    ; \
    \
    dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
    wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch"; \
    wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc"; \
    \
# verify the signature
    export GNUPGHOME="$(mktemp -d)"; \
    gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
    gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
    command -v gpgconf && gpgconf --kill all || :; \
    rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
    \
# clean up fetch dependencies
    apk del --no-network .gosu-deps; \
    \
    chmod +x /usr/local/bin/gosu; \
# verify that the binary works
    gosu --version; \
    gosu nobody true

ARG user=jenkins
ARG group=jenkins
ARG uid=1001
ARG gid=1001
ARG http_port=8080
ARG agent_port=50000
ARG JENKINS_HOME=/var/jenkins_home
ARG REF=/usr/share/jenkins/ref

ENV JENKINS_HOME $JENKINS_HOME
ENV JENKINS_SLAVE_AGENT_PORT ${agent_port}
ENV REF $REF

# Jenkins is run with user `jenkins`, uid = 1000
# If you bind mount a volume from the host or a data container,
# ensure you use the same uid
RUN mkdir -p $JENKINS_HOME \
  && chown ${uid}:${gid} $JENKINS_HOME \
  && addgroup -g ${gid} ${group} \
  && adduser -h "$JENKINS_HOME" -u ${uid} -G ${group} -s /bin/bash -D ${user} 

# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME $JENKINS_HOME

# $REF (defaults to `/usr/share/jenkins/ref/`) contains all reference configuration we want
# to set on a fresh new installation. Use it to bundle additional plugins
# or config file with your custom jenkins Docker image.
RUN mkdir -p ${REF}/init.groovy.d

# jenkins version being bundled in this docker image
ARG JENKINS_VERSION
ENV JENKINS_VERSION ${JENKINS_VERSION:-2.222.4}

# jenkins.war checksum, download will be validated using it
ARG JENKINS_SHA=6c95721b90272949ed8802cab8a84d7429306f72b180c5babc33f5b073e1c47c

# Can be used to customize where jenkins.war gets downloaded from
ARG JENKINS_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war

# Could use ADD but this one does not check Last-Modified header neither does it allow to control checksum
# See https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -o /usr/share/jenkins/jenkins.war \
  && echo "${JENKINS_SHA}  /usr/share/jenkins/jenkins.war" | sha256sum -c -

ENV JENKINS_UC https://updates.jenkins.io
ENV JENKINS_UC_EXPERIMENTAL=https://updates.jenkins.io/experimental
ENV JENKINS_INCREMENTALS_REPO_MIRROR=https://repo.jenkins-ci.org/incrementals
RUN chown -R ${user} "$JENKINS_HOME" "$REF"

# For main web interface:
EXPOSE ${http_port}

# Will be used by attached agents:
EXPOSE ${agent_port}

ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log

# Download and place scripts needed to run
RUN curl https://raw.githubusercontent.com/jenkinsci/docker/master/jenkins-support -o /usr/local/bin/jenkins-support && \
    curl https://raw.githubusercontent.com/jenkinsci/docker/master/jenkins.sh -o /usr/local/bin/jenkins.sh && \
    curl https://raw.githubusercontent.com/jenkinsci/docker/master/tini-shim.sh -o /bin/tini && \
    curl https://raw.githubusercontent.com/jenkinsci/docker/master/plugins.sh -o /usr/local/bin/plugins.sh && \
    curl https://raw.githubusercontent.com/jenkinsci/docker/master/install-plugins.sh -o /usr/local/bin/install-plugins.sh

COPY --chown=${user} entrypoint.sh /entrypoint.sh

RUN chmod +x /usr/local/bin/install-plugins.sh /usr/local/bin/plugins.sh /usr/local/bin/jenkins.sh /bin/tini /usr/local/bin/jenkins-support
RUN chmod +x /entrypoint.sh

# Stay root, the entrypoint drops down to User jenkins via gosu
ENTRYPOINT ["/entrypoint.sh"]

entrypoint.sh



#!/bin/sh

# By: Brandon Mitchell <public@bmitch.net>
# License: MIT
# Source Repo: https://github.com/sudo-bmitch/jenkins-docker

set -x

# configure script to call original entrypoint
set -- tini -- /usr/local/bin/jenkins.sh "$@"

# In Prod, this may be configured with a GID already matching the container
# allowing the container to be run directly as Jenkins. In Dev, or on unknown
# environments, run the container as root to automatically correct docker
# group in container to match the docker.sock GID mounted from the host.
if [ "$(id -u)" = "0" ]; then
  # get gid of docker socket file
  SOCK_DOCKER_GID=`ls -ng /var/run/docker.sock | cut -f3 -d' '`

  # get group of docker inside container
  CUR_DOCKER_GID=`getent group docker | cut -f3 -d: || true`

  # if they don't match, adjust
  if [ ! -z "$SOCK_DOCKER_GID" -a "$SOCK_DOCKER_GID" != "$CUR_DOCKER_GID" ]; then
    groupmod -g ${SOCK_DOCKER_GID} -o docker
  fi
  if ! groups jenkins | grep -q docker; then
    usermod -aG docker jenkins
  fi

  #If you run on MacOS
  if ! groups jenkins | grep -q staff; then
    usermod -aG staff jenkins
  fi
  # Add call to gosu to drop from root user to jenkins user
  # when running original entrypoint
  set -- gosu jenkins "$@"
fi

# replace the current pid 1 with original entrypoint
exec "$@"

Build and run our Jenkins image:



# We need to create a directory for Jenkins to save his data to
# Since to container runs with UID:GID 1001:1001, the folder also needs to get correct permissions 
mkdir $HOME/jenkins && chown 1001:1001 $HOME/jenkins

docker build -t myjenkins .
docker run -d \
           -v $HOME/jenkins:/var/jenkins_home \ 
           -v /var/run/docker.sock:/var/run/docker.sock \
           -p 8080:8080 \
           --name jenkins \
           --network dns-bridge \
           --restart unless-stopped \
           myjenkins

What does this do?

The mount to $HOME/jenkins makes sure you don't have to reconfigure jenkins every time you kill/stop the jenkins container. It's just convenience.
We mount the docker.sock into Jenkins, so Jenkins can build our docker images without having it's own functioning docker-runtime. This concept is called docker in docker.
The rest should be self-explanatory, we publish jenkins port so we can access it from the host, attach it to the network bridge we created and make sure the container restart on failure, unless we stop it ourselves

Gitlab



#Where you want to store data
export GITLAB_HOME=$HOME/gitlab

docker run -d \
  -p 443:443 -p 80:80 -p 22:22 \
  --name gitlab \
  --network dns-bridge \
  --restart unless-stopped \
  -v $GITLAB_HOME/config:/etc/gitlab \
  -v $GITLAB_HOME/logs:/var/log/gitlab \
  -v $GITLAB_HOME/data:/var/opt/gitlab \
  gitlab/gitlab-ce:latest

And yes, I know, this breaks the rule of "build it yourself", but I can't for the life of me get gitlab running on alpine yet, so for this demonstration I beg you to bear with me this one time.
Of course, I will update this post accordingly, once I got gitlab to work smoothly on alpine :)

If you don't want the containers to play around in your filesystem, just remove the mounts.
Note: Again, you'll have to set everything up again, if you restart the containers without their data persisted somewhere

The Gitlab omnibus image takes a while to boot, you can monitor it via docker logs -f gitlab to see when it's ready to use.
After it's booted up, you can open the Gitlab UI in your favorite browser and set a password for the user root.
Then you are ready to create some repos :)

More info on the Gitlab Docker Images can be found here

Demo Repository

Now that our shiny new Gitlab is running, let's create a demo repository here and commit the following demo Dockerfile to it:



FROM alpine:latest

RUN echo '#!/bin/sh' > /tmp/hello.sh && \
    echo 'echo "Hello $1"' >> /tmp/hello.sh && \
    chmod +x /tmp/hello.sh && \ 
    chown nobody. /tmp/hello.sh 

USER nobody
ENTRYPOINT ["/bin/sh"]
CMD ["/tmp/hello.sh", "Me!"]

This is a very simple container that just echos a greeting, which will suffice to demonstrate our Git-Hook.

Add, commit and push the file for now.

Demo-Job

Now is the time to go back to our Jenkins demo-job, we wanted to set up earlier.

Now that we have source to build, we can use it in our job.
Create a pipeline with your favorite name here with the following content:



node {
    stage("Git Checkout") {
        git credentialsId: 'gitlab', url: 'http://gitlab/root/demo'
    }
    stage("Docker Build") {
        app = docker.build('awesome-image')
    }
    stage("Docker Push") {
        docker.withRegistry("http://registry.local:5000") {
            app.push("awesome-tag-${env.BUILD_NUMBER}")
            app.push("latest")
        }
    }
}

As you probably saw, we are missing two things:

credentials for gitlab
the registry "registry.local:5000"

Jenkins Gitlab Credentials

Go here and add the credentials you set for gitlab earlier:

username and password are obvious. ID is the "shortname" you will use in your pipelines. In the example it is "gitlab".
Description is just that, a human readable description for easy distinction of the various credentials you can have.

As for the rest of the configuration, like the plugins and docker runtime, you can take a look at the previous post as a guideline.

Registry

To fire up and connect our registry to our dns-bridge:



docker run -d -p 5000:5000 --name registry.local --network dns-bridge registry:2

Since we are running docker-in-docker, the push works from our host. So our host has to know the dns-name registry.local too.
Make sure it is in our hosts with



cat /etc/hosts

If it isn't there, just add it via



echo '127.0.0.1 registry.local' | sudo tee -a /etc/hosts

That should fix it. Now you can trigger the build and enjoy the show!

It should be a success, and you can docker pull it anytime from our registry :)

Alright, on to the home-stretch! We are done with all the prep-work, so let's get to what we actually wanted to achieve.

Set up the Endpoint for Jenkins

To enable Jenkins-Jobs to be triggered via HTTP-Request, you need to do a few things.

First, check the following box in your job and define a secret token.

Now we need an API-Token to authenticate with Jenkins in HTTP-Requests, without having to use the actual password of our admin-user.
You can create a token here
Save it somewhere you can copy it from later, because that little bugger is gone, once you navigate away from the current page.

As a nice benefit, you can monitor later on how many times the token has been used. Neat!

Set up the Webhook in Gitlab

This is it, the end of the road, we are near the finish line!

Gitlab is smart, they don't want to permit Webhooks to spam the local network of a gitlab instance to prevent damage. But we want to allow it to do so with Jenkins, because otherwise that would render this whole mechanic moot for us.
So go to the admin panel and in the "Outbound Requests" section, add "jenkins" to the "Whitelist to allow requests to the local network from hooks and services".
Don't check the box that allows all adresses in the local network! Just add jenkins to the whitelist

Now, to create a webhook, open your project and on the left hand side go to Settings -> Webhook and configure your webhook like this:

For easy copy and paste, take this template and fill in your values:
http://:@:<8080>/job//build?token=

Final Test

Now our interconnectivity between gitlab and jenkins is automated for this job and repository. So let's test it!

Modify your Dockerfile, commit and push it and a few seconds later you should have a docker image in your local registry that reflects your change!

Outro

I have to admit, this was a little bit of work to get running locally, especially the Jenkins-DIND Image for MacOS..
But I do think this made you and me a little bit smarter, so I hope this helps you on your journey to transition to containers.

If not, shoot me a message, leave a comment or ask questions.
If there is one thing I have, it's time, patience and a talent for "making things work" or breaking them :D

Next time I'll probably write about how we can scan our images for CVEs, that draft is nearly done :)

I work with Containers, Automation and Cloud Stacks for up to 10 years now, ask me anything

Raphael Habereder — Mon, 15 Jun 2020 13:30:30 +0000

While I continue to work on about 5 posts in parallel, I thought this would be fun to do.

While it normally only works with celebrities, in the worst case noone answers and I can shamefully delete this post. Then I'd have to wait for the day I become famous for crashing some thought-to-be-indestructible software by pure accident.

I am by no means an Expert in these fields (they are huuuuge), but I have been around the block a while. I'm the Kid noone talks to, because I take the whole automation thing too far and put anything that sounds fun into containers to deploy it on azure or my k8s clusters somewhere.

So if you got fun questions, or need something containerized/automated feel free to hammer away at your keyboard, I take requests/challenges :)

How to build a basic Docker CI/CD Pipeline with Jenkins

Raphael Habereder — Fri, 12 Jun 2020 09:23:20 +0000

In the last post we talked about the Dos and Don'ts of containers.

We established the following example hierarchy of images:



minimal-baseimage (for example ubuntu, alpine, centos)
|__ nginx-baseimage
|  |__ awesome-website-container
|__ quarkus-baseimage
|  |__ awesome-java-microservice
|__ python-baseimage
   |__ awesome-flask-webapp

Maybe you dabbled a bit and built a few containers.
A few weeks pass.
You are sick of updating everything yourself.
This needs to be automated in some way, you want to focus on the real thing, your awesome apps!

So let's build a simple pipeline with Jenkins that takes care of patching your Images, and later on, maybe even deploy them for you! Automagically.

Setup

Install jenkins with your favorite package manager:

Debian:



wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > \
    /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins openjdk-11-jdk-headless

Or if you are on MacOS:



brew cask install homebrew/cask-versions/adoptopenjdk8
brew install jenkins-lts
brew services start jenkins-lts

If you want to run Jenkins on docker, it's going to be a bit more complicated, but it's doable. It just hastened my aging process by about 200 years to make this work on MacOS too.



FROM alpine

USER root
RUN apk add --no-cache \
  bash \
  coreutils \
  curl \
  git \
  git-lfs \
  openssh-client \
  tini \
  ttf-dejavu \
  tzdata \
  unzip \
  openjdk11-jdk \
  shadow \ 
  docker 

# Install Gosu
ENV GOSU_VERSION 1.12
RUN set -eux; \
    \
    apk add --no-cache --virtual .gosu-deps \
        ca-certificates \
        dpkg \
        gnupg \
    ; \
    \
    dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
    wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch"; \
    wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$dpkgArch.asc"; \
    \
# verify the signature
    export GNUPGHOME="$(mktemp -d)"; \
    gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4; \
    gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
    command -v gpgconf && gpgconf --kill all || :; \
    rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
    \
# clean up fetch dependencies
    apk del --no-network .gosu-deps; \
    \
    chmod +x /usr/local/bin/gosu; \
# verify that the binary works
    gosu --version; \
    gosu nobody true

ARG user=jenkins
ARG group=jenkins
ARG uid=1001
ARG gid=1001
ARG http_port=8080
ARG agent_port=50000
ARG JENKINS_HOME=/var/jenkins_home
ARG REF=/usr/share/jenkins/ref

ENV JENKINS_HOME $JENKINS_HOME
ENV JENKINS_SLAVE_AGENT_PORT ${agent_port}
ENV REF $REF

# Jenkins is run with user `jenkins`, uid = 1000
# If you bind mount a volume from the host or a data container,
# ensure you use the same uid
RUN mkdir -p $JENKINS_HOME \
  && chown ${uid}:${gid} $JENKINS_HOME \
  && addgroup -g ${gid} ${group} \
  && adduser -h "$JENKINS_HOME" -u ${uid} -G ${group} -s /bin/bash -D ${user} 

# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME $JENKINS_HOME

# $REF (defaults to `/usr/share/jenkins/ref/`) contains all reference configuration we want
# to set on a fresh new installation. Use it to bundle additional plugins
# or config file with your custom jenkins Docker image.
RUN mkdir -p ${REF}/init.groovy.d

# jenkins version being bundled in this docker image
ARG JENKINS_VERSION
ENV JENKINS_VERSION ${JENKINS_VERSION:-2.222.4}

# jenkins.war checksum, download will be validated using it
ARG JENKINS_SHA=6c95721b90272949ed8802cab8a84d7429306f72b180c5babc33f5b073e1c47c

# Can be used to customize where jenkins.war gets downloaded from
ARG JENKINS_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war

# Could use ADD but this one does not check Last-Modified header neither does it allow to control checksum
# See https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -o /usr/share/jenkins/jenkins.war \
  && echo "${JENKINS_SHA}  /usr/share/jenkins/jenkins.war" | sha256sum -c -

ENV JENKINS_UC https://updates.jenkins.io
ENV JENKINS_UC_EXPERIMENTAL=https://updates.jenkins.io/experimental
ENV JENKINS_INCREMENTALS_REPO_MIRROR=https://repo.jenkins-ci.org/incrementals
RUN chown -R ${user} "$JENKINS_HOME" "$REF"

# For main web interface:
EXPOSE ${http_port}

# Will be used by attached agents:
EXPOSE ${agent_port}

ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log

# Download and place scripts needed to run
RUN curl https://raw.githubusercontent.com/jenkinsci/docker/master/jenkins-support -o /usr/local/bin/jenkins-support && \
    curl https://raw.githubusercontent.com/jenkinsci/docker/master/jenkins.sh -o /usr/local/bin/jenkins.sh && \
    curl https://raw.githubusercontent.com/jenkinsci/docker/master/tini-shim.sh -o /bin/tini && \
    curl https://raw.githubusercontent.com/jenkinsci/docker/master/plugins.sh -o /usr/local/bin/plugins.sh && \
    curl https://raw.githubusercontent.com/jenkinsci/docker/master/install-plugins.sh -o /usr/local/bin/install-plugins.sh

COPY --chown=${user} entrypoint.sh /entrypoint.sh

RUN chmod +x /usr/local/bin/install-plugins.sh /usr/local/bin/plugins.sh /usr/local/bin/jenkins.sh /bin/tini /usr/local/bin/jenkins-support
RUN chmod +x /entrypoint.sh

# Stay root, the entrypoint drops down to User jenkins via gosu
ENTRYPOINT ["/entrypoint.sh"]

entrypoint.sh



#!/bin/sh

# Stolen from: Brandon Mitchell <public@bmitch.net>
# License: MIT
# Source Repo: https://github.com/sudo-bmitch/jenkins-docker

set -x

# configure script to call original entrypoint
set -- tini -- /usr/local/bin/jenkins.sh "$@"

# In Prod, this may be configured with a GID already matching the container
# allowing the container to be run directly as Jenkins. In Dev, or on unknown
# environments, run the container as root to automatically correct docker
# group in container to match the docker.sock GID mounted from the host.
if [ "$(id -u)" = "0" ]; then
  # get gid of docker socket file
  SOCK_DOCKER_GID=`ls -ng /var/run/docker.sock | cut -f3 -d' '`

  # get group of docker inside container
  CUR_DOCKER_GID=`getent group docker | cut -f3 -d: || true`

  # if they don't match, adjust
  if [ ! -z "$SOCK_DOCKER_GID" -a "$SOCK_DOCKER_GID" != "$CUR_DOCKER_GID" ]; then
    groupmod -g ${SOCK_DOCKER_GID} -o docker
  fi
  if ! groups jenkins | grep -q docker; then
    usermod -aG docker jenkins
  fi

  #If you run on MacOS
  if ! groups jenkins | grep -q staff; then
    usermod -aG staff jenkins
  fi
  # Add call to gosu to drop from root user to jenkins user
  # when running original entrypoint
  set -- gosu jenkins "$@"
fi

# replace the current pid 1 with original entrypoint
exec "$@"

Build and run our Jenkins image:



# We need to create a directory for Jenkins to save his data to
# Since to container runs with UID:GID 1001:1001
# The folder also needs to get correct permissions set
mkdir $HOME/jenkins && chown 1001:1001 $HOME/jenkins

docker build -t myjenkins .
docker run -d \
           -v $HOME/jenkins:/var/jenkins_home \ 
           -v /var/run/docker.sock:/var/run/docker.sock \
           -p 8080:8080 \
           --name jenkins \
           --restart unless-stopped \
           myjenkins

If you run a different system (sorry, I can't provide them all for you, it would take me days :( ), there is probably a guide for you out there, just as simple as these few lines.

Open the Jenkins UI in your awesome browser of choice and enter the password you can find in the location that jenkins tells you.

If it's not there, these places are usually a safe bet:



Linux:
/var/log/jenkins/jenkins.log, 

MacOS:
~/.jenkins/secrets/initialAdminPassword

Docker:
docker logs jenkins

Hammer it in and go on to install the suggested plugins. Depending on your machine, it's now your final chance to grab a cup of coffee, before we dive in.

Plugins, plugins, plugins

Next, we need some awesome plugins.
Just go via the Jenkins GUI -> Manage Jenkins -> Manage Plugins
Select the tab "Available", put Docker into the filter in the upper right corner and select the following Plugins:

Docker Commons
Docker Pipeline
Docker API
Docker
docker-build-step

Install without restart and wait a bit.
These will set you up fine for your first simple docker Pipelines.

Now we have to configure jenkins to find the docker-runtime to build our images with.

This can be done here. After installing the plugins, you get the new section "Docker", where you can Add Docker Installations. So go ahead and push that button.
Give it a name, I chose "Docker CE 19.03" and leave the installation root empty. Jenkins should find docker on the $PATH itself.

Pipelines

On to the next step, let's create a pipeline.

Via Jenkins -> New Item you'll get to a page that will let you specify which kind of item you want to create. Select Pipeline, give the puppy a nice name and hit OK.

Scroll down until you see this:

Now let's get this show on the road!



node {   
    stage('Clone repository') {
        // Missing Credentials can be added via UI 
        // Look at the bottom of the box for a link called "Pipeline-Syntax"
        // If you don't have much Jenkins experience, 
        // there you can generate pipelines with a few Dropdowns and Textboxes
        git credentialsId: 'git', url: '<your git url>'
    }

    stage('Build image') {
        // If you have multiple Dockerfiles in your Project, use this:
        // app = docker.build("my-ubuntu-base", "-f Dockerfile.base .")

        app = docker.build("my-ubuntu-base")
    }

    stage('Test image') {
        app.inside {
            sh 'echo "Tests passed"'
        }
    }

    stage('Push image') {
        docker.withRegistry('http://registry.local:5000') {
            app.push("18.04-${env.BUILD_NUMBER}")
            app.push("latest")
        }
    }
}

This file can be copied easily, since you don't have to change a lot. If you want to go the extra mile, make it a parameterized job and put the variables in there, to be filled via REST for example.

You might have noticed it, but we push to a registry called "registry.local:5000".

If you don't want to push your images into dockerhub right away, or have no other registry of your own, we can fire one up real quick.



docker run -d --name --restart always registry.local -p 5000:5000 registry:2

To use this registry with a nice dns-name, just run this:



echo '127.0.0.1 registry.local' | sudo tee -a /etc/hosts

To make sure the registry works, you have to tell docker to allow it, as an "insecure registry".



Linux: echo '{ "insecure-registries": ["registry.local:5000"] }' | sudo tee -a /etc/docker/daemon.json

On Desktops you can add this via the Docker Preferences UI

To get back to our pipelines, if you remember our imaginary Image-Hierarchy, we would need 3 jobs:



minimal-baseimage
|__ nginx-baseimage
   |__ awesome-website-container

Go ahead and copy/hack away, I'll wait for you.

That was easy, right?

Automate it

Now let's be honest, nobody likes pushing build buttons regularly, so let's automate this.

Minimal Base Build-Schedule

Let's go to your minimal-baseimage job, the first in the hierarchy, which provides the minimal, but regularly patched, base-system for our infrastructure/middleware containers.

Look for the following setting and schedule the job to run regularly, for example to run daily at 8:00 in the morning:

A base-image probably isn't going to be patched every other minute, so a daily, or even weekly schedule would be just fine.

Update derived containers automatically

Now how do we get our derived images to build as well, once the minimal base-image is updated?

Like this, for example:

That setting would take care of updating our NGinX base-image, once the minimal-baseimage has been patched. Obviously only, if the build actually succeeds.

Create Deployables as often as possible

For images that contain actual source, which would be our deployables, those should get built pretty frequently.
We don't only want builds when the baseimage gets patched, but also when our codebase changes. So let's implement that.

here we have two triggers, the first makes sure our app gets updated once the patches ran down the chain and arrived on our nginx container.

Additionally, we poll our git, to trigger a build for incoming git commits.
This, as is usually the case with polling, works in the beginning, for small teams that don't push dozens of builds in 5 minutes. Depending on your circumstances this could already suffice.

Teams with a high push frequency will probably end up with a build containing multiple commits, which is probabyly undesired for eventual testing stages (or the blame-game if the build breaks :P).

If you want a build for every push, reliably, you will have to look at your git repo-tool and check if your tool maybe provides post-push webhooks.

The set up Jenkins for that, the config could look like this:

This allows your job to be called via REST, if the specified token is provided.

Maybe there even is a cool plugin for your existing toolset, it would actually shock me if there wasn't. Jenkins existing Plugin-Base is enormous, there are plugins for pretty much anything.

Cleanup

After a short while, your Jenkins Host could look like this:

You should probably think about regularly cleaning up your Jenkins-Host via docker system prune -af to save space.

So we will do just that with another job:



node {   
    sh "docker system prune -af"
}

Add your schedule to run daily, just like you did before, and you are set!

Finishing Line

Congratulations, you have a completely independent build-pipeline for your images now. If you trigger the minimal-baseimage by hand, or they are triggered by their daily schedule, Jenkins should walk it's way all the way down to your awesome website-container and everything should be patched, pushed and ready to use!

Next time we can take a look at kubernetes and how we can implement a CI with tekton instead of Jenkins, if there is interest for the topic.
Or we could go and scan our images with anchore for CVEs.

Feel free to wish for something that interests you :)

The Dos und Don’ts of containers

Raphael Habereder — Wed, 10 Jun 2020 07:17:08 +0000

So I got a very nice question from a community member here, which I would like to pick up in this series.

It was as follows:
I’m going from being a low-code developer to full-stack.
I’m building my first web app and wondering how I go from ‘it runs on my machine’ to ‘ive got a multi-environment, self-healing, auto-scaling, well-oiled internet machine’.
What would you say are the key tasks to achieve with containers and when should they be done in the project?

That is a fantastic question which merits a whole series of posts to be perfectly honest. The topic of containers might seem intimidating at first.
Containers, Images, Operators, Orchestration-Tools, Ingresses, Observability, Monitoring, PaaS, CaaS, SaaS and many more buzzwords await you in the world of containers.
But fear not, there are many essentials you can always keep in mind, that I will try to cover as much as possible in this series.
These tasks/tips will get you into a great starting position, once you start tackling the topic of containers.

Let's split that question up into smaller chunks first.
We have the following three topics to talk about here:

The key tasks to achieve with containers
When they should be done
The actual transition from bare-metal/manual deployment to containers

1) The key tasks to achieve with containers

There are many benefits to containers, here is a small list of things my customers have said in the past, that were of "special importance to them".

Save money by using hardware as optimally as possible
Have software be more portable
Improve speed and quality of development

I think these reasons don't sound too bad. A lot can be already done, even without containers, but here is my take on it:

Save money by using hardware as optimally as possible
Containers require less overhead than the typical VM-Landscape many companies have today. By removing the hypervisor and sharing the same kernel between all containers, you suddenly get a lot more containers onto your hardware than VMs.
Not only do you save overhead, you also gain in speed. While VMs are just as scalable as containers, horizontally scaling a simple apache2 container is quite a bit faster than spinning up preimaged VMs.

Have software be more portable
Damn it, I forgot to install a JDK on Server X, now the microservice won't even start!
This won't happen with a container, since you package everything you need with your deliverable, the service.
If your container runs locally, it will run on Azure, GCS, Amazon ECS, or an on-premise PaaS (except if something is majorly borked).

Improve speed and quality of development
If you correctly set up your environments, a deployment to a different cluster/location/stage might just be a simple additional line:

kubectl config use-context aws
kubectl apply -f awesomeapp-deployment.yaml

Want to deploy on azure instead?

kubectl config use-context azure
kubectl apply -f awesomeapp-deployment.yaml

In my opinion, it doesn't really get easier or faster than that.
Quality, as always, is (probably) not to blame on the technology.

2) When they should be done

This can be answered pretty quick, I would say as early as possible. What I have learnt in many years of non-stop containerization is, "do it correct and right at the beginning". Go the extra mile, make your containers and surrounding infrastructure as seamless as possible. Don't skimp on things you think "might be overkill".
Automation/Optimization is boring, sometimes ugly and very cumbersome. But try to do it anyways, so you can reap the rewards as early as possible. You can easily strip out functionality you may not need afterwards. But adding onto a pipeline at the end almost always ends up in chaos.

3) The actual transition

What we want to achieve is, as was perfectly described before, "a well-oiled internet machine". So let's get started with the do's and don't's of containers with practical Dockerfiles.

Tip 1) Control

Have a single base-image, all your other images derive from. If you need something on top of it, make another base-image that derives from that. Put your actual stuff another layer down in final Images.

It could look like that:

minimal-baseimage (for example ubuntu, alpine, centos)
|__ nginx-baseimage
|  |__ awesome-website-container
|__ quarkus-baseimage
|  |__ awesome-java-microservice
|__ python-baseimage
   |__ awesome-flask-webapp

Why would you do that?
It's simple really, you need to update your images because of CVE's, patches or need something else in all your images? Update your top minimal-baseimage, kick off your CI and watch all your images getting updated to the same basic state.

Let's take a look at the path minimal -> nginx -> awesome-website.

This could be your Dockerfile for your minimal-baseimage:

FROM ubuntu:18.04

RUN apt-get -y update && \
    apt-get -y upgrade && \
    apt-get autoclean && \
    apt-get clean 


WORKDIR /opt
RUN chown nobody. /opt
USER nobody

This could be an image that installs nginx to host your webpage for example:

FROM my-ubuntu-base:18.04-somebuildnumber
USER root
RUN apt-get install -y nginx && \
    apt-get clean && \
    chown nobody. /usr/share/nginx/html
USER nobody
EXPOSE 80
STOPSIGNAL SIGTERM
CMD ["nginx", "-g", "daemon off;"]

And finally, this would be the actual container you deploy your website with:

FROM my-ubuntu-nginx-base:1.18.0-stable
COPY awesomefiles/index.html /usr/share/nginx/html/index.html

This might look overkill, if you could replace the whole chain by doing something like this:

FROM nginx
COPY static-html-directory /usr/share/nginx/html

But what is the difference, except copious amounts of effort for a single nginx-image with an awesome html-file?

Many things, but here are four important examples

You know and control what is inside your container
You can make sure it is updated and control the update process yourself
You already got the prerequisites to fix anything when it's broken, or your security scanner complains about "outdated packages" (Which in a commercial environment will happen a lot)
You control the security context inside your container, by example with using nobody instead of root in the official nginx image

Tip 2) Simplicity

As a rule of thumb, always keep in mind: "one function per container".
While multiple processes in a single container are possible and in some edge-cases may be perfectly reasonable, it will increase dramatically in difficulty to manage the whole thing.

Imagine the following processtree:

root@cc726267a502:/# pstree -ca
bash
  |-bash springboot.sh
  |   `-sleep 6000
  |-bash prometheus.sh
  |   `-sleep 6000
  |-bash grafana.sh
  |   `-sleep 6000
  |-bash h2.sh
  |   `-sleep 6000
  `-pstree -ca

What would happen if prometheus died? Nothing really, and you wouldn't even know or get informed by docker. The main process, in this case some kind of bash wrapperscript, would still run. So Docker would have no reason to restart the container or identify it as broken.

This would be the result:

root@cc726267a502:/# kill 347
root@cc726267a502:/# pstree -ca
bash
  |-bash springboot.sh
  |   `-sleep 6000
  |-bash grafana.sh
  |   `-sleep 6000
  |-bash h2.sh
  |   `-sleep 6000
  |-pstree -ca
  `-sleep 6000
[2]   Terminated              bash prometheus.sh

If you split all these processes into their own containers, your orchestration would notice if they died and spin them back up again. More on orchestration in another part, promised :)

There are also other very practical reasons as for why you should limit yourself to a single process:

Scaling containers is much easier if the container is stripped down into a single function. You need another container with your app in it? Spin one up somewhere else. If your container contains all the other apps, this complicates things and maybe you don't even want a second grafana or prometheus
Having a single function per container allows the container to be re-used for other projects or purposes
Debugging a simple container locally is way easier than pulling a gigantic god-container that was blown way out of proportions
Patching. Update your base-image, kick off your CI and you are good to go. Having to test for side-effects in other processes isn't fun
Above also holds true for rollbacks of changes. The update bricked your app? No problem, change the tag in your deployment descriptor back to the old one, and you are finished.

I could go on about security, too, but let's leave it at that, you probably get the point.

Important: I am not saying that multi-process containers are inherently bad. They can work and have their uses. As a beginner into containerization, I am just recommending you to keep it as simple as possible"

This tip won't come with a Dockerfile of the bad example really, since I don't want to encourage you starting the wrong way. Maybe later down the series we will tackle that process.

Tip 3) Updates

As hinted towards in Tip 1) already, keep your containers updated. Always, regularly, automatically. Modern CI-Tools can help with that, most of them have webhook integrations for Git that can trigger your build-jobs. Or if you don't want to implement too much infrastructure at the start, build a cronjob or something.
Just keep it automated. You should never have to patch a container yourself, because that increases the chance to just forget it.

Tip 4) Automation

Automate as much as possible. You don't want to fumble around in a live container. Update the source, build, push, run your CI. Never touch a running container afterwards, or you are in for a world of pain if a live-container behaves different from a freshly built one and you don't know why.
If you followed Tip 3), you already have a patch-pipeline in place, make use of that.

Tip 5) Safety

Keep your container safe. The saying goes "nobody is perfect", which fits right into containers. User root is bad news.
Your software should run as nobody, so a breach will get the intruder exactly nowhere, even if you forget to remap your usernamespace ids.

Use base images with slimmed down containers. Your containers should contain no curl, no compilers, no ping or nslookup. Nothing that can result in changes or load to your or other peoples infrastructure if someone breaks in.

Harden your runtime with best practices, remap your user namespace ids, scan your images regularly for vulnerabilities and keep privilege escalations like USER root to a minimum. You know, the things you would do with a good server too. Containers should be treated like a possible Botnet member just as well.

Tip 6) Reliability

This tip contains two things. Choose your base system wisely and stick to it as much as possible. For example alpine. While alpine has drawbacks, and sometimes doesn't work with what you have planned to run on it, it does provide the following advantages:

super small footprint (6MB vs. Ubuntus 74MB or CentOS whopping 237MB)
minimal attack surface from the outside, since alpine was designed with security in mind
Alpine Linux is simple. It brings it's own package manager and works with the OpenRC init system and script driven set-ups. It just tries to stay out of your way as much as possible

Build your container yourself. While there might be 100 containers that already do what you want to get done, you have no idea how frequently they are patched, or what else is in them. So build the container of your dreams yourself. It helps you to get into the routine of hardening, patching and optimizing them. You can never get enough experience in that regard. Always keep in mind, "you build it, you run it, you are liable for it".

Congrats you made it this far

I believe these few simple tips will get you a good headstart in developing healthy containers. Next time, we will take a look at how a CI for your containers could look like, by using Jenkins or Tekton.

If I left you with questions, as always, don't shy away from asking. There are no stupid questions, only stupid answers!

An Introduction to Quarkus

Raphael Habereder — Thu, 04 Jun 2020 22:08:16 +0000

The myth of slow Java

In the realm of Java, who hasn't heard one of these statements at least a dozen times:

"Java is slow"
"Java needs too much memory!"
"Java Services/Servers need too long to start!"
"Java isn't ready for microservices!"

I'm glad to tell you, with this post, that none of these are true anymore. Actually, they haven't been true anymore for a long time.
The last time I had a slow boot of an old-school Application Server was in the age of Tomcat or JBoss 4.x, and those are very, very old.

The solution to a non-existing problem

The - hopefully final - nail in the coffin of these myths is named Quarkus, or as the people behind this wonderful creation call it: "Supersonic Subatomic Java".

So what is it actually? Again, let's use the words of the creators: "A Kubernetes Native Java stack tailored for OpenJDK HotSpot and GraalVM, crafted from the best of breed Java libraries and standards."

Alright, that doesn't really tell us much, it also introduces another Buzzword: GraalVM.

GraalVM? What is that now?

Oracle created something wonderful with GraalVM, here is their sales-pitch on it:

"GraalVM is an ecosystem and shared runtime offering performance advantages not only to JVM-based languages such as Java, Scala, and Kotlin, but also to other programming languages such as JavaScript, Ruby, Python, and R. Additionally, it enables the execution of native code via an LLVM front-end, and WebAssembly programs on the JVM."

And let me tell you, those are not just empty promises! GraalVM is amazingly fast and combined with Quarkus, you are in for one hell of a joy-ride.

We want the numbers Mason!

While evaluating Quarkus for a Microservice Landscape, we did our homework first and compared three different environments:

Payara Micro + JVM
Quarkus + JVM
Quarkus Native Image

While Payara Micro is already considered "fast" and "small footprint", Quarkus and a Quarkus Native Image respectively, are considerably faster and even smaller in overhead.

Don't believe me? I'll back those claims up in just a second.

Here are some pictures of a loadtest we created. The application was equipped with JAX-RS and JPA, targeting a (brutally oversized) in-memory database (so we can reduce the impact of database speed or, god forbid, slow network traffic).

The test itself consisted of 5000 Requests in 5 concurrent threads and a minimal database payload (it was just a string tbh).

All we monitored were CPU-Load, Time and Memory-Usage of the service image itself.

It's not really a detailed or scientific test, but the numbers are sufficient enough to name a clear winner.

Payara Micro with OpenJDK 11.0.6 (Provided by GraalVM 20.0 CE):

Quarkus with OpenJDK 11.0.6 (Provided by GraalVM 20.0 CE):

Quarkus Native Image:

Amazing how fast a native Image can be, right?

As you can see, we got the following advantages, just by using quarkus native images:

we reduced the time to first request from 7.3 seconds, to 0.104 seconds. This would typically include the costly bootstrapping of a jvm/application-server runtime, as well as the preparation of the JAX-RS Endpoint
memory usage decreased from ~700 MByte to ~50 MByte
the CPU-Load went from a wild rollercoaster of 50%-90% to consistent 30% during the whole lifecycle

While working off the 5000 Requests hasn't sped up as drastically, the memory consumption consistently stayed very low.

If that isn't a noticeable improvement, I don't know what is.

So let's build something with Quarkus!

A demo project, including the following Dockerfile can be found here

Are you ready to dive in? So here we go!

First a basic Dockerfile to build some Java stuff

FROM quay.io/quarkus/centos-quarkus-maven:19.3.1-java11 AS build
# Since JVM has not been ported to alpines musl yet
# and quarkus still relies on gcc for native binaries
# We'll use the quarkus maven image as build-base


# Lots of workarounds and setup for graalvm and quarkus to build the native binary
# thankfully none of them end up in the final image
WORKDIR /usr/src/app
USER root
RUN chown -R quarkus /usr/src/app
USER quarkus
COPY --chown=quarkus app/ .

RUN ./mvnw clean package -Pnative


# Due to the very same reason of musl not playing along just yet
# we will use the redhat minimal image for delivery
FROM registry.access.redhat.com/ubi8/ubi-minimal

WORKDIR /app
# Since the user should always be nobody (imho) and the USER-Directive only affects RUN, CMD and ENTRYPOINT 
# But not WORKDIR, we have to modify ownership of the workdir
RUN chown nobody. /app

# Copy as nobody
COPY --from=build --chown=nobody /usr/src/app/target/*-runner /app/quarkusapp

# Set up permissions for user `nobody`
RUN chmod 775 /app \
  && chmod -R "g+rwX" /app \
  && chown -R nobody. /app

EXPOSE 8080
USER nobody

# Tell quarkus to listen on all interfaces, instead of localhost
CMD ["./quarkusapp", "-Dquarkus.http.host=0.0.0.0"]

This handles the build part for you.

You might be asking yourself now "Yeah, this is nice and all. But what if I need more than just your lame little JAX-RS Service? This doesn't really help me much..."

Don't worry friend, quarkus has a bootstrapping service, like most other cool cloud-ready frameworks today, where you can check some boxes and get the skeleton project ready to use!

All this does basically, is change the dependencies in the resulting pom.xml, which the quarkus-maven-plugin picks up and builds your image with.

    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-resteasy</artifactId>
    </dependency>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-junit5</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>io.rest-assured</groupId>
      <artifactId>rest-assured</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-resteasy-jsonb</artifactId>
    </dependency>

With this short, but hopefully sweet, little introduction, you should be ready to go and quarkus(ify?) your applications. And don't forget, always try to have fun doing it!

If I missed anything or you need some further information, don't shy away from leaving a comment, I'll make sure to leave none of you hanging :)

Zero To Kubernetes

Raphael Habereder — Tue, 26 May 2020 13:16:00 +0000

Elevator-Pitch

Who doesn't love kubernetes?
Who also doesn't love, if things just work without much tinkering, automagically?
If you do like both of these things, this might be a post for you!

While I like using k8s, I dislike the part about cleaning stuff up after long sessions of deploying dozens of containers.
Only to recreate the cluster again later. And damn it, I forgot to deploy grafana again...

So I automated the setup/teardown of this, using a few handy tools and some Bash Scripting to incorporate more laziness in my workflow.

Grand Reveal

I call it zero-to-k8s, or if you want a niftier name: 02k8s
Which is totally readable and everyone instantly understands what it means. Right? Right?!
I better leave the title as it is, I think.

So, what is it?

Essentially it is a glorified bash-script that downloads the typical binaries you need, for example kubectl, helm, k3d and the likes.
Everything you need to use kubernetes and a little more for comfort.

It then calls all those binaries, creates a k8s cluster and also deploys some workloads of tools you decided are worthy.
Deployment, Service, Ingress and all the other nice things you need for them.
Just one kubectl port-forward away from displaying in your favorite browser (as long as they are not w3m or lynx)

Where the laziness comes into play

There are two versions of the script, one having to be edited every time you want something to change.
And second, my favorite version (the lazy one) that pops up some dialogs and asks you annoying questions, so you don't have to do annoying editing every time.
Unfortunately we have to die one annoying death, since my mind-reading interface is not done yet.

Awesome Demo

Why don't I just give you the grand tour of how it works?
Behold the blue curses widget goodness:

The things I used:

k3s - A stripped down kubernetes runtime developed by rancher, or as they call it "a highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances"
k3d - Another tool made by rancher to deploy k3s inside docker.

The things you need:

WSL/Bash/MacOS ZSH (bless you if you get to work on a real Linux)
Docker (the whole "k3s in docker" approach probably won't work without it) 
Dialog (if you want to be as lazy as possible)
Access to the internetz (Or a very patient IT-Sec Department that downloads stuff for you)

I don't like it, make it go away!

To kill your cluster and thanos-snap it away (50% of the time it works all the time, or so I've been told), all you need is one line:

./bin/k3d delete --name dev

No harm, no foul, you do you my friend. I promise I won't take it personally!
As every good technician, the entire blame is on the tools, obviously.

What do I want to do with this thing?

To be honest, all I want is to continue to automate as much as possible, to enjoy a coffee during all the boring tasks that just take unnecessary amounts of time.
And learn of new tools and probably automate them away too.
So if you got something that just has to go in there, don't shy away from shooting me a message.

I hope some of you find this as helpful as I do, let's go and automate the rest of the world!