DEV Community: Boris Quiroz

Github as Helm repository

Boris Quiroz — Fri, 28 Jun 2024 02:39:43 +0000

Originally posted in my blog

For my personal projects, I use Github as a Helm repository. This is a simple way to share Helm charts with the community. In this post, I'll show you how to create a Helm repository using Github Pages either for personal use or for your organization.

Assumptions

This post will assume your already:

Know what kubernetes is
Have a basic understanding of helm
Have a Github account and repository
Have coded a Helm chart

Steps

Having all the above already in place, there are some things we need to do before we can use Github as a Helm repository. In particular, we need to enable Github Pages for our repository and add at least one index.html file to it. According to this documentation, the index.html file shouldn't be needed, but in reallity it is necessary for Helm to recognize the repository as a Helm repository.

Once having that and assuming our first chart is ready, we need to start packaging and adding it to the repository. The following steps will guide you through the process. All the directory structure assumed below is based on my own repository.

Package the chart

$ helm package mychart
$ mv mychart-0.1.0.tgz helm/charts

Update the index

$ helm repo index helm/charts --url https://boris.github.io/kubernetes/helm/charts
$ git add helm/charts/*
$ git commit -m "Add mychart-0.1.0"
$ git push

With the above we're now ready to add our repository to Helm and install our chart:

$ helm repo add boris https://boris.github.io/kubernetes/helm/charts
$ helm repo list
NAME                    URL
ealenn                  https://ealenn.github.io/charts
bitnami                 https://charts.bitnami.com/bitnami
kubernetes-dashboard    https://kubernetes.github.io/dashboard/
argo                    https://argoproj.github.io/argo-helm
boris                   https://boris.github.io/kubernetes/helm/charts/
$ helm install mychart boris/mychart

🐍 Testing exit codes with Pytest

Boris Quiroz — Sat, 10 Oct 2020 22:45:33 +0000

As almost every other sysadmin I know, python is the language of choice to do some scripting. And, as we all know, testing is an important part of any development.

My code is a simple parser that interacts with some Linux commands using subprocess, which instead of raising an exception it returns something != 0. In fact, the exit code is 2. How to test that with pytest? This is what I did (which might not be the best way to do it):

The parser is pretty simple, this is the cli.py:

from argparse import ArgumentParser

def create_parser():
    parser = ArgumentParser()
    parser.add_argument('list', help='List something')
    ...
    return parser

The tests:

import pytest
from parser import cli

@pytest.fixture
def parser():
    return cli.create_parser()

def test_something_to_test(parser):
    with pytest.raises(SystemExit) as e:
        # The command to test
        parser.parse_args(['some', 'commands'])
    # Here's the trick
    assert e.type == SystemExit
    assert e.value.code == 2

Microk8s: Unable to connect to the server: x509: certificate has expired or is not yet valid

Boris Quiroz — Mon, 04 May 2020 00:14:07 +0000

This is an error that you might see once every 10 years, so it's very likely you'll forget the solution.
The behavior is the following: When trying to do anything that interacts with API server you'll get the following error:

boris@ubuntu:~$ microk8s kubectl get all --all-namespaces
Unable to connect to the server: x509: certificate has expired or is not yet valid: current time 2020-05-03T23:53:06Z is after 2020-05-03T16:38:01Z

How to solve it?

Easy, just renew your certificates. But first, it's a good idea to check expiration time of current installed certificates:

boris@ubuntu:~$ sudo microk8s.refresh-certs -c
The CA certificate will expire in 0 days.

Ok, we're a but out-of-date, let's renew them:

boris@ubuntu:~$ sudo microk8s.refresh-certs -i
Backing up certificates under /var/snap/microk8s/1385/var/log/ca-backup/
Creating new certificates
Signature ok
subject=/C=GB/ST=Canonical/L=Canonical/O=Canonical/OU=Canonical/CN=127.0.0.1
Getting CA Private Key
Signature ok
subject=/CN=front-proxy-client
Getting CA Private Key
1
Creating new kubeconfig file
Stopped.
Started.

The CA certificates have been replaced. Kubernetes will restart the pods of your workloads.
Any worker nodes you may have in your cluster need to be removed and re-joined to become aware of the new CA.

Discuss: Would you join a dev.to IRC channel?

Boris Quiroz — Sun, 05 Jan 2020 02:39:30 +0000

This might sound a bit old-school, but IRC is still one of the tools I use on a daily basis so would be interesting to have some tech and non-tech conversations with the community in freenode or somewhere else.

Opening the discussion on this matter.

Create VPC using awscli

Boris Quiroz — Sun, 29 Dec 2019 23:35:37 +0000

The main idea of this post is to have an easy way to remind how to setup a VPC using awscli, and it's not intended to explain how a VPC works and what are the benefits of them.

The setup

Assuming that you already have awscli in place and configured, this is what we should do in order to have a working VPC:

Create a VPC with a /16 subnet (it can be /17, /18, etc. but never less than /16). I'll be using 10.88.0.0/16 for this example.
Create, at least, in different availability zones:
- Two public subnet: 10.88.0.0/24 and 10.88.1.0/24
- Two private subnet: 10.88.2.0/24 and 10.88.3.0/24
Create an Internet gateway and attach it to the VPC.
Create an Elastic IP Address.
Create a NAT gateway, attach to it the Elastic IP and place the NAT gateway in the public subnet.
Create a couple of routing tables:
- Assign public subnet to Internet gateway with destination of 0.0.0.0/0
- Assign the private subnet to NAT gateway with destination of 0.0.0.0/0

How it should be done.

This is the full list of commands that reflects the scenario described in the setup section. I won't explain each command, but feel free to ask in the comments:

aws ec2 create-vpc --cidr-block 10.88.0.0/16
aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.88.0.0/24 --availability-zone us-east-1e
aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.88.1.0/24 --availability-zone us-east-1f
aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.88.2.0/24 --availability-zone us-east-1e
aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.88.3.0/24 --availability-zone us-east-1f
aws ec2 create-internet-gateway
aws ec2 attach-internet-gateway --vpc-id $VPC_ID --internet-gateway-id $IGW
aws ec2 create-route-table --vpc-id $VPC_ID
aws ec2 create-route --route-table-id $RTB --destination-cidr-block 0.0.0.0/0 --gateway-id $IGW
aws ec2 allocate-address --domain vpc
aws ec2 create-nat-gateway --subnet-id $PUBLIC_SUBNET --allocation-id $EIP
aws ec2 create-route-table --vpc-id $VPC_ID
aws ec2 create-route --route-table-id $RTB_PRIVATE --destination-cidr-block 0.0.0.0/0 --gateway-id $NGW
aws ec2 associate-route-table --subnet-id $PUBLIC_SUBNET0 --route-table-id $RTB_PUB
aws ec2 associate-route-table --subnet-id $PUBLIC_SUBNET1 --route-table-id $RTB_PUB
aws ec2 associate-route-table --subnet-id $PRIVATE_SUBNET2 --route-table-id $RTB_PRIV
aws ec2 associate-route-table --subnet-id $PRIVATE_SUBNET3 --route-table-id $RTB_PRIV

Tips and tricks.

As a lot of reference to previously created resources is needed, I found useful to create a file with the following format:

export VPC_ID="vpc-<some_long_id>"
export PRIV_SUBNET="subnet-<some_long_id>"
export PUB_SUBNET="subnet-<some_long_id>"

And time to time I run source file_with_var so I can easily access them using the $.

Using YAML to manage Python configurations

Boris Quiroz — Mon, 02 Dec 2019 01:30:48 +0000

On my last weekend project I needed to talk to an API using an AUTH_KEY which should be private but should be included in the request instead of using any auth method.

The request I needed to run was:

requests.get('https://<URL>/<SOMETHING>/<SOMETHING_ELSE>.json/?auth_key=YOUR_AUTH_KEY')

So I decided to use pyyaml, and it's very easy to use! Here's a short example of the python file

import json
import yaml

file = open('config.yml', 'r')
cfg = yaml.load(file, Loader=yaml.FullLoader)

url = 'https://<URL>/<SOMETHING>/<SOMETHING_ELSE>.json/?auth_key=' + cfg['auth']['key']
request = requests.get(url)

# then, everything else

Then, the config.yml is something like this:

auth:
  key: MY_AUTH_KEY

And not I can have my config.yml stored only in my laptop or, even better, in some secret place like a vault or something but not publicly accessible to anybody in GitHub.

Enjoy.

[Halp!] How do you take notes?

Boris Quiroz — Sat, 05 Oct 2019 14:54:21 +0000

Notes are important, as brain is not able to retain all the information. In particular technical notes, as they can help us to avoid making the same mistake twice. Or three times.

Typically, I use a regular Castelli/Moleskinke/Other (any other on sale) regular note taking notebook using an old-school tool called pen. But this proven method is not fulfilling all my needs, which are:

Notes must be clear (this is on my side, and solved pretty well so far)
Notes must be searchable

And that's where the downside of old-school method appears: There's no efficient way to search for something in a 80 pages note taking notebook. The good thing about using pen and paper is that notes seems to be stored clearly in my brain, but not bullet proof.

I've also tested digital method: I wrote a small python app that, by using tags and/or date, I can search for notes I've taken, and I've been using it for the last couple of months. The problem with this is that I use the search option more often as information is not so well retained in my brain. Maybe I just need some training?

What do you use? How do you take notes? Do you see any disadvantages in your method?

Any help or comments are welcome.

Running tor using docker

Boris Quiroz — Fri, 19 Jul 2019 19:58:21 +0000

If you want to connect to tor network without using tor browser, docker is the solution!

Just copy this code into a Dockerfile:

FROM alpine:edge

RUN apk update
RUN apk add tor

RUN echo "Log notice stdout" >> /etc/torrc
RUN echo "SocksPort 0.0.0.0:9150" >> /etc/torrc

EXPOSE 9150

CMD tor -f /etc/torrc

Build it with docker build -t tor -f Dockerfile . and run it with:

docker run -d --rm --name tor -p 9150:9150 tor

But, if you don't want to do the build just run:

docker run -d --rm --name tor -p 9150:9150 boris/tor:alpine

After that, you'll have a tor proxy running in your 127.0.0.1:9150 so go ahead and configure your browser of choice to use a SOCKS proxy on 127.0.0.1:9150 and you'll go from this:

To this:

Enjoy your privacy :)

PS: Code for my Dockerfile is here and the image in Docker hub is here

Ubuntu: Remove disabled snaps

Boris Quiroz — Thu, 11 Jul 2019 03:02:17 +0000

In October 2015, Ubuntu 15.10 was shipped with something called snappy, which is the "software deployment and package management system developed by Canonical" (copy/paste from Wikipedia).

By now, most of Ubuntu users are not aware of snaps until they run a df -h and they see something like:

/dev/loop1                   174M  174M     0 100% /snap/spotify/34
/dev/loop4                    55M   55M     0 100% /snap/core18/1049
/dev/loop3                   141M  141M     0 100% /snap/gnome-3-26-1604/88
/dev/loop5                   4.2M  4.2M     0 100% /snap/gnome-calculator/406
/dev/loop7                    89M   89M     0 100% /snap/core/7270
/dev/loop8                   2.8M  2.8M     0 100% /snap/telegram-cli/25
/dev/loop9                   1.0M  1.0M     0 100% /snap/gnome-logs/61
/dev/loop10                   36M   36M     0 100% /snap/gtk-common-themes/1198
/dev/loop11                  436M  436M     0 100% /snap/libreoffice/133
/dev/loop12                  150M  150M     0 100% /snap/gnome-3-28-1804/63
/dev/loop13                   89M   89M     0 100% /snap/core/7169
/dev/loop14                   57M   57M     0 100% /snap/snapcraft/3059
/dev/loop15                   15M   15M     0 100% /snap/gnome-characters/292
/dev/loop16                  181M  181M     0 100% /snap/spotify/35
/dev/loop17                  3.8M  3.8M     0 100% /snap/gnome-system-monitor/95
/dev/loop20                  141M  141M     0 100% /snap/gnome-3-26-1604/90
/dev/loop21                  419M  419M     0 100% /snap/libreoffice/134
/dev/loop2                   3.8M  3.8M     0 100% /snap/gnome-system-monitor/100
/dev/loop22                   15M   15M     0 100% /snap/gnome-characters/296
/dev/loop23                  150M  150M     0 100% /snap/gnome-3-28-1804/67
/dev/loop24                   43M   43M     0 100% /snap/gtk-common-themes/1313
/dev/loop0                    55M   55M     0 100% /snap/core18/1055

The thing is, when an snap is installed the previous version is not uninstalled or removed, and you manually need to do that. Also, keep in mind that "updates" are, in fact, the install of a new version... So by doing the math, in a few months your df -h will be a mess.

To identify with more detail snaps installed in my computer, the best option is:

$ snap list --all                      
Name                  Version                     Rev   Tracking  Publisher       Notes
core                  16-2.39.3                   7270  stable    canonical✓      core
core                  16-2.39.2                   7169  stable    canonical✓      core,disabled
core18                20190627                    1049  stable    canonical✓      base,disabled
core18                20190704                    1055  stable    canonical✓      base
gnome-3-26-1604       3.26.0.20190705             90    stable/…  canonical✓      -
gnome-3-26-1604       3.26.0.20190621             88    stable/…  canonical✓      disabled
gnome-3-28-1804       3.28.0-10-gaa70833.aa70833  67    stable    canonical✓      -
gnome-3-28-1804       3.28.0-10-gaa70833.aa70833  63    stable    canonical✓      disabled
gnome-calculator      3.32.1                      406   stable/…  canonical✓      -
gnome-characters      v3.32.1+git2.3367201        292   stable/…  canonical✓      disabled
gnome-characters      v3.32.1+git2.3367201        296   stable/…  canonical✓      -
gnome-logs            3.32.0-4-ge8f3f37ca8        61    stable/…  canonical✓      -
gnome-system-monitor  3.32.1-2-ga7c19eaeff        95    stable/…  canonical✓      disabled
gnome-system-monitor  3.32.1-3-g0ea89b4922        100   stable/…  canonical✓      -
gtk-common-themes     0.1-16-g2287c87             1198  stable/…  canonical✓      disabled
gtk-common-themes     0.1-22-gab0a26b             1313  stable/…  canonical✓      -
libreoffice           6.2.5.2                     134   stable    canonical✓      -
libreoffice           6.2.5.2                     133   stable    canonical✓      disabled
snapcraft             3.6                         3059  stable    canonical✓      classic
spotify               1.1.5.153.gf614956d-16      35    stable    spotify✓        -
spotify               1.1.0.237.g378f6f25-11      34    stable    spotify✓        disabled
telegram-cli          1.4.5                       25    stable    marius-quabeck  -

And here's something interesting: The Notes column. Maybe we can use that column to filter and remove unused or disabled snaps? Yes, we can! And I do it with the following script:

#!/bin/bash
set -eu

snap list --all | awk '/disabled/{print $1, $3}' |
    while read s r; do
        sudo snap remove "$s" --revision="$r"
    done

As usual, I hope it helps :)

Unlimited git repositories

Boris Quiroz — Fri, 28 Jun 2019 18:54:35 +0000

Many developers owns one or more instances in any of the many "cloud providers" out there. And most of them uses git to track their project. For this, github or gitlab are pretty good alternatives but, is there any other alternative?

Yes, you can use nothing or go for Bitbucket or manage your own! Which is, in fact, something ridiculous simple and you just need to have an instance/virtual machine running anywhere ¹.

The process:

In your local machine (read: your laptop), create your project repository and git init it:

local: ~ $ cd ~/Code/sample
local: ~/Code/sample $ git init

On the other hand, on your remote server let's init a git bare repository:

remote: ~ $ mkdir ~/git
remote: ~ $ cd ~/git
remote: ~/git $ git init --bare sample.git

Now, assuming that you can reach your remote service using a fancy name, like dev.mydomain.io, let's configure our local using that name:

local: ~ $ git remote add origin boris@dev.mydomain.io:/git/sample.git

After doing that, just use your git repository as usual.

Anywhere reachable by ssh ↩