DEV Community: César M. Cristóbal

Secrets in ArgoCD with Sops

César M. Cristóbal — Wed, 07 Sep 2022 06:46:46 +0000

ArgoCD is a tool that implements the gitops philosophy for deploying applications on Kubernetes. It is a declarative, Git-based deployment system that uses a simple, human-readable manifest file to define the desired state of your application. In this article, we will explore how to use ArgoCD with Sops to manage secrets in a gitops workflow.

We use sops with GCP KMS, the first thing we need is a service account with roles/cloudkms.cryptoKeyDecrypter role and create a secret on Kubernetes:

kubectl create secret generic google-sa --from-file sa.json

The idea is to use Argo Config Management Plugin to decrypt the secret and generate a plane yaml with all Kubernetes objects (deployment, services, ...) and use Cumstom Tooling to install Sops and other tools required.

In the process we have encountered several problems, and one of them is that it is necessary to use yq to correctly format the output yamls and avoid errors such as: error converting YAML to JSON: yaml: line 4: did not find expected ',' or ']'.

These are the values to install argo-cd by Helm. Configuration assumes that all secrets are in secrets.enc file coding with yaml.

server:
  config:
    configManagementPlugins: |
      - name: sops
        init:
          command: ["/bin/sh", "-c"]
          args: ["echo '---' > secrets.yaml && sops -d --input-type yaml --output-type yaml secrets.enc >> secrets.yaml"]
        generate:
          command: ["/bin/sh", "-c"]
          args: ["source /virtualenv-python/bin/activate; pip install yq; cat *.yaml | yq -y"]
repoServer:
  volumes:
    - name: custom-tools
      emptyDir: {}
    - name: virtualenv-python
      emptyDir: {}
    - name: google-sa
      secret:
        secretName: google-sa
        items:
          - key: sa.json
            path: sa.json
  volumeMounts:
    - mountPath: /usr/local/bin/sops
      name: custom-tools
      subPath: sops
    - mountPath: /usr/local/bin/jq
      name: custom-tools
      subPath: jq
    - mountPath: /etc/secrets/sa.json
      name: google-sa
      subPath: sa.json
    - mountPath: /virtualenv-python
      name: virtualenv-python
  env:
    - name: GOOGLE_APPLICATION_CREDENTIALS
      value: /etc/secrets/sa.json
  initContainers:
    - name: custom-tools
      image: alpine:3.8
      command: ["/bin/sh", "-c"]
      args:
        - wget https://github.com/mozilla/sops/releases/download/v3.7.3/sops-v3.7.3.linux.amd64;
          chmod a+x sops-v3.7.3.linux.amd64;
          mv sops-v3.7.3.linux.amd64 /custom-tools/sops;
          wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64;
          chmod a+x jq-linux64;
          mv jq-linux64 /custom-tools/jq;
      volumeMounts:
        - mountPath: /custom-tools
          name: custom-tools
    - name: virtualenv-python
      image: python:3.7
      command: ["/bin/sh", "-c"]
      args:
        - python3 -m venv /virtualenv-python
      volumeMounts:
        - mountPath: /virtualenv-python
          name: virtualenv-python

helm upgrade sops argo/argo-cd --values values-sops.yaml

And an application example to try it.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: secrets
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/Callepuzzle/manifests
    targetRevision: main
    path: poc-argocd
    plugin:
      name: sops
  destination:
    name: ''
    namespace: ''
    server: 'https://kubernetes.default.svc'

Grafana multi-tenant configuration with Terraform

César M. Cristóbal — Fri, 11 Feb 2022 12:08:08 +0000

A way to configure a multi-tenant environment in Grafana is to use organization to split each tenant. But, how can I configure this by IaC?

Grafana provides an active provisioning system that uses config files. Data sources and dashboards can be defined via files which are version controlled.

There are many tools to manage these config files:

Grafana provisioning allows the configuration of data sources, plugins, dashboards and alert notification channels. All of these “objects” can be created in a specific organization.

This is great, what else would you like?
I would like a little bit more. What happens with the organization or the users? Can I configure them by IaC?
Yes, you can, and Terraform is going to help with that.

Grafana provider

Grafana has an official Terraform provider which includes resources for users and organizations.

Multi-tenant configuration

For managing resources in different organizations with Terraform you have to configure Grafana’s provider with the organization ID.

For example:

provider "grafana" {
  url  = "http://127.0.0.1:3000"
  auth = "admin:admin"
  org_id = 1
}

So, the idea is having two different providers using alias. The first creates an organization and an admin user with the principal admin user. And the second uses organization and users created in the previous step.

Example:

provider "grafana" {
  url  = "http://127.0.0.1:3000"
  auth = "admin:admin"
  alias = "admin"
}
provider "grafana" {
  url  = "http://127.0.0.1:3000"
  auth = "admin_org_2:pass_org_2"
  org_id = 2
  alias = "config"
}

Full example

Enough theory, let’s take a practical example. For this you need:

Terraform
Kind
Cloning this repository: https://github.com/jilgue/medium-grafana-multi-tenant
Kubie (not necessary)

Deploy Grafana in Kubernetes cluster:

$ kind create cluster
$ kubie ctx kind-kind
$ cd 010-environment/010-grafana
$ terraform init
$ terraform apply
$ terraform output admin_password
$ kubectl port-forward service/grafana 3000

Our Grafana is accessible from http://127.0.0.1:3000. Let’s create a new organization with its admin user and resources (a folder for this example)

$ cd ../../020-client-1/010-grafana-config/
$ terraform init
$ terraform apply
$ terraform output password

Now logging with admin or client-1 user and switching the organization we will see the folder created.

http://localhost:3000/dashboards?orgId=2

How to create custom pod with Kubebuilder

César M. Cristóbal — Fri, 11 Feb 2022 11:59:55 +0000

In this post we’ll implement a simple Kubernetes controller using the kubebuilder.

Kubebuilder has a book but I think that it is too complex for beginning users. I’m going to try to do it easier. We’ll implement a simple operator which manage a pod.

Install Kubebuilder

Kubebuilder doesn’t support Go 1.17, so we need to install Go 1.16. I decided to use goenv to manager Go versions.

Remove previous Go version
Install goenv: Install goenv: https://github.com/syndbg/goenv/blob/master/INSTALL.md
Install Go 1.16: $ goenv install 1.16.8
Use this version: $ goenv global 1.16.8
Install Kubebuilder:

$ curl -L -o ~/.local/bin/kubebuilder https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)
$ chmod a+x ~/.local/bin/kubebuilder

Initialize a project and create new API

We have to follow three steps to create a new white project with one custom resource.

$ mkdir medium-kubebuilder-pod
$ cd !$
$ git init
$ go mod init simplepod
$ kubebuilder init --domain callepuzzle.com
$ git add .
$ git commit -m "init"
$ kubebuilder create api --group medium --version v1alpha1 --kind SimplePod
$ git add .
$ git commit -m "create api"

Great, we have all the necessary scaffolding for our project.

If we now run make install, kubebuilder should generate the base CRDs under config/crd/bases and a few other files for us. Running make run should now allow us to launch the operator locally.

$ make install
/home/cesar/projects/k8s-operator/medium-kubebuilder-pod/bin/controller-gen "crd:trivialVersions=true,preserveUnknownFields=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go: creating new go.mod: module tmp
Downloading sigs.k8s.io/kustomize/kustomize/v3@v3.8.7
go get: added sigs.k8s.io/kustomize/kustomize/v3 v3.8.7
/home/cesar/projects/k8s-operator/medium-kubebuilder-pod/bin/kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/simplepods.medium.callepuzzle.com created$ kubectl get customresourcedefinitions.apiextensions.k8s.io simplepods.medium.callepuzzle.com 
NAME                                CREATED AT
simplepods.medium.callepuzzle.com   2021-10-13T15:47:52Z$ kubectl apply -f config/samples/medium_v1alpha1_simplepod.yaml$ kubectl get simplepods.medium.callepuzzle.com 
NAME               AGE
simplepod-sample   9s

Ok, we can create “simplepod” resources but it doesn’t do anything, doesn’t have any logic.

Make our custom resource

In api/v1alpha1/simplepod_types.go is defined struct of our resource.

$ git diff api/v1alpha1/simplepod_types.go config/samples/medium_v1alpha1_simplepod.yaml
diff --git a/api/v1alpha1/simplepod_types.go b/api/v1alpha1/simplepod_types.go
index f5f3fde..e4f0630 100644
--- a/api/v1alpha1/simplepod_types.go
+++ b/api/v1alpha1/simplepod_types.go
@@ -29,7 +29,7 @@ type SimplePodSpec struct {
        // Important: Run "make" to regenerate code after modifying this file

        // Foo is an example field of SimplePod. Edit simplepod_types.go to remove/update
-       Foo string `json:"foo,omitempty"`
+       Command string `json:"command,omitempty"`
 }

 // SimplePodStatus defines the observed state of SimplePod
diff --git a/config/samples/medium_v1alpha1_simplepod.yaml b/config/samples/medium_v1alpha1_simplepod.yaml
index 671c617..dd8cda0 100644
--- a/config/samples/medium_v1alpha1_simplepod.yaml
+++ b/config/samples/medium_v1alpha1_simplepod.yaml
@@ -4,4 +4,4 @@ metadata:
   name: simplepod-sample
 spec:
   # Add fields here
-  foo: bar
+  command: ls

Every time we change the struct of our resource we have to run make install to regenerate the manifests.

Now, we implement the logic of our operator. Create a pod object and execute the command given by simplepod object.

$ git diff controllers/simplepod_controller.go
diff --git a/controllers/simplepod_controller.go b/controllers/simplepod_controller.go
index 2f13668..89b63d0 100644
--- a/controllers/simplepod_controller.go
+++ b/controllers/simplepod_controller.go
@@ -18,7 +18,10 @@ package controllers

 import (
        "context"
+       "strings"

+       corev1 "k8s.io/api/core/v1"
+       metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
        "k8s.io/apimachinery/pkg/runtime"
        ctrl "sigs.k8s.io/controller-runtime"
        "sigs.k8s.io/controller-runtime/pkg/client"
@@ -36,6 +39,7 @@ type SimplePodReconciler struct {
 //+kubebuilder:rbac:groups=medium.callepuzzle.com,resources=simplepods,verbs=get;list;watch;create;update;patch;delete
 //+kubebuilder:rbac:groups=medium.callepuzzle.com,resources=simplepods/status,verbs=get;update;patch
 //+kubebuilder:rbac:groups=medium.callepuzzle.com,resources=simplepods/finalizers,verbs=update
+//+kubebuilder:rbac:groups=core,resources=pods,verbs=get;list;watch;create;update;patch;delete

 // Reconcile is part of the main kubernetes reconciliation loop which aims to
 // move the current state of the cluster closer to the desired state.
@@ -47,16 +51,62 @@ type SimplePodReconciler struct {
 // For more details, check Reconcile and its Result here:
 // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.8.3/pkg/reconcile
 func (r *SimplePodReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
-       _ = log.FromContext(ctx)
+       log := log.FromContext(ctx)

-       // your logic here
+       var instance mediumv1alpha1.SimplePod
+       errGet := r.Get(ctx, req.NamespacedName, &instance)
+       if errGet != nil {
+               log.Error(errGet, "Error getting instance")
+               return ctrl.Result{}, client.IgnoreNotFound(errGet)
+       }
+
+       pod := NewPod(&instance)
+
+       _, errCreate := ctrl.CreateOrUpdate(ctx, r.Client, pod, func() error {
+               return ctrl.SetControllerReference(&instance, pod, r.Scheme)
+       })
+
+       if errCreate != nil {
+               log.Error(errCreate, "Error creating pod")
+               return ctrl.Result{}, nil
+       }
+
+       err := r.Status().Update(context.TODO(), &instance)
+       if err != nil {
+               return ctrl.Result{}, err
+       }

        return ctrl.Result{}, nil
 }

+func NewPod(pod *mediumv1alpha1.SimplePod) *corev1.Pod {
+       labels := map[string]string{
+               "app": pod.Name,
+       }
+
+       return &corev1.Pod{
+               ObjectMeta: metav1.ObjectMeta{
+                       Name:      pod.Name,
+                       Namespace: pod.Namespace,
+                       Labels:    labels,
+               },
+               Spec: corev1.PodSpec{
+                       Containers: []corev1.Container{
+                               {
+                                       Name:    "busybox",
+                                       Image:   "busybox",
+                                       Command: strings.Split(pod.Spec.Command, " "),
+                               },
+                       },
+                       RestartPolicy: corev1.RestartPolicyOnFailure,
+               },
+       }
+}
+
 // SetupWithManager sets up the controller with the Manager.
 func (r *SimplePodReconciler) SetupWithManager(mgr ctrl.Manager) error {
        return ctrl.NewControllerManagedBy(mgr).
                For(&mediumv1alpha1.SimplePod{}).
+               Owns(&corev1.Pod{}).
                Complete(r)
 }

Every time we change the struct of our resource we have to run make install to regenerate the manifests.

Now, we implement the logic of our operator. Create a pod object and execute the command given by simplepod object.

$ git diff controllers/simplepod_controller.go
diff --git a/controllers/simplepod_controller.go b/controllers/simplepod_controller.go
index 2f13668..89b63d0 100644
--- a/controllers/simplepod_controller.go
+++ b/controllers/simplepod_controller.go
@@ -18,7 +18,10 @@ package controllers

 import (
        "context"
+       "strings"

+       corev1 "k8s.io/api/core/v1"
+       metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
        "k8s.io/apimachinery/pkg/runtime"
        ctrl "sigs.k8s.io/controller-runtime"
        "sigs.k8s.io/controller-runtime/pkg/client"
@@ -36,6 +39,7 @@ type SimplePodReconciler struct {
 //+kubebuilder:rbac:groups=medium.callepuzzle.com,resources=simplepods,verbs=get;list;watch;create;update;patch;delete
 //+kubebuilder:rbac:groups=medium.callepuzzle.com,resources=simplepods/status,verbs=get;update;patch
 //+kubebuilder:rbac:groups=medium.callepuzzle.com,resources=simplepods/finalizers,verbs=update
+//+kubebuilder:rbac:groups=core,resources=pods,verbs=get;list;watch;create;update;patch;delete

 // Reconcile is part of the main kubernetes reconciliation loop which aims to
 // move the current state of the cluster closer to the desired state.
@@ -47,16 +51,62 @@ type SimplePodReconciler struct {
 // For more details, check Reconcile and its Result here:
 // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.8.3/pkg/reconcile
 func (r *SimplePodReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
-       _ = log.FromContext(ctx)
+       log := log.FromContext(ctx)

-       // your logic here
+       var instance mediumv1alpha1.SimplePod
+       errGet := r.Get(ctx, req.NamespacedName, &instance)
+       if errGet != nil {
+               log.Error(errGet, "Error getting instance")
+               return ctrl.Result{}, client.IgnoreNotFound(errGet)
+       }
+
+       pod := NewPod(&instance)
+
+       _, errCreate := ctrl.CreateOrUpdate(ctx, r.Client, pod, func() error {
+               return ctrl.SetControllerReference(&instance, pod, r.Scheme)
+       })
+
+       if errCreate != nil {
+               log.Error(errCreate, "Error creating pod")
+               return ctrl.Result{}, nil
+       }
+
+       err := r.Status().Update(context.TODO(), &instance)
+       if err != nil {
+               return ctrl.Result{}, err
+       }

        return ctrl.Result{}, nil
 }

+func NewPod(pod *mediumv1alpha1.SimplePod) *corev1.Pod {
+       labels := map[string]string{
+               "app": pod.Name,
+       }
+
+       return &corev1.Pod{
+               ObjectMeta: metav1.ObjectMeta{
+                       Name:      pod.Name,
+                       Namespace: pod.Namespace,
+                       Labels:    labels,
+               },
+               Spec: corev1.PodSpec{
+                       Containers: []corev1.Container{
+                               {
+                                       Name:    "busybox",
+                                       Image:   "busybox",
+                                       Command: strings.Split(pod.Spec.Command, " "),
+                               },
+                       },
+                       RestartPolicy: corev1.RestartPolicyOnFailure,
+               },
+       }
+}
+
 // SetupWithManager sets up the controller with the Manager.
 func (r *SimplePodReconciler) SetupWithManager(mgr ctrl.Manager) error {
        return ctrl.NewControllerManagedBy(mgr).
                For(&mediumv1alpha1.SimplePod{}).
+               Owns(&corev1.Pod{}).
                Complete(r)
 }

Then, install our changes and run the operator:

$ go get k8s.io/api/core/v1@v0.20.2
$ make install
$ kubectl delete -f config/samples/medium_v1alpha1_simplepod.yaml
$ kubectl apply -f config/samples/medium_v1alpha1_simplepod.yaml
$ make run
(in another terminal)
$ kubectl get pod
NAME               READY   STATUS      RESTARTS   AGE
simplepod-sample   0/1     Completed   0          3s
$ kubectl logs simplepod-sample 
bin
dev
etc
home
proc
root
sys
tmp
usr
var

Entire final code: https://github.com/jilgue/medium-kubebuilder-pod

Running Nextcloud on OPNsense

César M. Cristóbal — Fri, 11 Feb 2022 11:52:26 +0000

In this article we’re going to see how to run Nextcloud on a OPNsense.

The first question is:

Why do you do this?

Well, I haven’t used Google services for many years, except maps. I haven’t found a good alternative to Google Maps. I had a Nextcloud installed on a virtual machine hosted in Hetzner. It’s ok, it worked perfectly. But I needed more resources, more space for photos and videos, so I decided to migrate Nextcloud to my home.

I have a OPNsense router and firewall for my home. It has enough memory, so why have another device running when I can just use Nextcloud?

The next question is:

Can OPNsense run Nextcloud?

At first, I thought about using Docker, but the FreeBSD port of Docker project hasn’t been updated in 6 years.

The second idea was to try to install Nextcloud on FreeBSD using the OPNsense ports. Bad idea. Nextcloud has a lot of dependencies, it’s too complex to install and impossible to maintain.

And then I found this article https://yom.iaelu.net/2020/05/freebsd-using-docker-and-kubernetes/. Really? Can I have a virtual machine inside OPNsense and run Docker inside this virtual machine?

Perfect, welcome to Inception.

Create a Debian virtual machine

The project we’re going to use is https://github.com/churchers/vm-bhyve

# cd /usr/ports/sysutils/vm-bhyve/
# make install clean
# cd /usr/ports/sysutils/grub2-bhyve/
# make install clean
# sysrc vm_enable="YES"
# mkdir -p /srv/vm
# sysrc vm_dir="/srv/vm"
# vm init
# curl -o /srv/vm/.templates/debian.conf https://raw.githubusercontent.com/churchers/vm-bhyve/master/sample-templates/debian.conf
# vm switch create -p -a 192.168.23.0/24 bridge
# vm create -t debian -s 80G debian

Ok, at this point we need to enabled the interface created by vm-bhyve and to allow the network traffic in OPNsense.

All ready to install Debian:

# vm install debian debian-10.9.0-amd64-netinst.iso
# vm console debian

In the network configuration step, we have to select manual configuration and set a static IP.

Install Nextcloud

Perfect, we have a Debian machine with its own IP. The good thing about this method is that I can use the same Ansible playbook to install Nextcloud that I used in previous machine hosted in Hetzner:

[In the network configuration step, we have to select manual configuration and set a static IP.
Install Nextcloud

Perfect, we have a Debian machine with its own IP. The good thing about this method is that I can use the same Ansible playbook to install Nextcloud that I used in previous machine hosted in Hetzner:

https://github.com/CallePuzzle/ansible-playbook-nextcloud-debian

Switchable graphics Nvidia / Intel in Linux

César M. Cristóbal — Fri, 11 Feb 2022 11:38:56 +0000

Many laptops have two graphics; Intel, which saves power, and Nvidia, which gives better performance.

There are several options to use switchable graphics, PRIME, Bumblebee, nvidia-xrun…

None of these programs functions perfectly, there is always a problem. For example, Nvidia is still active in the background so there is not any power saving or Nvidia works but with any issue (games in Steam doesn’t work or not well)

At the end I choose to play any video game and then find a way to disable Nvidia when I want.

Following Archlinux Nvidia guide, I created an Ansible playbook to enable and disable Nvidia

---
- name: Nvidia DRM switch (disable)
  hosts: localhost
  become: yes
  tasks:

    - name: X11
      ansible.builtin.file:
        path: /etc/X11/xorg.conf.d/10-nvidia-drm-outputclass.conf
        state: absent

    - name: SDDM
      blockinfile:
        path: /usr/share/sddm/scripts/Xsetup
        insertafter: |
          #!/bin/sh
          # Xsetup - run as root before the login dialog appears
        block: ""

    - name: GRUB
      ansible.builtin.lineinfile:
        path: /etc/default/grub
        regexp: '^GRUB_CMDLINE_LINUX_DEFAULT='
        line: GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet resume=/dev/mapper/Vol-swap"
      register: grub

    - name: Modules
      ansible.builtin.lineinfile:
        path: /etc/mkinitcpio.conf
        regexp: '^MODULES='
        line: MODULES=()
      register: modules

    - name: Backlist modules
      blockinfile:
        path: /etc/modprobe.d/blacklist.conf
        block: |
          blacklist nvidia
          blacklist nvidia_modeset
          blacklist nvidia_uvm
          blacklist nvidia_drm
    - name: Re-generate the grub.cfg
      ansible.builtin.command: grub-mkconfig -o /boot/grub/grub.cfg
      when: grub.changed

    - name: Re-generate image
      ansible.builtin.command: mkinitcpio -P
      when: modules.changed

---
- name: Nvidia DRM switch (enable)
  hosts: localhost
  become: yes
  tasks:

    - name: X11
      ansible.builtin.copy:
        src: "{{ playbook_dir }}/10-nvidia-drm-outputclass.conf"
        dest: /etc/X11/xorg.conf.d/10-nvidia-drm-outputclass.conf
        owner: root
        group: root
        mode: '0644'

    - name: SDDM
      blockinfile:
        path: /usr/share/sddm/scripts/Xsetup
        insertafter: |
          #!/bin/sh
          # Xsetup - run as root before the login dialog appears
        block: |
          xrandr --setprovideroutputsource modesetting NVIDIA-0
          xrandr --auto
    - name: GRUB
      ansible.builtin.lineinfile:
        path: /etc/default/grub
        regexp: '^GRUB_CMDLINE_LINUX_DEFAULT='
        line: GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet resume=/dev/mapper/Vol-swap nvidia-drm.modeset=1"
      register: grub

    - name: Modules
      ansible.builtin.lineinfile:
        path: /etc/mkinitcpio.conf
        regexp: '^MODULES='
        line: MODULES=(nvidia nvidia_modeset nvidia_uvm nvidia_drm)
      register: modules

    - name: Re-generate the grub.cfg
      ansible.builtin.command: grub-mkconfig -o /boot/grub/grub.cfg
      when: grub.changed

    - name: Re-generate image
      ansible.builtin.command: mkinitcpio -P
      when: modules.changed

The changes require a reboot the laptop but since I don’t play every day this issue is acceptable for me.

There are two more commands to execute when you are using the battery:

# echo '\_SB.PCI0.PEG0.PEGP._OFF' > /proc/acpi/call
# rmmod nvidia

These commands increase three or four hour using the battery, I will research how to add them to my /etc/acpi/handler.sh script to run automatically.

How to save power consumption in Linux

César M. Cristóbal — Fri, 11 Feb 2022 11:31:50 +0000

Powertop is a great tool to save battery power and with the help of another tool, acpid, to observe the acpi event and configure it automatically.

Acpid comes with predefined actions for triggered events. By default, these actions are defined in /etc/acpi/handler.sh

The action “ac_adapter” has a default option

*)
                logger "ACPI action undefined: $2"

So, whether or not you are plugged in the ac, you can see the log with:

journalctl -f

ene 12 20:54:49 msi root[3888]: ACPI action undefined: ACPI0003:00

In this case you must to change the /etc/acpi/handler.sh with:

ac_adapter)
        case "$2" in
            ACPI0003:00)

Saving the changes and unplug the ac, the following appears in the log:

ene 12 21:00:31 msi root[6101]: AC unpluged

The change works. Now you can add the powertop commands. This is my handler.sh:

ac_adapter)
        case "$2" in
            ACPI0003:00)
                case "$4" in
                    00000000)
                        powertop --auto-tune
                        echo 'on' > '/sys/bus/usb/devices/3-2/power/control' # USB USB Receiver [Logitech]
                        echo 'enabled' > '/sys/class/net/wlo1/device/power/wakeup'
                        echo 'enabled' > '/sys/bus/usb/devices/usb1/power/wakeup'
                        echo 'enabled' > '/sys/bus/usb/devices/usb2/power/wakeup'
                        echo 'enabled' > '/sys/bus/usb/devices/usb3/power/wakeup'
                        echo 'enabled' > '/sys/bus/usb/devices/usb4/power/wakeup'
                        echo 'enabled' > '/sys/bus/usb/devices/3-2/power/wakeup'
                        echo 'enabled' > '/sys/bus/usb/devices/3-5/power/wakeup'
                        echo 'enabled' > '/sys/bus/usb/devices/3-10/power/wakeup'
                        logger 'AC unpluged'
                        ;;
                    00000001)
                        echo 'disabled' > '/sys/class/net/wlo1/device/power/wakeup'
                        echo 'disabled' > '/sys/bus/usb/devices/usb1/power/wakeup'
                        echo 'disabled' > '/sys/bus/usb/devices/usb2/power/wakeup'
                        echo 'disabled' > '/sys/bus/usb/devices/usb3/power/wakeup'
                        echo 'disabled' > '/sys/bus/usb/devices/usb4/power/wakeup'
                        echo 'disabled' > '/sys/bus/usb/devices/3-2/power/wakeup'
                        echo 'disabled' > '/sys/bus/usb/devices/3-5/power/wakeup'
                        echo 'disabled' > '/sys/bus/usb/devices/3-10/power/wakeup'
                        logger 'AC pluged'
                        ;;

Remember, this doesn’t work if you start the laptop without an ac. How do you solve it?
Inspired by the following post I created a new systemd unit:

/usr/lib/systemd/system/acpid-boot.service

[Unit]
Description=ACPI boot handle
Documentation=man:acpid(8)

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/acpi-boot-handle.sh

[Install]
WantedBy=multi-user.target

And the script /usr/local/sbin/acpi-boot-handle.sh contains:

#!/usr/bin/env bash
set -euo pipefail

if [[ $(acpi -a | grep on) ]]
then
        onBit=1
else
        onBit=0
fi
/etc/acpi/handler.sh ac_adapter ACPI0003:00 foo 0000000$onBit

In this way you simulate the acpi action when the laptop starts.

In my case a MSI Modern 14 the battery has 3 hours more when all of the powertop options are enabled.

Como ejecutar varios tests en Terratest

César M. Cristóbal — Fri, 11 Feb 2022 09:31:27 +0000

Terratest es un framework para ejecutar test en Terraform. En este caso yo tenía un módulo muy sencillo que compone un nombre y quería probar que todas las posibles entradas funcionasen.

A la hora de ejecutar varios tests en Terratest, lo primero que se me vino a la cabeza fue crear varias funciones con cada caso que quería probar:

package test

import (
    "testing"

    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestTerraformComplete(t *testing.T) {
    t.Parallel()

    terraformOptions := &terraform.Options{
        TerraformDir: "../",

        Vars: map[string]interface{}{
            "type": "avs",
            "company": "hb01",
            "environment": "hub01",
            "project": "shs01",
            "location": "weu",
            "function": "dns",
            "num": "01",
        },
    }

    terraform.InitAndApply(t, terraformOptions)

    name := terraform.Output(t, terraformOptions, "name")
    assert.Equal(t, name, "avs-hb01-hub01-shs01-weu-dns-01")
}

func TestTerraformNoFunction(t *testing.T) {
    t.Parallel()

    terraformOptions := &terraform.Options{
        TerraformDir: "../",

        Vars: map[string]interface{}{
            "type": "avs",
            "company": "hb01",
            "environment": "hub01",
            "project": "shs01",
            "location": "weu",
            "num": "01",
        },
    }

    terraform.InitAndApply(t, terraformOptions)

    name := terraform.Output(t, terraformOptions, "name")
    assert.Equal(t, name, "avs-hb01-hub01-shs01-weu-01")
}

Con la sorpresa de que apareció un mensaje de error como este:

TestTerraformNoFunction 2020-08-11T22:35:11+02:00 logger.go:66: avs-hb01-hub01-shs01-weu-dns-01
    terraform_azure_test.go:52: 
            Error Trace:    terraform_azure_test.go:52
            Error:          Not equal: 
                            expected: "avs-hb01-hub01-shs01-weu-dns-01"
                            actual  : "avs-hb01-hub01-shs01-weu-01"

                            Diff:
                            --- Expected
                            +++ Actual
                            @@ -1 +1 @@
                            -avs-hb01-hub01-shs01-weu-dns-01
                            +avs-hb01-hub01-shs01-weu-01
            Test:           TestTerraformNoFunction
--- FAIL: TestTerraformNoFunction (4.74s)
TestTerraformComplete 2020-08-11T22:35:11+02:00 logger.go:66: avs-hb01-hub01-shs01-weu-dns-01
--- PASS: TestTerraformComplete (4.77s)
FAIL
exit status 1
FAIL    test    4.780s

Después de revisarlo no veía error en el código, pero parecía que estuviese mezclando las variables de las dos funciones.

¿Cuál era el problema? Pues resultó ser el “t.Parallel()”. Yo sé de Golang lo justo para pasar el día pero tiene toda la pinta de que el ejecutar cada paso en paralelo Terraform se vuelve un poco loco.

Solución:

Quitar la ejecución en paralelo

package test

import (
    "testing"

    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestTerraformComplete(t *testing.T) {

    terraformOptions := &terraform.Options{
        TerraformDir: "../",

        Vars: map[string]interface{}{
            "type": "avs",
            "company": "hb01",
            "environment": "hub01",
            "project": "shs01",
            "location": "weu",
            "function": "dns",
            "num": "01",
        },
    }

    terraform.InitAndApply(t, terraformOptions)

    name := terraform.Output(t, terraformOptions, "name")
    assert.Equal(t, name, "avs-hb01-hub01-shs01-weu-dns-01")
}

func TestTerraformNoFunction(t *testing.T) {

    terraformOptions := &terraform.Options{
        TerraformDir: "../",

        Vars: map[string]interface{}{
            "type": "avs",
            "company": "hb01",
            "environment": "hub01",
            "project": "shs01",
            "location": "weu",
            "num": "01",
        },
    }

    terraform.InitAndApply(t, terraformOptions)

    name := terraform.Output(t, terraformOptions, "name")
    assert.Equal(t, name, "avs-hb01-hub01-shs01-weu-01")
}

Ejecutar los tests desde una misma función

package test

import (
    "testing"

    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestTerraform(t *testing.T) {
    t.Parallel()

    Complete(t)
    NoFunction(t)
}

func Complete(t *testing.T) {

    terraformOptions := &terraform.Options{
        TerraformDir: "../",

        Vars: map[string]interface{}{
            "type": "avs",
            "company": "hb01",
            "environment": "hub01",
            "project": "shs01",
            "location": "weu",
            "function": "dns",
            "num": "01",
        },
    }

    terraform.InitAndApply(t, terraformOptions)

    name := terraform.Output(t, terraformOptions, "name")
    assert.Equal(t, name, "avs-hb01-hub01-shs01-weu-dns-01")
}

func NoFunction(t *testing.T) {

    terraformOptions := &terraform.Options{
        TerraformDir: "../",

        Vars: map[string]interface{}{
            "type": "avs",
            "company": "hb01",
            "environment": "hub01",
            "project": "shs01",
            "location": "weu",
            "num": "01",
        },
    }

    terraform.InitAndApply(t, terraformOptions)

    name := terraform.Output(t, terraformOptions, "name")
    assert.Equal(t, name, "avs-hb01-hub01-shs01-weu-01")
}

Espero que os sirva de ayuda :)

Desplegar Nextcloud en Docker usando Ansible

César M. Cristóbal — Fri, 11 Feb 2022 08:46:05 +0000

Nextcloud nos permite tener nuestro propio servidor de archivos a lo Dropbox. Pero a mí lo que más me gusta es poder tener mi calendario y mis contactos independiente del todopoderoso Google.

Vamos a ver como podemos desplegar Nextcloud en una Debian usando Docker y todo ello automatizado con Ansible.

El esquema que vamos a tener es una máquina Debian con tres contenedores dentro:

haproxy
nextcloud
mariadb

Requisitos

Tener acceso root a la máquina Debian.
Tener instalado Ansible.

A mí personalmente me gusta instalar Ansible en un virtualenv de python:

$ virtualenv3 venv/py3
$ source venv/py3/bin/activate
(py3)$ pip install ansible

Instalamos los roles necesarios:

(py3)$ ansible-galaxy install jilgue.ansible_role_docker_nextcloud
(py3)$ ansible-galaxy install jilgue.ansible_role_docker_haproxy

Playbook

A continuación vamos a generar la configuración de Ansible.

Necesitamos generar el inventario:

nextcloud ansible_ssh_host=192.168.56.20

También la configuracion del haproxy:

global
    daemon
    maxconn 256

defaults
    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend http-in
    bind *:80
    mode http
    default_backend servers

backend servers
    server server1 nextcloud:80 maxconn 32

Y por último el playbook:

---
- hosts: all
  become: true
  become_method: sudo

  vars:
    nextcloud_trusted_domains: 192.168.56.20
    nextcloud_ports: []
    haproxy_cfg_template_path: "{{ playbook_dir }}/"
    haproxy_links:
      - nextcloud
    haproxy_ports:
      - 80:80

  roles:
    - { role: nextcloud, tags: [ 'nextcloud' ] }
    - { role: haproxy, tags: [ 'haproxy' ] }

Y con esto tendríamos todo, al aplicar el playbook y entrar en la url http://192.168.56.20/ podremos ver nuestro nextcloud instalado.

El usuario y contraseña por defecto son admin / admin.

Podéis ver la configuración de mi propio servidor en github.

Autenticar nginx mediante el método service-to-service en GCP

César M. Cristóbal — Fri, 11 Feb 2022 08:35:22 +0000

Imaginad que estamos montado una serie de microservicios en Cloud Run donde el endpoint es un nginx y no queremos tener el resto de servicios públicos.

Google nos provee de un método de autenticación service-to-service. Perfecto, ¿cómo hacemos eso en nginx? Pues usando el módulo ngx_http_auth_request_module y el njs scripting language.

El código que necesitamos en nginx es el siguiente:

js_include conf.d/auth.js;

server {
    listen 80 default_server;
    server_name _;

    location / {
        auth_request /_oauth2;
        auth_request_set $authorization $sent_http_authorization;
        proxy_set_header Authorization $authorization;
        proxy_pass https://patinando-int-api-tfezajqgva-ew.a.run.app;
    }


    location = /_oauth2 {
        internal;
        js_content introspectAccessToken;
    }

    location = /_oauth2_send_request {
        internal;
        proxy_set_header  Metadata-Flavor Google;
        proxy_pass http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience=https://patinando-int-api-tfezajqgva-ew.a.run.app;
    }
}

function introspectAccessToken(r) {

    r.subrequest("/_oauth2_send_request",
                 function(reply) {
                     if (reply.status == 200) {
                         r.headersOut['AUTHORIZATION'] = 'Bearer ' + reply.responseBody
                         r.status = 204;
                         r.sendHeader();
                         r.finish();
                     } else {
                         r.return(401);
                     }
                 }
                );
}

En nuestro caso nuestro servicio interno es https://patinando-int-api-tfezajqgva-ew.a.run.app y si lanzamos un wget obtenemos un bonito 403:

HTTP request sent, awaiting response… 
 HTTP/1.1 403 Forbidden
 Date: Sun, 03 May 2020 11:36:33 GMT
 Content-Type: text/html; charset=UTF-8
 Server: Google Frontend
 Content-Length: 295
2020–05–03 11:36:33 ERROR 403: Forbidden

Lo que estamos haciendo con este código es decir que toda petición se tiene que autorizar con /_oauth2 que es una llamada interna y contiene el código de la función en javascript.

El código javascript a su vez llama a /_oauth2_send_request para obtener el token.

Para poder usar la cabecera que seteamos en javascript con

r.headersOut[‘AUTHORIZATION’] = ‘Bearer ‘ + reply.responseBody

es necesario añadir estas líneas en el location principal

auth_request_set $authorization $sent_http_authorization;
proxy_set_header Authorization $authorization;

Si ahora hacemos peticiones al nginx la api nos devolverá un 200:

HTTP/1.1 200 OK
 Server: nginx/1.17.10
 Date: Sun, 03 May 2020 11:44:56 GMT
 Content-Type: application/json
 Content-Length: 5436
 Connection: keep-alive
 vary: Authorization
 x-powered-by: PHP/7.4.5
 cache-control: no-cache, private

Referencias:

https://www.nginx.com/blog/validating-oauth-2-0-access-tokens-nginx/

Configurar Emacs usando Doom

César M. Cristóbal — Fri, 11 Feb 2022 08:02:07 +0000

¿Por qué Emacs?

Emacs fue el primer IDE que aprendí a usar, por aquel entonces todos programábamos en la misma máquina conectándonos por ssh.

Más tarde nos "modernizamos" y empezamos a usar PhpStorm. Así que cuando dejé Php de lado y empecé a programar en Terraform, Ansible y Python, opté por usar PyCharm ya que es un productor de la misma compañía y el cambio iba a ser más sencillo.

Ahora bien el mundo JetBrains tiene su pega, para cada lenguaje necesitas un maldito editor distinto, y cuando empecé a hacer cosillas en Go me negué a instalar otro más.

Vale, Emacs mola mucho y todo lo que tu quieras pero configurarlo es un dolor. Por eso opté por probar Doom.

Doom

Doom es un framework para configurar Emacs.

Básicamente nuestra configuración se guarda en ~/.doom.d/ donde tenemos tres ficheros:

init.el: desde el cual activas o desactivas extensiones
packages.el: desde el que puedes instalar paquetes de MELPA, ELPA o emacsmirror
config.el: donde va tu configuración personal

y ejecutando ~/.emacs.d/bin/doom sync se generará nuestra configuración en ~/.emacs.d/.

Instalación

Seguimos los pasos del getting started y al arrancar por primera vez Emacs veremos algo como esto:

Configuración

Después de jugar un poco me di cuenta de que había demasiados paquetes instalados que hacían demasiadas cosas que no sabía usar y me “estorbaban”.

Me habían aconsejado fuertemente que probase la extensión evil pero de momento prefiero el modo de edición de emacs “puro”. Me quedo sin galletas :(

Así que comenté las siguientes extensiones:

evil +everywhere
+bindings +smartparens

Además de los lenguajes que fuese a utilizar.
Añadimos la configuración para borrar los espacios en blanco al guardar:

;; Hooks
(add-hook ‘before-save-hook ‘delete-trailing-whitespace)

Arrancamos el servidor, de manera que luego podamos abrir un fichero desde otra terminal:

;; Server
(server-start)

$ emacsclient -n .bashrc

Añadimos keybindings que para mí son necesarios:

;; Keybindings
(global-set-key “\M-p” ‘backward-paragraph)
(global-set-key “\M-n” ‘forward-paragraph)
(global-set-key (kbd “C-7”) ‘undo)
(global-set-key (kbd “C-x p”) ‘counsel-projectile-switch-project)

Y con esto ya estaría para funcionar.

Autocompletado

Company ya viene por defecto y en principio con activar lsp sería suficiente. Aunque esta parte aún la tengo algo verde y hay cosas que no terminan de ir fino.

NeoTree

Es una extensión para ver el árbol de ficheros y directorios de un proyecto. Para activarlo con su extensión de iconos hay que realizar los siguientes cambios:

instalar “all-the-icons”

(package! all-the-icons)

descomentar “neotree” del init.el
añadir la siguiente configuración:

;; NeoTree
(setq neo-theme (if (display-graphic-p) ‘icons ‘arrow))
(global-set-key (kbd “C-x t”) ‘neotree-show)
(setq projectile-switch-project-action ‘neotree-projectile-action)

Mi configuración al final ha quedado así:

Ver repositorio en GitHub