DEV Community: Iván Moreno

How to setup a remote development environment with code-server and nextcloud

Iván Moreno — Wed, 09 Feb 2022 19:07:03 +0000

In this tutorial we are going to setup a complete remote work environment with the ability to sync our workspace directory across multiple devices using nextcloud and code-server as web editor.

We can use this work environment if we need to use a powerful computer using a cloud server or just to keep our environment from anywhere at any time on the internet.

Prerequisites

In this tutorial you need to have the following components

A kubernetes cluster (I recommend to use k3s.io)
A nextcloud installation (you can install via helm within the same k8s cluster)
A k8s storage class where you can save your files

All kubernetes manifest files are stored in this repository feel free to clone and customize

System Architecture

This is the system architecture used in this tutorial

We are going to use a kubernetes storage class to save our files and share between code-server and nc-client deployments.

Configure nextcloud sync client

In order to sync all out files across multiple devices we are going to use nextcloud as sync system, in this tutorial we used the nextcloudcmd command to sync our files using a docker container.

The docker used syncs the nextcloud remote directory every X seconds, you need to set the NC_USER, NC_PASS and NC_HOST in the k8s/conf/nextcloud-client-conf.yaml file, the NC_PASS is a secret file, so you'll need to encode as base64 (example: echo -n 'verysecretpass' | base64).

Create the Persistent Volume Claim for nextcloud client



kubectl apply -f k8s/01-pvc-nextcloud-client.yaml

Create the environment variables for the nc-client container



kubectl apply -f k8s/conf/nextcloud-client-conf.yaml

Create the nc-client deployment



kubectl apply -f k8s/conf/nextcloud-client-conf.yaml

This container will sync the files in the nextcloud server every 300 seconds

Configure vscode on the server

In this tutorial we are using the linuxserver/code-server which includes the web version of visual studio code (oss version) and is highly customizable due to s6-overlay system. This docker image allow us to extend the capabilities of the base image using docker mods.

You can setup your environment creating your own docker mod, a docker mod can bootstrap your environment and install your required packages such as kubectl, helm, awscli, etc. Here is an example of my docker mod used to bootstrap my work environment, this mod install the required packages and bootstrap the config files in the container.

If you want to setup your own docker mod I highly recommend to check the docker-mods documentation for more info.

Docker mods

In this tutorial I'm using my own docker mod, this docker mod install the required packages such as psql, awscli, python, kubectl, helm, zsh, etc. This docker mod also bootstrap the config files such a gitconfig, aws credentials, zshrc file, ssh keys, kube config file, etc. This files are mounted in the code-server pod then the bootstrap scrip will copy these files in her location with the correct owner and permissions.

We are using the linuxserver/mods:code-server-extension-arguments to install the required vscode extensions in the code-server pod. The docker mods can be set in the DOCKE_MODS variable separated by |. Check the list of official mods for code-server



apiVersion: v1
kind: ConfigMap
metadata:
  name: code-server-configmap
  namespace: prod
data:
...
  DOCKER_MODS: "linuxserver/mods:code-server-extension-arguments|ghcr.io/ivanmorenoj/lsio-mods:code-server-ws"
...

Configure code-server on k8s

Create the Persistent Volume Claim for code-server deployment



kubectl apply -f k8s/02-pvc-code-server.yaml

Configure the code-server values such the user and sudo password, the docker mods and the config files.



kubectl apply -f k8s/conf/code-server-conf.yaml

Create the code-server deployment, you can modify the resource request and limits according to your available compute resources



kubectl apply -f k8s/04-code-server.yaml

Configure the ingress

In this step we are going to setup the ingress in order to access to our code-server installation through a FQDN domain with TLS endpoint.



kubectl apply -f k8s/05-ingress.yaml

NOTE: Make sure you have pointed your domain to your k8s cluster and configured your firewall to access to your installation.

Results

When you finished the setup you'll have a remote environment with your files synchronized in a remote server with nextcloud just like this:

Now you can edit, test and build your projects in your browser from anywhere at anytime, you only need a stable internet connection and a web browser.

Understand how linux containers works with practical examples

Iván Moreno — Sun, 18 Apr 2021 06:02:41 +0000

Nowadays a bast majority of server workloads run using linux containers because of his flexibility and lightweight but have you ever think how does linux containers works. In this tutorial we will demystify how does linux containers works with some practical examples. Linux containers works thanks two kernel features: namespaces and cgroups.

Linux Namespaces
Linux control groups (cgroups)
Container Fundamentals (key technologies)
- Process namespace fundamentals
- Filesystem Overlay FS fundamentals
- Networking Linux bridge fundamentals
- Control groups (cgroups) fundamentals
Create a container from scratch
Inspect Namespaces within a docker container
- Install docker CE
- Inspect Docker Network
- Inspect cgroups in a docker container
- Inspect overlay fs in a docker container
- Inspect docker process namespace
Conclusion

Linux Namespaces

A namespace wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource. Changes to the global resource are visible to other processes that are members of the namespace, but are invisible to other processes. One use of namespaces is to implement containers. [1]

Currently the linux kernel have 8 types of namespaces:

Namespace	Isolates
cgroup	Cgroup root directory
IPC	System V IPC, POSIX message queues
Network	Network devices, stacks, ports, etc.
Mount	Mount points
PID	Process IDs
Time	Boot and monotonic clocks
User	User and group IDs
UTS	Hostname and NIS domain name

Linux control groups (cgroups)

Cgroups allow you to allocate resources — such as CPU time, system memory, network bandwidth, or combinations of these resources — among user-defined groups of tasks (processes) running on a system. You can monitor the cgroups you configure, deny cgroups access to certain resources, and even reconfigure your cgroups dynamically on a running system. [2]

Container Fundamentals (key technologies)

In this section we gonna make some practices with the following key technologies that make possible the usage of containers in linux:

Process namespace fundamentals
Filesystem Overlay FS fundamentals
Networking Linux bridge fundamentals
Control groups (cgroups) fundamentals

NOTE: This tutorial was made using a VM with 1GB of ram and 1vCPU using debian 10 buster with kernel 4.19.0-16-amd64. All commands were executed using root privileges

Process namespace fundamentals

A process namespace isolate a running command from the host. Let's see how to implement a process namespace in linux.

List process namespaces

$ lsns -t pid

Get the PID of the current terminal

$ echo $$ # parent PID

Launch a new zsh terminal using namespaces

$ unshare --fork --pid --mount-proc zsh
$ sleep 300 &
$ sleep 300 &
$ sleep 300 &
$ sleep 300 &
$ sleep 300 &
$ top

See the process tree from the parent

$ ps f -g <PPID>

List namespaces

$ lsns -t pid

Filesystem Overlay FS fundamentals

Containers need tohave a filesystem, one of the most used filesystem for containers is overlay who can mount with layers and merge in a single directory, the lower layers are read only and all changes are made on the upper layer. Let's see how does overlay fs works.

Create directories

$ cd /tmp
$ mkdir {lower1,lower2,upper,work,merged}

Create some files in lower directories

$ echo "Lower 1 - original" > lower1/file1.txt
$ echo "Lower 2 - original" > lower2/file2.txt

Create overlay FS

$ mount -t overlay -o lowerdir=/tmp/lower1:/tmp/lower2,upperdir=/tmp/upper,workdir=/tmp/work none /tmp/merged

Create, modify files

$ cd /tmp/merged
$ echo "file created in merged directory" > file_created.txt
$ echo "file 1 modified" > file1.txt

Umount overlay fs

$ cd /tmp
$ umount /tmp/merged

Inspect lower and upper dirs

$ find -name '*.txt' -type f 2>/dev/null | while read fn; do echo ">> cat $fn"; cat $fn; done

Networking Linux bridge fundamentals

Linux container uses network namespaces to isolate the network from the host, this is possible implementing a bridge interface that acts like network switch, and every container connect to that interface with his own ip address. Let's see how does linux bridge and network namespaces works.

Create a Network Virtual bridge

$ ip link add br-net type bridge

List Network Interfaces

$ ip link

Assign an IP Address to bridge interface

$ ip addr add 192.168.55.1/24 brd + dev br-net

Bring UP the bridge interface

$ ip link set br-net up

Create 2 Network Namespaces

$ ip netns add ns1
$ ip netns add ns2

Create a Virtual Ethernet cable pair

$ ip link add veth-ns1 type veth peer name br-ns1
$ ip link add veth-ns2 type veth peer name br-ns2

Assign veth to namespaces

$ ip link set veth-ns1 netns ns1
$ ip link set veth-ns2 netns ns2
$ ip link set br-ns1 master br-net
$ ip link set br-ns2 master br-net

Assign IP address to veth within namespaces

$ ip -n ns1 addr add 192.168.55.2/24 dev veth-ns1
$ ip -n ns2 addr add 192.168.55.3/24 dev veth-ns2

Bring UP veth interfaces within Namespaces

$ ip -n ns1 link set lo up
$ ip -n ns2 link set lo up
$ ip -n ns1 link set veth-ns1 up
$ ip -n ns2 link set veth-ns2 up

Bring UP bridge veth in the local host

$ ip link set br-ns1 up
$ ip link set br-ns2 up

Configure default route within namespaces

$ ip -n ns1 route add default via 192.168.55.1 dev veth-ns1 
$ ip -n ns2 route add default via 192.168.55.1 dev veth-ns2

Enable IP forward in the host

$ sysctl -w net.ipv4.ip_forward=1

Configure MASQUERADE in the host for 192.168.55.0/24 subnet

$ iptables -t nat -A POSTROUTING -s 192.168.55.0/24 ! -o br-net -j MASQUERADE

Check connectivity within namespaces

$ ip netns exec ns1 ping -c 3 192.168.55.3 # ping ns2
$ ip netns exec ns2 ping -c 3 192.168.55.2 # ping ns1
$ ip netns exec ns1 ping -c 3 192.168.55.1 # ping br-net gateway
$ ip netns exec ns2 ping -c 3 192.168.55.1 # ping br-net gateway
$ ip netns exec ns1 ping -c 3 1.1.1.1 # ping internet
$ ip netns exec ns2 ping -c 3 1.1.1.1 # ping internet

Control groups (cgroups) fundamentals

Control groups or cgroups are used by containers to limit the usage of resource in the host machine. Let's see how does cgroups works.

Create cgroups directory

$ mkdir -p /mycg/{memory,cpusets,cpu}

Mount cgroups directory

$ mount -t cgroup -o memory none /mycg/memory
$ mount -t cgroup -o cpu,cpuacct none /mycg/cpu
$ mount -t cgroup -o cpuset none /mycg/cpusets

Create new directories under CPU controller

mkdir -p /mycg/cpu/user{1..3}

Assign CPU shares to every user (This example uses 1vCPU)

# 2048 / (2048 + 512 + 80) = 77%
$ echo 2048 > /mycg/cpu/user1/cpu.shares
# 512 / (2048 + 512 + 80) = 19%
$ echo 512 > /mycg/cpu/user2/cpu.shares
# 80 / (2048 + 512 + 80) = 3%
$ echo 80 > /mycg/cpu/user3/cpu.shares

Create artificial load

$ cat /dev/urandom &> /dev/null &
$ PID1=$!
$ cat /dev/urandom &> /dev/null &
$ PID2=$!
$ cat /dev/urandom &> /dev/null &
$ PID3=$!

Assign process to every user

$ echo $PID1 > /mycg/cpu/user1/tasks
$ echo $PID2 > /mycg/cpu/user2/tasks
$ echo $PID3 > /mycg/cpu/user3/tasks

Monitoring process

$ top

Create a container from scratch

So far we know how does linux namespaces works, now lets create a container using overlayfs, network namespaces, cgroups and process namespaces from scratch. Let's see how a linux container is created.

Download and extract debian container fs from docker

$ docker pull debian
$ docker save debian -o debian.tar
$ mkdir debian_layer
$ mkdir -p fs/{lower,upper,work,merged}
$ tar xf debian.tar -C debian_layer
$ find debian_layer -name 'layer.tar' -exec tar xf {} -C fs/lower \;

Create bridge interface

$ ip netns add cnt
$ ip link add br-cnt type bridge
$ ip addr add 192.168.22.1/24 brd + dev br-cnt
$ ip link set br-cnt up
$ sysctl -w net.ipv4.ip_forward=1
$ iptables -t nat -I POSTROUTING 1 -s 192.168.22.0/24 ! -o br-cnt -j MASQUERADE

Create overlay Filesystem from debian container fs

$ mount -vt overlay -o lowerdir=./fs/lower,upperdir=./fs/upper,workdir=./fs/work none ./fs/merged

Mounting Virtual File Systems

$ mount -v --bind /dev ./fs/merged/dev

Launch process namespace within fs/merged fs

$ unshare --fork --pid --net=/var/run/netns/cnt chroot ./fs/merged \
    /usr/bin/env -i PATH=/bin:/usr/bin:/sbin:/usr/sbin TERM="$TERM" \
    /bin/bash --login +h
# Mount proc within container
$ mount -vt proc proc /proc

Connect the container with br-cnt

$ ip link add veth-cnt type veth peer name br-veth-cnt
$ ip link set veth-cnt netns cnt
$ ip link set br-veth-cnt master br-cnt
$ ip link set br-veth-cnt up
$ ip -n cnt addr add 192.168.22.2/24 dev veth-cnt
$ ip -n cnt link set lo up
$ ip -n cnt link set veth-cnt up
$ ip -n cnt route add default via 192.168.22.1 dev veth-cnt
$ ip netns exec cnt ping -c 3 1.1.1.1

Mount cgroup

$ mkdir /sys/fs/cgroup/memory/cnt
$ echo 10000000 > /sys/fs/cgroup/memory/cnt/memory.limit_in_bytes
$ echo 0 > /sys/fs/cgroup/memory/cnt/memory.swappiness
$ CHILD_PID=$(lsns -t pid | grep "[/]bin/bash --login +h" | awk '{print $4}')
$ echo $CHILD_PID > /sys/fs/cgroup/memory/cnt/tasks

Run commands within container

$ apt update
$ apt install nginx procps curl -y
$ nginx
$ curl 127.0.0.1:80
$ curl 192.168.22.2:80 # from host
$ cat <( </dev/zero head -c 15m) <(sleep 15) | tail

Clean all

$ umount /proc # within container
$ exit # within container
$ umount -R ./fs/merged
$ ip link del br-veth-cnt
$ ip link del br-cnt
$ ip netns del cnt # grep cnt /proc/mounts

Inspect Namespaces within a docker container

Fortunately for us there is a program that simplifies the usage of containers, for us this program is docker who manage the life-cycle of running a container. Let's see how does docker implement the namespaces running a container.

Install docker CE

Install docker community edition from official script in get.docker.com

$ curl -fsSL https://get.docker.com -o install_docker.sh
$ less install_docker.sh # optional
$ sh install_docker.sh
$ usermod -aG docker $USER
$ newgrp docker # Or logout and login

Inspect Docker Network

Create a bridge network using docker

$ docker network create mynet

Inspect bridge network, see subnet using IP

$ BR_NAME=$(ip link | grep -v '@' | awk '/br-/{gsub(":",""); print $2}')
$ ip addr show ${BR_NAME}

Inspect Docker bridge network, see subnet using docker

$ docker network inspect mynet | grep Subnet

Run an nginx web server

$ docker run --name nginx --net mynet -d --rm -p 8080:80 nginx

Inspect network namespace from nginx container

Create symlink from /proc to /var/run/netns

$ CONTAINER_ID=$(docker container ps | awk '/nginx/{print $1}')
$ CONTAINER_PID=$(docker inspect -f '{{.State.Pid}}' ${CONTAINER_ID})
$ mkdir -p /var/run/netns/
$ ln -sfT /proc/${CONTAINER_PID}/ns/net /var/run/netns/${CONTAINER_ID}

Check network interface within namespace

$ ip netns list
$ ip -n ${CONTAINER_ID} link show eth0

Check IP address of nginx container

$ ip -n ${CONTAINER_ID} addr show eth0
$ docker container inspect nginx | grep IPAddress

Check port forwarding from 8080 to 80

$ iptables -t nat -nvL

Inspect cgroups in a docker container

Run a Ubuntu container with limited resources

$ docker run --name test_cg --memory=10m --cpus=.1 -it --rm ubuntu

See cgroup fs hierarchy

$ CONTAINER_ID=$(docker container ps --no-trunc | awk '/test_cg/{print $1}')
$ tree /sys/fs/cgroup/{memory,cpu}/docker/${CONTAINER_ID}

See attached task to container cgroup

$ docker container top test_cg | tail -n 1 | awk '{print $2}' # container parent PID
$ cat /sys/fs/cgroup/{memory,cpu}/docker/${CONTAINER_ID}/tasks # the same as container parent PID

Monitoring the container

$ docker container stats test_cg

Generate CPU load

$ cat /dev/urandom &> /dev/null

Generate Memory load

$ cat <( </dev/zero head -c 50m) <(sleep 30) | tail

Inspect overlay fs in a docker container

Run a ubuntu container with limited resources

$ docker run --name test_overlayfs -it --rm debian

NOTE: The merged layer is the actual container Filesystem

Inspect lower layers with tree and less

$ docker container inspect test_overlayfs -f '{{.GraphDriver.Data.LowerDir}}' | awk 'BEGIN{FS=":"}{for (i=1; i<= NF; i++) print $i}' | while read low; do tree -L 2 $low; done | less

Inspect upper layer (It's empty)

$ docker container inspect test_overlayfs -f '{{.GraphDriver.Data.UpperDir}}' | while read upper; do tree $upper; done | less

Run command withing the container

$ apt update && apt install nmap -y

Inspect (again) upper layer (now it's not empty)

$ docker container inspect test_overlayfs -f '{{.GraphDriver.Data.UpperDir}}' | while read upper; do tree $upper; done | less

Inspect docker process namespace

Run docker container

$ docker run --name test_ps -it --rm ubuntu

Launch process within container

$ sleep 600 &
$ sleep 600 &
$ sleep 600 &
$ sleep 600 &
$ sleep 600 &
$ top

See container tree process from container

$ CONTAINER_PID=$(docker container top test_ps | sed -n '2p' | awk '{print $2}')
$ ps f -g ${CONTAINER_PID}

List PID namespaces

$ lsns -t pid

See process using docker

$ docker container top test_ps

Conclusion

In this tutorial we create our first container from scratch understanding what happen behind the scenes when we run a container. I hope this tutorial helps you to understand the technologies behind the linux containers.

Source code

How to deploy pihole and wireguard on kubernetes (DNS over HTTPS)

Iván Moreno — Mon, 01 Mar 2021 13:51:43 +0000

In this tutorial we are going to deploy a DNS ad-blocker with pihole using a recursive DNS as upstream server accessible in LAN network and internet through wireguard vpn. We utilized K3S as kubernetes provider.

Assumptions

To follow this tutorial you need to have:

A computer running any linux distribution (in this tutorial I'm using debian buster)
This computer needs to have at least 1 Gb of ram and 2 cores.
This computer needs to have an static ip address (in my case 192.168.10.10, LAN: 192.168.10.1/24)
Configure the firewall to accept inbound connections in the following ports:
- 53/tcp
- 53/udp
- 80/tcp
- 443/tcp
- 51820/udp
In order to connect from internet, you need to open the following ports in your ISP's Router:
- 80/tcp (Pihole Web Interface HTTP)
- 443/tcp (Pihole Web Interface HTTPS)
- 51820/tcp (Wireguard VPN)
In order to connect to pihole via web UI from internet you need to have a FQDN that points to your Network (in this tutorial I'm using google domains, you can use duckdns if you don't have a domain)
A functional K3S cluster (See Installation guide). This tutorial should work with K8S, but you need to deploy some extra features like Traefik Ingress, Klipper Service Load Balancer
You need to have kubectl to perform operations in the K3S cluster

System Architecture

In the following image you can see the diagram that we want to implement. We can access to pihole in our local network (192.168.10.1/24) and access to admin interface through internet, finally we can access to cluster resources via VPN trough Wireguard or just route dns queries.

Resources

All manifests are in this GitHub repository feel free to clone and customize

Methodology

Namespaces

The first thing that we need to configure is the namespace we need to create 2 namespaces: pihole for pihole, unbound and wireguard. cert-manager to deploy a cert manager in order to enable HTTPS on pihole UI interface.

$ kubectl apply -f k8s/01-namespaces.yaml

Dynamic DNS

If you don't have assigned an static ip address, you can point to your public ip provided by your ISP using a dynamic dns, in my case I'm using a google domains, so i can update the subdomain with curl. I created a cronJob that updates every 5 min the public ip assigned by my ISP.

$ kubectl apply -f k8s/02-dynamic-dns-google.yaml

Create directories

To store config files you need to create 2 directories in the master node.

$ sudo mkdir -p /var/lib/{pihole,wireguard}

Pihole

Pihole (as you probably already know) is a dns add blocker. In order to configure Pihole and make accessible in our LAN network we need to configure a LoadBalancer service in 53/tcp-udp that binds in the local host. The web UI will be accessible through Ingress controller (If you don't want that just use a NodePort service in pihole-ui-svc)

This manifest comes with three different configurations:

Pihole with upstream servers
Pihole with unbound
Pihole with DNS over HTTP(S) with cloudflare DNS servers

Common configurations:

Persistent volume type hostPath (/var/lib/pihole)
Persistent Volume claim
Config Map (Timezone, admin email and upstream dns)
Secret (Web Password)
Pihole Deployment
Cluster IP Services (Pihole UI, Pihole DNS)
LoadBalancer Services (Pihole DNS bind to host)

You need to configure:

Time Zone (pihole-configmap)
Admin email (pihole-configmap)
Web Password (pihole-secret)

Pihole with upstream servers

This configuration forwards the dns queries to an upstream server like cloudflare dns 1.1.1.1 and google dns 8.8.8.8 with no encryption.

To apply this configuration use the following command

$ kubectl apply -f k8s/03-pihole-upstream-dns.yaml

Pihole with unbound

Unbound is a validating, recursive, caching DNS resolver. In order to resolve a dns query unbound queries to the root domain (.com), then queries to subdomain (example.com) and keeps doing til it found the desire domain (pihole.example.com). Is very useful to avoid some type of tracking in the popular dns servers (like google). In this tutorial we utilize this image mvance/unbound that comes pre-configured to works as recursive dns.

To apply this configuration use the following command

$ kubectl apply -f k8s/03-pihole-unbound.yaml

Test Unbound installation

Pihole with DNS over HTTP(S) with cloudflare DNS servers

This configuration forward DNS queries to cloudflare dns server over https, ie. all queries are being encrypted since they are using TLS to perform the connection with cloudflare. Your ISP does not know what DNS request you are making.

To apply this configuration use the following command

$ kubectl apply -f k8s/03-pihole-DoH-cloudflare.yaml

Test Pihole installation

WireGuard

WireGuard is a modern VPN server that works at kernel level. In this tutorial we utilize the vpn to access at LAN resources from internet or just to encrypt the dns queries to our pihole installation from internet.

This manifest comes with:

Persistent volume type hostPath (/var/lib/wireguard)
Persistent Volume claim
Config Map (Timezone, server url, number of peers, etc)
Wireguard StatefulSet
LoadBalancer Service (Wireguard listen port 51820/udp)

You need to configure:

Timezone (wireguard-configmap)
Server URL (wireguard-configmap)
Number of peers (wireguard-configmap)

$ kubectl apply -f k8s/04-wireguard.yaml

Wireguard Client config

You can see the logs of your wireguard pod to get the QR code for each peer as is showed in the following image

You can also copy the config files for each peer located at /var/lib/wireguard/config/peer<N>/peer<N>.conf

Extra (Ingress Configuration for Pihole UI)

This steps describe how to make the Pihole Web User Interface available from internet HTTP/HTTPS using Traefik Ingress. If you don't want that, just use a NodePort service in pihole-ui-svc.

Deploy cert-manager

We need to deploy a cert manager to serve TLS certificates for HTTPS.

$ kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml

Deploy Cluster Issuer (Let's Encrypt)

This manifest deploy a cluster issuer that points to Let's encrypt. You need to configure the email to get notifications.

$ kubectl apply -f k8s/05-cluster-issuer-letsencrypt.yaml

Ingress

This manifest comes with the domain of your pihole host. You need to configure the subdomain previously configured.

$ kubectl apply -f k8s/06-ingress.yaml

How to use

You can change your LAN DNS servers to point to your pihole installation. If you're in a VPN by default you route all of your traffic trough wireguard including your dns queries. If you only want to route your dns queries through wireguard you should only route the following routes in AllowedIPS: 10.0.0.0/8, 192.168.10.0/24 (replace whit your local network CIDR).

If you have been following this tutorial so far you will see the following resources on your kubernetes cluster

And the Pihole admin User Interface accessible through internet via FQDN

How to connect to kubernetes internal network using WireGuard

Iván Moreno — Fri, 13 Nov 2020 07:12:35 +0000

When you are testing your deployments in a kubernetes cluster on the cloud you have a few options to expose your services outside world, for example you can use a NodePort service, but also you need to configure the firewall rules for each NodePort service, the other type of service that you can use is LoadBalancer however each of them is billed by cloud provider. To solve this problem you can use a vpn running within your k8s cluster, this vpn can be exposed outside the cluster with a NodePort or LoadBalancer service. As client you can access to you kubernetes internal network using service FQDN in your local machine.

In this tutorial we gonna setup a pod that run wireguard server, this wireguard will be configured with the kube-dns service and generate cliente credentials automatically the diagram will be like this:

Assuming that you are in a testing k8s cluster in the cloud with multiple namespaces and services.

First we need to know the kube-dns IP address with the following command

$ kubectl -n kube-system get svc | grep kube-dns | awk '{print $3}'
# output example: 10.124.0.10

In order to isolate wireguard server from another apps, we need to create a wireguard namespace named wireguard

apiVersion: v1
kind: Namespace
metadata:
  name: wireguard
  labels:
    name: wireguard

To store wireguard config files, we need a persistent volume, in my case i’m using a gke managed service that provides me a storage class, so i’m gonna create a persistent volume claim to that storage class.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pv-claim-wireguard 
  namespace: wireguard
spec:
  storageClassName: "standard"
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10M

The next thing to configure is the environment variables of wireguard server, this will be do with a config map. The kube-dns IP from steps earlier will be set in PEERDNS field.

apiVersion: v1
kind: ConfigMap
metadata:
  name: wireguard-configmap
  namespace: wireguard
data:
  PUID: "1000"
  PGID: "1000"
  TZ: "America/Mexico_City"
  SERVERPORT: "31820"
  PEERS: "2"
  PEERDNS: "10.124.0.10"
  ALLOWEDIPS: "0.0.0.0/0, ::/0"
  INTERNAL_SUBNET: "10.13.13.0"

Now we can create the wireguard server pod, this pod needs to be privileged with NET_ADMIN and SYS_MODULE capabilities and needs to mount /lib/modules directory from the host. The image used is ghcr.io/linuxserver/wireguard from linuxserver.io

apiVersion: v1
kind: Pod
metadata:
  name: wireguard
  namespace: wireguard
  labels:
    app: wireguard
spec:
  containers:
  - name: wireguard
    image: ghcr.io/linuxserver/wireguard
    envFrom:
    - configMapRef:
        name: wireguard-configmap 
    securityContext:
      capabilities:
        add:
          - NET_ADMIN
          - SYS_MODULE
      privileged: true
    volumeMounts:
      - name: wg-config
        mountPath: /config
      - name: host-volumes
        mountPath: /lib/modules
    ports:
    - containerPort: 51820
      protocol: UDP
    resources:
      requests:
        memory: "64Mi"
        cpu: "100m"
      limits:
        memory: "128Mi"
        cpu: "200m"
  volumes:
    - name: wg-config
      persistentVolumeClaim:
        claimName: pv-claim-wireguard 
    - name: host-volumes
      hostPath:
        path: /lib/modules
        type: Directory

Finally to access to wireguard server, we need to create a service, this service could be a NodePort or LoadBalancer, in my case i used a NodePort service on port 31820, take in mind that you probably need to configure a firewall rule to access at this service.

kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: wireguard
  name: wireguard-service
  namespace: wireguard
spec:
  type: NodePort
  ports:
  - port: 51820
    nodePort: 31820
    protocol: UDP
    targetPort: 51820
  selector:
    app: wireguard

This configurations are in a single file wireguard-pod.yaml to execute just apply the file with kubectl command

$ kubectl apply -f wireguard-pod.yaml

The container generate a QR code for each peer, these QR appears in the logs of the pod, to see just type the following command

$ kubectl -n wireguard logs wwireguard

The output will be like this

In order to connect to wireguard server download mobile app of install in your local machine. See wireguard.com

You can scan the code with the mobile app or copy the config file in your computer at ~/peer1.conf

$ kubectl -n wireguard exec wireguard -- cat /config/peer1/peer1.conf > ~/peer1.conf

Now you can utilize the config file to activate the vpn. With NetworManager you can import the config file

$ nmcli connection import type wireguard file ~/peer1.conf

And activate or deactivate the connection

$ nmcli connection up peer1 
$ nmcli connection down peer1

Finally to access a ClusterIP service within k8s cluster just use the IP of ClusterIP service or use the FQDN of the service using the following rule

<clusterip-service>.<namespace>.svc.cluster.local

Check the output of dig in a FQDN inside a remote k8s cluster, note that the query is answered by kube-dns IP inside the k8s cluster.

For example to access a ClusterIP service named thingsboard-service in the namespace thingsboard at 9090 port from our local machine through wireguard vpn:

http://thingsboard-service.thingsboard.svc.cluster.local:9090

And the output in our local environment

Conclusion

This method is very useful for a managed kubernetes service in the cloud in a development environment because we can test our services without configure a nodeport for each service and his respectively firewall rule.

WARNING: Only use this method in a development environment, don't use in a production environment

Source Code

How to bypass a firewall using a VPS on AWS

Iván Moreno — Sun, 20 Sep 2020 07:00:30 +0000

When you browse at school or work you probably have experienced some problems browsing in certain websites or using certain applications. Is a common practice to block some kind of pages on internet at work or school environment also they block another kind of traffic such as torrents, adult-content or they just permit web traffic, you school or work don't want you to browse sites they haven't approved.

Types of firewall blocking

There are certain levels of firewall blocking, the most simple is based on blocking certain kind of web-pages. The other type of firewall blocking is based on protocols, they just allow certain kind of traffic such as web traffic, ssh, dns, etc. Almost every school or work place have both kind of firewall blocking, but there is more sophisticated blocking levels, in this article we only focus on this type of firewall blocking.

Who wants to bypass a firewall?

There are some many types of users who wants to bypass a firewall for different reasons, this reasons could be:

Wants to browse in certain kind of web pages
Wants to use specific application that are blocked such a torrent
Needs to access a school or work environment behind a firewall

There are different methods to bypass a firewall according to their usage. For the first kind of user who only wants to browse in blocked websites, the method could be a VPN or proxy. For the second kind of user who wants to use specific non web application, the method could be a VPN. The last kind of user wants to expose some service running behind firewall such a ssh server running at school or workplace, the method for this user could be ssh port forwarding or VPN with port forwarding on server-side.

Disclaimer

This tutorial was made for informational and educational purposes only. I cannot be held responsible for any misuse of the given information. If you plan to use the information for illegal purposes, please leave this tutorial now.

Before begin

In this tutorial we are gonna setup a VPN server, Proxy server and SSH server to bypass a firewall, normally some services are't blocked by the firewall such http, https, ssh and DNS it means that we can setup any service on these ports for example:

Squid proxy on port 80/tcp
SSH server on port 22/tcp
OpenVPN on port 443/tcp
Wireguard VPN on port 53/udp

Each of these services have advantages and disadvantages and serves a specific purpose for example you can use ssh tunnel as sock proxy, forward ports through ssh tunnel using ssh port forwarding this is useful but in some application and mobile devices this method is not enough. Using a proxy y very useful if you only want to browse in blocked web pages. A vpn is the best solution to navigate without restrictions, it works on desktop and mobile devices, you can forward traffic from internet to client or make available a private subnet connecting client to client through VPS server using a gateway device such raspberry pi. In this tutorial I setup 2 vpns, wireguard and openvpn, in my personal opinion wireguard is better than openvpn but wireguard does't work with tcp protocol, that's why i set wireguard at 53/udp and openvpn at 443/tcp one advantage of openvpn is his protocol that utilizes SSL/TLS for key exchange so in a packet analysis will appears like https traffic.

Configure VPS on AWS

For all services we need a virtual private server (VPS), this vps could be setup on any public cloud provider, in my case i chose aws because they offer one year of free trial on aws EC2.

The first step is create an account in aws, create a VPS, configure a key pair and security group. I choose ubuntu 20.04 with the following configuration.

In order to avoid a change os ip when the VPS restart i recommend to associate a static ip address, this can be set in elastic ip option as shown in the following image

Access to VPS

We need to download the key pair created steps earlier and move to ~/.ssh directory

$ mv ~/Downloads/aws.pem ~/.ssh/

Change file permissions to be only readable for user owner

$ chmod 400 ~/.ssh/aws.pem

Access to remote VPS

$ ssh -i ~/.ssh/aws.pem ubuntu@<public_ip_address>

Update the system

$ apt-get update && apt-get upgrade -y

Install needed packages

$ apt-get install wireguard squid fail2ban qrencode apache2-utils -y

Configure fail2ban

fail2ban is a tool that helps to prevent automated brute force attacks in services exposed to internet.

Add sshd jail in /etc/fail2ban/jail.d/sshd-jail.conf

[sshd]
enabled   = true
maxretry  = 3
findtime  = 1d
bantime   = 4w
ignoreip  = 127.0.0.1/8

Reload fail2ban service

$ systemctl restart fail2ban

Check sshd jail

$ fail2ban-client status sshd

Configure SSH server

In order to setup a ssh port forwarding we need to configure some options in /etc/ssh/sshd_config to allow TCP Forwarding and Gateway Ports

AllowTcpForwarding yes
GatewayPorts yes

Reload the ssh server

$ systemctl restart sshd

Now you can setup port forwarding through ssh tunnels, check this tutorial to create a systemd service with ssh tunnel.

Configure squid proxy on 80/tcp

Squid is a http proxy, it can be configured on port 80 to bypass firewall.
Configure squid proxy to listen in port 80 and basic user authentication.

Create user and password

$ htpasswd -c /etc/squid/passwd client1

Generate config file, first create a backup of original file

$ mv /etc/squid/squid.conf /etc/squid/squid.conf.bk

Generate squid config file in /etc/squid/squid.conf

$ cat <<EOF > /etc/squid/squid.conf
acl localnet src 0.0.0.1-0.255.255.255  # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8             # RFC 1918 local private network (LAN)
acl localnet src 100.64.0.0/10          # RFC 6598 shared address space (CGN)
acl localnet src 169.254.0.0/16         # RFC 3927 link-local (directly plugged) machines
acl localnet src 172.16.0.0/12          # RFC 1918 local private network (LAN)
acl localnet src 192.168.0.0/16         # RFC 1918 local private network (LAN)
acl localnet src fc00::/7               # RFC 4193 local private network range
acl localnet src fe80::/10              # RFC 4291 link-local (directly plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/passwd
auth_param basic children 5
auth_param basic realm Squid Basic Authentication
auth_param basic credentialsttl 2 hours
acl auth_users proxy_auth REQUIRED
http_access allow auth_users

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

include /etc/squid/conf.d/*

http_access allow localhost

http_access deny all

http_port 80

coredump_dir /var/spool/squid

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims
refresh_pattern \/Release(|\.gpg)$ 0 0% 0 refresh-ims
refresh_pattern \/InRelease$ 0 0% 0 refresh-ims
refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims
refresh_pattern .               0       20%     4320
EOF

Restart squid service

$ systemctl restart squid

Now you can configure a proxy in your browser, system or phone.

Configure Wireguard at 53/udp

Wireguard is modern VPN protocol that works on kernel-side, unfortunately only works on UDP ports and most of firewalls blocks outgoing udp connections except in some services such a DNS on 53/UDP so we'll configure wireguard to listen in 53/UDP.

First of all, we need to enable ip forwarding

$ sysctl -w net.ipv4.ip_forward=1

Make changes permanent uncomment this line net.ipv4.ip_forward=1 in /etc/syctl.conf

$ sed -i '/net.ipv4.ip_forward/s/^#//g' /etc/sysctl.conf

Most of systemd based distros uses systemd-resolve to resolve dns queries, this is a problem if we want to setup wireguard to listen on 53/udp, so we need to disable systemd-resolve, first of all check what process is listen on port 53/UDP.

$ ss -ulpn4

As we can see in the following image, the process who listen on port 53/UDP is systemd-resolve

Disable systemd-resolve

$ systemctl disable --now systemd-resolved

Set static dns server adding this lines on /etc/resolv.conf

nameserver 1.1.1.1
nameserver 1.0.0.1

Now you can check again who is using port 53/udp with this command

$ ss -ulpn4

A this point we need to generate a config file for wireguard server.
Before create a config file check what is you default interface in my case is eth0, modify wireguard config file according to your interface name

$ ip -o -4 route show to default | cut -d ' ' -f 5

Create wireguard config file at /etc/wireguard/wgserver.conf

[Interface]
Address = 10.20.0.1/24
ListenPort = 53
PrivateKey = <your_private_key>
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
SaveConfig = true

Generate wireguard private key and put in /etc/wireguard/wgserver.conf

$ sed -i "s/<your_private_key>/$(wg genkey)/" /etc/wireguard/wgserver.conf

Enable wireguard server

$ sudo systemctl enable --now wg-quick@wgserver

Generate client template at ~/wgclients/template.conf

[Interface]
PrivateKey = <private_key>
Address = 10.20.0.xxx/32

[Peer]
PublicKey = <server_public_key>
Endpoint = <public_ip_address>:53
AllowedIPs = 0.0.0.0/0, ::/0

Get server public key, replace this in <server_public_key> in ~/wgclients/template.conf file

$ wg show wgserver | awk '/public key:/{print $3}'

Set server public ip address in ~/wgclients/template.conf

$ sed -i "s/<public_ip_address>/$(curl -s ifconfig.me/ip)/" ~/wgclients/template.conf

Generate client and copy template

$ mkdir -p ~/wgcliets/client1
$ cp ~/wgclients/template.conf ~/wgcliets/client1/client1.conf

Generate client keys

$ cd ~/wgclients/client1
$ wg genkey | tee private-key | wg pubkey > public-key

Copy private key and paste in ~/wgclients/client1.conf file and set ip address. Add client public key in /etc/wireguard/wgserver.conf and tunnel ip address

[Peer]
PublicKey = <client_public_key>
AllowedIPs = 10.20.0.10/32

Reload config on-the-fly

$ su -c "wg addconf wgserver <(wg-quick strip wgserver)"

To check wireguard connection, download wireguard android app in PlayStore. Generate QR code from client config file and import in wireguard app.

$ qrencode -t ansiutf8 < ~/wgclients/client1/client1.conf

Now you can check connection typing ifconfig.me at mobile web browser it will appears the server public ip address

Configure OpenVPN at 443/tcp

OpenvPN is a vpn that uses a custom security protocol that utilizes SSL/TLS for key exchange so it could be set in HTTPS port since every web site uses SSL/TSL the firewall cannot block that port.

Install OpenVPN with automated script

$ git clone https://github.com/Nyr/openvpn-install.git
$ cd openvpn-install
$ bash openvpn-install.sh

Configure using the following parameters:

Use IPV4 address in eth0 in my case 172.31.14.144
Put public ipv4 address
Protocol: TCP
Port: 443
DNS: 1.1.1.1

Configurations is showed in the following image

The client config file will be placed on /root directory, copy to home

$ mkdir ~/ovpnclients
$ mv /root/client1.ovpn /home/ubuntu/ovpnclients/
$ chown ubuntu:ubuntu ~/ovpnclients/client1.ovpn

Download config file to your pc and transfer to android app

$ scp -i ~/.ssh/aws.pem ubuntu@<public_ip_address>:~/ovpnclients/client1.ovpn ~/ovpnclients

Import file in android app and connect, type ifconfig.me it will appears the server public ip address.

Conclusion

These methods will work in almost all scenarios, please use them at your own risk because these methods can open a security breach in your work environment.

How to install and configure your own VPN server in GCP with Wireguard

Iván Moreno — Sun, 16 Aug 2020 01:44:59 +0000

Security is one of the most important things now days specifically in enterprise environments, a vpn helps to encrypt traffic from client to internet. Wireguard is a vpn protocol than works on the kernel side and acts like a network interface, is one of the most modern vpn protocols it’s based in public and private key exchange just like ssh does. A vpn can connect different host with encrypted connection throught internet, this topology can connect a simple client in an android app or even connect different hosts across datacenters, for example to connect workers in Kubernetes/swarm cluster in different datacenters regions with encrypted connection without open any public API on internet, wireguard also can be used to connect different home based host across internet only opening one UDP port and use the vpn server as encrypted bridge between clients.
In this tutorial we focus on implementation of vpn server on Google Cloud Platform (GCP) with wireguard, this setup will use a centos 8 on the server-side, and the configuration of one client in android device.

Server-side setup

In the server side we will use CentOS 8 in GCP, the steps are:

Create a virtual instance
Setup ssh keys
Install Wireguard
Configure Wireguard Server
Configure clients

Create virtual host in GCP

In this step we need to have an account in Google Cloud Platform and create a vm instance in compute engine. In this step you need to select the vm resource of you preference, in my case I choose a general purpose virtual machine with E2 series processor, I select the e2-small configuration with 2 vCPU and 2GB of ram, finally I choose centos 8 with 20 GB of ssd storage, the configuration is showed in next image.

Setup ssh keys

In order to access to our virtual machine, we need to create a ssh key, the ssh key are create with this command

ssh-keygen -t rsa -f ~/.ssh/[KEY_FILENAME] -C [USERNAME]

And, restrict the access of the key

chmod 400 ~/.ssh/[KEY_FILENAME]

Get public ssh key and copy
Copy directly from command line output

cat ~/.ssh/[KEY_FILENAME].pub

Copy with xclip from command line

cat ~/.ssh/[KEY_FILENAME].pub | xclip -selection c

Paste the public key in GCP. Go to Compute Engine -> Metadata -> SSH keys -> Edit -> Add item, then paste generated public ssh key
Then paste in GCP and test ssh connection

ssh -i ~/.ssh/[KEY_FILENAME] [USER]@[PUBLIC_IP_ADDRESS]

Install Wireguard

Before install wireguard we need to update the system

sudo dnf update

Then, we need to install some repositories

sudo yum install elrepo-release epel-release

Finally, install wireguard kmod and wireguard tools

sudo yum install kmod-wireguard wireguard-tools

Configure Wireguard Server

Before creating the wireguard config file is necessary to generate public and private key
First change to root user

sudo su -

Go to /etc/wireguard directory

cd /etc/wireguard

Limit default file permission of root user

umask 077

Generate public and private key

wg genkey | tee private-key | wg pubkey > public-key

Create a wireguard config file in /etc/wireguard/wgserver.conf and add the following lines

[Interface]
Address = 10.50.0.1/24
SaveConfig = true
PostUp = firewall-cmd --zone=public --add-port 50555/udp && firewall-cmd --zone=public --add-masquerade && firewall-cmd --zone=trusted --add-interface=wgserver && firewall-cmd --zone=trusted --add-masquerade
PostDown = firewall-cmd --zone=public --remove-port 50555/udp && firewall-cmd --zone=public --remove-masquerade && firewall-cmd --zone=trusted --remove-interface=wgserver && firewall-cmd --zone=trusted --remove-masquerade
ListenPort = 50555
PrivateKey = <private key>

Config explained

Address: the address of subnet in wireguard interface, it must be a private ip address class a, b or c
SaveConfig: Update the config file when it added more users
PostUp: This setting is executed when the interface initializes, in this case configure the firewall
PostDown: This setting is executed when the interfaces shutdown, in this case remove firewall configuration
ListenPort: The port where wireguard is listen (wireguard only listen udp ports)
PrivateKey: Wireguard private key, this key was generated one step above in /etc/wireguard/private-key copy and paste in /etc/wireguard/wgserver.conf

Add firewall rules in GCP

Go to VPC network -> Firewall -> Create new firewall rule
To grant access to wireguard server add in source IP ranges 0.0.0.0/0, in protocols and ports add udp port on 50555 (ListenPort) then create the rule

Enable wireguard server at boot with systemd

systemctl enable --now wg-quick@wgserver

Check the status

systemctl status wg-quick@wgserver

The output will be like this

Configure clients

To create peers, we need to create a folder wgclients and one template for all clients
The ~/wgclients/template.conf file will be like this

[Interface]
PrivateKey = <client_private_key>
Address = 10.50.0.xxx/32
[Peer]
PublicKey = <server_public_key>
Endpoint = <server_public_ip_address>:<server_listen_port>
# to route all traffic through wireguard server 
# AllowedIPs = 0.0.0.0/0, ::/0
# to route only wireguard server subnet
AllowedIPs = 10.50.0.0/24

Then create a directory for each client, generate private and public key, copy template and replace client private key and address
Create client directory

mkdir ~/wgclients/client1

Copy template

cp ~/wgclients/template.conf ~/wgclients/client1/client1.conf

Generate client keys

wg genkey | tee private-key | wg pubkey > public-key

Copy client private key and paste in client config file

cat private-key | xclip -selection c

Edit ~/wgclients/client1/client1.conf and copy and paste client private key, the file will be like this

[Interface]
PrivateKey = QWERTfvCAJ5WgIqpCxOz9e7yYIzxOmB/PE1GBGNGJ29=
Address = 10.50.0.100/32
[Peer]
PublicKey = QWERTYvCAJ5WgIqpCxOz9e7yYIzxOmB/QWERTYNGJ20=
Endpoint = 32.54.69.87:50555
AllowedIPs = 10.50.0.0/24

Then add client public key to /etc/wireguard/wgserver.conf after Interface config add the following lines

[Peer]
PublicKey = QWERTYvCAJ5WgIqpCxOz9e7yYIzxOmB/QWERTYNGJ20=
# if client have static ip address put here, else omit the field
# Endpoint = 32.54.69.87:50555
AllowedIPs = 0.0.0.0/0, ::/0

Then reload config file in server-side

sudo su -c "wg addconf wgserver <(wg-quick strip wgserver)"

Check wireguard config status, will appers the new client

sudo wg show wgserver

Then test connection, in this case with an android app. First download wireguard android app on google play store, generate QR from client config file and load from app

Generate QR code from terminal

qrencode -t ansiutf8 < ~/wgclients/testclient/testclient.conf

Example of QR code generated from command line

If everything works fine, type ifconfig.me in android browser, it will appear the wireguard server address, from command line will appear the latest handshake and transfer data summary.

Check connection in the server-side

sudo wg show wgserver

Now you can add more android clients with the same method, for desktop clients I highly recommend to add PersistentKeepalive option in server and client side, in the desktop you can use NetworkManager to import a connection or use systemd based service, also you can implement client config in OpenWrt router.

How to encrypt an existing archlinux LVM installation [LVM on LUKS]

Iván Moreno — Sat, 15 Aug 2020 03:05:36 +0000

Many users need to secure their laptop, workstation or regular PC, this users want to protect their information. In Linux exist many cryptographic techniques to protect a hard disk, directory and partition, one of this techniques is Linux Unified Key Setup (LUKS) which uses the kernel device mapper subsystem via the dm-crypt module which make the encrypted device transparent to the user.

Assumptions

This tutorial only encrypt an existing LVM installation, the EFI and boot partitions are in a non-encrypted partition. In order to encrypt an existing LVM installation we need to have root permissions and an external temporally drive to allocate the entire volume group.
In this tutorial the system drive is called rootDrive and temporally drive is called tmpDrive, the LVM root partition (partition to encrypt) is called rootDriveXY, the temporally partition is called tmpDriveXY both are formatted as Linux LVM.
The entire system are installed in a single volume group, in this tutorial is called “vg-sys”.
You can realize the entire process without umount any partition because of flexibility of LVM, in summary this process can be do on-the-fly.

Warning

Only realize the process if you know what are you doing a bad execution can result in a system damage and data-loss. I recommend to realize a backup before starting this tutorial.

The specific case

I have an existing archlinux installation on LVM volumes, the entire system is in a single volume group “arch”, I have 5 logical volumes (swap, root, var, home and data) in a single physical volume, the entire disk have 3 partitions EFI (/dev/sdb1), boot (/dev/sdb2) and LVM (/dev/sdb3). In order to encrypt an existing LVM installation (/dev/sdb3) I have an empty temporally hard drive (/dev/sda1) with the minimum size of volume group allocated size, in my case the allocated size is 208 GiB (arch volume group), the temporally drive have 465 GiB (/dev/sda1), the temporally drive is formatted as Linux LVM, this setup is showing in the next image

Steps

The first step is to create a physical volume in temporally drive, extend the “vg-sys” volume group to temporally drive and move volume group to temporally drive.

Create physical volume



$ pvcreate /dev/tmpDriveXY

Extend "vg-sys" to temporally drive



$ vgextend vg-sys /dev/tmpDriveXY

Move volume group to temporally drive



$ pvmove /dev/rootDriveXY /dev/tmpDriveXY

Once the volume group is moved, reduce the rootDriveXY from volume group “vg-sys” and remove the physical drive to be encrypted /dev/rootDriveXY

Remove physical drive from volume group



$ vgreduce vg-sys /dev/rootDriveXY

Remove physical drive



$ pvremove /dev/rootDriveXY

Change the partition type of root drive, on GPT the code is 8309 for Linux LUKS



$ gdisk /dev/rootDrive

Change type with t, choose partition number
Set type 8309 for Linux LUKS
See changes whit p and write with w



$ gdisk -l /dev/rootDrive

Wipe the partition to prevent cryptographic attacks or unwanted file recovery

Create a temporary encrypted container



$ cryptsetup open --type plain -d /dev/urandom /dev/rootDeviceXY to_be_wiped

Verify



$ lsblk

Wipe with zeros, wait til "No space left on device" be patient, this step takes a long time



$ dd if=/dev/zero of=/dev/mapper/to_be_wiped bs=1M status=progress

Close wiped device



$ cryptsetup close to_be_wiped

A this point probably you want to check performance of encrypt algorithms, use the next command to run a benchmark



$ cryptsetup benchmark

I choose an aes-xts-plain64 cipher with a key size of 256 and sha256 hash. To encrypt the rootDriveXY use the next command

Encrypt device and set passphrase



$ cryptsetup -v --type luks --cipher aes-xts-plain64 --key-size 256 --hash sha256 --iter-time 2000 --use-urandom --verify-passphrase luksFormat /dev/rootDriveXY

Open encrypted device with passphrase, create a LVM physical volume, extend the “vg-sys” volume to encrypted physical volume and move the temporally drive to encrypted

Open encrypted volume



$ cryptsetup open /dev/rootDriveXY cryptarch

Create a physical LVM device



$ pvcreate /dev/mapper/cryptarch

Extend the "vg-sys" group



$ vgextend vg-sys /dev/mapper/cryptarch

Move volume group to encrypted drive



$ pvmove /dev/tmpDriveXY /dev/mapper/cryptarch

Now remove the temporally drive from LVM



$ vgreduce vg-sys /dev/tmpDriveXY

Remove physical volume from temporally drive



$ pvremove /dev/tmpDriveXY

Configure mkinitcpio, first edit /etc/mkinitcpio.conf file and change hooks like this



HOOKS=(base udev autodetect keyboard keymap consolefont modconf block encrypt lvm2 filesystems fsck)

Then recreate the initramfs image for linux-lts kernel



$ mkinitcpio -P linux-lts

Configuring the boot loader to unlock the encrypted root partition at boot, the following kernel parameter needs to be set by the boot loader, this can be set in /etc/default/grub

Get UUID of encrypted drive, in this case /dev/rootDriveXY NOT /dev/mapper/cryptarch



$ blkid

Add this line in GRUB_CMDLINE_LINUX option, in /etc/default/grub file



cryptdevice=UUID=rootDriveXY-UUID:cryptarch root=/dev/vg-sys/root

Uncoment GRUB_ENABLE_CRYPTODISK="y"
Regenerate /boot/grub/grub.cfg file



$ grub-mkconfig -o /boot/grub/grub.cfg

If everything goes fine reboot the system and in boot time the system should ask a password to open the encrypted rootDriveXY as shown in the next image

Finally I recommend to wipe the temporally drive to prevent unwanted data recovery.

How to setup WiFi hotspot in raspberry pi and connect with ESP8266

Iván Moreno — Fri, 14 Aug 2020 20:39:13 +0000

Basic connection between arduino and raspberry pi through WiFi

The Hardware is

Esp8266 with arduino sdk
Raspberry pi 3b+

System requirements

arduino-cli
picocom
python3

Configurations for Raspberry Pi

Install raspbian
Enable ssh server
Upgrade system
Configure keys for ssh server
Configure the raspberry pi as hotspot
Install python3 and set as default python interpreter
Copy server code

Configurations for ESP8266

Install esp8266 for arduino
Copy the code and upload to esp8266 board

Raspberry Pi Setup

Download raspbian buster lite from official repository

$ curl -LO http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2020-02-14/2020-02-13-raspbian-buster-lite.zip

Write to sd card

$ unzip -p 2020-02-13-raspbian-buster-lite.zip | sudo dd of=/dev/mmcblk0 status=progress

Connect the raspberry pi via Ethernet and HDMI, startup the system and access with user=pi and password=raspberry. Enable ssh service

$ sudo systemctl enable --now ssh

Check what is the ip address and logout from the system

$ ip addr show eth0 # copy ip address
$ exit

Connect via ssh

$ ssh pi@{ip_address}

Add user with the same privileges and groups to pi user, generate password and delete the pi user

$ sudo useradd -m -s /bin/bash --groups $(groups pi | cut -d ' ' -f 4- --output-delimiter ',') new-user

Create password to new-user

$ sudo passwd new-user

Add new-user to sudoers file

$ echo "new-user ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/010_new-user-nopasswd

$ su new-user

Check if sudo works

$ sudo apt update

Check groups

$ groups new-user

If everything works, logout and access to new-user with ssh, first logout

$ exit

Generate keys to new-user and copy to raspberry pi

$ ssh-keygen -f ~/.ssh/rpi-key

Copy public key to raspberry pi

$ ssh-copy-id -i ~/.ssh/rpi-key.pub new-user@{raspberry-pi-ip}

Now access to new-user on raspberry pi

$ ssh new-user@{raspberry-pi-ip}

Remove pi user

$ sudo userdel -rf pi

Update and upgrade the system

$ sudo apt update && sudo apt upgrade -y

Reboot the system

$ sudo reboot

Setup hotspot

Create an access point in raspberry pi

 Diagram                                  +- RPi ---------+
                                      +---+ 192.168.10.15 |          +- ESP8266 ---+
                                      |   |     WLAN AP   +-)))  (((-+ WLAN Client |
                                      |   | 192.168.7.1   |          | 192.168.7.x |
                                      |   +---------------+          +-------------+
                                      .
                                      .
                              .
                 +- Router -----+     |
                 | Firewall     |     |   +- PC#2 --------+
(Internet)---WAN-+ DHCP server  +-LAN-+---+ 192.168.10.11 |
                 | 192.168.10.1 |     |   +---------------+
                 +--------------+     |
                                      |   +- PC#1 --------+
                                      +---+ 192.168.10.10 |
                                          +---------------+

Install required packages

$ sudo apt-get -y install hostapd dnsmasq netfilter-persistent iptables-persistent

Enable wireless access point

$ sudo systemctl unmask hostapd
$ sudo systemctl enable hostapd

Define the wireless interface IP configuration
Configure wlan0 interface in /etc/dhcpcd.conf
Add the following lines

  interface wlan0
    static ip_address=192.168.7.1/24
    nohook wpa_supplicant

Enable ip forwarding in /etc/sysctl.conf and uncomment the following line

 net.ipv4.ip_forward=1

Allow traffic between clients on this foreign wireless network

$ sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Make persistent changes

$ sudo netfilter-persistent save

Configure the DHCP and DNS services for the wireless network
First create a backup

$ sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig

Edit /etc/dnsmasq.conf and add the following lines

  interface=wlan0 
  dhcp-range=192.168.7.2,192.168.7.40,255.255.255.0,24h
  domain=wlan
  address=/gw.wlan/192.168.7.1

Set WiFi country
Localization Options -> Change WLAN Country -> choose your country -> finish

$ sudo raspi-config

Ensure wireless operation

$ sudo rfkill unblock wlan

Configure access point in /etc/hostapd/hostapd.conf
Add the following lines

  country_code=MX
  interface=wlan0
  ssid=rpi-hotspot
  hw_mode=g
  channel=8
  macaddr_acl=0
  auth_algs=1
  ignore_broadcast_ssid=0
  wpa=2
  wpa_passphrase=rasp-p@s2w0rd
  wpa_key_mgmt=WPA-PSK
  wpa_pairwise=TKIP
  rsn_pairwise=CCMP

Reboot the system and check connection with new access point

Install python3 and set as default interpreter

$ sudo apt install -y python3
$ sudo rm /usr/bin/python
$ sudo ln -s /usr/bin/python3 /usr/bin/python

Copy pythonServer to raspberry pi

ESP8266 Configurations

Install required packages

arduino-cli
picocom

Install esp8266 arduino core
First add additional boards in ~/Arduino/arduino-cli.yml and add the following lines

# arduino-cli.yaml
board_manager:
  additional_urls:
    - http://arduino.esp8266.com/stable/package_esp8266com_index.json

Update index

$ arduino-cli --config-file ~/Arduino/arduino-cli.yml core update-index

Search esp8266 boards

$ arduino-cli --config-file ~/Arduino/arduino-cli.yml core search esp8266

Install esp8266 boards

$ arduino-cli --config-file ~/Arduino/arduino-cli.yml core install esp8266:esp8266

Compile and upload the sketch, assuming that port of esp8266 is /dev/ttyUSB0

$ arduino-cli compile -b esp8266:esp8266:nodemcuv2 -u -p /dev/ttyUSB0 arduinoClient/

Test communications

Init server in raspberry pi

$ python ~/pythonServer/server.py

Init serial monitor for esp8266 board

$ picocom -b 115200 /dev/ttyUSB0

Raspberry pi output

  Intialize socket server, listen port: 35000
  Connection from: ('192.168.7.17', 59552)
  From connected user: Init connection from ESP8266
  Write response -> hi from server
  From connected user: hi from client
  Write response -> send quit sequence
  Recive disconnect from client
  Close connection...

Arduino output

  Wait for WiFi... .....
  WiFi connected
  IP address:
  192.168.7.17
  Connecting to 192.168.7.1:35000
  Wait response from server ...
  Server response: hi from server
  You response -> hi from client
  Server response: send quit sequence
  You response -> quit!

Source Code

How to connect raspberry pi with HC-05 bluetooth module + arduino programm

Iván Moreno — Fri, 14 Aug 2020 20:10:22 +0000

Basic connection between arduino and raspberry pi through Bluetooth

The hardware

Raspberry pi 3b+ (it has inbuilt Bluetooth)
Arduino micro
HC-05 Bluetooth module

System requirements

arduino-cli
picocom
python 3

Configurations for Bluetooth module HC-05

Configure the module as master
Set PIN

Configurations for Raspberry Pi

Install and configure raspbian
Configure python3 as default python interpreter
Install Bluetooth packages
Pair with HC-05 module
Copy Bluetooth code

Configurations for arduino

Install avr boards
Make physical connections
Copy arduino code and upload to arduino board

HC-05 Setup

First connect Bluetooth module to USB serial and press EN button, then send AT commands.

Test AT commands in serial port from USB to serial converter. The default baud rate for AT commands is 38400

$ echo -e "AT\r" | picocom -b 38400 -qrx 1000 /dev/ttyUSB0
OK

Set with AT commands

$ echo -e "AT+NAME=ARDUINOBT\r" | picocom -b 38400 -qrx 1000 /dev/ttyUSB0
OK

Set password

$ echo -e "AT+PSWD=1379\r" | picocom -b 38400 -qrx 1000 /dev/ttyUSB0
OK

Set default baud rate

$ echo -e "AT+UART=115200,1,0\r" | picocom -b 38400 -qrx 1000 /dev/ttyUSB0
OK

Disconnect and test settings

Raspberry Pi Setup

Once installed raspbian and make the basic configurations, install Bluetooth required packages.

$ sudo apt install -y pi-bluetooth bluetooth bluez picocom blueman python3-pip

First edit Bluetooth service in /etc/systemd/system/dbus-org.bluez.service
Add the following lines

  ExecStart=/usr/lib/bluetooth/bluetoothd -C
  ExecStartPost=/usr/bin/sdptool add SP

Reload systemd units

$ sudo systemctl daemon-reload

Enable Bluetooth service

$ sudo systemctl enable --now bluetooth

Edit /etc/modules-load.d/modules.conf to load rfcomm module automatically
Add the following line

rfcomm

Reboot the system

Pair with HC-05 module

Access to Bluetooth console

$ sudo bluetoothctl

Pair with Bluetooth module

[bluetooth]# agent on
[bluetooth]# scan on
  Discovery started
  [NEW] Device 98:D3:31:50:4A:C1 98-D3-31-50-4A-C1
  [CHG] Device 98:D3:31:50:4A:C1 LegacyPairing: no
  [CHG] Device 98:D3:31:50:4A:C1 Name: ARDUINOBT
  [CHG] Device 98:D3:31:50:4A:C1 Alias: ARDUINOBT

[bluetooth]# scan off
  [CHG] Controller B8:27:EB:80:2D:06 Discovering: no
  Discovery stopped

[bluetooth]# pair 98:D3:31:50:4A:C1
  Attempting to pair with 98:D3:31:50:4A:C1
  [CHG] Device 98:D3:31:50:4A:C1 Connected: yes
  Request PIN code
  [agent] Enter PIN code: 1379
  [CHG] Device 98:D3:31:50:4A:C1 UUIDs: 00001101-0000-1000-8000-00805f9b34fb
  [CHG] Device 98:D3:31:50:4A:C1 ServicesResolved: yes
  [CHG] Device 98:D3:31:50:4A:C1 Paired: yes
  Pairing successful
  [CHG] Device 98:D3:31:50:4A:C1 ServicesResolved: no
  [CHG] Device 98:D3:31:50:4A:C1 Connected: no

[bluetooth]# trust 98:D3:31:50:4A:C1
  [CHG] Device 98:D3:31:50:4A:C1 Trusted: yes
  Changing 98:D3:31:50:4A:C1 trust succeeded

[bluetooth]# exit

Create a serial device

$ sudo rfcomm bind rfcomm0 <device's MAC>

If everything works, connect with picocom and test communication in both sides

Install pyserial library for python3

$ sudo pip3 install pyserial

Copy pythonClient dir to raspberry pi and execute client.py script

$ python pythonClient/client.py

Arduino Setup

Physical connections

Connections for HC-05 module

GND -> GND Arduino
VCC -> VCC Arduino
RX -> D14 Arduino
TX -> D15 Arduino
EN -> N/C

Compile and upload the sketch

Upload the code to arduino board, considering serial port of arduino micro is /dev/ttyACM0

$ arduino-cli compile -b arduino:avr:micro -u -p /dev/ttyACM0 arduinoBT-HC05/

Test Communications

Raspberry pi output example

developer@raspberrypi:~ $ python pythonClient/client.py
  Enter your message below.
  Insert "exit" to leave the application.
  You message >> hi from raspberry pi
  Server Response >> hi

  You message >> test connection
  Server Response >> test connection from arduino

  You message >> exit

Arduino Output example

$ picocom -b 115200 /dev/ttyACM0
  Terminal ready
  Server response: hi from raspberry pi
  You response -> hi
  Server response: test connection
  You response -> test connection from arduino

  Terminating...
  Skipping tty reset...
  Thanks for using picocom

DEV Community: Iván Moreno

How to setup a remote development environment with code-server and nextcloud

Prerequisites

System Architecture

Configure nextcloud sync client

Configure vscode on the server

Docker mods

Configure code-server on k8s

Configure the ingress

Results

Understand how linux containers works with practical examples

Table of contents

Linux Namespaces

Linux control groups (cgroups)

Container Fundamentals (key technologies)

Process namespace fundamentals

Filesystem Overlay FS fundamentals

Networking Linux bridge fundamentals

Control groups (cgroups) fundamentals

Create a container from scratch

Inspect Namespaces within a docker container

Install docker CE

Inspect Docker Network

Inspect cgroups in a docker container

Inspect overlay fs in a docker container

Inspect docker process namespace

Conclusion

How to deploy pihole and wireguard on kubernetes (DNS over HTTPS)

Assumptions

System Architecture

Resources

Methodology

Namespaces

Dynamic DNS

Create directories

Pihole

Pihole with upstream servers

Pihole with unbound

Pihole with DNS over HTTP(S) with cloudflare DNS servers

Test Pihole installation

WireGuard

Wireguard Client config

Extra (Ingress Configuration for Pihole UI)

Deploy cert-manager

Deploy Cluster Issuer (Let's Encrypt)

Ingress

How to use

How to connect to kubernetes internal network using WireGuard

Conclusion

How to bypass a firewall using a VPS on AWS

Types of firewall blocking

Who wants to bypass a firewall?

Disclaimer

Before begin

Configure VPS on AWS

Access to VPS

Install needed packages

Configure fail2ban

Configure SSH server

Configure squid proxy on 80/tcp

Configure Wireguard at 53/udp

Configure OpenVPN at 443/tcp

Conclusion

How to install and configure your own VPN server in GCP with Wireguard

Server-side setup

Create virtual host in GCP

Setup ssh keys

Install Wireguard

Configure Wireguard Server

Add firewall rules in GCP

Configure clients

How to encrypt an existing archlinux LVM installation [LVM on LUKS]

Assumptions

Warning

The specific case

Steps

How to setup WiFi hotspot in raspberry pi and connect with ESP8266

Basic connection between arduino and raspberry pi through WiFi

The Hardware is

System requirements