DEV Community: Harry Tang

How to install & configure Longhorn for Kubernetes

Harry Tang — Wed, 07 May 2025 18:02:31 +0000

Introduction

Longhorn is a free, open-source, lightweight, reliable, and easy-to-use distributed block storage system for Kubernetes cluster. This guide will show you how to install and configure Longhorn for your k8s cluster.

Prerequisites

A k8s cluster >=v1.25.
Helm v3.
Basic k8s knowledge.

Step-by-step Guide

Create the longhorn-system namespace and configure the Helm repository.

kubectl create ns longhorn-system
namespace/longhorn-system created

helm repo add longhorn https://charts.longhorn.io
"longhorn" has been added to your repositories

helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "longhorn" chart repository
Update Complete. ⎈Happy Helming!⎈

Create the longhorn.values.yaml for storing the Longhorn configuration values:
```
# https://artifacthub.io/packages/helm/longhorn/longhorn?modal=values
```
The first setting we want to adjust is the Data Locality; set it to best-effort to have Longhorn try to keep a replica on the same node as the attached volume:
```
persistence:
  defaultDataLocality: 'best-effort'
```
If you are using microk8s, you will also need to set the kubeletRootDir point to your kubelet directory:
```
csi:
  kubeletRootDir: /var/snap/microk8s/common/var/lib/kubelet
```

The final values should look like below:

# https://artifacthub.io/packages/helm/longhorn/longhorn?modal=values    
persistence:
  defaultDataLocality: 'best-effort'
csi:
  kubeletRootDir: /var/snap/microk8s/common/var/lib/kubelet

Now it is time to install the Longhorn system:

helm install longhorn longhorn/longhorn \
  --namespace longhorn-system \
  --values longhorn.values.yaml
NAME: longhorn
LAST DEPLOYED: Wed Jul 24 07:16:08 2024
NAMESPACE: longhorn-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Longhorn is now installed on the cluster!

Please wait a few minutes for other Longhorn components such as CSI deployments, Engine Images, and Instance Managers to be initialized.

Visit our documentation at https://longhorn.io/docs/

Check the status of the installation:

kubectl get pods -n longhorn-system
NAME                                                READY   STATUS    RESTARTS   AGE
csi-attacher-5f8b9cbbdc-2dbdg                       1/1     Running   0          45s
csi-attacher-5f8b9cbbdc-hq7ps                       1/1     Running   0          45s
csi-attacher-5f8b9cbbdc-k2tpq                       1/1     Running   0          45s
csi-provisioner-6d85c78995-b4b4r                    1/1     Running   0          45s
csi-provisioner-6d85c78995-lvvwb                    1/1     Running   0          45s
csi-provisioner-6d85c78995-nkm7g                    1/1     Running   0          45s
csi-resizer-677c8b7c75-h9kc7                        1/1     Running   0          45s
csi-resizer-677c8b7c75-qbvxl                        1/1     Running   0          45s
csi-resizer-677c8b7c75-wppts                        1/1     Running   0          45s
csi-snapshotter-868c6c774d-4z57t                    1/1     Running   0          45s
csi-snapshotter-868c6c774d-9t2cd                    1/1     Running   0          45s
csi-snapshotter-868c6c774d-ps5lp                    1/1     Running   0          45s
engine-image-ei-b0369a5d-4jqzf                      1/1     Running   0          47s
instance-manager-84b0d2d8f37aaeabfa5785c886a57710   1/1     Running   0          47s
longhorn-csi-plugin-bsv4m                           3/3     Running   0          45s
longhorn-driver-deployer-6dff48688b-4497d           1/1     Running   0          62s
longhorn-manager-ft96r                              1/1     Running   0          62s
longhorn-ui-966cf4bb4-5shp9                         1/1     Running   0          62s
longhorn-ui-966cf4bb4-lx2th                         1/1     Running   0          62s

By default, the longhorn storage class is set to 3 replicas for redundancy. This setting is stored at the longhorn-storageclass config map. If your k8s cluster has less than three nodes, set it to 1 replica:
```
kubectl get cm longhorn-storageclass -n longhorn-system -o yaml | sed 's/numberOfReplicas: "3"/numberOfReplicas: "1"/' | kubectl apply -f -
configmap/longhorn-storageclass configured
```

Setup a recurring job to trim the filesystem periodically to reclaim disk space, create a trim-filesystem.recurring-job.yaml file:

apiVersion: longhorn.io/v1beta2
kind: RecurringJob
metadata:
  name: trim-filesystem
  namespace: longhorn-system
spec:
  concurrency: 1
  cron: 0 0 * * *
  name: trim-filesystem
  task: filesystem-trim
  groups:
    - default

Create another recurring job to periodically purge removable snapshots snapshot-cleanup.recurring-job.yaml:

apiVersion: longhorn.io/v1beta2
kind: RecurringJob
metadata:
  name: snapshot-cleanup
  namespace: longhorn-system
spec:
  concurrency: 1
  cron: 12 0 * * *
  groups:
    - default
  name: snapshot-cleanup
  task: snapshot-cleanup

Deploy all tow recurring jobs:

kubectl apply -f trim-filesystem.recurring-job.yaml 
recurringjob.longhorn.io/trim-filesystem created

kubectl apply -f snapshot-cleanup.recurring-job.yaml 
recurringjob.longhorn.io/snapshot-cleanup created

By default, Longhorn does not delete orphan resources automatically, but it does provide the setting for us to do so, set orphan-auto-deletion to true:

kubectl -n longhorn-system get setting orphan-auto-deletion -o yaml | sed 's/value: "false"/value: "true"/' | kubectl apply -f -
setting.longhorn.io/orphan-auto-deletion configured

kubectl -n longhorn-system get setting orphan-auto-deletion
NAME                   VALUE   AGE
orphan-auto-deletion   true    53m

Conclusion

Now that you have the Longhorn system installed in your Kubernetes cluster with a fine-tuned configuration. Consider following the Best Practices to improve your system further.

Source: https://harrytang.xyz/blog/how-to-longhorn-k8s.

Deploy Nextcloud on Kubernetes

Harry Tang — Wed, 12 Mar 2025 20:49:14 +0000

Introduction

Nextcloud is an open-source data storage solution that you can self-host at home. This article shows how I deployed Nextcloud on my home Kubernetes cluster.

Prerequisites

A Kubernetes cluster.
MariaDB (see Deploy MariaDB Galera with MaxScale on Kubernetes).
MinIO (see Deploy MinIO in Kubernetes with Tenant Separation).
FluxCD.
Sealed Secrets.
SMTP provider.
Cert Manager.
CloudFlare Tunnel.

Step-by-step Guide

I used FluxCD to manage my YAML files in Git, following the GitOps approach. However, you can also use standard Helm commands for the HelmRelease.

Prepare the ENVs

I set up the environment variables for Nextcloud's user credentials, SMTP mail server, and MinIO credentials:

```bash
export NEXTCLOUD_ADMIN_USER=harry
export NEXTCLOUD_ADMIN_PASSWORD=$(openssl rand -base64 32)
export SMTP_HOST=smtp.eu.mailgun.org
export SMTP_USER=nextcloud@notify.harrytang.com
export SMTP_PASSWORD=YOUR_SMTP_PASSWORD
export MINIO_ACCESS_KEY=YOUR_MINIO_ACCESS_KEY
export MINIO_SECRET_KEY=YOUR_MINIO_SECRET_KEY
```

Create Sealed Secrets

    kubectl -n nextcloud create secret generic nextcloud-secrets --dry-run=client --from-literal=nextcloud-password=$NEXTCLOUD_ADMIN_PASSWORD \
      --from-literal=nextcloud-username=$NEXTCLOUD_ADMIN_USER \
      --from-literal=smtp-host=$SMTP_HOST \
      --from-literal=smtp-username=$SMTP_USER \
      --from-literal=smtp-password=$SMTP_PASSWORD \
      --from-literal=minio-access-key=$MINIO_ACCESS_KEY \
      --from-literal=minio-secret-key=$MINIO_SECRET_KEY \
      -o yaml | kubeseal --format=yaml > sealed-secret.yaml

Create the Certificate

I use a wildcard certificate to access Nextcloud both from home and remotely via the domains homecloud.harrytang.com and nextcloud.harrytang.com, respectively.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: harrytang-com
  namespace: nextcloud
spec:
  secretName: harrytang-com-tls
  dnsNames:
    - 'harrytang.com'
    - '*.harrytang.com'
  issuerRef:
    name: letsencrypt
    kind: ClusterIssuer

Deploy Ingress

I use Nginx Ingress to expose Nextcloud within my home network (by adding an A record to my domain):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nextcloud
  namespace: nextcloud
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 100G
    nginx.ingress.kubernetes.io/proxy-read-timeout: '86400'
    nginx.ingress.kubernetes.io/proxy-send-timeout: '86400'
    nginx.ingress.kubernetes.io/proxy-connect-timeout: '86400'
spec:
  tls:
    - hosts:
        - homecloud.harrytang.com
      secretName: harrytang-com-tls
  rules:
    - host: homecloud.harrytang.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nextcloud
                port:
                  number: 8080

Config CloudFlare Tunnel

Configure Cloudflare Tunnel to acess your Nextcloud remotely and securely:

...
  ingress: # https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/configure-tunnels/origin-configuration/#tls-settings 
    - hostname: nextcloud.harrytang.com
      service: http://nextcloud.nextcloud:8080
...

Deploy Nextcloud

Finally, after setting everything up, I deployed Nextcloud:

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: nextcloud
  namespace: nextcloud
spec:
  interval: 24h
  url: https://nextcloud.github.io/helm/
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: nextcloud
  namespace: nextcloud
spec:
  interval: 24h
  chart:
    spec:
      chart: nextcloud
      version: '~6.6.4'
      sourceRef:
        kind: HelmRepository
        name: nextcloud
        namespace: nextcloud
      interval: 24h
  values:
    replicaCount: 1
    livenessProbe:
      initialDelaySeconds: 180
    readinessProbe:
      initialDelaySeconds: 180
    nextcloud:
      update: true
      host: nextcloud.harrytang.com
      trustedDomains:
        - nextcloud.harrytang.com
        - homecloud.harrytang.com
      existingSecret:
        enabled: true
        secretName: nextcloud-secrets
      mail:
        enabled: true
        fromAddress: noreply
        domain: harrytang.com
      objectStore:
        s3:
          enabled: true
          bucket: nextcloud
          usePathStyle: true
          host: minio.nextcloud.svc.cluster.local
          existingSecret: nextcloud-secrets
          secretKeys:
            accessKey: minio-access-key
            secretKey: minio-secret-key
          ssl: false
          port: 80
      configs:
        proxy.config.php: |-
          <?php
          $CONFIG = array (
            'trusted_proxies' => array(
              0 => '127.0.0.1',
              1 => '10.0.0.0/8',
            ),
            'forwarded_for_headers' => array('HTTP_X_FORWARDED_FOR'),
          );
        zz.config.php: |-
          <?php
          $CONFIG = array (
            'trusted_domains' => array(
              0 => 'localhost',
              1 => 'nextcloud.harrytang.com',
              2 => 'homecloud.harrytang.com',
            ),
          );
      phpConfigs:
        zz-custom.ini: |-
          memory_limit=1024M
          upload_max_filesize=100G
          post_max_size=100G
          max_execution_time=86400
          max_input_time=86400
      extraVolumes:
        - name: private-ca
          configMap:
            name: private-ca
            items:
              - key: private-ca.pem
                path: private-ca.crt
      extraVolumeMounts:
        - name: private-ca
          mountPath: /usr/local/share/ca-certificates
          readOnly: true
    lifecycle:
      postStartCommand:
        - update-ca-certificates
    persistence:
      enabled: true
      accessMode: ReadWriteMany
    phpClientHttpsFix:
      enabled: true
    cronjob:
      enabled: false
    internalDatabase:
      enabled: false
    externalDatabase:
      enabled: true
      type: mysql
      host: maxscale-galera.nextcloud.svc.cluster.local
      database: nextcloud
      existingSecret:
        enabled: true
        secretName: mariadb
        usernameKey: mariadb-username
        passwordKey: mariadb-password

Conclusion

Running Nextcloud on your home lab Kubernetes cluster provides a secure and private way to store your data. You can store unlimited photos and videos for yourself and your family while accessing them from anywhere.

DEV Community: Harry Tang

How to install & configure Longhorn for Kubernetes

Introduction

Prerequisites

Step-by-step Guide

Conclusion

Deploy Nextcloud on Kubernetes

Introduction

Prerequisites

Step-by-step Guide

Conclusion

References