K3S

Looking for a straightforward, no-nonsense guide to Kubernetes and K3s? Here is everything you need to know—from core node architecture to everyday manifests and commands—all in one place.

K3S Architecture

A K3S cluster works with nodes, a node representing a machine like a VPS. A node can have 2 roles:

• Server/Master node: responsible for managing the cluster.
• Agent/Worker node: responsible for running pods.

Although a node can handle both roles, it is generally considered a best practice to keep them separate and never run workload pods on the master node.

Each type of node manages specific processes:

Master Node

• kube-apiserver: The conductor of the cluster. All communication goes through it (for instance, kubectl talks to it when you run a command).
• etcd: The memory of the cluster. It is a distributed key-value database (like SQLite by default in K3S).
• kube-scheduler: Looks at newly created pods and decides which worker node should run them.
• kube-controller-manager: Ensures that the current state of the cluster matches the desired state defined in manifests.

Worker Node

• kubelet: Receives instructions from the master node and applies them to manage the pods on the local node.
• kube-proxy: Manages network routing inside the node.

Kubernetes Workflow

1. You send a manifest (YAML) via kubectl to the API server.
2. The API server writes this state to etcd.
3. The scheduler sees a new task, chooses a worker node, and requests that node's kubelet to execute it.
4. The kubelet instructs the container runtime to start the container.

Kubernetes Node Commands

Command	Description
kubectl get nodes	Get all nodes in the current cluster
kubectl top node	Get resource usage of nodes
kubectl describe node <node_name>	Get detailed information about a node
kubectl cordon <node_name>	Mark a node as unschedulable

Kubernetes Resource Commands

Command	Description
kubectl get <resource>	Get all resources (pods, services, secrets...)
kubectl get <resource> <name>	Get a specific resource
kubectl describe <resource> <name>	Get detailed information on a resource

---

Labels and Annotations

In Kubernetes, a label is a key-value pair attached to objects (like pods). They can be displayed by adding --show-labels to kubectl get pods.

You can also filter pods by a specific label value:

kubectl get pods --selector="version=1" -n <namespace>

Keep in mind that labels are scoped to a namespace, so you cannot use labels as a substitute for namespaces.

ConfigMap

A ConfigMap is an object used to store non-confidential data in key-value pairs. Since it is stored in plain text, it should never be used for sensitive data like passwords or tokens.

How to create a ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  VERSION: "2"
  LOG_LEVEL: "DEBUG"

How to use it

For a single value, or if the environment variable name differs from the ConfigMap key:

spec:
  containers:
  - name: my-app
    image: my-app:v1
    env:
    - name: PURGE_MONTHS 
      valueFrom:
        configMapKeyRef:
          name: budget-config
          key: PURGE_THRESHOLD_MONTHS

For multiple values at once:

spec:
  containers:
  - name: my-app
    image: my-app:v1
    envFrom:
    - configMapRef:
        name: app-config

Other Usages, CLI Arguments:

spec:
  containers:
    - name: purge
      image: my-app:v1
      env:
        - name: PURGE_MODE
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: mode
      command: ["java", "-jar", "app.jar"]
      args: ["--operation=$(PURGE_MODE)"]

Mounted File:

First, define a ConfigMap where the key acts as a file name and the value acts as the file content:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config-file
data:
  application-prod.yaml: |
    spring:
      datasource:
        url: jdbc:postgresql://postgres:5432/mydb
    app:
      purge:
        threshold: 12
    logging:
      level:
        root: WARN

Then, mount it inside your pod spec:

spec:
  containers:
  - name: my-app
    image: my-app:v1
    volumeMounts:
    - name: config-volume
      mountPath: "/app/config" 
      readOnly: true
  volumes:
  - name: config-volume
    configMap:
      name: app-config-file

Secrets

Secrets are designed for sensitive data such as passwords, OAuth tokens, or SSH keys.

How to create a Secret

While secrets can be declared in YAML (base64 encoded), creating them via the CLI is very common:

kubectl create secret generic db-access \
  --from-literal=DB_PASSWORD='MyPassword' \
  --from-literal=DB_USERNAME='dbuser'

How to use it

spec:
  containers:
  - name: my-app
    envFrom:
    - secretRef:
        name: db-access

Services

Pods are ephemeral by nature—their IP addresses change whenever they restart. Services provide a stable network interface and a fixed IP address to access a set of pods.

There are 3 main types of services:

• ClusterIP: Exposes the service on a cluster-internal IP. Only reachable from within the cluster.

• NodePort: Exposes the service on a static port on each node's IP, allowing external traffic to reach it.

• LoadBalancer: Provisions an external load balancer (if supported by the cloud provider or environment) to expose the service publicly.

Ingress

An Ingress manages external HTTP/HTTPS access to services within a cluster, typically acting as a reverse proxy and handling TLS termination (HTTPS).

There are two main components:

• The Ingress Controller: The actual reverse proxy running in the cluster (e.g., Traefik by default in K3S).

• The Ingress Resource: Rules defining how domain names map to internal Kubernetes services.

Example of an Ingress resource:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  namespace: my-app
  annotations:
    kubernetes.io/ingress.class: traefik
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
    - hosts:
        - my-app.domain.app
      secretName: my-app-tls
  rules:
    - host: my-app.domain.app
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app-service
                port:
                  number: 8080

Storage

Persistent storage in Kubernetes relies on three core concepts:

• StorageClass (SC): Defines how storage is provisioned (e.g., local disk, cloud disk).

• PersistentVolume (PV): A piece of storage provisioned in the cluster.

• PersistentVolumeClaim (PVC): A request for storage made by a user/application.

On K3S, local-path is the default StorageClass, which automatically provisions a folder on the node's local disk when a PVC is created:

NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local-path (default)   rancher.io/local-path   Delete          WaitForFirstConsumer   false                  108d

1. Create the PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
  namespace: my-app
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

1. Mount the PVC in your deployment spec:

spec:
  containers:
  - name: my-db
    image: postgres:15
    volumeMounts:
      - name: my-storage
        mountPath: /var/lib/postgresql/data
  volumes:
    - name: my-storage
      persistentVolumeClaim:
        claimName: my-pvc

Authentication and Authorization

Command	Description
kubectl get roles,clusterroles -A	See all Roles and ClusterRoles
kubectl get rolebindings,clusterrolebindings -A	See all RoleBindings and ClusterRoleBindings
kubectl describe clusterrole admin	Show details about a specific Role/ClusterRole
kubectl auth can-i create deployments	Check if you have permission to create deployments
kubectl auth can-i --list	See all your available permissions in the cluster