Deploying High Availability SafeLine WAF on K3s(Part 4)

Here are the three of my previous posts under this topic:

• Deploying k3s cluster
• Deploying nginx-ingress
• Installing the nfs-provisioner Component and SafeLine WAF via HelmChart

Nginx-Ingress Integration with SafeLine WAF for Domain Protection

Choosing Nginx-Ingress

After logging into the Open Source SafeLine WAF service, you need to configure the protection sites that need to be integrated with the WAF and the upper-level proxy forwarding IP or domain name after WAF integration in the protection sites section.

All integration sites are accessed and forwarded through the safeline-tengine service. Since SafeLine WAF is deployed in a Kubernetes cluster, even if the safeline-tengine service is exposed via NodePort, if the domain name directly points to the public IP + NodePort port of the node, the tengine service will not be able to obtain the X-Forwarded-For header information. It can only obtain network IP, which only displays the internal POD IP of Kubernetes, not the real public IP of the client.

Example of Nginx access logs without the X-Forwarded-For field, showing only internal IP 10.42.1.1:

10.42.1.1 - - [16/Apr/2024:16:52:36 +0800] "www.abc.com" "GET /message-ajaxGetMessage-0.html HTTP/1.1" 200 0 "https://www.abc.com/my/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" ""

Therefore, only by using Nginx-Ingress to forward to the WAF's safeline-tengine service can the X-Forwarded-For header be obtained, allowing the real public IP address of the client to be retrieved. Only with the client's real IP can blocking and whitelist/blacklist rules be implemented.

In the "Protection Site" - "Proxy Configuration" section of the SafeLine WAF console, set the "Source IP Retrieval Method" to "Retrieve from HTTP Header" and set the Header value to: X-Forwarded-For.

Example of tengine access logs retrieving the X-Forwarded-For field from nginx-ingress:

192.168.3.45 - - [16/Apr/2024:16:52:36 +0800] "www.abc.com" "GET /message-ajaxGetMessage-0.html HTTP/1.1" 200 0 "https://www.abc.com/my/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36" "222.234.923.56"

In addition to Nginx-Ingress, k3s' built-in Traefik can also achieve client real IP retrieval, but some modifications are required. Users who are interested can study this further.

Example Nginx-Ingress Configuration

This document uses the configuration of the www.abc.com domain as an example. Besides configuring the protection site in SafeLine WAF, you also need to create a resource manifest file for the domain in Nginx-Ingress. For detailed explanations of Kubernetes resource manifests, refer to the Kubernetes official documentation.

In addition to Nginx-Ingress configuration, similar protection site configurations need to be done in the SafeLine WAF console. Refer to the SafeLine WAF official documentation for details.

This manifest file includes the following two aspects:

1. Domain Certificate File Resource Manifest, which is of the Kubernetes secret type. Example (certificate content is base64 encoded, the example values are not real):

apiVersion: v1
kind: Secret
metadata:
  name: www-abc-com
  namespace: safeline
type: kubernetes.io/tls
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tDQpNSUlHYXpDQ0JWT2dBd0lCQWdJUkFQeWFPRDdFM2dicFZ3a2RRZWJ4M2Iwd0RRWUpLb1pJaHZjTkFRRUxCUUF3DQpYREVMTUFrR0ExVUVCaE1DUTA0eEdqQVlCZ05WQkFvVEVWZHZWSEoxY3lCRFFTQk1hVzFwZEdWa01URXdMd1lEDQpWUVFERENoWGIxUnlkWE1nVDFZZ1UyVnlkbVZ5SUVOQklDQmJVblZ1SUdKNUlIUm9aU0JKYzNOMVpYSmRNQjRYDQpEVEl6
  tls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb1FJQkFBS0NBUUVBZ0FYRDRSTW44RGozdVNFOGczbVQzK3hYeDFDRzhJTURkdmdvQVB6N2gxbHVPY1lDCjdnYnUxbFFaVEphVno1YzNtcXF6YXpsK0Vtdjh2Y2hQQS9FYWI3TEVpV1A0ZUZpd0VXZFU3NVVqeTMxR3BnT2kKRnVzS1RidXhPN1gvc0ZNbmtrdDFvbjI5N1Vrc2JCNG1iV3BKa0RMbU0xUHc5bFpuK21TWmNQbXp0L3dmcW5SZgpqbGx3MDZ3M2w0eCtTRFpyd2syMVdLY2NWRExqdGp5TGNFbXlrNTIyTUIyVGhZck1uNjh5bHA5UG5vYndNTEx4CmE1WFozUkFtL0NPeTd2TG5FZXdFb

1. Ingress Resource Manifest, which binds the specified domain, domain certificate, and Ingress backend forwarding service. Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: www-abc-com-ingress
  namespace: safeline
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - www.abc.com
    secretName: scmttec-com
  rules:
  - host: www.abc.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: safeline-tengine
            port:
              number: 80

Applying the Configuration and Checking Access

Apply the two created YAML files using the kubectl command:

kubectl apply -f www-abc-com-secret.yaml
kubectl apply -f www-abc-com-ingress.yaml

Check the created resources using kubectl commands:

kubectl get secrets -n safeline
# Output:
NAME                             TYPE                 DATA   AGE
www-abc-com                      kubernetes.io/tls    2      15d

kubectl get ingress -n safeline
# Output:
NAME                         CLASS   HOSTS          ADDRESS         PORTS     AGE
www-abc-com-ingress          nginx   www.abc.com    10.43.109.170   80, 443   7d5h

Finally, access the bound domain via the web, and if the page loads and forwards correctly, the configuration is successful!

Testing Attack Protection

For example, if the domain www.abc.com is used, it first needs to resolve to the IP of the node server where the WAF in the k3s cluster is located.

Example attack command:

curl -v https://www.abc.com/\?id\=3%20AND%201\=1

Expected output:

<!DOCTYPE html>
<html lang="zh">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width,initial-scale=1" />
    <title>Request poses a threat, blocked</title>
...

If the standard SafeLine WAF interception page is displayed, it indicates successful interception. Corresponding attack event notifications will also appear in the "Attack Events" menu in the SafeLine WAF console.