Pipy: Protecting Kubernetes Apps from SQL Injection & XSS Attacks

Injection attacks sliding down to 3rd position in 2021 have been on the OWASP Top 10 list for many years and SQL Injection (SQLi) is a common injection technique used for attacking websites and web applications. Applications that do not cleanly separate user input from database commands are at risk of malicious input being executed as SQL commands. Successful injection attacks can lead to unauthorized access to sensitive data such as passwords, credit card details, and personal user information. Many recent high-profile data breaches have resulted from SQL injection attacks, resulting in reputational damage and regulatory fines.

The common and normal workaround which you might have seen with other solutions is using a list of regular expressions to filter out traffic, which works for some cases but falls short with some complex inputs or escaped inputs. We are not trying to bash Regular Expressions, they are good and have their own use cases, but they aren't a good fit for such use cases.

This blog post will demonstrate how to use Pipy an open-source programmable proxy to harden the security by adding an extra layer of shield to your applications which might otherwise be vulnerable to such attacks.

In a previous blog post announcing the latest release of Pipy 0.70.0, it was introduced that Pipy added the support of extensions called Native Module Interface (NMI) and we will be using Pipy NMI to develop a module to integrate with mature and stable open-source library libinject to scan incoming traffic against SQLi and XSS attacks before it reaches the application.

For the demo application, we will be using an open-source classical LAMP stack Vulnerable Mama Shop (VMS) which is intentionally developed to have SQLi flaws. We will have fun together by first hacking the basic application to demonstrate the SQLi attacks, then we will harden the application security by adding Pipy as a sidecar to block certain SQLi attacks.

Pre-requisites

This blog post assumes you have access to:

• Running Kubernetes cluster (dev box)
• kubectl

The demo source code is available on GitHub and can be downloaded from pipy-sqli-demo repository.

Deploy Kubernetes cluster and Vulnerable Mama Shop App

To run the demo locally, we recommend k3d a lightweight wrapper to run k3s (Rancher Lab’s minimal Kubernetes distribution) in docker.

$ k3d cluster create my-cluster -p 8080:30060@server:0

In the above command, we are creating a cluster with a single server and mapping port 30060 of the k3d container to the local 8080 port, usage of this port will become clear in the later steps of this tutorial.

Deploy Vulnerable Mama Shop App

We are going to deploy a simple online store application that comes installed with

• Apache Webserver
• MariaDB
• PHP app that runs on Apache and connects to the MariaDB database

1. Use the text editor of your choice and create a YAML file called 1-app.yaml with the following contents:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vms
spec:
  selector:
    matchLabels:
      app: vms
  template:
    metadata:
      labels:
        app: vms
    spec:
      containers:
        - name: vms
          image: naqvis/mamashop
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: vms
spec:
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30060
  selector:
    app: vms
  type: NodePort

1. Deploy the app

$ kubectl apply -f 1-app.yaml

1. Confirm that pod is up and running, as indicated by the value Running in the STATUS column. It can take 30-40 seconds for them to fully deploy, so it’s useful to run the command again to confirm all pods are running before continuing to the next step.

$ kubectl get pods
NAME                   READY   STATUS    RESTARTS   AGE
vms-6658dd4478-hn246   1/1     Running   0          2m12s

1. Query the Servcice

$ kubectl get svc vms
NAME   TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
vms    NodePort   10.43.244.253   <none>        80:30060/TCP   5m43s

Still remember at the beginning we did port mapping for the K3d container: 8080=>30060? Keen users may also find that we have created a NodePort service for our Vulnerable Mama Shop web application, and the node port is 30060.

Open the app in your browser by visiting the URL http://localhost:8080

Hack the App

Mama Shop is rather very basic, it comes with only 3 pages. The main page (shown above) is where a user can query items for sale through a drop-down box, a customer login page, and an about page. Play around with these 3 pages to see what each does and how each page works normally.

Try submitting a category and see how items are listed. The following shows a listing of items from the Drinks category.

As you may have noticed, selecting a category and hitting submit doesn't change the URL, so the application is doing a POST request and we will need some tool to intercept or view the traffic the web page is generating.

Normally you will be using a proxy tool like Blurp suite or OWASP ZAP to intercept and modify the request to the application. But to keep things simple, we will be using Firefox browser Developer Console to view and modify the requests.

Go back to your browser (Firefox in this demo) and turn on Web Developer Tools either via Tools | Browser Tools | Web Developer Tools menu option or via short-cut key Option+Cmd+I on Mac or similar on your OS. Move to the Network tab, and hit submit button on a web page to view the network traffic.

We can see all of the requests along with all details, a web page makes to the webserver to display the page. Hit Resend button at the top right of Request details window displayed at the bottom right.

Network action window shows that to browse items in a category, a URL encoded HTTP POST with a single parameter catid is sent to welcome.php. To test for SQL injection, it is common to modify user input and send a single quotation mark like '.

Modify Form Parameter

We will find out if the input is properly escaped with a simple experiment: changing the catid to a SQL query string, to see if it will cause a syntax error which may be shown on the webpage.

catid=' or 1 = 1 ; --

Make above change and hit send button at the bottom right of network action window.

Whoa, we are upto something, welcome.php function for displaying category items have a SQL injection vulnerability. The error message also tells us the database is MariaDB. And error is telling us that there is a syntax issue near or 1 = 1. Using our imaginations, we can assume the database query is something like

SELECT item_name, item_category, item_description from items_table where item_category = ' or 1 = 1 ; -- ;

If we were to change the catid ending to " or 1 = 1 -- ;, that should fix the quoting issue. It should select all rows from the database, which is useful in a hack.

The structure of the assumed SQL query is probably correct, although it is not necessarily the exact query that the developer has used. This is sufficient for devasting attacks such as dumping customer information.

Extract User Data

The customer login page tells us that the application contains customer data and this is likely stored in some sort of customer or user table.

Based on our information gathered so far we can make use of the MariaDB INFORMATION_SCHEMA and the SQL Union operator to retrieve database and tables details.

Change catid parameter to below to retrieve database name, which will be further used to retrieve table details.

catid=1000 union select database(), "A" , "B" from dual

The following shows the response from Vulnerable Mama Shop containing the database name, appdb.

Now we have database name, we can obtain the tables in the database

catid=1000 union select table_name,version, table_comment from information_schema.TABLES where table_schema = 'appdb'

So we get a list of all tables, We can safely assume users table in the database contains usernames and passwords.

We need to retrieve list of columns users table have and we will need to ensure that we use same number of columns as queried from products table for UNION to work.

catid=1000 union select table_name, COLUMN_NAME, DATA_TYPE from information_schema.COLUMNS where table_name = 'users'

Now that we know there are a total of six columns in the users table to contains details like firstname, lastname, nric, password, email are available from users table.

We have gathered enough information to dump out a users listing. The following input can be used to dump out a listing of users with their firstname, password and email address.

catid=1000 union select firstname, nric, email from users LIMIT 7, 100

The values for the LIMIT and offset can be worked out by observing how many item entries there are in a normal request. The offset value should remove these items. The LIMIT can be set a sufficiently large value so that customer records can be dumped out.

Play around and use these gathered information to see if you can login with these credentials.

Using Pipy Sidecar to block certain SQLi attacks

SQL injection occurs when unvalidated user input is mixed with SQL instructions, so developers should pay more attention to escape user input (such as use of parameterized queries), but you – the Kubernetes engineer – can also help avoid SQL injection by preventing this attack from reaching the app. That way, even if the app is vulnerable, attacks can still be stopped.

There are multiple options for protecting your apps, but for this blog post we will focus on injecting a sidecar container to proxy all of the traffic and deny any request that is detected as SQLi attacks.

For demonstration purposes, we are following the approach of manually injecting sidecar, but in reality, manually deploying proxies as sidecars isn’t the best solution. The complexity of your apps and architecture might require more fine–grain control. If your organization requires Zero Trust and has a need for end–to–end encryption, consider a service mesh like osm-edge a light weight, ultra fast, low resources, highly extensible, Service Mesh Interface (SMI) compatible, built for edge and cloud computing service mesh.

Deploying Pipy Sidecar

1. Create a YAML file called 2-app-sidecar.yaml with the contents below, and check out these noteworthy components:

• A sidecar container running Pipy is started on port 8000.
• The Pipy process forwards all traffic to the app.
• Requests URI or POST body containing SQLi is declined
• The service for the app routes all traffic to the Pipy sidecar container first.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vms
spec:
  selector:
    matchLabels:
      app: vms
  template:
    metadata:
      labels:
        app: vms
    spec:
      containers:
        - name: vms
          image: naqvis/mamashop
          ports:
            - containerPort: 80
        - name: pipy # <-- sidecar
          image: naqvis/pipy-nmi
          env:
            - name: PIPY_CONFIG_FILE
              value: /etc/pipy/nmi/nmi.js
          ports:
            - containerPort: 8000
          volumeMounts:
            - mountPath: /etc/pipy/nmi
              name: pipy-pjs
      volumes:
        - name: pipy-pjs
          configMap:
            name: sidecar
---
apiVersion: v1
kind: Service
metadata:
  name: vms
spec:
  ports:
    - port: 80
      targetPort: 8000 # <-- the traffic is routed to the proxy
      nodePort: 30060
  selector:
    app: vms
  type: NodePort
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: sidecar
data:
  nmi.js: |-
    pipy({
      _rejected: undefined,
    })
      .import({
        __is_sqli: 'lib-inject',
        __is_xss: 'lib-inject',
        __sqli_fingerprint: 'lib-inject',
      })
      .listen(8000)
      .demuxHTTP().to(
        $=>$
            .use('/etc/pipy/modules/inject-nmi.so')
            .handleMessage(() => _rejected = (__is_sqli || __is_xss))
            .branch(
                () => _rejected === true, (
                $=>$
                .handleMessageStart(_ =>
                  console.log(`SQL Injection found with Fingerprint: ${__sqli_fingerprint}`))
                .replaceMessage(new Message({ status: 403 }, 'Forbidden'))
                ),
                () => _rejected === false, (
                  $=>$.muxHTTP().to(
                      $=>$.connect('localhost:80')
                  )
                )
            )
      )

1. Deploy the side

$ kubectl apply -f 2-app-sidecar.yaml

Wait for pods to be up and ready

$ kubectl get pods
NAME                  READY   STATUS    RESTARTS   AGE
vms-945f6f85c-8v7sb   2/2     Running   0          8m48s

Test the Sidecar

Test whether the sidecar is filtering traffic by returning to the app and trying the SQL injection again. Pipy sidecar blocks the request before it reaches the app!

Sidecar is blocking the traffic by returning 403 Forbidden. Run few more requests with any of the previous SQLi attempts we made and we will be getting 403 back in response.
We can validate that by looking at the logs of Pipy side car.

Conclusion

The purpose of learning about offensive techniques is to enable developers, defenders and security professionals to protect the confidentiality, the integrity and availability of critical information assets and services.

Kubernetes is not secure by default. This blog post made use of Vulnerable Mama Shop, a simple application based on LAMP stack to demonstrate SQL injection attacks and how one can use techniques like SideCar to protect against them.

But as the complexity of your apps and architecture grows, you might require more fine–grain control over your services. And if your organization requires Zero Trust and has a need for end–to–end encryption like mTLS, you should consider a service mesh like osm-edge a light weight, ultra fast, low resources, highly extensible, Service Mesh Interface (SMI) compatible, built for edge and cloud computing service mesh. When you have communication between services (east–west traffic), a service mesh allows you to control traffic at that level.