Keda com SQS e HTTP Scaler no EKS [Lab Session]

KEDA is a Kubernetes-based Event Driven Autoscaler. With KEDA, you can drive the scaling of any container in Kubernetes based on the number of events needing to be processed.
KEDA is a single-purpose and lightweight component that can be added into any Kubernetes cluster. KEDA works alongside standard Kubernetes components like the Horizontal Pod Autoscaler and can extend functionality without overwriting or duplication. With KEDA you can explicitly map the apps you want to use event-driven scale, with other apps continuing to function. This makes KEDA a flexible and safe option to run alongside any number of any other Kubernetes applications or frameworks.
https://keda.sh/

Keda Architecture

---

Repositórios

• Repositório de infraestrutura (terraform) - EKS-Istio-Karpenter
• Repositório keda
• Repositório K6

---

Lab architecture

---

Deploy do EKS

git clone https://github.com/paulofponciano/EKS-Istio-Karpenter.git
cd EKS-Istio-Karpenter
tofu init
tofu plan --var-file variables.tfvars
tofu apply --var-file variables.tfvars

Utilizando SQS Scaler

• Criar fila SQS e IAM policy:

aws sqs create-queue --queue-name keda-sqs --region REGION

aws iam create-policy --policy-name keda-sqs-policy \
  --policy-document file://policy-sqs.json

• Criar IAM role para service account (IRSA):

eksctl create iamserviceaccount --name keda-operator \
  --namespace keda \
  --cluster pegasus \
  --attach-policy-arn arn:aws:iam::ACCOUNTID:policy/keda-sqs-policy \
  --region REGION \
  --profile default \
  --approve

kubectl describe sa keda-operator -n keda

• Keda deploy via helm:

helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda \
  --set serviceAccount.create=false \
  --set serviceAccount.name=keda-operator

Veja que definimos 'create service account' como false, pois já criamos anteriormente e iremos utilizar aquela.

kubectl get pods -n keda

• Deployment de teste e ScaledObject:

kubectl apply -f deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx-deployment
spec:
  replicas: 0
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx

kubectl apply -f scaledobject.yaml

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: aws-sqs-queue-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    name: nginx-deployment
  minReplicaCount: 0
  maxReplicaCount: 5
  pollingInterval: 5
  cooldownPeriod:  25
  triggers:
  - type: aws-sqs-queue
    metadata:
      queueURL: https://sqs.REGION.amazonaws.com/ACCOUNTID/keda-sqs
      queueLength: "15"
      awsRegion: "REGION"
      identityOwner: operator

Podemos ver que ainda não existe nenhum pod criado pelo deployment que fizemos. Isso porque o numero de réplicas é definido em '0' e a fila SQS está vazia, assim o keda não irá acionar a criação:

kubectl get pods | grep nginx

• Colocando mensagens na fila SQS:

for i in `seq 35`; do
  aws sqs send-message --queue-url 'https://sqs.REGION.amazonaws.com/ACCOUNTID/keda-queue' \
    --message-body "testmessage" \
    --region REGION \
    --no-cli-pager \
    --output text
done

Agora temos mensagens na fila. No parâmetro 'queueLength' do ScaledObject está definido '15', assim o keda já deve escalar nosso deployment:

kubectl get pods | grep nginx

Assim temos o keda trabalhando no kubernetes de acordo com a demanda na fila SQS.

Utilizando HTTP Scaler

• Deploy addon http:

helm install http-add-on kedacore/keda-add-ons-http --namespace keda
kubectl get pods -n keda

• Deployment de teste com ingress e ScaledObject:

kubectl apply -f deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-2
  name: nginx-deployment-2
spec:
  replicas: 0
  selector:
    matchLabels:
      app: nginx-2
  template:
    metadata:
      labels:
        app: nginx-2
    spec:
      containers:
      - image: nginx
        name: nginx-2
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service-2
  labels:
    app: nginx-2
    service: nginx-service-2
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: nginx-2
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: nginx-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 443
      name: http
      protocol: HTTP
    hosts:
    - "demo-ingress.pauloponciano.digital"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - "demo-ingress.pauloponciano.digital"
  gateways:
  - nginx-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: keda-add-ons-http-interceptor-proxy.keda.svc.cluster.local
        port:
          number: 8080

kubectl apply -f http_scaledobject.yaml

kind: HTTPScaledObject
apiVersion: http.keda.sh/v1alpha1
metadata:
    name: nginx-http-scaledobject
spec:
    hosts:
    - demo-ingress.pauloponciano.digital
    targetPendingRequests: 1
    scaleTargetRef:
        deployment: nginx-deployment-2
        service: nginx-service-2
        port: 80
    replicas:
        min: 0
        max: 5

• Gerando requests com k6:

request.js

import { check } from 'k6';
import http from 'k6/http';

export const options = {
  scenarios: {
    constant_request_rate: {
      executor: 'constant-arrival-rate',
      rate: 100,
      timeUnit: '1s', // 100 iterations per second, i.e. 100 RPS
      duration: '120s',
      preAllocatedVUs: 100, // how large the initial pool of VUs would be
      maxVUs: 100, // if the preAllocatedVUs are not enough, we can initialize more
    },
  },
};

export function test(params) {
  const res = http.get('http://demo-ingress.pauloponciano.digital');
  check(res, {
    'is status 200': (r) => r.status === 200,
  });
}

export default function () {
  test();
}

Executamos o script e podemos observar o keda fazendo o scaling do nosso deployment de acordo com as requisições e com o que foi definido no ScaledObject:

k6 run request.js --vus=1000 --duration=60m

---

Happy building!