DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» is a community of 966,904 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

Create account Log in
Cover image for Using multiple Ingresses in Azure AKS
Xacobe
Xacobe

Posted on

Using multiple Ingresses in Azure AKS

In the company I'm currently working we work in several different projects. My job is to deploy and maintain dev and staging environments for each project, and some of them were already deployed. They used one AKS for each environment, which is more expensive than sharing one for all the environments. I found it a little tricky to find how to do this on google, so I'm going to explain it in this post and I hope someone finds it useful.

First, create a new namespace for Ingress:
kubectl create ns ingress-nginx

We are going to use Helm to install Ingress in our cluster. Also the Azure CLI (az) to interact with our Azure resources and, of course, kubectl.

Let's start by adding the Ingress' repo to Helm:
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

And installing Ingress through Helm one time for each needed environment:
helm install blog-dev ingress-nginx/ingress-nginx --set controller.ingressClass=blog-dev --namespace ingress-nginx --set controller.replicaCount=1 --set rbac.create=true

Now ge can go check the services to see if we have all our controllers with their public IPs assigned.

pic1

Now, we create a new namespace for each environment. Deploy all your stuff there as you would normally do it. If you are just testing you could deploy just a nginx. Now take a look at our Ingress routing file, let's assume something like this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: blog-dev-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    kubernetes.io/ingress.class: blog-dev
spec:
  rules:
  - http:
      paths:
        - path: /
          backend:
            serviceName: nginx-service
            servicePort: 80

Check line 7: we have the ingress.class anotation, which points our Ingress file to one of our controllers. If you remember when we installed our controllers with Helm, --set controller.ingressClass=blog-dev sets the class of our Ingress controller. Just point each deployment's Ingress to a different controller, and if you did good you can access all your environments through the LoadBalancer's IPs found with kubectl get svc -n ingress-nginx

We now have all our environments accessible through IP, but usually you will need SSL, or you just want a URL to allow your team to access the environments easily.

Follow this commands:
az network public-ip list --query "[?ipAddress!=null]|[?contains(ipAddress, 'YOUR_IP_HERE')].[id]" --output tsv (keep quotes in YOUR_IP_HERE)

Put the output of the previous command in this:
az network public-ip update --ids OUTPUT --dns-name blog-dev

This will associate the IP from your LoadBalancer with a URL with the following shape: blog-dev.westeurope.cloudapp.azure.com. (Well, westeurope or any other region.)

Now edit your ingress to only accept connections from your new URL:

spec:
  rules:
  - host: blog-dev.westeurope.cloudapp.azure.com
    http:
      paths:
        - path: /
          backend:
            serviceName: nginx-service
            servicePort: 80

And we are done! In the same AKS, we have n environments, all of them with different namespaces, different URLs and, therefore, different IP addresses. These are completely isolated environments, and it could be nice to add some RBAC access roles, to forbid blog's developers to access, e.g, the camunda environments.

Hope this was useful to you, if so share and like it!

Top comments (2)

Collapse
 
rythmoftheredman profile image
Mateusz Pydych

Amazing work @xac0 ! I worked with @aboutroots and it helped us a lot that time.
It's worth remarking, that a few things changed, so you might want to update this piece of art.

  1. kubernetes.io/ingress.class: blog-dev annotation is getting deprecated. For new API compatible ingress declaration example have a look below (or in docs)
apiVersion: networking.k8s.io/v1 #changed
kind: Ingress
metadata:
  name: {{ include "HELPERS.name" . }}
spec:
  ingressClassName: my_ingresclass #changed
  rules:
  - host: {{ .Values.domain }}
    http:
      paths:
      - path: /
        pathType: Prefix #changed
        backend:
          service:
            name: {{ template "HELPERS.name" . }}-service
            port:  #changed
              number: 80
Enter fullscreen mode Exit fullscreen mode

ingress controller has been installed as follows (here are docs for additional sets)

helm install my-nginx-ingress ingress-nginx/ingress-nginx \
 --namespace ingress-nginx \
 --set controller.replicaCount=1 \
 --set controller.ingressClassResource.name=my_ingresclass \
 --set controller.ingressClassResource.enabled=true \
 --set controller.ingressClassResource.default=false \
 --set controller.ingressClassResource.controllerValue=k8s.io/my_ingresclass \
 --set controller.nodeSelector."beta\.kubernetes\.io/os"=linux \
 --set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux
Enter fullscreen mode Exit fullscreen mode

Note that --set controller.ingressClass=blog-dev is not valid anymore at least for my fresh installation:

NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
nginx-ingress-dev-xd    ingress-nginx   1               2022-02-04 23:30:01.907296 +0100 CET    deployed        ingress-nginx-4.0.16    1.1.1     
Enter fullscreen mode Exit fullscreen mode

rest is pretty much fine. Many thans for good work!

Collapse
 
aboutroots profile image
MichaΕ‚ Popiel

That was great, and actually saved my day. I believe that you can even further improve the process by setting up the static IPs beforehand (with a proper names etc) instead of relying on the auto-generated ones, and than add line
--set controller.service.loadBalancerIP="YOUR_IP"
to the helm install command. Thanks!

πŸ‘‹ Hey, my name is Noah and I’m the one who set up this ad. My job is to get you to join DEV, so if you fancy doing me a favor, I’d love for you to create an account.

If you found DEV from searching around, here are a couple of our most popular articles on DEV: