Performing a canary release, rather it be for A/B testing or controlled rollout, is a common practice within software deployments. Istio enables canary routing, through a plethora of HTTP Match Requests, but for my particular use case, I wanted to route to the canary release based on a particular request header. Internally, this enables us to dark deploy, run a/b tests, and enable developers to test features before cutting our production traffic over.
Prerequisites:
- Istio installed namespace
Architecture:
For this POC, I built a sample app which simply returns “test-a” if it's using the “a” image tag, or “test-b” if using the “b” tag. When a user makes an HTTP request call, if it has the header version: canary
, it will route to the canary release, otherwise, it will route to the production release. All source code is linked below.
Setup
With istio installed namespace, we need to deploy two deployments of our application, which is fronted by a service. For simplicity, I’ve redacted the config to only the essential pieces:
# Production Release
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: app-canary-testing
name: app-production
spec:
replicas: 1
selector:
matchLabels:
app: app-canary-testing
template:
metadata:
labels:
app: app-canary-testing
version: production
spec:
containers:
- image: danquack/canary-testing:a
name: canary-testing
ports:
- containerPort: 9000
name: http
protocol: TCP
# Canary Release
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: app-canary-testing
name: app-canary
spec:
replicas: 1
selector:
matchLabels:
app: app-canary-testing
template:
metadata:
labels:
app: app-canary-testing
version: canary
spec:
containers:
- image: danquack/canary-testing:b
name: canary-testing
ports:
- containerPort: 9000
name: http
protocol: TCP
A couple of the key differences between these two deployments is the image tag (prod: a, canary: b) and the pod version label, which we will utilize later.
With the deployment defined, next is setting up the service:
apiVersion: v1
kind: Service
metadata:
labels:
app: app-canary-testing
name: app-canary-testing
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
app: app-canary-testing
Istio Configuration
To configure routing we are going to configure three resources, which will define ingress, subsets (groups driven from labels), and routing rules.
Ingress
Configure a Istio Gateway to route traffic for our hostname app.local
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: canary-testing
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- app.local
port:
name: http
number: 80
protocol: HTTP
Destination Rule
Destination rules are designed to define traffic policies for load balancing or group individual versions of services. With subsets, we can define individual groups driven from the label we set on our pod. This config states for our service hostname, group the pods based on the label version
:
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: canary-testing
spec:
host: app-canary-testing
subsets:
- labels:
version: canary
name: canary
- labels:
version: production
name: production
Virtual Service
Now that all the core pieces are in place, we can configure the joining definition, which will allow us to route the traffic in the desired manner. When the traffic comes through the ingress gateway, istio will check our virtual service, to determine where to send that packet. In this setup, because the desired response from a requestor who sends an HTTP request with the header version: canary
is to route to the canary pod, we will configure the matching header, and then route to the service with the matching subset created from the above Destination Rule.
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: canary-testing
spec:
gateways:
- canary-testing
hosts:
- app.local
http:
- match:
- headers:
version:
exact: canary
name: canary-route
route:
- destination:
host: app-canary-testing
subset: canary
- name: Production Route
route:
- destination:
host: app-canary-testing
subset: production
Testing
To verify our deploy is working, first, grab the ingress host:
$ kubectl -n istio-system get service istio-ingressgateway
Then well try a sample curl call to our production endpoint:
$ curl http://$INGRESS_HOST -H "Host: app.local"
test-a
Then append the http header, version: canary
, which will verify your canary release
$ curl http://$INGRESS_HOST -H "Host: app.local" -H "version: canary"
test-b
All source code included can be found on Github:
Top comments (0)