Setup NGINX Ingress Controller to Implement Canary Releases

We'll go through a step by step guide to implement Canary Releases using Ingress Controller!

ingress-nginx

ingress-nginx Ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer

To use, add ingressClassName: nginx spec field or the kubernetes.io/ingress.class: nginx annotation to your Ingress resources.

Requirements

Kubernetes: >=1.20.0-0

Get repo Info

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

Install chart

Important: only helm3 is supported

kubectl create ns nginx
helm install --set controller.service.type=NodePort --set controller.admissionWebhooks.enabled=false --namespace nginx [RELEASE_NAME] ingress-nginx/ingress-nginx

Canary

Ingress Nginx has the ability to handle canary routing by setting specific annotations, the following is an example of how to configure a canary deployment with weighted canary routing.

For advanced monitoring of your Ingress resources, consider using the Ingress Monitor Controller. It integrates seamlessly with your Kubernetes environment, providing real-time monitoring of Ingress endpoints.

Create namespace for Canary Demo

kubectl create ns canary-demo

Create your main deployment and service

This is the main deployment of your application with the service that will be used to route to it

echo "
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: production
  namespace: canary-demo
  labels:
    app: production
spec:
  replicas: 1
  selector:
    matchLabels:
      app: production
  template:
    metadata:
      labels:
        app: production
    spec:
      containers:
      - name: production
        image: registry.k8s.io/ingress-nginx/e2e-test-echo@sha256:6fc5aa2994c86575975bb20a5203651207029a0d28e3f491d8a127d08baadab4
        ports:
        - containerPort: 80
        env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
---
# Service
apiVersion: v1
kind: Service
metadata:
  name: production
  namespace: canary-demo
  labels:
    app: production
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  selector:
    app: production
" | kubectl apply -f -

Create the canary deployment and service

This is the canary deployment that will take a weighted amount of requests instead of the main deployment

echo "
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: canary
  namespace: canary-demo
  labels:
    app: canary
spec:
  replicas: 1
  selector:
    matchLabels:
      app: canary
  template:
    metadata:
      labels:
        app: canary
    spec:
      containers:
      - name: canary
        image: registry.k8s.io/ingress-nginx/e2e-test-echo@sha256:6fc5aa2994c86575975bb20a5203651207029a0d28e3f491d8a127d08baadab4
        ports:
        - containerPort: 80
        env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
---
# Service
apiVersion: v1
kind: Service
metadata:
  name: canary
  namespace: canary-demo
  labels:
    app: canary
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  selector:
    app: canary
" | kubectl apply -f -

Create Ingress pointing to your main deployment

Next you will need to expose your main deployment with an ingress resource, note there are no canary specific annotations on this ingress

echo "
---
# Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: production
  namespace: canary-demo
  annotations:
spec:
  ingressClassName: nginx
  rules:
  - host: echo.prod.mydomain.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: production
            port:
              number: 80
" | kubectl apply -f -

Create Ingress pointing to your canary deployment

You will then create an Ingress that has the canary specific configuration, please pay special notice of the following:

• The host name is identical to the main ingress host name

• The

nginx.ingress.kubernetes.io/canary: "true" 

annotation is required and defines this as a canary annotation (if you do not have this the Ingresses will clash)

• The

nginx.ingress.kubernetes.io/canary-weight: "50" 

annotation dictates the weight of the routing, in this case there is a "50%" chance a request will hit the canary deployment over the main deployment

echo "
---
# Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: canary
  namespace: canary-demo
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  ingressClassName: nginx
  rules:
  - host: echo.prod.mydomain.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: canary
            port:
              number: 80
" | kubectl apply -f -

Testing your setup

You can use the following command to test your setup (replacing INGRESS_CONTROLLER_IP with your ingresse controllers IP Address)

for i in $(seq 1 100); do curl  -s  echo.prod.mydomain.com:$INGRESS_CONTROLLER_PORT | grep "Hostname" ; done

You will get the following output showing that your canary setup is working as expected:

Hostname: production-5c5f65d859-phqzc
Hostname: canary-6697778457-zkfjf
Hostname: canary-6697778457-zkfjf
Hostname: production-5c5f65d859-phqzc
Hostname: canary-6697778457-zkfjf
Hostname: production-5c5f65d859-phqzc
Hostname: production-5c5f65d859-phqzc
Hostname: production-5c5f65d859-phqzc
Hostname: canary-6697778457-zkfjf
Hostname: production-5c5f65d859-phqzc

Let us know if you found this useful or in case if you have any tips!