Canary Deployment Setup for Web Applications: Automation and Monitoring

You rolled out a new version, and within half an hour — an avalanche of errors and a drop in conversion? Canary deployment prevents such scenarios: the new version gradually receives real traffic, and if metrics degrade, it automatically rolls back. We implement these schemes turnkey — from a simple Nginx config to automated rollout with Prometheus. Over years of practice, we have conducted more than 50 successful releases with canary. Our experience shows: without canary deployment, the risk of failure during a release increases fivefold.

Canary deployment — gradual traffic shifting to a new version: first 1–5% of users, then 10%, 25%, 50%, and finally 100%. It allows detecting issues on real traffic before full transition. Importantly, we guarantee that when the error threshold is exceeded (usually >1%), rollback happens in seconds. This approach provides three advantages: reduction of MTTR from hours to minutes, the ability to A/B test directly in production, and complete absence of downtime. Compare: with blue-green deployment, you maintain two full environments, while canary requires 30% fewer resources. According to the definition, Canary deployment is a rollout strategy where a new version of an application gradually receives real traffic (Wikipedia).

What Problems Does Canary Deployment Solve?

• Early detection of regressions on a small traffic share: errors that unit tests missed will appear on 1% of users, not all.
• Instant rollback without full redeployment: just change the weight to 0% — and users return to the old version.
• Testing new features on real users without staging costs: canary can be targeted to specific groups (by cookie or geo).

For example, on one project we discovered that a new API version caused an N+1 query to the database — on 5% of traffic latency increased by 200%. Canary automatically rolled back the version, and we fixed the issue without a mass outage.

Manual weight change in Nginx means 5 minutes of downtime to edit the config and reload. Automated rollout with metric checks reduces this to zero. We implement a pipeline that decides itself: increase weight or rollback. In Kubernetes with NGINX Ingress, rollback is 10 times faster — just delete the canary ingress.

Implementation: From Nginx to Kubernetes and AWS

Traffic is distributed between stable and canary versions: 95% of requests go to the old version, 5% to the new one. Load balancer or proxy determines which upstream to use.

Nginx split_clients

# /etc/nginx/nginx.conf
split_clients "${remote_addr}${http_user_agent}" $upstream_pool {
    5% canary; # 5% → new version
    * stable; # 95% → old version
}

upstream stable {
    server 10.0.0.10:8080;
}

upstream canary {
    server 10.0.0.11:8080; # new version
}

server {
    location / {
        proxy_pass http://$upstream_pool;
    }
}

To change the percentage — edit the config and reload Nginx: nginx -s reload.

Canary via Cookie (sticky routing)

# The user always hits the same version
map $cookie_canary $upstream_canary {
    "1" canary;
    default stable;
}

# Or force enable for testers
map $http_x_canary_override $upstream_override {
    "true" canary;
    default $upstream_canary;
}

How to Set Up Canary Deployment in Kubernetes?

---
# stable-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-stable
spec:
  replicas: 10
  selector:
    matchLabels:
      app: myapp
      version: stable
  template:
    metadata:
      labels:
        app: myapp
        version: stable
    spec:
      containers:
        - name: myapp
          image: registry/myapp:v1.0.0
---
# canary-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-canary
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
      version: canary
  template:
    metadata:
      labels:
        app: myapp
        version: canary
    spec:
      containers:
        - name: myapp
          image: registry/myapp:v1.1.0
---
# canary-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "5" # 5% traffic
spec:
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            backend:
              service:
                name: myapp-canary-svc
                port: { number: 80 }
---

Manage weight via kubectl: kubectl annotate ingress myapp-canary nginx.ingress.kubernetes.io/canary-weight=25 --overwrite. When fully transitioning, update the stable deployment and delete canary: kubectl delete ingress myapp-canary and kubectl delete deployment myapp-canary.

AWS: Weighted Target Groups

import boto3
elbv2 = boto3.client('elbv2')

def set_canary_weight(listener_arn: str, stable_tg: str, canary_tg: str, canary_weight: int):
    """stable_weight + canary_weight must sum to 100"""
    stable_weight = 100 - canary_weight
    elbv2.modify_listener(
        ListenerArn=listener_arn,
        DefaultActions=[{
            'Type': 'forward',
            'ForwardConfig': {
                'TargetGroups': [
                    {'TargetGroupArn': stable_tg, 'Weight': stable_weight},
                    {'TargetGroupArn': canary_tg, 'Weight': canary_weight},
                ],
                'TargetGroupStickinessConfig': {
                    'Enabled': True,
                    'DurationSeconds': 3600,  # stickiness 1 hour
                }
            }
        }]
    )

Automated Canary with Metric Analysis

# canary-rollout.py
import time
import boto3
import requests

PROMETHEUS_URL = "http://prometheus:9090"

def get_error_rate(version: str, duration: str = "5m") -> float:
    query = f'rate(http_requests_total{{version="{version}",status=~"5.."}}[{duration}]) / rate(http_requests_total{{version="{version}"}}[{duration}])'
    r = requests.get(f"{PROMETHEUS_URL}/api/v1/query", params={"query": query})
    result = r.json()["data"]["result"]
    return float(result[0]["value"][1]) if result else 0.0

def progressive_rollout():
    steps = [5, 10, 25, 50, 75, 100]
    canary_weight = 0
    for target_weight in steps:
        print(f"Setting canary weight to {target_weight}%")
        set_canary_weight(LISTENER_ARN, STABLE_TG, CANARY_TG, target_weight)
        # Wait and check metrics
        time.sleep(300)  # 5 minutes per step
        error_rate = get_error_rate("canary")
        print(f"Canary error rate: {error_rate:.2%}")
        if error_rate > 0.01:  # >1% errors
            print(f"Error rate too high ({error_rate:.2%}), rolling back!")
            set_canary_weight(LISTENER_ARN, STABLE_TG, CANARY_TG, 0)
            return False
    print("Canary rollout complete!")
    return True

How Does Canary Deployment Implementation Happen?

1. Analysis of current architecture and traffic (1–2 days).
2. Designing the canary scheme: choose tool (Nginx, K8s Ingress, AWS ALB) (1 day).
3. Configuration setup and deployment (2–4 days).
4. Integration with monitoring (Prometheus, Grafana, Datadog) (1–2 days).
5. Testing and team training (1–2 days).
6. Deployment and support during first days (1 day).

Total: from 5 to 10 business days depending on complexity.

What Is Included in the Work

Over 7 years of experience, 50+ projects — our team is certified and ready to take on your project. Contact us for a consultation — we will assess your project in one day. Order turnkey Canary Deployment setup and get zero-downtime releases.

Timelines

• Nginx canary on VPS: 1–2 days
• Kubernetes NGINX Ingress canary: 2–3 days
• Automated rollout with metrics: 3–5 days