A/B Deployment in Kubernetes: Setting Up Traffic Splitting and Collecting Metrics

A/B Deployment: where traffic splitting breaks

Traffic splitting in Kubernetes is configured in half an hour: weights on the ingress balancer, two sets of pods, a switch. A/B Deployment breaks not on the config, but on the fact that two versions of the service live simultaneously.

While the new version is taking traffic, the old one continues writing to the same data schema — if the migration has moved ahead and there is no backward compatibility, some requests fail specifically on the old branch. Externally, the release looks successful: the build passed, pods are Ready, logs are quiet.

Sticky sessions during rollout add a second class of failures.

A user is bound to a specific pod for the duration of a scenario, and the pod is taken out of rotation a minute later — the scenario breaks off midway, the cart loses state, and reproducing this in a test environment is nearly impossible.

Metrics at this moment are blind: the aggregated error rate is blurred across both versions, and it cannot tell which branch is degrading and which is holding the load.

The third failure is rolling back a release manually by phone.

The on-call engineer checks with the analyst whether the numbers can be trusted, traffic all this time goes to the problematic version, and the weights are switched by a person in the console: without a change history and without checking that the previous version is compatible with the current state of the data schema.

We break down how this is closed: how to separate migrations and releases in time, keep sessions during weight changes, and collect metrics by which the rollback decision is made without a phone call.

What managed traffic splitting gives

When traffic is distributed by weights on the route, rather than by switching "roll out everything at once", the share of the new version becomes a controllable parameter.

In Istio this is described declaratively: some requests go to the new version, the rest — to the current one, and the weight is changed by a single edit of the rule without rebuilding images.

Two measurable consequences follow. The failure rate during release drops: the new version first sees a small share of live traffic, not the entire flow, so a defect affects a limited circle of sessions.

Service recovery time is measured in minutes — rollback returns the weights without recreating pods or breaking established connections.

Comparing versions on live traffic stops being a dispute about hypotheses. Requests are split by variant, metrics are collected in parallel, so the difference in latency, error rate, and conversion is visible on the same scenarios.

Release observability rests precisely on this: the release has graphs by which the decision to expand the share or roll back is made based on data, not on the on-call engineer's feeling.

Managed rollback is available throughout the entire rollout and does not require the participation of the release author — it is enough to remove the traffic share. Both variants remain live as long as needed, and the second is removed after the metrics have converged.

Traffic splitting options: comparison of approaches

The choice of traffic splitting method determines how long rollback will take and which metrics will have to be checked at each step.

Below are working alternatives: they are not mutually exclusive, but require different degrees of control over routing and different maturity of observability.

In practice, the options are combined: weight-based routing sets the share, canary steps control its growth, and feature flags insure a separate heavy component that cannot be rolled back together with the entire build.

Implementing A/B Deployment: work stages

Traffic splitting between versions lives at the shared entry to the cluster, so an error in routing rules hits not the experiment but all users at once.

We follow a fixed order: first we describe how traffic flows now, then we design the scheme, and only then we enable it in production.

1. Cluster routing audit. We build a map of entry points, service objects, and rules: what already handles distribution, which settings are configured outside the repository manually, and which application versions actually receive traffic.
2. Designing the traffic splitting scheme. We choose the splitting criterion — request share, header, or user group label — and fix which rules should trigger when one of the versions is unavailable. The scheme goes into declarative manifests, not manual edits in the UI.
3. Splitting setup. We deploy rules and weights, bind the user session to a version, and limit the trial version's share so that it does not take on all the load in case of failure.
4. Metrics and observability. We tag requests with a version attribute so that metrics are calculated separately: error rate, response time, and depth of completion of key scenarios for each version individually.
5. Load testing. We run a profile close to real traffic: we look at the actual distribution, behavior when one version fails, and the time to switch weights.
6. Rollback plan. We fix which signal is considered an emergency and by which command all traffic returns to the stable version in one step — without rebuilding and redeploying.
7. Handover to operations. We provide documentation on the scheme, a policy for launching new experiments, and instructions for the on-call shift: how to read metrics and what to do in case of deviation.

Output artifacts: manifests, diagrams, policies

The work ends not with a presentation, but with a set of files in your repository and on the wiki. We hand over everything needed so that your team can maintain traffic splitting between versions itself — selection rules, shares, and incident analysis on call.

• Routing configurations — manifests with version weights, selection rules by request headers and session attributes. Changing the traffic share is an edit of one parameter, without rebuilding the application.

• Access policies and roles — who changes weights, who only reads metrics, who rolls out the second version; permissions are granted on a least-privilege basis.

• Traffic flow diagram — from the external balancer to the pods of each version, with marked metric collection points and where the selection rule triggers.

• Metric set and comparison rules — which indicators we consider a signal that a version is lagging, on which observation window, and with what delay they reach the on-call engineer.

• Release policy — the order of rolling out the second version, criteria for expanding the share, conditions and time for returning all traffic to the previous version.

• On-call shift instructions — step-by-step scenarios: a metric went negative, the second version stopped responding, the experiment needs to be urgently removed.

• Handover checklist — what your team goes through together with our engineers on the first independent release.

Case study: traffic splitting with metrics before and after

Before: manual switching and rollback by instructions

The client is a service with a constant request flow, where every release was done manually: the weight of the new version in the balancer was edited through a panel, and the state was compared using application logs and support messages.

One release took 11 manual steps — from editing the configuration to checking on some nodes. Rollback took from 25 to 40 minutes: returning the weight, restarting a pod, and warming up the cache were done according to instructions, and all this time some users received errors.

During release windows, the share of responses with 5xx codes reached 1.8%, and the cause was determined only after the fact.

After: comparing options on live traffic

Next, at the ingress gateway we deployed Istio and separated the versions into two groups: one receives traffic by weight, the second by header, to test specific users without affecting others.

The weight is changed by editing one resource, not through a panel, and is removed in one step — rollback now takes about 40 seconds without a restart.

Metrics are now collected separately for each group: error rate, share of slow responses, scenario completion depth, and number of failures at the checkout stage.

Comparison is done over identical time windows, so it shows not "became better" but exactly how much the version differs.

After three months of releases: manual operations dropped from eleven to two, the error rate in production fell to 0.2%, and 94% of releases went through without returning to the previous version — rollback stopped being a standard part of the release.

How canary release differs from blue-green

Canary release splits traffic between the old and new version by shares — 5, 25, 50 percent — and increases them as metrics improve. Rollback speed here is measured in seconds: returning the weight to zero moves the flow to the previous version without rebuilding and restarting.

Blue-green deployment keeps two full environments and switches all traffic in one motion — by changing the service selector or routing rule.

Rollback is just as fast, but the entire audience switches at once, so the error is visible to everyone, and open connections are broken when the environment changes.

Resource cost during release differs most of all. The canary circuit needs capacity only for the current traffic share, and it grows with the weight; blue-green needs a full duplicate of the environment, which sits idle throughout preparation and checks.

The data schema is the main watershed. Canary release keeps both versions in production simultaneously, so the schema is changed by expansion: a new field is added as optional, the old version ignores it, and only after it is retired does the code migrate the data.

Blue-green allows migration within the switch window, but the shared database remains, and an incompatible change breaks one of the sides.

For comparing metrics, canary mode is more convenient: traffic is split by version label, and conversion, error rate, and latency are compared on a homogeneous sample.

Blue-green gives an "all or nothing" result — the different base makes it hard to gather statistics, and the decision is made after the fact, not based on data.

Common technical doubts before starting

Traffic splitting at the cluster entry does not require rewriting the service: the versions remain two separate deployments, and the route by header or cookie decides where the request goes. Below are the things most often clarified at the evaluation stage.

Can I keep a shared database between versions?

Yes, as long as the schema is backward compatible: we add new fields as optional, do not remove old ones while the second version is running. We split a breaking change into two steps — first both versions read both structures.

How reliable is release statistics?

A stable binding of a person to a group is needed — a cookie or identifier, otherwise one user will fall into both groups and the metrics will mix. We calculate the sample size before launch, otherwise the difference will be within noise.

What to do with already configured routing?

We preserve existing rules: we layer the A/B route on top with higher priority and intercept only the needed header or traffic share. Before enabling, we test priorities on the staging environment.

Are the environments compatible with each other?

We keep staging and production on the same cluster versions and set of add-ons, otherwise routing behaves differently. We store manifests as uniform sets — differences only in values.

Let's discuss the task: what is needed from your side

For the estimate to be substantive, not "by eye", we need a minimum of initial data about your environment. It is enough to describe it in words and confirm with access — then we will assemble a traffic splitting scheme and an implementation plan for your release cycle.

• Routing scheme as it is: ingress or service mesh, which weight rules are already configured, whether there are canary releases, and how the flow is currently distributed between versions.
• Target release metrics: by which signals we consider a version successful — error rate, response latency, conversion of the key scenario, depth of the business event.
• Access to the environment: test and production clusters, permissions to change routes and policies, a separate environment for rehearsing weight switching.
• Experiment boundaries: which versions we compare, whether user retention in one group between sessions is needed, how to handle authorized and guest sessions.
• Risk budget: allowable traffic share to the new version, condition for automatic rollback, and the time within which switching must occur.
• Observability: which metric and tracing collectors are already in place, where logs are written, whether there are ready dashboards and alerts for group divergence.
• Environment constraints: network modes between services, encryption requirements, change policy in production, and windows when release is allowed.

Describe the task in free form — we will send an estimate of work on traffic splitting, a configuration scheme, and a step-by-step implementation plan with rollback points.