Deployment frequency drops: where the release pipeline breaks
Deployment frequency does not drop at the moment of rollout, but long before it. Changes queue for approval between teams, the release window shifts, and dozens of edits accumulate in the branch that no one has tested together.
The metric here is an indicator of the throughput of the entire release pipeline, not development activity.
The next bottleneck of release is manual rollout. The build is started manually, artifacts are transferred by copying, environment parameters are edited directly on servers. Every step rests on the memory of a single engineer and fails exactly when that engineer is unavailable.
The third reason is environment unreadiness: the test environment diverges from production in configuration and data, schema migrations are run by a separate script, and the rollout result is checked by eye.
Thus the long delivery cycle grows: days pass from a ready change to the user, during which the release has time to become outdated.
The cost is failures during release: large changes are poorly isolated, the cause of failure takes longer to find, and rollback to the previous version is not prepared, so the next rollout is postponed.
The release pipeline starts working against the product: the less often we release, the more expensive each release. We break down how this is closed — from order in approvals to automatic rollout and checks.
What a managed release flow gives: effect in measurable numbers
A managed release flow is measured not by the number of rollouts, but by predictability. We look at a set of DORA metrics: delivery frequency, time to production, time to restore, and change failure rate.
Below is what exactly changes when release goes through a pipeline with checks, rather than as a manual operation under a specific engineer.
- Share of automatic releases. Two manual steps remain — confirming the rollout window and reacting to an alert. The rest is done by the pipeline: build, test run, delivery to the environment, traffic switching. The share of automatic releases grows from 20–30% to 85% and above.
- Time to production. A change reaches the user in hours instead of days. The bottleneck stops being approval; it becomes the duration of the checks themselves — it is visible and can be reduced in a targeted way.
- Time to restore. Builds, migrations, and configurations are versioned, and rollback is a return to the previous artifact, not manual troubleshooting. If a failure used to take 40 minutes to fix, recovery fits into 5–10.
- Class of eliminated failures. Entire categories disappear: configuration divergence between environments, a forgotten migration, version mismatch on some servers. Logical errors in code remain — tests catch them before rollout.
- Cost of downtime. Service unavailability falls proportionally to recovery time: with the same number of incidents, the user sees failure several times less often, and support spends minutes instead of hours.
Deployment frequency grows without an increase in the number of incidents — every rollout follows one proven route. We take the numbers before and after from release logs and alerts, not from the team's feelings.
Rollout formats: comparing approaches for different teams
Rollout modes differ not by tool, but by the cost of error and the team's maturity.
A manual procedure with release confirmation, phased rollout to a share of the audience, and continuous delivery are not stages of "growing up", but different compromises between the pace of change and predictability of returning to the previous version.
| Rollout mode | When it fits | Limitations |
|---|---|---|
| Manual procedure | Rare releases, weak test coverage, high cost of failure | Rollout frequency runs into the human factor: the more manual steps, the longer the window between releases and the higher the share of errors |
| Phased rollout to part of the audience | Observability and metrics exist, an early signal of degradation is needed | Requires version management and a proven rollback; with a small share of the audience, a weak failure may go unnoticed |
| Continuous delivery | Automatic checks cover critical paths, rollback is a routine operation | Holds on discipline: small changes, a green set of checks, no manual approvals in the process |
In practice, modes coexist: critical services are rolled out in phases, auxiliary ones as a stream. The boundary between them is shifted by automatic checks — until critical paths are covered by them, growing deployment frequency increases risk, not pace.
What slows the transition
Changing the mode runs not into the tool, but into three things: lack of observability by release, an unproven rollback to the previous version, and large changes in one rollout.
They are closed one by one — first metrics and rollback, then splitting changes, and only then increasing frequency.
How to set up deployment frequency: stages of engineering work
Setting up deployment frequency begins not with the pipeline, but with analyzing how changes get to production today: where manual approvals stand, which checks run only on the release branch, how much time passes from commit to rollout. Without this picture, release automation only reinforces existing bottlenecks.
- Audit of the change flow. We count actual rollouts for the period, the share of failed releases, and the segments where changes wait longest. We get a list of constraints that are holding back frequency.
- Change flow diagram. We fix branching, the composition of environments, and mandatory checks, and remove duplicates. We agree on the diagram with the team before changes to the pipeline.
- Setting up release automation. We assemble the pipeline by stages: artifact, automated tests, data migrations, rollout to environments. Rollout becomes a repeatable action, not a manual operation.
- Run on the test environment. We trial the procedure on a copy of production: a series of training rollouts, including a deliberately problematic one, to see where it breaks.
- Rollback rules. We agree on what counts as a failed rollout, who makes the decision, and how the previous version is returned. We test rollback as strictly as a forward rollout.
- Handover to operations. We hand over the pipeline, the on-call procedure, and incident analysis; the team then makes changes to the flow themselves.
After this, rollout frequency grows not by speeding up work, but by removing waits and manual steps.
Set of artifacts: diagrams, policies, access, documentation
We assemble the set of documents and settings so that the customer's team runs releases themselves. Below is what remains in the repository and internal knowledge base after the work is handed over.
- Environment diagram: which environments exist, how their configurations differ, how changes move between them, and where secrets are stored.
- Access policies and roles: who launches rollout to the production environment, who only approves, how temporary access is issued to the on-call engineer and how it is revoked automatically.
- Rollback procedure: criteria for stopping a release, the procedure for returning the previous version, the person responsible for the decision, and the target recovery time.
- Release documentation: how the change list is assembled, where the version is recorded, how a release is linked to tasks and subsequent requests.
- Instructions for on-call engineers: typical failures, sequence of actions, escalation points, and signs under which on-call duty is handed over further.
- Build and check configurations: step settings, the set of mandatory checks before release, branching rules, and cross-review of changes.
- Process metrics: where release frequency and the share of failed rollouts come from, by which thresholds the team sees that the process is degrading.
Artifacts are updated in the same release as the change that affected them — otherwise, a month later, the documentation diverges from reality.
At acceptance, we go through the list together with the customer's engineers and make sure that release and rollback proceed without our prompting.
Case study: release frequency tripled without an increase in the number of failures
A product team released once every two weeks: the build was manual, checks stretched over three days, and rollback took an entire night shift. The share of successful rollouts held at 72%, and every fourth attempt ended in an incident investigation.
What was changed
We assembled build and delivery pipelines in GitLab CI: mandatory checks were moved to a separate stage, and manual steps were reduced to rollout confirmation.
Changes were split into small portions, and enablement was closed behind feature flags — turning off a flag replaced rebuilding the image during rollback. We set up result measurement: metrics for each run are calculated automatically, not from team surveys.
What became
- Deployment frequency: 1 release every two weeks → 6–8 per week.
- Share of successful rollouts: 72% → 96%.
- Rollback time: a night shift → 4 minutes.
- Path from change to production: 12 days → 1–2 days.
Deployment frequency tripled, but the class of failures did not change — there were fewer incidents because every change is small and rolled back with one action. The metrics remain on the team's dashboard, so degradation is visible within a day, not by the end of the quarter.
How does continuous delivery differ from continuous deployment?
Continuous delivery and continuous deployment are not two different pipelines, but two positions of the gate in one. In the first case, the build reaches an artifact ready for release and stops before production.
In the second, the same pipeline goes further: a change that has passed checks reaches users without manual confirmation.
The difference is not in tools, but in the boundaries of automation. With continuous delivery, the gate is a human decision: they look at metrics, choose the release window, and confirm the rollout.
Everything after confirmation is done by automation — rollout across nodes, health checks, return to the previous version if it deviates.
Continuous deployment requires that automation itself catches regressions: tests for critical paths, compatibility checks for data schema changes, canary rollout to a share of traffic with automatic rollback.
Plus feature toggles — code reaches the product, while enabling behavior remains a separate action. Otherwise, manual confirmation remains the only release control.
In practice, most teams live in continuous delivery and gradually shift the boundary: first build and tests are automated, then rollout, then rollback.
The deployment frequency metric grows even without abandoning the manual step — if release is one confirmation and rollback takes minutes, the human decision stops being a bottleneck.
In the task breakdown, we look at where your gates are, what the pipeline actually checks, and which checks to add so that some confirmations can be removed without increasing risk.
Technical concerns: rollbacks, migrations, access
Concerns about rollback, migrations, and access usually surface before the start — these are the right questions, not an obstacle. Below are short answers to those engineers ask most often: about the existing pipeline, automation risks, and return points.
What will happen to the existing pipeline?
We do not tear anything down or rewrite it blindly. First we capture the actual state: how artifacts are built, where configurations are stored, who rolls out today and by what procedure.
Automation is layered on top of the working pipeline, switching goes service by service, and the previous method remains a return point until the migration is complete.
What about data migrations?
Migrations are separated from code rollout and made backward compatible: the schema changes in two steps — first the new is added, removal of the old happens after the release has settled.
Then rolling back a release does not require rolling back the schema: the previous version continues to work with the current data structure.
Who has the right to roll out?
Rights are tied to a role and environment, not to a person. Rollout to the production environment is confirmed by a second pair of eyes, access is issued for a limited time and closed automatically, and all actions go into the log. This removes dependence on a single engineer who "knew how it should be done."
What risks do we take with automation?
The main risk is reinforcing a chaotic process with automation: then errors will start reproducing faster. Therefore, we first fix the rollout procedure and rollback rules, and only then transfer them to the pipeline.
Separately, we build in safeguards: stopping by metrics and a manual brake for the responsible engineer.
Let's discuss the task: what we need from you for assessment and a plan
To come back with a realistic assessment and plan, one breakdown and several artifacts are enough for us.
Deployment frequency cannot be increased "on the industry average": the bottleneck may sit in manual release approval, in a long build, or in thin test coverage of critical paths — and the treatment is different in each case. Therefore, we collect the source data before assessment, not after.
- diagram of the current pipeline: the path from a change in the repository to production, with manual steps and approval points;
- release statistics for the last quarter — number of deployments, share of rollbacks, time from change to rollout;
- composition of environments and synchronization rules between them, who gets access and how;
- build and artifact storage: how long the run takes, whether there is dependency caching, what the version retention policy is;
- tests: what is checked automatically, what manually, which paths remain uncovered;
- rollback policy: how you return the previous version and who makes that decision;
- constraints: regulations, release windows, security and approval requirements.
This is enough to come back with a stage-by-stage work plan, control points, and the expected metric shift. We agree on the order of work before the start so that the first result is visible in a short segment, not at the end of the project.
The next step is to describe the task in writing or on a call: we will name where we will start and what will change first.







