Automatic deployment: where manual change release breaks
Manual change release breaks not on complex logic but on repetitive operations.
A developer builds a package locally, copies it to the server, and edits configuration according to an instruction — and already here environments diverge: staging has one set of variables, production another.
Then the divergence surfaces at the worst possible moment: a failure during release looks like a code error, though the cause is a missing parameter.
The second source of failures is the release instruction itself. It has dozens of steps, some are performed "from memory," some are outdated after the previous release.
The human factor in release manifests not in carelessness but in the fact that the order of actions is held in one engineer's head. A forgotten migration step or skipped cache warm-up causes a crash after rollout, not before it.
The third is the release window. To avoid disturbing users, releases are shifted to night or weekends, when neither support nor analytics is around.
Diagnostics at such a time is more expensive, rollback is done manually according to the same instruction, and every error stretches the window by hours. The less often we release, the larger the changes and the higher the cost of failure.
Automatic deployment on merge to main removes exactly these three points: the environment is assembled from a single description, the steps are performed by the pipeline, and rollback is a return to the previous verified state. Let's break down how this is covered.
What automatic deployment gives: time, failures, downtime
Manual release is stretched not by the build or checks but by waiting: you need to lift the freeze, synchronize migrations, make sure the configuration on staging matches the reference, and gather people for this.
The pipeline performs the same steps with one command after merging to main — the time to production becomes the length of the build, not the length of approvals.
The first measurable effect is the time from merge to production. It stops depending on who is on call today: the same set of checks runs for both a small fix and a release of a dozen tasks, so you can release more often and in smaller increments.
Release frequency does not grow by itself: release stops being an event, and small portions of changes are easier to verify and faster to roll back.
The second is the class of failures that stops reaching users altogether.
Environment divergence, a forgotten migration, different artifact versions on staging, configuration edits bypassing the repository — this is caught before production because the path of changes is single and reproducible, and checks run automatically, not "if someone remembered."
The third is downtime due to release. When rolling out via Kubernetes, instances are replaced one by one, and an unready one does not receive traffic: an update stops being a downtime window.
Rollback is symmetric — returning to the previous version takes as long as the release and does not require manual actions under pressure.
Change release modes: comparison and limitations
The release mode defines what serves as the trigger for rollout and how quickly a change reaches users.
Manual release according to an instruction leaves full control but keeps speed dependent on the on-call engineer's discipline; automatic deployment removes the human factor but requires mature pre-rollout checks.
The choice depends on the cost of error and the frequency of edits: the more expensive the failure, the smaller the release step. In practice, modes are combined — a version tag triggers rollout, and the user share limits the blast radius.
| Release mode | When it fits | Limitations |
|---|---|---|
| Manual release according to an instruction | rare changes, schema migrations, work under support | human factor, speed depends on the on-call engineer, manual rollback |
| Scheduled release | regular approval windows, reporting to business | changes wait for the window and accumulate into a large batch — the cause of failure takes longer to find |
| Release by version tag | deliveries to closed environments, need to roll back to a specific build | tag discipline is required, the step of creating them remains manual |
| Release by merge to the main branch | frequent small merges, short-lived branches | fast and complete checks are required, otherwise failure reaches production along with the code |
| Partial release to a share of users | risky changes, edits in high-load paths | metrics and traffic splitting by cohorts are required, incidents are harder to reproduce |
Practical guideline: automatic deployment pays off where there are already tests, one-step rollback, and observability across key paths. Without them, a version tag and a user share only mask the absence of control.
Implementation step by step: audit, design, setup, verification
An audit of the repository and environments takes more time than setting up the pipeline itself, and that is normal: until it is clear which branches live long, where secrets are stored, and how the staging environment differs from production, automation will only cement the existing order.
We look at the merge history, build time, and manual steps from the "how to release" instruction — and then design the release scheme from branch to environment.
Order of work:
- Repository audit. Branch structure, protected branches, merge history, tests in the pipeline, availability of a tag or commit to roll back to.
- Environment design. How many environments are needed, who writes to them, which data does not leave the environment. We fix exactly where the release leads.
- Release rules. What counts as a publication event: which merge to the main branch triggers deployment, which conditions are mandatory, how a hotfix is handled separately.
- Pre-release checks. Build, tests, migrations, configuration validation. Publication does not start until the mandatory set is green.
- Rights and secrets. Roles, keys, access scopes: the process gets exactly the rights it needs, not full access to the environment.
- Trial release. We run a real change, measure build and rollback time, deliberately break the build — and make sure rollback and notifications worked.
- Team handover. Procedures, documentation, review with engineers: further edits are made without our involvement.
What we hand over: configuration files and procedures
What is handed over is not "access to the control panel" but a kit the team uses to conduct releases themselves.
Everything lives in the repository next to the code and goes through the same checks as a regular change — so the configuration does not diverge from reality a month after launch.
- Release pipeline description: release configuration files in the repository — build stages, test runs, artifact publication. Any edit is recorded in history.
- Environment design: staging, pre-production, production — which variables differ, where checks are mandatory, what limits access to data.
- Access matrix: who merges changes to the main branch, who approves release, who has production rights. Roles are described explicitly, not granted from memory.
- Secrets policy: where keys are stored, how and when they are rotated, what is guaranteed not to end up in logs and built artifacts.
- Rollback procedure: how to return the previous image, how long it takes, who makes the decision and how the result is verified.
- Release procedure: rollout windows, requirements for change size, on-call and shift handover.
- Team documentation: pipeline map, breakdown of typical failures, checklist for the first independent release.
The kit remains a working instruction, not a project report: it is used with every change, and it also shortens investigation time when a release goes wrong.
Case study: release by merge without night shifts and rollbacks
The “Severny Kontur” team kept release manual: releases were deployed at night because a downtime window during the day was unacceptable.
A release took 40–60 minutes from three engineers, rollback was done by hand and took about two and a half hours, and a failure in production occurred in roughly every fourth release.
What changed
Merging to the main branch now triggers the pipeline: building an immutable artifact with the commit number, migrations as a separate step with reversibility checks, rolling out in parts across nodes with automatic metric control.
If the error rate on the new version goes beyond the boundary, rollout stops and traffic returns to the previous artifact — without human involvement.
What we got
A release takes 10–12 minutes and happens at any time of day; night shifts and on-call duty are gone.
Failure frequency dropped to one case per twenty releases, recovery — 3–4 minutes instead of hours, because rollback is done by returning to the previous artifact, not by step-by-step investigation.
The team moved from a weekly release to several releases per day: preparation stopped being an event, changes in one release became smaller and therefore are analyzed faster.
Merge to the main branch or version tag: what to choose
Release by merge to the main branch is the simplest setup: branch protection rules allow changes only through a merge request; after approval and successful checks, the pipeline builds the artifact and deploys it to the target environment.
The version here is tied to the commit, so at any moment you can see exactly which set of changes is running in production. Rollback is redeploying the previous artifact, not a manual edit on the server, and it takes minutes.
Release by version tag separates the moment of build from the moment of merge. Changes can accumulate in a branch for as long as needed, but only the commit that has a tag goes to production.
This gives control over exactly what is released and allows maintaining several lines simultaneously — for example, a fix for a previous version when the main branch has already moved ahead.
The price is discipline: the tag must be placed consciously, and the difference between "merged" and "released" becomes explicit and requires maintenance.
The fork is decided by the number of supported versions and audit requirements. If the product lives on one line and deployment happens when ready, the merge-to-main-branch scheme is simpler and cheaper to maintain.
If several versions live in parallel for different customers, there is a release approval procedure, or a reproducible set of artifacts is needed for a specific delivery — you cannot do without a version tag, otherwise you will have to keep manual records of what went where.
A separate case is an already deployed scheme on long-lived branches, where there are permanent lines of development, release preparation, and urgent fixes.
There is no need to break it in one step: first we fix the main branch protection rules and mandatory checks in the merge request, then we switch build pipelines to trigger from merge, and close long-lived branches as their contents are transferred. Tags remain as release markers and a rollback point.
A practical compromise is to build and deploy by merge to the main branch, and mark with a tag the commit that went to a specific environment or delivery.
Then the release history is read by tags, rollback relies on the previous tag, and the team does not waste time synchronizing branches manually.
Common technical questions about release setup
Automatic release stumbles not on the build itself but on the questions an engineer asks before starting: what will happen to the database, how to roll back, where secrets go. Below are short answers to doubts that most often surface when designing such a scheme.
How do database migrations work?
Schema changes are moved to a separate step before code deployment and written according to the expand-contract principle: first we add a column or table, the code writes to both versions for some time, and only then we remove the old one.
Long changes are tested on a copy of the production database — locking a table for minutes stops the entire service.
What about backward compatibility and rollback?
The rule is simple: each code version works with both the old and the new schema. Then rollback is switching to the previous image, not restoring the database from a copy.
Migrations are not rolled back: the schema remains expanded, extra fields are removed in a separate release when the old code no longer runs anywhere.
What if checks fail?
Merging to main only triggers the pipeline: build, tests, security checks, responsible person's confirmation. Until a red check turns green, the release does not proceed — it cannot be bypassed with administrator rights; that is a separate logged action. A failed check costs one stopped build, not a night rollback in production.
Who gets access to secrets and the right to release?
Secrets are stored in a vault and injected at execution time — they do not end up in build variables, logs, or artifacts. The right to release is separated from the right to merge: a developer merges, but the responsible person confirms deployment.
Every access to a secret and every release is written to a journal — you can see who deployed what and when.
Let's discuss your task: what to send for assessment
The more precise the inputs at the start, the fewer clarifications in the middle of the work — and therefore the estimate will not drift.
Below is a list of what actually affects the scope of automatic deployment setup: if you send this in one message, we will return with a plan and timelines by stage.
- Repository and branch structure: how many there are, what naming convention is used, whether the main branch is protected and who has the right to merge into it.
- Current build: one job or several, how long a full build takes, whether dependencies and artifacts are cached between runs.
- Delivery environments: how many there are (test, pre-production, production), whether a separate manual gate before production is needed and who passes it.
- Access and secrets: where keys and tokens currently live, whether there is a secrets vault, whether you have your own task runners in a closed environment.
- Tests as a condition for merge: which runs are mandatory, what to do on failure — block the merge or only stop deployment.
- Rollback requirements: how many previous releases to keep ready for return and how long the return should take.
- Environment limitations: prohibition of outbound connections, requirements for access logs and build history storage.
Describe your task — we will return with an estimate and work plan: we will fix the stages, divide areas of responsibility, and determine how we will verify the result on the very first release.