AWS CloudFormation: Where Manual Cloud Configuration Breaks
Manual cloud configuration does not break on the first launch, but on the third or fourth change. An engineer adds a rule to a security group, spins up an instance, attaches tags — and repeats this in dev, stage, and prod from memory.
A week later, the environments diverge: logging is enabled in stage but not in prod, the image version is updated on one environment while the previous one remains on another. No one knows which configuration was working at the moment of the incident.
Configuration drift is invisible until the first serious failure. When prod goes down, recovery turns into archaeology: some of the settings have settled in notes, some — in the head of the engineer who happens to be on vacation.
Recovery time grows not because the team works slowly, but because the environment cannot be reproduced from scratch.
Infrastructure as code removes this at the model level: the description of an environment lives in templates, changes undergo review and enter version control rather than the console.
AWS automation through CloudFormation makes every environment the result of applying the same description — from networks and load balancers to access rights.
Environment reproducibility no longer depends on who exactly built the environment and in what order they pressed buttons.
We examine how this is addressed in practice: where the boundaries between templates run and what to do with resources that are already configured manually.
What Changes After Moving to Infrastructure Templates
The first environment assembled from a description sets a reference point: if previously an environment was built manually over several days and each one came out slightly different, now deployment takes 15-20 minutes.
The environment reproduces the network, rights, security groups, and service parameters — without the “forgotten” settings from the checklist.
Next, the very nature of rollbacks changes.
Configuration drift between environments — the most common source of “it works locally, but not in prod” — disappears as a class: the share of rollbacks for this reason drops from half of releases to isolated cases, because all environments are assembled from a single template.
After a few weeks, recovery time noticeably decreases. The emergency environment is not repaired manually but recreated from the description — tens of minutes instead of hours spent editing configurations under load.
Rolling back to the previous version of the environment becomes as routine an operation as a release.
The cost of downtime ceases to be an unknown quantity. The unavailability window no longer stretches into diagnostics: it is limited by template application time, and this time is known in advance and can be included in the service level agreement.
Over months, the effect accumulates: changes are verified before application, environments are spun up for each task and torn down the same way, and environments remain identical.
AWS CloudFormation here works not as a tool for the sake of a tool, but as an executable infrastructure description that lies in the repository next to the service code.
Ways to Describe Infrastructure: Comparison of Approaches
Declarative templates describe the desired state, and the tool itself brings the account to it — this removes the discrepancy between documentation and reality. But the choice of a specific approach determines how many engineering hours go into maintenance each month. Below are the options that are actually chosen between, and their limitations.
| Option | When it fits | Limitations |
|---|---|---|
| CloudFormation templates (YAML/JSON) | All infrastructure in one cloud, native stack rollback and integration with access roles are needed | Cumbersome syntax for large descriptions, rollback occurs entirely across the stack |
| Modules in a programming language (CDK, Pulumi) | The team writes code and wants reusable modules, loops, and type checking | A build step appears, library version discipline is needed |
| HCL descriptions (Terraform) | Several clouds and services from different providers in one environment | Separate state storage, locks are coordinated manually |
| Configuration scripts (Ansible) | Bringing already running machines to a uniform state, one-off operations | Do not store state, repeated runs produce different results |
Infrastructure modules remove the main pain point: a typical network, access roles, and logging are described once and connected via parameters.
Instead of a copy of the template for each environment, there is one module and a values file — an edit in it changes all environments at once, rather than three times manually.
The limitations of automation should be acknowledged in advance.
Resources created by applications, data in storage, and edits through the console remain outside the description — therefore the process includes regular comparison of the actual state with the template and the rule that the stack remains the single source of truth. Otherwise, the first manual change nullifies all the benefit of automation.
Implementation Sequence: From Audit to Handover of Procedures
The order of work here is more important than speed: a template assembled without inventory describes the wrong infrastructure, and pushing it straight to prod turns a mistake into downtime.
Therefore, the implementation stages proceed sequentially, and each next one relies on the result of the previous one.
-
Infrastructure audit. We go through accounts and regions, record the actual resources, the connections between them, and changes made manually bypassing descriptions. The output is an inventory ledger and a list of discrepancies with the documentation.
-
Dependency scheme. We distribute resources across stacks: network, compute, storage, access. We determine the creation order and what cannot be changed without downtime — this sets the boundaries of the CloudFormation templates.
-
Template assembly. We describe resources declaratively, parameterize environment differences, and move secrets separately from descriptions. Repeating blocks are folded into shared fragments — edits are made in one place.
-
Testing in an environment. We deploy a copy of the environment in an isolated setting and run the full cycle: creation from scratch, modification, deletion. We catch discrepancies with the expected change plan before the production environment.
-
Promotion to prod. We move resources under description management in stages, verifying the state after each step. The previous configuration is preserved until the result is confirmed — rollback remains possible.
-
Handover of procedures. We hand over template descriptions, the stack scheme, the edit order and rules for change verification, and rollback instructions. The customer's team makes changes themselves, without contacting us for every step.
Set of Templates, Schemes, and Handover Procedures
After implementation, the customer's team is left with a working repository containing infrastructure descriptions, not a set of screenshots from the console.
The package is designed for an engineer who did not participate in the work: using it, one can deploy an environment from scratch, see the connections between services, and safely make a change.
- Environment templates — CloudFormation descriptions for development, test, and production environments. They differ by parameters, not code copies: an edit in one place does not diverge across three versions.
- Dependency schemes — the order of creating and deleting resources, nested stacks, passing values between them. The scheme shows what a change to a specific resource will affect.
- Access policies — roles for deployment and for applications, permission boundaries, separation of manual edits and changes through a template.
- Change procedure — branch, build, review by a second engineer, approval for the production environment, rollback procedure to the previous version.
- Documentation — purpose of parameters, output values, instructions for deployment from scratch, known limitations and workarounds.
- Protections against accidental destruction — prohibition of stack deletion, policies for updating critical resources, regular reconciliation of the actual state with the description.
- Catalog of reusable fragments — typical blocks of network, storage, and permissions that are connected into new services instead of being copied.
We hand over the package with a walkthrough on the running infrastructure and annotations of where everything is located. After that, your team makes changes themselves.
Case Study: How Environment Deployment Time Was Reduced
The environment for acceptance and pre-production was assembled manually according to an instruction: networks, load balancer, databases, access rights.
Full deployment took 6–8 hours of work by two engineers, and discrepancies between environments surfaced during testing — some changes were rolled back with reverse edits directly in the console.
We described the infrastructure declaratively in Amazon Web Services CloudFormation templates: a separate stack for each environment, shared parameters moved to a separate layer, changes undergo review before application. Rollback stopped being a manual operation — returning to the previous stack takes one step.
Before and After
Deploying the environment from scratch was reduced from 6–8 hours to 20–25 minutes, and one engineer launches it, without waiting for the others.
The share of failures during rollout decreased approximately threefold: the environment is reproduced exactly, because changes come from the description, not from manual edits.
Recovery time after a configuration failure dropped from several hours to 10–15 minutes — the environment is simply reassembled.
A separate effect is new environments: a test environment for a task is spun up in half an hour instead of a week of approvals and assembly. Templates and parameters became a shared asset: the team adds environments by copying a layer, rather than repeating the instruction.
How Does AWS CloudFormation Differ from Manual Scenarios?
A script executes a sequence of commands: create, configure, connect. It does not know what has already been done, so a repeated run either fails on an existing resource or creates a duplicate.
AWS CloudFormation works on a different model — the template describes the desired state, not the order of actions, and the system itself brings the infrastructure to this description.
The key difference is state management. The stack stores information about the resources actually created, so reapplying the template calculates the difference between the description and reality and changes only that.
The same template is applied repeatedly with the same result, and a targeted parameter edit does not rebuild the entire environment from scratch.
Next, the template becomes the single source of truth. It resides in version control and undergoes review like code, so the change history shows who changed a specific parameter, when, and why.
Edits made manually in the console surface as a discrepancy with the template, rather than living in the infrastructure for years as a detail unknown to anyone.
Updates also proceed differently: before application, you can see what will be created, changed, or deleted, and if an error occurs, the stack returns to its previous state.
An unsuccessful run does not leave half of the environment in an unknown state — recovery and deployment of a new environment are performed according to the same scenario.
Technical Doubts Before Starting Work
Doubts before starting are usually technical: what will happen to the running environment, how to control edits that bypass templates, and who gets access to production.
There is already a running environment — should everything be rebuilt?
No. We describe resources in templates and connect them to existing settings without recreating them: services continue to run. After that, changes go only through the template, and the console remains for reading state.
Can it be implemented in parts?
Yes, by layers: network, data, applications. Each layer is a separate stack with explicit dependencies, so automating one area does not block the others and does not require stopping work.
How to track manual edits?
Drift detection shows changes made bypassing the template. They are either returned to code or deliberately recorded — no unknown deviations remain in the environment.
How to divide access rights?
Roles are tied to stacks and environments. Changes reach prod only through a pipeline with review; developers have the test environment and read access to the prod state.
What about rolling back changes?
Before application, we show the plan: what is created, changed, deleted. Rollback is a return to the previous version of the template; the stack state is brought to it.
Let's Discuss the Task and Return with an Implementation Plan
At the first review, we look not at the phrasing “we need code instead of manual edits,” but at the actual state of the infrastructure: what is already described in AWS CloudFormation templates, and what is still spun up manually from the console.
We clarify the number of environments and accounts, the separation of production and test environments, the team's access scheme, and how changes currently reach the production environment — by manual edit or through automated deployment.
Separately, we record what cannot be touched without extreme necessity: databases with data, network rules, domain bindings.
We return with a document, not verbal promises: it shows what the work consists of, how long each stage takes, and at which step you get an interim result.
- infrastructure map: which resources move under template management, which remain outside code and why;
- scope of work by stages — from preparing descriptions to migrating environments and configuring deployment;
- timelines for each stage and points where you see a working result, not a report;
- migration procedure with a rollback plan to the previous state at each step;
- rules for resource naming, splitting into reusable blocks, and storing parameters;
- list of checks that are run before applying changes to the production environment;
- acceptance criteria: what is considered completed at each stage and who confirms it.
Describe the task — the number of environments, the current deployment method, and access restrictions are enough for us to return with an estimate and a work plan.