Setting up AWS cloud infrastructure for production

Configuring AWS for production load: what breaks

Manual environment configuration almost never breaks at launch — it breaks during growth. The virtual machine was provisioned for a specific task: the instance class was chosen by eye, security group rules were adjusted on the fly, and automatic scaling was simply not configured. While the load is steady, this works and quietly accumulates debt.

The first thing to fail is not the server but the connections between services. The database connection pool is exhausted at peak, some requests wait for a connection to free up, response time grows, and client retries finish off what was still holding on.

There is no way to add instances under load: images, environment variables, and network rules live in the console and in the head of the engineer who assembled it all.

Next, environment drift becomes apparent. The production environment differs from the test one by edits accumulated over months: it cannot be deployed from scratch, and rollback to a previous state turns into night work with an unpredictable outcome.

That is why infrastructure bottlenecks are not visible until peak — monitoring is configured for the "alive or not" signal rather than queue saturation, and failure under peak load comes without warning.

We break down how this is addressed: declarative resource descriptions, metric-based scaling, separate roles and permissions instead of shared access, observability across key queues and headroom.

What well-built infrastructure delivers: measurable effect

Infrastructure should be evaluated by four measurable values, not by the number of services deployed. Below are benchmarks that show the work has actually been done, rather than simply "configured": for each item, we record the value before the start and after delivery.

• Speed of change deployment. The pipeline builds, tests, and delivers every change without manual involvement: the path from commit to production environment takes minutes, and rollback to the previous version is one command. Deployment stops being an event requiring a maintenance window and on-call duty.
• Recovery time after failure. Backups are regularly verified by restoring to a separate environment, and standby nodes are warmed up in another availability zone. Traffic switching is automatic, so recovery fits within tens of minutes rather than hours of manual troubleshooting.
• Reduction of manual operations. Environments are described declaratively and provisioned from code, so drift between environments disappears, and the share of edits through the web console drops to almost zero. A new environment is deployed with the same configuration as the production one.
• Eliminated class of failures. Automated pre-deployment checks and configuration control remove a whole group of failures — caused by unplanned manual edits and differing settings across environments. Incidents of this type simply stop appearing in the log.

In AWS, all four values are captured by native monitoring tools, so progress is visible on a dashboard rather than in "verbal" reports. If at least one metric has not moved, then a bottleneck remains and needs to be pushed further.

AWS configuration options: comparison of approaches

Let's look at deployment options that teams actually choose between in projects: they differ not in the set of services but in the way the environment is managed. The key criteria are how often the infrastructure changes and how costly it is to lose it.

In practice, the options do not compete but replace one another: a single account gives way to separated environments, on top of which cross-availability-zone redundancy is added.

The transition is not a rewrite from scratch but a transfer of already working settings into a description, so downtime at this step can be avoided.

Implementation stages: from audit to handover

Configuring AWS for production operation is not a one-time installation of services but a sequence of steps where each relies on the result of the previous one.

If you design the target configuration before the audit, some implicit connections and constraints will surface under load. Below are six stages with a verifiable result at each.

1. Infrastructure audit. We collect the actual state: what is deployed, how requests flow, where the configuration diverges from documentation, which permissions were granted with excess. The result is a dependency map and a prioritized list of bottlenecks.
2. Target configuration design. Isolated environments, network segmentation, access roles, secrets management, limits, and scaling rules. We agree on the design before work begins: changes here are cheaper than changes in code.
3. Implementation. We move configuration into code — environments are provisioned from templates, changes go through review, and manual edits in the console are excluded. The infrastructure state becomes reproducible, and drift is visible immediately.
4. Failure testing. We break things deliberately: shut down a node, revoke access, sever a connection between segments. We observe whether the system recovers on its own and how long it takes — before this happens by accident.
5. Documentation. Design, environment variables, recovery procedure, service owners, and control points. A new engineer can provision the environment from it without verbal explanations.
6. Handover to support. We walk through the change and on-call regulations with your team, transfer access, and record the escalation procedure and responsibility boundaries.

After that, changes follow the same order: design, implementation, testing — so the infrastructure does not become overgrown with manual exceptions.

Deliverable set of documents and configurations

A working infrastructure without documentation is a black box that is frightening to change.

Therefore, together with the environment, the customer receives a set of documents that allows it to be developed and operated without us: a new engineer figures things out from the documents, not from chat messages.

• Infrastructure as code — networks, subnets, security groups, load balancers, and databases are described in the repository. Any change goes through review and remains in history; no manual edits are left in the console.
• Access policies and roles — a "who can do what" matrix, least-privilege permissions, separate roles for services, on-call engineers, and build. Secrets are stored in a secure vault with rotation.
• Network and data flow diagram — which services communicate with what, where private and public segments are, how entry points are arranged, and what is closed from external access.
• Backup and recovery regulations — what is backed up, how often, where it is stored, how recovery is verified, and how long it takes based on actual measurements.
• Infrastructure documentation — architecture diagram, environment descriptions, variables and dependencies, deployment and release rollback procedures.
• Monitoring and alerts — a set of metrics and trigger thresholds, routing notifications to on-call staff, analysis of typical incidents and actions for them.
• Knowledge transfer to the team — joint walkthroughs and paired on-call: your engineers go through the path "change → deployment → rollback" with their own hands.

The set remains in your repository and internal knowledge base — not in someone else's account and not in the contractor's head.

Case study: peak load without incidents and manual edits

A store with seasonal sales: traffic grew tenfold in minutes, and the infrastructure on AWS scaled manually. The on-call engineer provisioned virtual machines from a checklist, edited load balancer routing, and monitored graphs at night.

Recovery after a failure took about 40 minutes, and almost every action remained manual — from changing group sizes to restarting dependencies.

What was changed

The infrastructure was described as code: network, access permissions, databases, and queues are assembled from the repository, not in the console.

Autoscaling was switched from a daily average to current load, application health checks were added, and behavior for an availability zone failure was specified in advance. Changes go through review and are repeated one-to-one in the test environment.

Metrics before and after: recovery time — from 40 minutes to 3–4 minutes; share of operations requiring manual work — from 70% to 9%; incidents at peaks — five per quarter versus zero over six months of observation.

Images and configurations are versioned, so rollback to the previous release takes minutes, not hours.

Behavior at peaks became predictable: capacity is added as request volume grows, and excess is removed after the decline, so storage and compute costs do not grow along with traffic. The engineer gets involved in solving new tasks, not firefighting.

Service resilience holds only as long as redundancy designs and regulations are updated together with the product.

How does a managed service differ from your own server?

The "service or own server" debate is usually not about technology but about where the boundary of responsibility lies.

In the managed model, the provider maintains physical nodes, fault tolerance, updates, and backups, while you are responsible for configuration, access, and the data schema.

On your own servers, the entire stack — from hardware to application patches — remains with the team and cannot be outsourced beyond the contract.

Managed services remove work where the task is standard: relational database, message queue, cache, object storage.

Replacing a failed node, replication, and minor version updates happen without your engineers' involvement — in AWS, this is built into the design of the services themselves.

The team is left with what truly affects the product: schema and indexes, permission separation, backup strategy, and recovery verification.

Own servers win where deep control over the environment is needed: non-standard kernel parameters, specific network configuration, licensed software with strict binding, regulator requirements for data placement.

Another case is stable high load with a predictable profile: then the node fleet is tuned to your task, rather than the architecture being adjusted to the limitations of a managed platform.

The compromise has a price. The deeper you use a provider's unique capabilities, the more expensive migration becomes: lock-in grows along with the amount of routine work removed.

The flip side of your own servers is the size of the responsibility zone: on-call duties, security updates, monitoring, and failure recovery become your permanent line item, not an entry in someone else's regulations.

In practice, it is almost always a hybrid: managed services where there is no differentiating value, own nodes where control and predictability are needed. The decision is made at the design stage, when the load profile, data requirements, and acceptable downtime are clear.

Technical concerns before starting work

Engineers and technical leads ask almost the same questions before starting: what happens to availability during changes, what the first stage will be, how reproducible the environments are, who is responsible for the environment after handover. We answer before work begins — the process design depends on this.

Can implementation be done in parts?

Yes, the environment is broken down into stages: network and access, code build and delivery, observability, and backup. Each stage addresses its own task and works independently — you can stop after any of them and continue later.

Will environments be reproducible?

The infrastructure is described declaratively and stored in a version control system.

An environment is provisioned from a single description, and differences are moved into parameters — the test environment can be redeployed and get the same configuration as the production environment.

What about responsibility after handover?

We transfer infrastructure descriptions, access policies, diagrams, and documentation, and train the team. The support period is agreed separately: until your engineers are ready to make changes independently, we stay in touch.

Can a change be rolled back?

Yes. The configuration is versioned, and rollback is a return to the previous description and its application. We restore data from snapshots if the change affected storage.

Let's discuss the task and come back with an estimate and plan

To make the estimate based on facts rather than assumptions, a few inputs are enough to start. Gather what you have at hand — draft diagrams and notes will do; you won't have to reconstruct the documentation.

• List of services and their role in the product: what serves users, what runs in the background.
• Current load profile — peak requests, data volumes, seasonal spikes.
• Requirements for environments: how many environments are needed and how they differ in access.
• Constraints: deadlines, fixed integrations, security and audit requirements.
• Target metrics: acceptable downtime, recovery speed, backup retention depth.
• Release process — who deploys changes and how often, whether automatic rollback is needed.

In response, you receive a working package, not general words.

A target architecture design with network boundaries and access roles, the order of stages with points where the result is already visible, the scope of work with a division of responsibility zones, and a list of what remains with your team after handover: infrastructure as code descriptions, backup policies, and recovery regulations.

Describe the task in free form — we will come back with an estimate, implementation plan, and scope of work.