Containers on AWS: ECS, autoscaling, and CI/CD for production

AWS ECS Setup: Where Containers Fail in Production

Container failures in production rarely look like a process crash—usually it is degradation at the junction of components.

The database connection pool is not aligned with the idle timeout on the load balancer: the connection closes before the application releases it, and after reconnection the database receives a burst of requests from all tasks at once.

Version updates are the second most frequent source of failures. A new task definition revision starts while the old one is still taking traffic: the readiness check responds before the application has warmed the cache, and some requests fail.

Schema migrations that each task runs at startup, with several instances, turn into a race for the same tables—the update does not roll itself back.

The third scenario is a load spike: scaling reacts to the metric with a delay, and the surviving nodes take the entire peak.

Resource shortages on them knock tasks out by memory, they go into restart, and the service loses even more capacity—an avalanche at the worst possible moment.

Configuring AWS ECS is primarily about service behavior under failure: how tasks survive connection breaks, how they are drained during updates, what metric they scale by, and what happens when resources are insufficient. We examine how this is addressed in each scenario.

What Changes After Moving Services to Managed Containers

While a service lives on individual servers, a version release is a coordinated window, a nighttime call, and manual deployment, while rollback is a separate operation under pressure.

In managed containers, the version is described declaratively: an image in the registry, a task definition with parameters, and a new revision is rolled out gradually.

Rollback is a return to the previous image tag, minutes instead of hours, and you can release even every day, not once per sprint.

Peak load is no longer handled manually: the scaling policy is tied to a metric—CPU load, request queue length, number of messages in the broker—and adds tasks by trend, not when the service is already down.

The decline is handled by scaling back so as not to keep excess resources. The load balancer drains traffic from an unhealthy instance and returns it only after a successful health check—degradation of one instance does not turn into a general outage.

Automatic recovery removes a whole class of failures. A failed task is replaced by the scheduler without on-call intervention; when a node fails, its tasks move to healthy ones.

Previously this was an incident with manual investigation; now it is a log entry that no one wakes up for.

Downtime is measured not by feeling but by the share of failed requests over a period and the time to full recovery.

This share depends on how carefully AWS ECS is configured: automatic restarts, health checks, and limits on the number of restarts keep it in tenths of a percent where manual on-call produced minutes of unavailability.

Implementation Order: From Architecture Audit to Handover to Support

The work follows a sequence where each next step relies on the agreed result of the previous one—this way changes reach production predictably, rather than in one big release.

Below is a typical order for a container cluster; timelines depend on the state of the current architecture and the number of services.

1. Infrastructure audit. We analyze where containers run, how images are built, what limits are set, and how many operations are performed manually. The output is a dependency map and a list of bottlenecks that cause the architecture to fail under load.

2. Target architecture design. We agree on how everything will be arranged afterward: division into services and environments, placement in the cluster, network boundaries, roles and access, rollback strategy.

   The design becomes the configuration specification—contentious details are resolved before work begins, not at rollout time.

3. Autoscaling setup. We tie policies to metrics that reflect real load, not indirect signs. Scaling triggers within minutes, not after the request queue has grown; we separately set upper and lower bounds so as not to go into endless growth or shut down the service prematurely.

4. Build and delivery pipeline. We set up automatic build, checks, and rollout with rollback to the previous image in one command. Manual steps remain only where human judgment is needed.

5. Load testing. We run a profile close to peak: we see how autoscaling performs, where the database hits its limit, which requests degrade first. Based on the results, we adjust thresholds and limits according to actual measurements.

6. Handover to operations. We provide documentation on the architecture and policies, update and rollback procedures, access, and on-call instructions. The customer's team conducts the first release with our support—handover is confirmed by practice, not by a list of files.

Access Policies, Network Designs, and Release Procedures

A cluster assembled without fixed rules lasts until the first new engineer: permissions are granted 'to make sure it works,' the network is described only in the author's memory, and a new service version is released manually from the console.

We deliver infrastructure with documentation that allows it to be handed over to another team without reconstructing context. Below is the package the customer receives based on the results of the work.

• Roles and permissions — description of service roles for tasks, separation of access between build, runtime, and on-call engineer, the principle of least privilege, and abandonment of long-lived keys in favor of temporary credentials.
• Network access design — which subnets are public, which are private, where the load balancer and entry points are located, what exactly security groups allow, and why the boundaries are drawn that way.
• Release procedure — change order, set of automated checks before release, who approves, and how an emergency fix is performed bypassing the usual window.
• Rollback plan — criteria for stopping a release, procedure for returning to the previous task version, and the time within which the service again accepts traffic.
• Autoscaling policies — which metrics expand and shrink the number of tasks, what bounds are set and what justifies them.
• Access matrix — which employees have rights to what and how rights are revoked when a person leaves or a contractor changes.
• On-call instructions — typical failures, what to check first, where logs and metrics go, whom to contact outside working hours.

How We Cut Release Time and Eliminated Failures During Peaks

The project came to us with a typical picture: an application on several always-on servers, manual release according to instructions from an internal knowledge base, and traffic peaks handled by an on-call engineer—phone in hand.

A version reached production in 4-5 hours and was accompanied by a short service interruption, while a failed release was rolled back manually for several more hours.

What We Changed in Deployment

We replaced manual steps with immutable images and described the infrastructure as code on AWS: how many tasks to keep under baseline load, by what indicator to add new ones, and after what pause to scale back.

We switched deployment to gradual replacement—the new version first takes part of the traffic, and only with a clean error graph is the previous group taken out of service.

Metrics Before and After

• version release: 4-5 hours with a service interruption → 12-18 minutes without stopping the service;
• failure recovery: 5-6 hours of manual investigation → 20-30 minutes by rolling back to the previous image;
• share of releases that resulted in an incident: every third → one in twenty.

The first serious peak came a few weeks after the migration: load grew severalfold in a quarter of an hour.

Automation added tasks, distributed them across availability zones, and returned to the baseline number after the decline—response time remained normal, and the on-call engineer intervened only to check the graphs. Previously such a scenario ended in failures and manual configuration edits.

Common Technical Doubts Before Starting Work

Before starting, engineers rarely ask 'does ECS work at all'—they are interested in integration with what is already deployed. We have collected the questions that most often come up at the assessment stage, with short answers to the point.

The architecture is almost always hybrid: containers live alongside the previous services until those are migrated—the transition does not turn into flipping a switch.

Will there be downtime during migration?

No downtime window is required: the new version comes up in parallel with the old one, passes the health check, and only after that does the load balancer direct traffic to it. If it fails, we return the previous image.

How is container security handled?

Images are built on a minimal base and scanned for vulnerabilities during build. Tasks run under separate roles with access only to the required resources; secrets are injected at launch from a vault rather than stored in the image.

What about operations and support after handover?

We set up logs, metrics, and alerts before handover: consumption, number of tasks, and response time are visible. Autoscaling is tied to load—capacity grows during peaks and does not sit idle during quiet hours. After that—support according to the agreed scope: updates and incident investigation.

Let's Discuss the Task and Come Back with an Estimate and Plan

To come back with a work estimate rather than generalities, we need a minimum of context about your infrastructure. Below is the review order: what to send, what we will look at ourselves, and what will be included in the implementation plan and support.

• Current production architecture: which services are already in containers, what runs on a managed orchestrator, and what remains on individual servers.
• How images are built and rolled out: where steps are done manually, who starts the release, and how a failed release is rolled back.
• Load profile: peaks by hour and day of week, how many requests one service instance handles, and at what threshold you hit the ceiling.
• Fault tolerance requirements: how many availability zones are needed, what downtime you consider acceptable, and what should recover without on-call intervention.
• Access and constraints: accounts, network segmentation, security policies, compliance requirements, and who on your side makes infrastructure decisions.
• What hurts most: long rollouts, growing costs for storing logs and images, failures under load, or lack of observability.

The work plan will include the target service placement architecture, task configurations and autoscaling rules, a build and delivery pipeline with checks at each step, observability with clear alert thresholds, and a rollback procedure.

We describe support separately: what we do, what remains with your team, and how we transfer knowledge during the work.

Describe the task in free form and attach what you have on hand—we will come back with an estimate and plan.

What to Do If Some Services Are Already Deployed on Servers?

A full migration is not mandatory—in most systems, a partial one is more appropriate.

Containerization pays off where a service is stateless and released frequently, while databases, message brokers, and file storage are better left on the previous machines or moved to managed services.

The migration order is dictated by dependencies, not convenience.

First, stateless services: an ECS service comes up alongside, a share of traffic is gradually shifted to it through a target group, latencies and error rates are compared—and only then are the old instances shut down.

Next come background workers without incoming traffic, and last is the stateful part according to a separate plan.

The mixed architecture rests on a shared network: containers and remaining servers end up in the same perimeter and see each other by internal addresses, so integrations do not need to be rewritten.

Releases go through a single build pipeline—otherwise there will be two inconsistent release processes. Standard AWS tools are sufficient for this: an image registry, versioned task revisions, and target groups for smooth traffic shifting.

We mitigate risks in stages, not all at once: first rollback—the old platform and previous task revision remain operational, then access and network rules, then data durability during node failure.

We enable autoscaling after metrics stabilize; otherwise it masks overloads instead of revealing them.