Ansible Role Instead of Manual Configuration: Where Environments Diverge
While there are only a few environments, configuration lives in three places: in the repository, in the engineer's head, and in the command history on the server.
A manual edit — connect to the server, adjust a parameter, restart a service — takes minutes and almost never makes it back into the repository. A month later, no one can say why the test environment behaves differently from production.
Then environment divergence starts to cost more. A release that passed in test fails in production at a step that was already considered verified: a different path to the directory, a different user, a different dependency version.
The investigation proceeds not by application logic but by inventory — what is configured where and exactly how it differs. Configuration drift accumulates unnoticed: each individual edit looks harmless.
Manual server configuration scales poorly: a new environment is assembled from memory, not from a description, and the result depends on who spun it up. Rollback becomes equally complicated — there is no way to return to the previous state if it was never recorded anywhere.
Ansible Role turns configuration into a description: the role defines the server state — files, services, permissions, and order of application — and produces the same result across all environments.
Below we break down how this is addressed: from role structure to checks that catch divergence before release.
Measurable Effect: Build, Recovery, Share of Manual Edits
The effect of Ansible roles is verified not by the declarative nature of the description but by four numbers that we capture on the same environment before and after.
The source is run logs, change history, and service availability metrics; assessments like "it became more convenient" do not go into the report.
- Environment build time. Measured from startup to the moment services accept traffic and pass checks. An environment from scratch is brought up with a single command instead of a chain of manual steps — hours of waiting turn into tens of minutes.
- Service recovery time. Roles are idempotent: a repeated run returns the node to the described state after a partial failure. Recovery comes down to reinstalling a known set of roles, not to analyzing what was configured manually.
- Share of manual edits. We calculate it from change history: edits directly on the server go away, only role descriptions and variable values change — divergence between environments stops accumulating.
- Eliminated class of deployment failures. A forgotten dependency, a different package set, differing paths and parameters are described once and verified by a run, not by the on-call engineer's memory.
Reusable configuration blocks provide one set of roles for all environments: the developer environment, test, and production are assembled from identical parts and differ only in variable values.
A new service is connected as another block, not as a copy of someone else's configuration — so changes reach all environments in a single run.
Ansible Role, Script, or Monolithic Configuration: What to Choose
The choice of approach is determined not by taste but by the number of environments and the frequency of edits. While there is only one environment and it is touched once a year, a monolithic configuration honestly fulfills the task.
As soon as dev, stage, and prod appear with different parameters, and edits happen every week, any manual step becomes a source of divergence between environments.
| Option | When it fits | Limitations |
|---|---|---|
| Custom modular sets | Multiple environments, different service composition, security rules tailored to your specifics | Requires time to design role interfaces and discipline in variable naming |
| Ready-made external sets | Typical services: database, web server, observability, task scheduler | Pulls in dependencies, updated by a third-party author, some parameters have to be overridden |
| Monolithic configuration | One environment, a narrow scope of edits, short environment lifetime | No reuse, any edit affects everything at once, testing can only be done as a whole |
| Deployment scripts | One-off migrations, wrappers over external utilities, non-standard sequence of steps | No idempotency or rollback, a repeated run on an already configured environment breaks the state |
Configuration versioning is what distinguishes a managed environment from a set of scripts in a shared folder.
Roles and environment descriptions live in a version control system, changes go through review, and a release is tagged: rollback to a previous state takes one command, not reconstruction from memory.
The practical criterion is simple — if more than one person works on the infrastructure or there are more than two environments, take the modular approach and supplement it with ready-made external sets where the task is typical.
Implementation Order: From Environment Audit to Configuration Handover
The order of work is not set by formalities: a role assembled "from memory" breaks on the first environment that is not identical. Therefore, we first record the actual state, then design the scheme, and only after that write the configuration code.
- Infrastructure audit. We compile a list of servers, their roles in the system, OS and service versions, and find manual edits that live outside the repository and will surface on the first run.
- Server inventory. We arrange machines into groups and environments, and move difference parameters into variables: the same set of roles describes both the test and production environments, differing by values, not by copies of code.
- Configuration scheme. We divide tasks into reusable roles with explicit dependencies and determine what goes into a role, what goes into environment variables, and what remains outside its boundaries.
- Configuration and run. We write roles designed for repeated execution: a change is applied once, a repeated run breaks nothing and shows no changes.
- Verification on a clean environment. We bring up an environment from scratch and run the configuration without accumulated state — only this shows whether the result is reproducible and whether there are hidden dependencies on the old machine.
- Rollback procedure. We version roles, tag releases, record the procedure for returning to the previous version, and verify it on the test environment before handover.
- Handover to the customer's team. We deliver the repository, the decomposition scheme, instructions for running and adding new servers, and conduct a walkthrough with the engineers who will maintain the configuration going forward.
What Remains in the Repository After Implementation
Project handover does not end with a deployed environment.
The repository retains a description of the machine fleet, role sets, and the rules for working with them — what an on-call engineer can use to bring up an environment from scratch without hunting down the author of the changes. Below is what exactly is recorded by the time of delivery.
- Fleet inventory. A list of machines, groups, and their purposes: what kind of node it is, what task it solves, what it is connected to.
- Configuration sets. Each Ansible Role with a description of input variables, default values, and scope of application — it is clear what can be changed and what will affect related nodes.
- Configuration documentation. How to enable a parameter, where it is overridden, and what will happen to already running machines on the next run.
- Access policies. Which keys and accounts the control node uses, who is authorized to change roles, and how permissions are granted to a new engineer.
- Rollback procedure. Role versioning, the return point, the procedure for returning to the previous set, and verification of nodes after it.
- On-call shift instructions. Standard operations, analysis of common failures, escalation procedure, and the boundaries beyond which a developer is called.
- Handover to maintenance. A joint walkthrough with your team, answers on non-obvious points, and the procedure for making further edits.
Case Study: Reducing Divergence Between Environments and Accelerating Releases
The project is a service with three environments and 40+ hosts that were configured manually according to a checklist before implementation. Each environment lived its own life: differences in package versions, paths, and environment variables surfaced already during deployment. Preparing a new environment took two days and required two engineers.
We decomposed the configuration into Ansible Role: a separate role for each component — web layer, queue, database, monitoring — with a unified set of variables and per-environment value files.
What was common to all environments was moved into a base role, and differences were described with inventory variables rather than copies of tasks. The run became idempotent: a repeated execution does not touch what is already in the desired state.
What Changed in Numbers
The share of manual edits after deployment decreased from 60% to 5% — the remaining ones concern data that we intentionally do not automate. Environment preparation time was reduced from two days to 40 minutes: inventory and one run.
The number of deployment failures fell from eight to one per month — divergences stopped accumulating. The configuration registry shrank by almost a factor of three: instead of copies of tasks per environment, only role descriptions and value files remained.
Role verification is built into the continuous delivery pipeline: before deployment, a run on the test inventory, so an error in a task is visible before the production environment.
Rollback is returning to the previous role and rerunning: the environment is brought back to its previous state in a single run.
How Does Idempotent Configuration Differ from Running a Script?
A script executes commands in sequence and does not know what has already been done. A repeated run will again create a user, append a line to the config, restart the service — and the already configured system will go off the rails.
On a test environment this is usually not noticeable; on production, the second run is exactly what exposes the problem.
An Ansible role describes not a sequence of actions but a target state: which packages should be installed, which files should exist with the specified content, which services should be enabled.
Before each step, the engine compares the description with the actual system state and applies only the difference. If a package is already installed, the step is marked as completed without changes — the command is not executed at all.
This is also where change verification comes from: a run in preview mode shows exactly what will be changed without touching anything. An empty report means that the system state matched the description — this also serves as confirmation that the role is written correctly.
The practical effect is that a repeated run is safe at any moment. The role can be applied on a schedule, run across the entire machine fleet, catch up nodes that have fallen out of service, and erase manual edits: the configuration converges to one described state. Recovery after a failure comes down to running the same role.
A caveat: idempotency does not appear by itself. Shell command calls, archive extraction, and schema migrations are wrapped in conditions and explicit checks — otherwise the role looks declarative but behaves like a script.
Concerns About Timelines, Rollback, and Version Compatibility
Usually the same set of questions comes up before the start: will we make it on a large fleet, what happens with an unsuccessful deployment, how do different versions coexist across environments, who gets access, and who maintains the roles going forward.
A fleet of hundreds of machines — how long does a run take?
Playbooks are applied in parallel by inventory groups, so the time depends not directly on the number of hosts but on the network and package manager speed. We define groups and limits in advance so the run is predictable.
How do you roll back an unsuccessful deployment?
The role is versioned together with the playbook: deployment is a tag, not manual edits on servers. Rollback is switching to the previous tag and rerunning; the state returns in one run.
What if environments have different versions of the tool?
The role checks facts about the target system and selects the appropriate task branch — one codebase covers both old environments and the current branch without forks or copies.
How is server access arranged?
A service user with minimal privileges and sudo only for the necessary commands, keys in an encrypted secrets store, rotation on a schedule.
Who maintains the configurations after handover?
Ansible roles are transferred to the customer's repository with variable descriptions and inventory examples. Updates go through separate branches with a run in the test environment — we catch regressions before production.
Let's Discuss the Task: What to Send for an Estimate and Plan
To make the work estimate concrete and the implementation plan feasible, we need input data about the current state of the infrastructure. Below is the minimum we use to calculate the scope, order of stages, and risks. If something is missing, just write that — we will suggest what to replace it with.
- Environment diagram — which environments exist, how they differ, and how they are connected to each other and to production.
- Current configurations — existing Ansible Role and deployment scenarios, or the manual instructions and scripts used to bring up environments now.
- Access constraints — how keys and service accounts are issued, whether a jump host is needed for connection, and who owns the accesses.
- Change windows and release rules — when production may be changed, who approves a release, and how you roll back now.
- Target metrics — how long bringing up an environment from scratch should take, what share of steps you want removed from manual work, and how quickly rollback is needed.
- Build and delivery — where the artifact is built, how it reaches the environments, what is already automated, and what is done manually.
- Incident history — what broke during deployment and what workarounds have taken hold in the team.
This is enough for us to come back with a work estimate, a stage-by-stage implementation plan, and a list of risks. Describe the task in free form — we will ask clarifying questions ourselves.