GitOps repository: a structure that withstands growth
A GitOps repository is not a folder with manifests but the source of truth for infrastructure.
While its structure emerges organically, each environment lives by its own rules, and the divergence between the declarative description and the actual state of the cluster is discovered during an incident, not at review.
The most common failure is manual configuration changes made outside the repository. An engineer adjusted limits or added a secret directly, synchronization overwrites the change a day later, and the service crashes where it worked yesterday.
The reverse scenario is no better: automation is disabled, drift accumulates for months, and rollback turns into archaeology—it is unclear which configuration version was even working.
The second source of pain is environment desynchronization. Development, testing, and production environments were described by copying, differences are not recorded anywhere, and behavior in the test environment stops predicting anything.
A new engineer spends a week finding where the truth lies: in manifests, deployment scripts, or the head of the person who set it all up.
A well-designed structure removes these failures: the declarative description of environments gains shared base layers and targeted overrides, application configurations are separated from platform ones, and any change goes through a merge request and leaves a history.
Next, we will examine how such a structure is organized and which conventions keep it in order as the team grows.
What Changes in the Team's Work After Establishing Structure
During the first week after moving environment descriptions into the repository, the team spends time migrating configurations and setting up the synchronization policy: Argo CD compares the desired state in the repository with the actual state in the cluster and reconciles divergences itself.
Deployment stops being a manual sequence of steps—a change enters the environment branch, passes validation, and goes to the environment without an on-call engineer's involvement.
After a month or a month and a half, the effect is visible in releases.
Previously, deployment depended on who was on shift and whether they remembered the sequence of actions; now the sequence is the same for all environments, and the time from merging a change to production drops from hours to minutes.
Change transparency follows: repository history shows who, when, and why someone changed the configuration of a specific environment, not just the fact of an edit on the server.
Then what is rarely accounted for in advance becomes apparent—configuration recovery.
If an environment diverged from its description after a manual live edit, it is enough to revert the commit or wait for reconciliation to the target state, rather than reconstruct settings from memory and logs.
The class of failures where an environment differs from production by an undocumented parameter goes away along with the reduction in manual operations.
Release predictability is reinforced at a new level: a temporary environment for testing a hypothesis or a permanent one for a new landscape is spun up by copying the structure, not from scratch.
Deployment speed stops depending on team size—the number of environments grows, not the volume of manual work.
Monolithic, Split, or Hybrid Scheme: What to Choose
A repository scheme is primarily an answer to the question of who changes environment descriptions and how often.
While there is one product and environments differ only in a set of parameters, a monolithic scheme provides the shortest path from edit to application: a single validation pipeline, one change history, predictable rollback.
As soon as there are more environments and different teams are allowed to make edits, a shared repository starts to get in the way: a change to the test environment goes through the same approval cycle as production, and it is almost impossible to separate permissions at the level of individual files.
Splitting by environment removes this but brings description drift and manual release transfer between environments.
| Scheme | When it fits | Limitations |
|---|---|---|
| Monolithic | One product, one or two teams, environments differ by parameters | Shared validation cycle for any edits, identical permissions for everyone, mixed history |
| Split by environment | Need for test environment independence and different access rules | Duplicated descriptions, environment divergence, manual change transfer |
| Hybrid | Platform and products evolve in different teams at different speeds | Requires explicit contracts between layers and versioning of shared modules |
The hybrid scheme addresses both limitations: platform descriptions and product descriptions live separately, and the connection goes through shared versioned modules.
The choice is determined by the number of teams, required access isolation, and change frequency, not by infrastructure size.
Setup Stages: From Audit to Handover to the Team
The audit begins not with the repository but with the question of how changes get into the cluster today. We collect an inventory: how many environments there are, where manifests sit next to code, and where they are edited directly in the environment.
Usually already at this step two or three sources of divergence are visible—the future repository structure is what closes them.
- Infrastructure audit. Inventory of environments, applications, and connections between them; separately, we list everything that is currently changed manually and is not described declaratively.
- Environment scheme. We determine how many environments are needed, what goes into a shared base layer, and what remains an override—by branch or by directory per environment.
- Naming conventions. Unified rules for directories, application names, releases, tags, and labels—so both automation and people find what they need without digging through history.
- Change control. Protected branches, mandatory review, automatic schema validation, and synchronization rules: editing in the cluster outside the repository becomes impossible.
- Application validation. Run on a copy of the environment: deployment from scratch, rollback to the previous state, behavior during conflict and loss of connection to the repository.
- Handover to the team. Walkthrough of the structure with engineers, regulations for connecting a new environment and application, and an assigned owner for review.
After handover, the repository lives without us: a new service is added by template, rollback takes one revision, and the cluster state always matches what is recorded in the repository.
Handover Artifacts: Diagrams, Rules, Regulations
The repository describes the desired state but does not explain why it is exactly that way and what to do when synchronization diverges.
Therefore the handover package is not only manifests and an environment directory but a set of artifacts by which a new shift or contractor starts work without correspondence with the author. Each item below is verifiable: if it is missing, acceptance is not closed.
- Infrastructure diagram — connections between clusters, environments, image registry, and external services; separately noted which resources are managed from the repository and which are created manually and why.
- Environment descriptions — list of environments, purpose of each, differences in value sets, and the order of change promotion from test to production.
- Access policies — repository roles, merge rights to protected branches, procedure for granting temporary cluster access, and secret rotation rules.
- Change regulations — path of an edit from branch to automatic synchronization, who approves, in which windows risky changes are made, and how rollback to the previous state is performed.
- On-call shift instructions — typical failures (state divergence, stuck synchronization, failed release), sequence of actions, and escalation point.
- Documentation handover — where decisions and their rationale are stored, how they are tied to configuration changes, and who is responsible for keeping them current.
- Recovery procedure — validated scenario for restoring an environment from the repository, including what to do when an external dependency is unavailable.
Such a package reduces onboarding from weeks to days and removes the class of failures where no one knows why the environment is configured that way.
Case Study: Eliminated Manual Environment Edits and Accelerated Deployment
The client came with four environments and twelve engineers: changes to clusters were made manually through the control panel, and the repository with manifests lived its own life.
After a month, there were more divergences than matches, and no one could say which state was considered correct.
Before: 40-Minute Release and On-Call Engineer as Courier
A version release took 35–45 minutes: building manifests, manually editing values for test and production environments, and eyeballing validation.
Configuration errors accounted for about 30% of all failures, and recovery after an unsuccessful release stretched to 25–40 minutes because rollback was done by the same hands and in the same order.
The on-call engineer woke up two or three times a night, and half the calls were not about code but about diverged parameters.
After: Environments Are Described Declaratively, Synchronization Runs Itself
We separated environments into directories with shared base layers and overrides only where environments truly diverge. State synchronization with the repository was assigned to an operator on Argo CD with automatic reconciliation and a separate rule for rollback. Any edit now passes structure and policy validation before application.
Release was reduced to 6–8 minutes, rollback to 90 seconds, because the previous version is returned from the signed repository state rather than rebuilt.
The share of configuration-related failures fell from 30% to 5%, and all of them are caught at the validation stage before application. Night calls for the on-call engineer became three times fewer: only application failures remained, not work with parameters.
What to Do If Configuration Has Already Diverged?
Configuration that has diverged over years of manual edits cannot be fixed by a single switch to declarative description.
First, an inventory of environments is needed: we capture the actual state—what is really deployed, which values differ from the description in the repository, and what is not recorded anywhere at all.
The result is a divergence register where each item has an owner and a status of “described correctly / described incorrectly / not described.”
Next, the method of entering order is chosen—and here what matters more is not speed but the acceptable risk of downtime.
| Option | When it fits | Limitations |
|---|---|---|
| Recreating the environment from the description | Intermediate environments where data can be lost | Downtime is unacceptable for the production environment |
| Gradual migration service by service | Production is running, shutdown is excluded | Longer; some services are temporarily managed manually |
| Parallel environment with traffic switching | There is spare capacity and dependencies are portable | Requires a rollback plan and validation on real data |
The procedure for establishing cleanliness is the same for all options: first the tool is put into observation mode without automatic synchronization and only shows divergences, then synchronization is enabled one service at a time.
Argo CD in this mode is convenient because divergence is visible before it changes anything. Manual edits are closed off by access policy: a change bypassing the repository is automatically rolled back.
As a result, the environment state is restored from the repository, and any change passes review.
Technical Doubts Before Starting Work
Before starting, the client's team asks the same set of questions—and these are the right questions: the answers determine what the repository will become in six months.
GitOps repository structure is not a directory layout but the speed of answering “where to change,” “who has the right,” and “how to revert it.”
Will the work proceed without downtime?
Yes. Environment descriptions are assembled alongside the running environment: first test and pre-production, reconciling the actual state with manifests, and only then production. Divergences are visible before switching.
How will team permissions be separated?
A directory per service, the directory owner in CODEOWNERS, cluster access through roles. A team changes only its own resources; platform resources (network policies, base resources) are closed to others.
One repository or several?
We look at the number of teams and release frequency. Up to ten services with a shared cycle—one repository with directories by environment; independent teams—separate repositories for applications and platform, linked by versions.
How reversible is the solution?
The state is entirely in version control: returning to the previous commit brings the environment to that same state, images are immutable—rollback rebuilds nothing. Descriptions remain with you in readable form.
What happens after handover?
We hand over the repository with conventions, environment templates, and regulations for adding a service, and train the team. Our engineer remains available until support transitions to your specialists.
Let's Discuss the Task and Return with an Estimate and Plan
So that the estimate is based on facts rather than assumptions, it is enough to prepare several initial data points about environments for the conversation. Nothing needs to be formalized in advance—what the team already has is enough: lists, accesses, and on-call observations. Below is what helps move faster from discussing the task to an implementation plan.
- List of environments: how many there are, what each is for, how configurations differ, and who is authorized to change the environment state.
- Current release method: how a version gets to production—manually, via a build pipeline, or with on-call scripts, and where this most often breaks.
- Repository structure: one shared or several, what is already described declaratively, and what is configured only manually in the control panel.
- On-call pain points: which incidents repeat, how long rollback takes, and what has to be clarified over chat.
- Team roles: who owns application descriptions, who owns infrastructure, whether permission isolation is needed, and whether changes require validation before application.
- Environment requirements: secret storage, rules for promoting a version between environments, and whether automatic rollback is needed after an unsuccessful application.
The further order of collaboration is simple: we review this data, fix the target repository structure and conventions, and return with an estimate of work and a phased implementation plan.
Describe the task—we will suggest where to start so that the first transition happens without stopping releases.