Why a private Docker Registry breaks as the team grows
As long as images are pulled from a public source, deployment depends on someone else's service. Image pull limits hit at the worst possible moment — when several branches are building in parallel and agents request base layers all at once.
The build doesn't fail immediately: it slows down, some steps time out, and the pipeline has to be restarted manually.
The second failure is quieter and therefore more costly. Tags get overwritten, and the same name on different environments ends up pointing to different bytes: the tester verifies something other than what the release will build.
Image drift between environments isn't visible in logs; it shows up in incidents — 'it worked in testing, but not in production.' Build reproducibility is lost because no one knows exactly what is inside the image.
Next, rolling back a release by image breaks. As long as the required layers are in the agent's local cache, everything holds; when the agent moves or the tag is overwritten, the previous version simply isn't there.
All that's left is to rebuild the previous commit from memory and hope the dependencies resolve to the same versions.
The cost is hours of downtime per team and missed release windows. This isn't fixed by discipline but by changing the storage point: your own image registry with immutable tags, access separation, and image checks before deployment. Let's break down how this is addressed.
What a managed image registry provides
A single point for serving images removes the main cause of discrepancies between environments.
A private Docker Registry collects artifacts in one place: the application is built once, and then all environments — development, testing, production — receive the same immutable layer.
The 'it works on my machine' argument ends because there's nothing to compare: the artifact identifier is the same across all environments, rather than being rebuilt on every machine.
Caching layers next to compute resources reduces release deployment time. Base layers are downloaded once; when the application is updated, only the changed part is transferred, and deployment no longer depends on the availability of public repositories.
The typical effect is that deployment takes a few minutes instead of dozens, and teams stop duplicating traffic during parallel builds.
Version control rests on tag policy: a release gets an immutable label, and movable tags are closed for writing so that a different artifact doesn't end up under the same name.
From this grows reproducible rollback — reverting to the previous release is done by switching the label and takes minutes, without rebuilding and without searching for the right build in someone else's directory.
Scanning on upload and on a schedule changes the vulnerability picture: the policy doesn't let an image with critical findings into production, so the share of artifacts with known vulnerabilities falls rather than grows along with the number of services.
A role-based access model adds order: a team sees and deploys only its own projects, actions are logged, and access is revoked when an employee leaves.
Image storage options: comparing modes
The choice of image storage mode determines two things: how quickly a team can roll back to the previous release and how predictably the registry itself grows.
The differences between options aren't in hardware but in who is responsible for availability, updates, access rights, and cleanup of old tags.
| Option | When it fits | Limitations |
|---|---|---|
| Local registry on your own hardware | Data must not leave your environment; you have your own machines and engineers ready to maintain the service | Updates, backups, and disk space are on your side; a node failure stops builds until a second instance is up |
| Cloud managed registry | The team is distributed and you don't want to maintain infrastructure; you need a quick start and access from different networks | Permission setup and scanning are limited by the provider's capabilities; registry size and request count require cleanup discipline |
| Hybrid scheme with replication | Builds happen inside the network, while distribution goes to external sites; you need both speed and availability | Two storage points: tag drift during replication failure, harder incident investigation, need a "who is the source" policy |
The limitations of all three modes manifest in the same way: you run out of space or a node goes down — builds stop, and there's nowhere to roll back to. That's why roles and scopes are configured alongside the registry right away, instead of giving the team shared write access.
Next, we break down how this is addressed in Harbor: project-based access separation, automatic image scanning on upload, and tag-based cleanup rules.
Implementation stages: from audit to handover
The work proceeds in an order that turns on the registry without stopping current deployments: first we record what already exists, then design storage and permissions, and only then migrate images.
- Audit of build pipelines and image inventory. Which services are built, where base layers come from, who pushes and how, how many tags live in the old storage. The result is a list of repositories and a unified naming rule.
- Registry scheme. Projects and quotas, lifecycle policies (how many tags we keep, when we clean up), storage placement, and replication. We agree on the scheme before deployment, otherwise data will have to be migrated again.
- Deployment and connection. We set up Harbor in a fault-tolerant configuration, configure TLS, integration with the corporate user directory, and backup.
- Access rights. Roles by project, separate service accounts for pipelines, separation of read and write, anonymous access closed.
- Image migration. We transfer while preserving digests and verifying integrity — nothing needs to be rebuilt, and deployments aren't interrupted.
- Deployment verification. We run real builds and deployments: vulnerability scanning, a policy to block critical findings, rollback to the previous tag.
- Handover of the registry to the team. Regulations, scheme, recovery procedure, and training on a live environment — after that, the team runs the registry itself.
Registry configurations, access policies, and regulations
After implementation, the customer's team has not only a working registry but also a complete set of artifacts with which it can be developed and maintained without us.
Configuration files are stored in git alongside deployment manifests, so differences between environments are set by values rather than by manually editing templates. What exactly we hand over:
- Registry configurations: storage parameters, external connections, upload size limits, enabled vulnerability checks, and layer retention periods.
- Access policies: rules at the project and repository level, requirements for signed images, prohibition of overwriting immutable tags.
- Team roles and rights: a "team — project — action" matrix, binding to the corporate directory, separate accounts for build agents.
- Image update regulations: frequency of rebuilding base layers, procedure for releasing security fixes, responsible parties for approval.
- Cleanup and retention rules: how many latest images per tag remain, what leaves the registry automatically, how to preserve rollback capability.
- Backup scheme: which metadata and artifacts are copied, at what frequency, and how recovery is verified on a separate environment.
- Instructions for teams: how to get access, how to publish an image, where to turn in case of an incident, and how to read the scan report.
Each document is tied to a specific file and version rather than describing a desired state. The handover is accompanied by a dry run: we deploy the registry from scratch using these materials and compare the result with the working environment.
Case study: how deployment time and incident count were reduced
The team maintained about forty services: builds ran in a continuous integration pipeline, and images were stored in a public registry without access policies.
Deployment to the test environment took an average of 35–40 minutes, and half the time went to manually substituting tags and rebuilding after unclear errors.
Versions diverged across environments: the developer pushed latest, the tester brought up the wrong image, and the on-call engineer couldn't quickly tell what was actually running.
Discrepancies between environments surfaced three to four times a week, and investigating an image-related incident — rolling back the wrong version, an incompatible library — took two to three hours.
We deployed a private registry on Harbor: projects by team and environment, roles in RBAC, separate service accounts for the pipeline, immutable tags, and a retention rule that cleans old layers but keeps the latest builds for rollback.
Vulnerability scanning runs on image push; a build with critical findings is blocked until fixed, and an exception is issued explicitly and for a limited period.
Three months later: deployment to the environment takes about six minutes instead of forty, because the environment pulls the image by immutable tag. Version discrepancies between environments dropped to zero.
The share of images with known critical vulnerabilities fell from 22% to 0.4% — isolated cases remain where the update depends on an external vendor. No image-related incidents were recorded over two quarters.
At the same time, rollback remains manageable: the needed builds are stored, the registry log shows who pushed the image and when, so investigation takes minutes rather than hours.
What to do if the image registry is already deployed manually
A working homegrown registry doesn't need to be rebuilt from scratch — it is migrated to a managed platform while preserving the previous image serving addresses. We start with an inventory: without it, migration turns into guessing tags at random. Then comes a phased switchover in which deployments don't stop.
Image inventory is based on facts, not on a list in the engineer's head: the registry returns a list of repositories and tags, and it is checked against access logs, references in cluster configurations, and build steps.
The output shows which images production and test environments pull, which duplicate each other by layers, and which have never been requested. Usually a noticeable portion of tags turns out to be dead weight — and this immediately sets the real scope of work.
We migrate the existing registry with a dual-run period: Harbor is set up alongside, proxying to the old address is configured, and it remains the source until the last consumer is switched over.
Builds are moved to service accounts limited to their own project, and shared keys are revoked as migration progresses — a seamless transition without a downtime window.
We put permissions and storage in order right away: projects by team, roles from read-only to registry administrator, synchronization with the user directory, quotas, and a retention rule — the last ten tags per repository, release tags immutable.
We don't delete legacy tags blindly: they are retired based on access metrics, and freed layers are collected by the garbage collector.
Common technical concerns when switching to your own registry
The main concern when moving away from an external registry isn't 'how to configure it' but 'what happens when it stops responding'. Below are direct answers about availability, rewriting build pipelines, and responsibility after handover.
What happens if the registry becomes unavailable?
Running containers continue to work: the image is already on the node, and restart happens from the local cache. We keep the registry in two instances at different sites, with state in replicated storage.
A single machine failure doesn't stop builds and deployments, and the health check removes the node from load balancing on its own.
Will build pipelines have to be rewritten?
No. The registry address and credentials are moved to environment variables — a line in the build agent settings changes, while the step logic stays the same. During the transition, the old mirror runs in parallel, so the switchover happens without stopping deployments.
How secure is a private registry?
Access is built on roles: a team sees only its own projects, and read and write permissions are separated.
Images are scanned for vulnerabilities on upload, the policy blocks deployment of builds with critical findings, and automation uses short-lived credentials instead of a shared password for everyone.
What about support after handover?
We hand over configurations, placement schemes, update and recovery regulations, and access is issued to your side. Engineers remain on support with an agreed response time, but not as the only ones who can maintain the registry: after training, your team performs the operation.
Let's discuss the task and come back with an estimate and plan
To come back with an estimate and work plan for the registry, we need a minimum of inputs: where images are currently stored, how many teams publish, and what requirements there are for access and retention.
Usually a conversation with our engineer and access to the current configuration is enough — there's no need to analyze the entire environment end to end.
- A list of services and teams that publish and pull images — the project scheme and isolation between them depend on it.
- Volumes and current storage: how many tags live in the registry and what must be preserved during migration.
- Role requirements: who only reads, who publishes, who needs administrator rights and for how long.
- Retention rules: how many latest versions to keep, what leaves the registry without rollback capability.
- Image vulnerability check policy: block publication or only record the result.
- The customer-side responsible person and the window in which work on the shared environment is allowed.
In response, you receive a registry work plan: stages, changes to existing builds, checkpoints and acceptance criteria, as well as an implementation estimate broken down by stage — without after-the-fact work. The plan is agreed with your tech lead, and if necessary, with security.
After approval, we start with the test environment: we set up the registry, migrate the pilot team, and look at build and image pull speed. Then comes the phased migration of the remaining teams, training of on-call engineers, and documentation handover. Describe the task in a few words — we'll come back with an estimate and plan.