TeamCity Setup: Where the Build Pipeline Breaks
TeamCity setup does not start with the build screen, but with a question: how much time passes from commit to a working environment.
If the answer is “an hour or more,” the bottleneck is almost always at the handoffs — the agent queue, manual steps between build and deployment, and diverged environments.
Slow builds are rarely explained by weak hardware. More often, one agent serves several projects, jobs queue up, and build wait time outweighs compilation with tests itself.
Manual steps in release are the second point of loss.
While the artifact is moved manually or someone waits for the person who knows what to click, change delivery moves in bursts: no one takes risks at the end of the week, and by the beginning of the next one a batch of edits accumulates that is harder to validate.
Non-reproducible environment produces the most expensive class of failures: everything is green locally, but not on the agent. Different tool versions, warmed caches, and environment variables turn debugging into a search through hypotheses.
Separately — flaky checks. A test that fails intermittently trains the team to ignore red status, and a real breakage goes into the main branch along with it.
That is why we look at the pipeline as a system: where it waits, where it depends on a person, and where the result does not repeat. Below we discuss how this is addressed — from configuration described as code in Kotlin DSL to cloud agents that scale under load.
TeamCity Setup: What Changes in Speed and Reliability
TeamCity setup does not start with installing the server, but with an inventory of what is still done manually in the project. Then the build is described as code in Kotlin DSL, and heavy steps move to cloud agents: they spin up under load and do not sit idle between releases.
How Change Validation Time Is Reduced
Caching dependencies and artifacts between runs and parallelizing independent steps reduce build time severalfold — change validation on a typical service goes from tens of minutes to single digits.
A developer gets the result for their branch without waiting in the common queue on a single agent, so the build queue stops being a bottleneck as the team grows.
How the Class of Failures from Manual Operations Disappears
A manual step breaks a release with the same predictability: they forgot a migration, built from the wrong branch, deployed the wrong configuration.
Each such action becomes a step in the build chain with pre-deployment checks — the share of manual operations drops, and release stability grows without night shifts.
The result is measurable: build time and change validation speed decrease, team downtime due to broken builds goes away, and rollback to the previous artifact becomes a routine operation.
Cloud and Self-Hosted Build Agents: Choosing the Mode of Operation
An agent is a machine that actually executes build steps, and its mode of operation determines both queue wait time and pipeline behavior at peaks.
TeamCity manages cloud and self-hosted agents the same way, so changing the mode is a matter of configuration, not rewriting build scenarios.
| Option | When It Fits | Limitations |
|---|---|---|
| Cloud agents | Steady flow, rare heavy jobs, no desire to buy hardware | Cold start hurts short builds, the environment is ephemeral — cache warms up again |
| Self-hosted fleet | Need for a closed perimeter or local registry, stable utilization | Idle during quiet hours, images and patches on your side, capacity limited by the number of machines |
| Hybrid scheme | Baseline flow on premises, peaks go to the cloud | Requires rules for routing jobs to pools and separate accounting |
When a Self-Hosted Agent Fleet Is Needed
Self-hosted machines are justified if the build needs access to a closed perimeter, local registry, or a staging environment without an external address.
The second sign is a sustained flow: if agents are constantly busy during working hours, the fleet provides predictable queue time. It also solves build environment isolation: pools are assigned to a team or job type, and someone else’s build does not take resources from yours.
How Peak Build Load Is Calculated
We calculate peak load from the run log: a graph of concurrent builds over a representative period, an upper percentile instead of the average, and headroom for team growth.
Then we compare it with the number of agents and hold time — it shows how many machines are missing in the worst window and how many are usually idle. For a hybrid scheme, the same calculation sets the threshold after which jobs automatically go to the cloud pool.
Work Stages: From Build Audit to Handover of Procedures
The work is structured so that builds remain working at every step: first we capture the current state, then migrate build descriptions to code, and only after validation on a production copy do we move the team to the new pipeline in TeamCity. The sequence of stages does not depend on project size — the scope changes, not the order.
- Build audit — inventory of configurations, measurement of each build’s time, search for duplicate checks and steps that depend on a specific agent’s state.
- Migration of build descriptions to Kotlin DSL — configurations become code: an edit passes review before merge, and rollback is done by reverting the previous revision of the file.
- Environment layout — which branches build into what, where artifacts are stored, what permissions accounts have, and where secrets come from.
- Cloud agents — pools for job types, autoscaling against peak queue, isolation of agents from each other.
- Validation on a production copy — we run the full build and deployment path on the copy and compare it with the current pipeline.
- Handover of procedures — how to add a step, roll back a released version, and what to do if an agent fails.
Environment Layout and Migration of Build Descriptions
The layout answers the question of what runs where and where the result goes: repositories, agent pools, artifacts, permissions. Until it is agreed upon, migrating descriptions to code turns into rewriting the same logic for each project.
Validation on a Production Copy
The copy replicates the configuration of production servers, so it reveals discrepancies in environment variables and services unavailable from the agent. We switch over only after the old and new pipelines produce matching results.
What You Keep: Configurations, Layouts, Procedures
Implementation ends not with a report, but with a set of working artifacts in your environment.
TeamCity configurations, environment layouts, and procedures remain with you as readable files and documents, not a closed archive — you can edit them and hand them to a new engineer without us.
Everything lives in your version control system next to the code: changes go through review and are rolled back like a regular commit.
- Build configuration — descriptions in Kotlin DSL in your repository: steps, triggers, dependencies between builds, environment parameters. Edits go through a branch and review, not through the web UI.
- Build documentation — what each step does, which artifacts it produces, which checks are mandatory before deployment.
- Environment layout — which environments exist, how they differ, where each branch points, and how the artifact is promoted to the production environment.
- Access policies — roles and permissions: who runs a build, who approves deployment, which agents are allowed to which storages. Plus the procedure for granting and revoking access.
- Maintenance procedures — how to add a step, update an agent, clear the cache, and roll back a release, with an explicit boundary of responsibility between you and us.
- Cloud agent connection guide — how to bring up a new agent, what labels and limits it has, and how to verify that it landed in the correct pool.
Such a handover removes the main risk — dependence on a person who “knows how everything works here.” Configuration is versioned, policies are described, and the procedure is tested on live releases, not written from memory.
Case Study: Reduced Build Time and Removed Manual Steps
The project is a monorepo for four teams: Kotlin services, a shared library, and a mobile build. The pipeline lived in TeamCity, but was configured through the web UI: steps were edited manually, configurations were duplicated between projects, and no one reviewed changes.
What It Was Before
The build took 24–28 minutes: half of the steps ran sequentially, although they did not depend on each other. The release was assembled manually — 11 operations from tagging to editing environment variables, each taking 3–4 minutes and relying on the responsible person’s memory.
Errors were predictable: a forgotten tag, the wrong build profile, diverged configurations. Every third build on the main branch failed, and debugging took more than 20 minutes — the log mixed steps from all projects.
How We Measured the Result and What We Considered a Failure
We took metrics from build history for 60 working days before and after: median duration of a successful build, failure rate on the main branch, number of manual operations in the release checklist, and time from commit to artifact publication.
We considered any failure not caused by code a failure: unavailable agent, dependency version conflict, incorrect environment parameter. Developer-caused failures were counted separately — otherwise the metric would measure code quality, not the pipeline.
The build description was migrated to Kotlin DSL in the repository, agents were made cloud-based and single-job, steps were parallelized up to six, and dependency caching was added.
Median build time — 8 minutes, failure rate — 4%, manual operations in release — one, from commit to artifact — 14 minutes.
Kotlin DSL or Build via Web UI: What to Choose?
Short answer: if the pipeline changes more often than a couple of times per quarter and several engineers work on it — describe the build as code in Kotlin DSL.
If the configuration is stable and edited by one person, TeamCity’s web UI is enough: a settings repository would add work without payoff.
In Kotlin DSL, the build description lives in the same repository as the code.
Any change — a step, dependency, agent parameter — comes through a merge request: you can see line by line what was changed and why, and who approved it, while rollback restores the previous state from history.
Configuration versioning and build change review stop being a verbal agreement — they are built into the process.
The web UI wins on speed: configuring by clicks is faster for similar projects, the edit history is stored inside the system, and for a small team this is enough.
The limitation shows up as you grow: shared templates are duplicated manually, and to understand why a build behaves differently than six months ago, you can only use the log inside the server.
You can migrate gradually, without rewriting everything at once. The project stays in the web UI, while the shared parts are moved to code: templates, environment parameters, launch rules. Then pipelines are migrated one by one — with validation that build behavior has not changed.
The criterion is simple: many projects, branches, and mandatory approval of edits — Kotlin DSL; one or two stable pipelines and one responsible person — web UI. A mixed mode also works: it builds up the practice of configuration versioning where it delivers value.
Common Concerns: Access, Secrets, Build Rollback
The concerns that most often come up before the start are almost always about one thing: what happens if something breaks or leaks. Below are five questions we discuss with the client before the first commit to TeamCity, and our default answers.
Where are build secrets stored and will they leak into logs?
Secrets do not go into the repository with the build description — they are stored in the server’s secured vault or in an external secrets manager and are injected into a specific step only during execution.
Values are masked in output, so the token is not visible in logs, artifacts, or the UI to those who do not have rights to the project. The permission to edit a secret is separated from the permission to run the build.
How are environments separated so a test does not go to prod?
Each environment has its own set of variables and access keys, and deployment to prod goes only from a protected branch and with approval from the responsible person.
The artifact is built once and promoted between environments, not rebuilt: what passed the checks goes to prod unchanged. This way, “works on test, fails in prod” stops being a class of failures.
What is available to build agents and does anything remain after the build?
The agent receives only the parameters needed for the current step and a separate read-only repository key — without write permission. Cloud agents are created per task and deleted after it, so nothing remains between projects or between runs.
How are agents updated and won’t the entire pipeline stop?
Updates roll out in waves: first one agent is updated and validated on non-production builds, then the rest. We verify agent-server version compatibility before the mass update — otherwise the entire pipeline stops, and that is exactly the failure that costs the most in downtime.
What to do if the build description is corrupted or someone broke it?
The build description history is stored in the version control system, so rollback is a commit revert, not a manual edit from memory.
Additionally, we keep the last working artifact and its launch parameters: the release can be rebuilt from the same commit even if the pipeline is broken.
Let's Discuss the Task: What Is Needed for an Estimate and Work Plan
To come back with a realistic estimate and plan, rather than a “from and to” range, we need a snapshot of your current infrastructure.
For TeamCity setup, it gives an understanding of the scope of work, stages, and risks — what can be moved to cloud agents, what will have to remain on self-hosted machines, and how to make the transition without stopping releases.
- Build inventory: how many configurations and projects are in the environment, which languages and runtimes, where artifacts are stored, which builds are no longer used and can be retired.
- Current build pipeline: what triggers it, which stages code goes through from commit to production, where manual steps and manual approvals remain.
- Access requirements: roles in repositories and the image registry, where secrets are stored and who owns them, whether a separate network perimeter is needed for build agents.
- Execution environment: self-hosted or cloud agents, which licenses and quotas already exist, whether there are network, storage, and build duration constraints.
- Acceptance criteria: what we consider the result — build time, the share of checks that run automatically, and the ability to roll back to the previous artifact in one step.
- Handover procedure: who accepts the work, whether your team needs procedures and instructions, who is responsible for the pipeline after launch.
Describe the task in free form — by these points or however convenient. We will come back with an estimate, a stage plan, and timelines, and will address contentious points on migration and access separately.