Autoscaling CI/CD runners: setup and optimization

Why builds queue up: runner autoscaling in practice

A static pool of build agents is sized for an average commit flow. Each agent runs one task at a time, so a burst — a series of commits to one branch, a mass merge, a scheduled nightly run — instantly consumes the available workers.

Tasks that started immediately a minute ago now queue for builds and wait for an agent with matching labels.

The queue grows faster than the load. With an eight-minute build and six agents, the twentieth task starts not in eight minutes but in about two hours: its queue position depends on the number of tasks ahead, not on their complexity. Peak load is always higher than average — and a static pool is not designed for it.

Then a feedback spiral kicks in: pipeline wait time grows, results arrive late, the developer switches to another task and loses context.

An error in the main branch is found several merges later, the number of reruns increases, and the queue becomes self-sustaining — the pipeline stops working as a quality control tool.

Increasing the pool "by eye" does not solve the problem: outside peaks agents sit idle, and the estimated peak still turns out higher than the actual one. You need a pool that grows with queue length and shrinks after the queue clears — runner autoscaling. We'll break down how it works and what it delivers in practice.

What runner autoscaling delivers: measurable effects

Feedback speed

A static pool is built for peak: during quiet hours half the agents sit idle, and during a mass deployment a queue forms. Autoscaling spins up runners for the actual task flow, and a run starts immediately after changes are pushed. The time from commit to check results drops from tens of minutes to single digits.

Runner utilization

Scaling by queue length and the number of busy agents keeps exactly as many workers as are needed right now: Kubernetes adds and removes them based on a metric, without manual duty. Utilization grows many times over because the reserve for peak is no longer kept idle.

Build downtime

Ephemeral runners live for one run and are deleted — there are no accumulated caches or environment drift that cause an agent to drop out of service. A faulty worker is replaced automatically, and the build does not wait for manual intervention.

Deployment predictability

A clean, identical environment for every run removes build time variance between machines. Deployments are planned against a stable time budget, not a worst-case scenario.

Eliminated class of failures

Failures from exhausting a static pool disappear: tasks lost from the queue, agent wait timeouts, "stuck" runs on a degraded worker. What remains are code failures — and those are exactly what the pipeline should catch.

Scaling modes: fixed pool, on-demand, hybrid

The three schemes differ not in the number of machines but in where the agent comes from and how much time passes between task submission and its start.

A fixed pool keeps a fixed set of workers: agent startup latency is close to zero, but the scaling ceiling equals the pool size — peaks queue up, and during idle time resources are occupied and images must be updated on all nodes at once.

On-demand scaling creates an agent for a specific task and removes it afterward — there is no idle time, and environments can be heterogeneous.

The price is a cold start: agent startup latency consists of resource allocation, registration in the system, and image preparation, so a stream of small commits starts to hit a wait.

A hybrid scheme keeps a small warm pool for baseline load and adds dynamic agents at peaks, but it requires eviction rules — otherwise stuck agents and double image costs appear.

Comparison of schemes

In practice, the mode is chosen by queue profile: if tasks arrive in waves and are short — hybrid; if the flow is steady and predictable — fixed pool.

Implementation step by step: queue audit, scheme, validation, handover

The work order is structured so that the scheme relies on measurements, not assumptions. First we capture actual queue data and the build profile, then set calculated scaling limits, validate behavior under load, and hand over the configured scheme to your team.

1. Queue audit. We collect task wait time, the share of idle workers, and hourly peaks — showing where the queue grows from insufficient capacity and where from slow builds.
2. Build profile. We break down jobs by duration, memory consumption, and run frequency; short and heavy jobs go into different groups so they do not block each other.
3. Scaling limits. We set upper and lower bounds, the increment step, and the retention time for an idle worker: peaks clear quickly, and idle time does not accumulate extra machines.
4. Scheme and configuration. We describe launch rules, dependency cache, and access separation. Configuration is stored as code and moved between environments without manual edits.
5. Load validation. We run a series of parallel jobs and compare against baseline metrics: queue wait time, failure rate, and time to readiness of the first worker.
6. Handover to operations. We provide documentation, monitoring dashboards, and change procedures, walk through the scheme with your engineers, and support the first edits.

Configurations, schemes, and procedures that remain with you

We hand over not an archive of settings but a working loop: runner configuration tied to your repositories, a scaling scheme with thresholds, and maintenance procedures.

Each artifact comes with an explanation of what each parameter is responsible for and what will change if it is edited — your team will not have to reconstruct the logic from change history.

• Runner configuration files — tags, concurrent job limits, images, cache reuse rules; versioned in the repository and change-reviewed rather than edited by hand on the server.
• Scaling scheme — the metric by which workers are spun up, lower and upper bounds, idle time before removal, behavior under peak load and on node failure.
• Access policies — team and service account roles, per-project separation, restrictions on running jobs from external branches, temporary key lifetime.
• Maintenance procedures — what is updated and on what schedule, how queues and node saturation are checked, and in what order a faulty worker is removed.
• Team instructions — how to add a runner, change limits, connect a new repository, and roll back configuration to a previous version.
• Diagnostics map — metric collection and typical failures: the queue grows, jobs are not picked up, cache is not reused — with verification steps for each.
• Rollback procedure — which version is considered the baseline and how to return to it without stopping builds.

Acceptance is a joint run on your staging environment: we apply load, watch how scaling performs, and adjust thresholds on the spot. After that, your team makes changes according to the instructions.

Case study: how we eliminated build queues and pipeline downtime

Before implementation, builds ran through a fixed pool of four agents for all repositories. In the morning hours load tripled, tasks queued for 20+ minutes, and engineers either waited or opened parallel branches and later resolved merge conflicts.

The failed build rate held at 12% — mainly due to timeouts and resource shortages on overloaded agents.

We moved builds to an elastic pool in GitLab CI: agents spin up for the actual load and shut down after idle time, the limit on concurrent jobs is set at the project level rather than for the whole server.

Heavy checks and fast code checks were separated into different agent profiles, and dependencies are cached between runs.

Before-and-after metrics: queue wait — from 21 minutes to 40 seconds, number of concurrently running agents — from 4 to 34 at peak load, failed build rate — from 12% to 1.5%.

The full check run dropped from 34 to 12 minutes, and the time from a repository change to a ready artifact no longer depends on the time of day.

A separate class of failures also disappeared — the ones previously blamed on "flaky tests": agents no longer compete for resources, so timeout failures almost vanished. Engineers spend the freed-up time on tasks rather than manually restarting builds.

How does on-demand autoscaling differ from a reserve pool?

On-demand scaling creates an agent for a specific task, while a reserve pool keeps several agents already warmed up. The difference shows in three places: startup latency, image warm-up cost, and behavior during a task burst.

With on-demand scaling, an agent appears for each task: the scheduler selects a node, the image is pulled from the registry, and runtime dependencies start. Startup latency is the sum of these steps, and the main share is agent warm-up.

It is reduced by preloading image layers and caching the registry near the cluster, while idle resource consumption remains zero: you pay only for task execution time.

A reserve pool produces a different curve: agents are already running and warmed up, and a task starts almost immediately after submission. The price is constant resource consumption on idle agents, even when there is no load.

On a sudden burst, the reserve pool instantly provides only its own size; then on-demand scaling kicks in — just with a shorter queue.

Behavior during a decline also differs. With on-demand, extra agents leave after the last task, and the cluster returns to its baseline consumption. A pool lives until the end of its retention window: a short burst leaves warmed-up agents occupying resources for nothing.

In practice, a hybrid is assembled in Kubernetes: a small warmed-up minimum handles the regular flow, and autoscaling picks up the rest based on queue length.

The setup is validated by two metrics — time from task submission to agent start and the share of tasks that waited in the queue longer than the threshold.

Cache, secrets, and build isolation: technical constraints

Dynamic agents are disposable, so dependency cache and secrets live outside them: the cache is in shared storage with a key based on the dependency file checksum, and secrets are issued for a specific task.

Build isolation relies on each agent getting its own filesystem and not seeing artifacts from neighboring tasks.

The price of this scheme is the first run after idle time: the cache is not warmed up and there are no agents in the pool. Beyond that, the rules address the main concerns.

How do secrets reach a dynamic agent and not remain on it?

A secret is issued for the duration of one task and mounted into its environment rather than written into the agent image. After completion, the agent is destroyed together with its environment, access is revoked, and values in logs are masked. Therefore the number of concurrently running agents does not affect how long secrets live.

What happens if an agent crashes in the middle of a task?

The task is marked as interrupted and goes for a retry: unfinished steps run from the beginning, and completed artifacts are picked up from the cache. If the cause is resource shortage, the scheduler brings up a replacement. The retry limit prevents the failure from looping indefinitely.

Let's discuss your runner scheme and come back with an estimate

An estimate for implementing autoscaling relies on facts, not market averages: everyone's runner schemes differ, and configuration differences change both architecture and effort.

So we first collect baseline data — usually one conversation with the engineer who operates those runners is enough.

• Version control system and how jobs are triggered: by event, on schedule, or manually.
• Where the runners currently live and what manages them: standalone machines, cloud virtual machines, a cluster.
• Task profile — build, tests, artifact publishing — and their typical duration.
• Peak load: how many jobs arrive simultaneously and how often this repeats.
• Requirements for network zones, secrets, and access rights to repositories and image registries.
• What has already been configured and where exactly it hits a limit: jobs idling in the queue, insufficient capacity, manual machine provisioning.
• Change constraints: maintenance windows, approvals, isolated environments.

Based on this data, we describe the target runner scheme for your task flow and come back with an implementation estimate: a phased work plan, setup timelines, a rollout procedure without stopping current builds, and a list of what will be required on your side.

If data is insufficient, we will say so directly rather than build in a buffer. Describe the task, and we will come back with a plan.