Kubernetes Autoscaling Setup (KEDA — Event-Driven Autoscaling)

KEDA: why queues grow while nodes sit idle

The event stream exceeds processing speed — and the job queue grows faster than workers can drain it. At that moment, native autoscaling looks at CPU and memory consumption, while a handler waiting for a response from an external system or database barely loads the core.

From a metrics perspective, everything looks normal, nodes are one-third utilized, and jobs keep piling up.

Consumption metrics are also averaged over a window and reach the controller with a delay. By the time the value updates, the rule triggers, and a new instance comes up, several minutes pass — during which the queue grows several times over, and some jobs go into reprocessing. In this scenario, the reaction delay to a peak is measured in tens of minutes.

The only lever left is manual node addition: the on-call engineer sees the queue growing on a dashboard and scales manually. By the time new nodes are ready to take load, the peak has already passed — and capacity sits idle until the next spike.

As a result, between peaks the infrastructure runs idle, and during a spike it still cannot keep up.

Event-driven autoscaling removes this gap: the metric becomes not resource consumption but job queue length and event arrival rate. KEDA polls the event source directly and keeps exactly as many handlers as are needed to drain the queue — down to zero at idle. Below, we look at how this is addressed and what needs to be planned in advance.

What event-driven scaling provides

The scaling signal is usually taken from CPU and memory utilization — these are averaged values with a delay of minutes. During that time, the queue has time to grow, while pods are already being scaled down on the decline.

KEDA changes the decision source: instead of utilization, it reads queue length, consumer lag, and incoming event rate, so it can scale a workload down to zero replicas and back up again.

Four effects visible in metrics:

• Load reaction time. Handler instances come up according to the actual event stream, not a forecast based on average utilization — the spike is drained noticeably sooner.
• Cluster utilization. At idle, workloads go to zero replicas, and freed resources go to other tasks: the share of idle capacity drops without manual schedules.
• Eliminated failure class. As the backlog grows, the number of handlers grows, so jobs stop being lost due to queue overflow or timeout disconnect.
• Predictable resource consumption. The scaling profile is described by rules and thresholds, so it fits into capacity planning and load reporting.

Separately, it is worth locking down the boundaries: minimum and maximum replica counts, scaling step, and cooldown window after a decline. Without them, autoscaling turns into a seesaw that breaks both stability and consumption predictability.

Autoscaling modes: KEDA and native mechanisms

Several scaling mechanisms usually coexist in a cluster, and they are chosen by signal source: queue, resource consumption, calendar, or lack of space for pods.

KEDA covers scaling by external events, the native mechanism — by resource metrics, while schedules and the node level solve separate tasks.

The practical question is what notices load growth first. If it is queue length or consumer lag, resource metrics will trigger later: the pod is already overloaded, but the scale has not yet increased. Below — the conditions under which each option is justified and where its limit lies.

The layers complement each other: an event trigger quickly increases the number of pods, the resource one holds the baseline, and the node level adds capacity. Next — how to assemble this without conflicts between mechanisms.

Implementation stages: from load audit to handover

The sequence of work is the same for one service and for a dozen: first we measure, then we design, and only then do we enable autoscaling in production. Blindly copying someone else's thresholds results in either idle replicas or an event avalanche on the very first peak.

1. Load profile capture. We collect metrics on queues, task processing time, and daily request dynamics over a representative period. The output is a table of peaks and troughs that shows where manual scaling can no longer keep up.

2. Designing rules and priorities. We determine which signals control the replica count and which only inform the on-call engineer. With multiple sources, we set priority: the queue length rule overrides the utilization rule.

3. Calculating scaling thresholds. We calculate at which metric value we add a replica and at which we remove one, with allowance for application startup time. We separate the thresholds so the replica count does not oscillate at the boundary.

4. Configuring KEDA and event sources. We connect triggers to queues and the broker, set minimum–maximum boundaries, the stabilization window, and behavior when the source is unavailable. We separately describe cold start, when there are no events at all.

5. Load testing. We run the profile on a staging environment that mirrors production and look at the actual reaction: how many replicas came up, how long it took, and whether there was a rollback. The result is a report with before-and-after graphs.

6. Handover to operations. We hand over configurations, rule diagrams, and instructions for the on-call shift: how to switch the service to manual control, where to look when the queue is stuck, and which alerts are critical. Threshold changes are reversible, so rolling back to previous values takes minutes.

What remains with the team: KEDA configurations and regulations

Autoscaling is not a single setting in a cluster but a set of artifacts that live alongside the service code and change with it.

We hand over to the team not a “working autoscaler” but something they can manage independently: without reading someone else's mind and without asking us every time a threshold changes.

• Trigger configurations. Manifests describing the event source, trigger threshold, polling interval, and decay period — as well as behavior when the metric is unavailable and rules for scaling to zero.
• Event flow diagram. Where the load signal comes from, through which intermediary, and what happens to accumulated messages when replicas scale to zero and when they scale back up.
• Access policies and roles. Service accounts for reading metrics, boundaries of controller permissions by namespace, and restrictions on manual scaling intervention.
• Operations manual. How to view the current scale and the reason for a decision, how to temporarily pin the replica count, and how to roll back the configuration to the previous one.
• On-call shift policy. Which alerts are critical, what to do when scaling is “stuck,” and where the boundary is between waiting and escalation.
• Checks in the build pipeline. Manifest validation, monitoring of minimum and maximum replica boundaries, and a load test for peak behavior.

Each artifact lives in the repository alongside the code, goes through review, and is rolled back like a regular change. Thresholds and rules change as load grows — without us and without stopping the service.

Case study: order processing under peak load

A store with seasonal sales: the order handler ran on a fixed number of instances sized for average traffic.

During peak hours, the order queue grew faster than it was drained, processing wait time stretched into tens of minutes, and the on-call engineer manually scaled up capacity — and still could not always keep up. At night, the same capacity sat almost entirely idle.

What we did

We connected KEDA and switched scaling from compute resource utilization to queue length: the number of handlers follows the real volume of unprocessed orders rather than an indirect indicator.

We added scale-to-zero for night windows, stabilization windows so a short spike does not cause jerky scaling, and an upper instance boundary as protection against uncontrolled growth if the event source fails.

We described the scaling scenarios declaratively and placed them alongside the service configuration, so changing thresholds does not require code edits and goes through the regular review cycle.

The on-call engineer is no longer part of the scaling loop: intervention remains only for emergency situations, and capacity is now allocated according to the actual order flow.

How KEDA differs from native autoscaling

The native autoscaler relies on CPU and memory utilization — an indirect signal. Metrics arrive with a delay, after the load has already hit, and queue handlers often do not load the CPU at all: they wait for a response from an external system. As a result, replicas are added late, and some of those brought up sit idle.

KEDA changes the signal source. It subscribes to an external queue, message broker, or event stream and reads it directly: how many messages are waiting and how far behind job consumers are.

Scaling starts when work appears, not when hardware heats up, so the spike is drained by the same instances already running.

The second difference is scale-to-zero. The native autoscaler does not go below one instance, while KEDA can scale to zero and scale up from zero on the first message.

For night windows, rare scheduled jobs, and test environments, this removes the constant background of unused resources.

The third is its place in the overall scheme. KEDA does not replace the native solution but complements it: it only supplies the metric, while the native autoscaler still determines the replica count.

Several triggers are configured — queue, schedule, load — and the one requiring more replicas is used.

In practice, this removes manual on-call work during peaks: replicas appear according to the actual volume of jobs, not the worst-case scenario. We separately test behavior when the metric source is unavailable — without that, autoscaling itself becomes a point of failure.

Technical concerns: limitations and compatibility

Concerns about autoscaling appear when the setup goes into the production loop: what is already in the cluster and how the system will behave if the metric source goes silent. Here are four questions engineers ask, and how we address them.

What should be done if a metric arrives incomplete or delayed?

The source may not return all values: some queues have no counter, polling delay, aggregation window. For this case, we set a rule — do not scale down replicas on an empty value, hold the current count, and add a fallback CPU threshold and schedule for known peaks.

Autoscaling remains operational during partial metric failure instead of getting stuck at the minimum.

How do we roll back if the behavior is unsatisfactory?

The configuration lives in the repository and is applied through regular deployment.

It is enough to remove the binding to the metric source for a specific application — and the object returns to native load-based scaling, while replica count rules and stabilization windows remain. Pods are not recreated, and rolling back one application does not affect the others.

Will the load on the metrics system increase?

Each scalable object polls the source at its own interval, and by default it is higher than the actual reaction speed. We bring the frequency down to what is needed, enable value caching and a stabilization window so decisions do not thrash on short spikes. The load on the metrics system grows predictably, not proportionally to the number of applications.

Let's discuss the task: what is needed for an estimate and plan

To come back with a scope estimate and implementation plan, we need initial data about the cluster, load, and current processes.

The more specific the inputs, the fewer assumptions in the timeline: almost all edits after the start are caused by details that were missing during the estimate.

What we request at the start:

• Cluster architecture: where it is hosted, how many worker nodes, who manages the control plane.
• Triggers: which metrics should start scaling — queue length, consumer lag, number of pending jobs.
• What we scale: list of services and queues, replica boundaries, and reaction time requirements.
• Deployment: how services get into the cluster, which environments exist, and how they differ.
• Constraints: provider quotas, node ceiling, maintenance windows, availability requirements.
• Observability: what is already collected in metrics and logs, and which thresholds on-call engineers rely on.
• Access and roles: how we organize access to the cluster and repositories, and who on your side accepts the result.
• Readiness criteria: what we consider success — reaction to a spike, behavior when a consumer fails, and return to previous settings.

The interaction process is simple: a task review with your engineers, access to the staging environment, a pilot of event-driven autoscaling on one service with before-and-after measurements, then phased enablement on the remaining queues. Describe the task — we will come back with an estimate and plan.