Kubernetes Cluster Autoscaler Setup (Node Autoscaling)
After deploying a Kubernetes cluster with HPA, you notice pods are scaling but new nodes are not added. Cluster Autoscaler logs are silent, pods hang in Pending. Typical case: on one project, a client lost 3 hours of production availability due to incorrect CA configuration. We solve such problems regularly — 5 years working with k8s on 30+ projects. Cluster Autoscaler (CA) automates node scaling: it adds compute resources when pods cannot be scheduled due to CPU or memory shortage, and removes nodes when load decreases. But its behavior heavily depends on cloud provider settings and scheduler policies.
CA works at the node level: it adds and removes nodes based on pending pods due to resource shortage. In combination with HPA or KEDA, this is a full cycle: load increases → HPA adds pods → pods don't fit → CA adds node → pods start. In reality, the process often breaks due to subtle settings.
Why Don't New Nodes Appear?
The most common scenario: CA is configured, pods are in Pending, but nodes are not added. The command kubectl describe pod <pending-pod> will show the exact reason. Typical causes:
-
The pod is unschedulable not due to resources, but due to affinity/taints. For example, if a pod uses
nodeSelector: gpu: "true"and the pool has no GPU instances, CA cannot help — it creates nodes only from existing node groups. - PodDisruptionBudget (PDB) blocks scale-down. CA tries to remove a node by evicting pods, but PDB prevents eviction. The node remains, no resource savings.
-
Scale-down blocked by system pods. CA does not remove nodes with pods lacking PDB, standalone pods, pods with annotation
cluster-autoscaler.kubernetes.io/safe-to-evict: "false"orhostPathvolumes. For diagnosis, usekubectl describe node <node> | grep -A5 "Annotations".
Diagnosis example
kubectl get pods --field-selector=status.phase=Pending kubectl describe pod pending-pod-name Pay attention to events and conditions. If the reason is 0/1 nodes are available: 1 Insufficient cpu, CA should trigger.
How Cluster Autoscaler Interacts with HPA?
HPA increases the number of pods under load. When current node resources are exhausted, new pods transition to Pending state, and CA adds a node. It is important to configure delays: --scale-down-delay-after-add=10m prevents CA from immediately removing a node after it appears, and --scale-down-unneeded-time=10m sets idle time before removal. This prevents fluctuations during peak loads. According to our data, correct delays reduce unnecessary scaling by 40%.
Configuration for Cloud Providers
AWS (EKS)
command:
- ./cluster-autoscaler
- --v=4
- --stderrthreshold=info
- --cloud-provider=aws
- --skip-nodes-with-local-storage=false
- --expander=least-waste
- --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster
- --balance-similar-node-groups
- --scale-down-delay-after-add=10m
- --scale-down-unneeded-time=10m--expander=least-waste selects the node group that most efficiently uses resources of the new node. Alternatives: random, most-pods, priority (with explicitly set node group priorities).
--balance-similar-node-groups balances nodes among similar groups — important for multi-AZ deployment to distribute nodes across availability zones.
GKE (Standard)
On GKE, Cluster Autoscaler is managed via gcloud when creating a node pool:
gcloud container node-pools create general \
--cluster=my-cluster \
--enable-autoscaling \
--min-nodes=1 \
--max-nodes=20 \
--machine-type=n2-standard-4On bare metal or kubeadm clusters, CA only works if there is an API for node management — for example, via cloud-controller-manager or a custom provider.
When to Choose Karpenter Instead of Cluster Autoscaler?
On AWS, Karpenter is a more flexible solution. It is not tied to pre-created Auto Scaling Groups: it selects the instance type based on pod requirements and provisions it via EC2 API. Difference: CA adds a node from a fixed node pool with a specific instance type, while Karpenter looks at requests of all pending pods and selects the cheapest suitable type — could be m5.large or c6i.2xlarge. Spot instances are natively supported.
Karpenter NodePool:
---
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: general
spec:
template:
spec:
requirements:
- key: karpenter.sh/capacity-type
operator: In
values: ["spot", "on-demand"]
- key: kubernetes.io/arch
operator: In
values: ["amd64"]
limits:
cpu: 1000
disruption:
consolidationPolicy: WhenUnderutilized
consolidateAfter: 1m
consolidateAfter: 1m — Karpenter aggressively consolidates: if nodes are underutilized, it moves pods and removes extra nodes. This reduces AWS costs by 30-50% according to our estimates. Karpenter reacts faster to changes: consolidation occurs within a minute, not 10 minutes as with CA.
Comparison of Cluster Autoscaler and Karpenter
| Parameter | Cluster Autoscaler | Karpenter |
|---|---|---|
| Instance types | Fixed from ASG | Dynamic selection |
| Spot instances | Via ASG | Native support |
| Consolidation | Basic | Aggressive (1 min) |
| Configuration | CA flags | Custom Resource |
| Providers | AWS, GCP, Azure | AWS (currently) |
Typical CA Delay Parameters
| Parameter | Default Value | Recommended | Description |
|---|---|---|---|
scale-down-delay-after-add |
10m | 10m | Delay after node addition before possible removal |
scale-down-delay-after-delete |
0s | 10s | Delay after node deletion |
scale-down-delay-after-failure |
3m | 3m | Delay after failed scaling |
scale-down-unneeded-time |
10m | 5-10m | Node idle time before removal |
How to Diagnose Scaling Issues?
- Check CA status:
kubectl get pods -n kube-system | grep cluster-autoscaler. - View logs:
kubectl logs -n kube-system deployment/cluster-autoscaler --tail=50. - Ensure metrics-server is running:
kubectl top nodes. - Verify pods have requests: CA does not scale pods without requests.
- Use
kubectl describe pod <pending-pod>to analyze events.
What Our Work Includes
- Audit of current cluster configuration and load.
- Strategy selection: CA or Karpenter.
- CA configuration with appropriate expander, delays, balancing.
- PDB configuration for system pods.
- Integration with monitoring (Prometheus+Grafana).
- Operational documentation and runbook.
Timelines and Cost
Basic CA setup for one cloud provider with standard parameters — from 1 day. Full setup with multi-AZ, PDB, monitoring — from 1 to 3 days. Migration to Karpenter — from 2 days. Cost is calculated individually after audit. By reaching out to us, you get a guarantee of correct Cluster Autoscaler operation in production. Contact us for a project assessment — we will help optimize cloud resource costs.
Our engineers bring 5+ years of Kubernetes experience, over 30 successful infrastructure automation projects. Proper CA configuration saves from $500 to $3,000 monthly depending on scale.
Monitoring
CA metrics in Prometheus: cluster_autoscaler_nodes_count, cluster_autoscaler_unschedulable_pods_count, cluster_autoscaler_last_activity. Alert on cluster_autoscaler_errors_total > 0 — a sign that CA cannot scale due to cloud provider API errors. Configure a Grafana dashboard for trend visualization.
Source: official Cluster Autoscaler documentation on GitHub