Setting Up Autoscaling for Mobile App Servers
We know what a 503 looks like on a user's screen after a push notification blast. When 3000 rps hit a single pod, the server crashes and the app store rating plummets. Autoscaling solves this, but configuring it requires architectural insight. In our practice, over 50 projects have seen a 30–50% reduction in infrastructure costs thanks to properly tuned HPA and KEDA. This article breaks down how to achieve zero-downtime for a mobile API with guaranteed SLAs.
How to Choose Between HPA and KEDA?
| Type | Description | When to Use |
|---|---|---|
| HPA | Scale by CPU/memory | Predictable load, standard metrics |
| VPA | Change requests/limits | JVM services with heap growth |
| Cluster Autoscaler | Add nodes | Cluster resource shortage |
| KEDA | Scale by external events | Queues, Kafka lag, push notifications |
For a mobile API with push notifications, KEDA reacts to load changes 2–3 times faster than HPA by CPU, because scaling starts before traffic arrives.
Types of Autoscaling and When to Apply Them
Horizontal Pod Autoscaler in Kubernetes adds pods under load and removes them when it subsides. The basic metric is CPU utilization, but for a mobile API, better metrics are p99 latency, request queue depth, or custom Prometheus metrics exposed via the Custom Metrics API.
Vertical Pod Autoscaler changes a pod's requests/limits. Useful for JVM services where memory grows as the heap warms up. However, VPA requires a pod restart on resource changes, so it's unsuitable for stateful services. Use VPA in recommendation mode initially to gather data.
Cluster Autoscaler adds/removes Kubernetes nodes in the cloud (AWS EC2, GCP GKE, Azure AKS). It works alongside HPA: HPA wants 5 pods but there's no room — Cluster Autoscaler adds a node.
KEDA scales based on external metrics: RabbitMQ queue length, Kafka lag, number of messages in Redis Streams. For a mobile app with a push notification queue: workers scale by the number of tasks in the queue, not by CPU.
Setting Up HPA for a Mobile API
The problem with standard CPU scaling: during a request spike, CPU first rises, then HPA decides to add a pod (15–30 seconds), the pod starts (another 10–30 seconds), and readiness probes pass. Total: 30–60 seconds before the new pod begins accepting traffic. By then, some mobile clients have already received a 503.
Solutions:
- Predictive scaling — scale out before the expected peak (send push → immediately scale out)
- ScaleUp faster, ScaleDown slower —
scaleUp.stabilizationWindowSeconds: 0(immediate scaling up),scaleDown.stabilizationWindowSeconds: 300(wait 5 minutes before scaling down to avoid thrashing) - MinReplicas: 2 — never drop to 1 pod to prevent downtime during rolling updates
apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: mobile-api-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: mobile-api minReplicas: 2 maxReplicas: 20 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 60 behavior: scaleUp: stabilizationWindowSeconds: 0 policies: - type: Pods value: 4 periodSeconds: 60 scaleDown: stabilizationWindowSeconds: 300 Step-by-step HPA configuration:
- Define the metric (CPU, memory, or custom via Prometheus Adapter).
- Set target utilization (50–70% for CPU).
- Configure behavior for scaleUp (fast) and scaleDown (slow).
- Set minReplicas >= 2.
- Test with load testing (e.g., k6 or Locust).
- Monitor latency and errors using SLO-based alerts.
Why CPU-Based Scaling Isn't Suitable for a Mobile API?
CPU lags behind requests — that's a fact. While HPA detects the peak and adds a pod, some clients already see a 503. For a mobile API, it's better to use p99 latency or request queue depth metrics. In our case with an iOS news app, we solved this with KEDA and an SQS queue — scaling started before traffic arrived, and 503s disappeared.
Cold Start Problem for Mobile Traffic
Go and Node.js start in 1–3 seconds — acceptable. JVM applications (Spring Boot) take 10–20 seconds. Lambda (serverless) cold starts take 500ms–3 seconds depending on runtime and package size.
For JVM: keep at least 2 pods hot at all times. GraalVM Native Image — starts in 0.1–0.3 seconds, but requires reflection configuration. Spring Boot 3 + GraalVM Native — a production-ready combination. Standard JVM instances cost ~$200/month, GraalVM ~$300, Provisioned Concurrency ~$500.
For serverless (AWS Lambda, Google Cloud Functions): Provisioned Concurrency keeps N instances warm. It's more expensive, but cold starts disappear for those instances.
Case: iOS news app. After an editorial push, 40,000 concurrent opens in 2 minutes. One pod on 2 vCPU handled 400 rps. HPA set to 60% CPU — by the time a pod was added, the peak load had passed. Solution: KEDA with a CloudWatch metric (SQS queue depth) — on push send, automatically added 8 pods before traffic arrived. Zero 503s on the next three sends.
| Solution | Startup Time | Suitable For | Cost |
|---|---|---|---|
| Standard JVM | 10–20 s | Stateful, large services | ~$200/mo |
| GraalVM Native | 0.1–0.3 s | Microservices, serverless | ~$300/mo |
| Provisioned Concurrency | 0 (warm) | Critical paths | ~$500/mo |
What's Included in Turnkey Setup
- Audit of current architecture and load testing.
- Configuration of HPA, VPA, Cluster Autoscaler, or KEDA.
- Custom metrics setup (Prometheus, CloudWatch, Datadog).
- Cold start optimization (GraalVM, Provisioned Concurrency).
- Documentation on scaling schemes.
- Monitoring and alerts based on SLO.
- One team training session.
- 2 weeks of support after delivery.
Turnkey setup available from $2,500, with timelines from 2 to 14 days depending on complexity. An average project saves $1,200 per month on cloud costs after autoscaling configuration.
Kubernetes official documentation on HPA
With the right configuration using HPA's behavior scale policies and VPA's recommender mode, cloud resource savings reach 40%, and operating costs drop by $500–2000 per month for an average project.