Serverless Warming: Cut P99 Latency by 40-60%

Cold start is the main issue of serverless functions in latency-sensitive applications. The first invocation after a period of inactivity takes 200ms–2s, and for Java it can reach 2 seconds. For APIs with thousands of requests per second, every millisecond counts. Reducing Lambda latency is crucial, and our solutions achieve significant latency reduction. We have been solving this with serverless warming for 5+ years, delivering projects for 20+ high-load APIs. Our approach combines scheduled warming, parallel warming, and provisioned concurrency to completely eliminate cold starts. Serverless warming, also known as lambda warming, ensures functions stay warm. We offer turnkey implementation: from audit to production monitoring.

How Cold Start Affects Latency

Cold start occurs when loading the function image, initializing the runtime, and executing global code. Duration depends on language, package size, and configuration. Typical values:

Even 200–300ms latency is unacceptable for real-time APIs. Warming keeps the function hot and avoids these pauses.

Scheduled Warming: Basic Warming Method

The simplest approach is to invoke the function every 5 minutes via CloudWatch Events / EventBridge so it doesn't cool down.

# lambda_warmer.py — ping function
import json

def handler(event, context):
    if event.get('source') == 'warming':
        # This is a ping from warmers, not a real request
        return {'statusCode': 200, 'body': json.dumps({'warm': True})}
    # Real function logic
    return process_request(event)

Terraform to create the rule:

# Terraform: CloudWatch rule for warming
resource "aws_cloudwatch_event_rule" "warmer" {
  name = "lambda-warmer"
  schedule_expression = "rate(5 minutes)"
}

resource "aws_cloudwatch_event_target" "warmer" {
  rule = aws_cloudwatch_event_rule.warmer.name
  arn = aws_lambda_function.api.arn
  input = jsonencode({"source": "warming"})
}

Limitation: Each EventBridge trigger launches only one concurrent instance. For multiple desired warm instances, we need N parallel invocations. EventBridge warming is a straightforward way to maintain one warm instance.

How to Warm Multiple Instances

We use asynchronous invocation with a delay:

import boto3
import asyncio

lambda_client = boto3.client('lambda')

async def warm_instance(function_name: str, instance_num: int):
    lambda_client.invoke(
        FunctionName=function_name,
        InvocationType='RequestResponse',
        Payload=json.dumps({
            'source': 'warming',
            'instance': instance_num,
            'sleep': 10  # Keep instance busy for 10 seconds
        })
    )

async def warm_function(function_name: str, concurrent_count: int = 5):
    """Launch N parallel warmup invocations"""
    tasks = [warm_instance(function_name, i) for i in range(concurrent_count)]
    await asyncio.gather(*tasks)

While one invocation keeps an instance busy, Lambda creates a new container for the next parallel invocation. Result: 5 warm instances. The cost of such warming is about $0.01 per day per 5 instances. Typical monthly savings from using parallel warming instead of provisioned concurrency can exceed $15. For one client — an e-commerce platform with Python functions and 10,000 requests per hour — we implemented parallel warming with 5 warm instances. Result: P99 latency dropped from 1.2s to 250ms, and warming costs were less than $0.50 per month. The client saved 40% on infrastructure by avoiding provisioned concurrency.

Provisioned Concurrency: When Warming Falls Short

The official solution from AWS is to reserve pre-initialized instances. It's more expensive but guarantees P99 latency without cold starts.

resource "aws_lambda_provisioned_concurrency_config" "api" {
  function_name = aws_lambda_function.api.function_name
  qualifier = aws_lambda_alias.live.name
  provisioned_concurrent_executions = 5
}

resource "aws_appautoscaling_target" "lambda_pc" {
  max_capacity = 20
  min_capacity = 2
  resource_id = "function:${aws_lambda_function.api.function_name}:live"
  scalable_dimension = "lambda:function:ProvisionedConcurrency"
  service_namespace = "lambda"
}

resource "aws_appautoscaling_policy" "lambda_pc_tracking" {
  policy_type = "TargetTrackingScaling"
  resource_id = aws_appautoscaling_target.lambda_pc.resource_id
  scalable_dimension = aws_appautoscaling_target.lambda_pc.scalable_dimension
  service_namespace = aws_appautoscaling_target.lambda_pc.service_namespace

  target_tracking_scaling_policy_configuration {
    target_value = 0.7 # 70% utilization of provisioned
    predefined_metric_specification {
      predefined_metric_type = "LambdaProvisionedConcurrencyUtilization"
    }
  }
}

Provisioned Concurrency provides the best latency, but for sudden traffic spikes, parallel warming is more cost-effective. Provisioned Concurrency costs about $0.00000417 per instance per second, which for 5 instances 24/7 amounts to roughly $16 per month.

Optimizing Initialization Code and SnapStart

Warming helps, but reducing the cold start itself is the best strategy:

# BAD: creating clients inside handler
def handler(event, context):
    dynamodb = boto3.resource('dynamodb')  # Every cold start
    db_client = psycopg2.connect(DSN)  # Creates connection
    ...

# GOOD: create clients at module level (once)
import boto3
import psycopg2

dynamodb = boto3.resource('dynamodb')  # Initialized at cold start
_connection = None  # Lazy connection pool

def get_connection():
    global _connection
    if _connection is None or _connection.closed:
        _connection = psycopg2.connect(DSN)
    return _connection

def handler(event, context):
    conn = get_connection()  # Reuses existing connection
    ...

For Java, AWS offers SnapStart: it creates a snapshot of the initialized state, reducing cold start from 1–2s to 100–200ms. This solution is enabled with one option. Learn more in the AWS documentation.

How We Choose a Warming Strategy

We select a combination of methods based on your load. The process includes:

1. Analysis — profile cold start, determine latency thresholds.
2. Design — choose the stack: EventBridge, Parallel warming, or Provisioned Concurrency.
3. Implementation — write warmer code, configure autoscaling.
4. Test — run load testing, compare latency before and after.
5. Deploy — integrate into CI/CD, set up monitoring.

What's Included in the Work

We provide a full package: audit of current architecture, design of warming strategy, implementation of warmer code, setup of monitoring and alerting, and operational documentation. After implementation, you get reduced P99 latency, infrastructure cost savings up to 30%, access to our expertise — 5+ years of serverless experience, 20+ successful projects. With over 5 years on the market, we have refined our warming strategies. We also provide team training and support for one month post-deploy. To select the optimal strategy, contact us. Get an audit of your serverless architecture.

Method Comparison and Results

Parallel warming is over 50x cheaper than Provisioned Concurrency for maintaining 5 warm instances, and P99 latency improved by 4.8x using parallel warming compared to no warming.