Serverless Monitoring Setup: Lumigo and Datadog Turnkey

Serverless monitoring: where observability breaks

Serverless monitoring breaks not where it is usually sought. The function executed, returned a response, and the logs are empty or contain a fragment without a request identifier.

The incident surfaces an hour later, when users are already writing to support, and the on-call engineer walks the chain from the queue, the gateway, the function, and the external service without a single end-to-end trace.

The root of the problem lies in the very nature of the serverless model. Each invocation lives in a separate isolated environment that the platform creates and destroys at its own discretion.

Observability of serverless functions breaks at three junctions: the request context does not survive the boundary between functions, the log collector starts after the first lines are already lost, and the cold start adds a segment to the response time that is nowhere displayed as a separate phase.

Then the accumulation effect kicks in. The loss of context in one function turns request tracing into a set of unrelated events: you can see that an invocation happened, but you cannot see with what parameters it arrived and what it returned.

When there are dozens of such functions, reconstructing the picture of an incident takes hours of manual correlation by time and indirect signs — and by that point the cause of the failure can no longer be reproduced.

The cost of such a blind spot is not only downtime. Without an end-to-end picture, the team cannot distinguish degradation of an external service from exhaustion of concurrent executions, and therefore fixes the symptom instead of the cause and gets a repeat failure a week later.

Below we break down how this is closed: exactly where to place the collection of metrics and traces so that every request remains traceable from the entry point to the last invocation.

What serverless monitoring delivers: the effect in numbers

Serverless monitoring starts with three independent telemetry streams: collecting function metrics (cold starts, duration, errors, concurrency), distributed tracing of an end-to-end request through the gateway, queue, and function, and call logging with correlation by request ID. While these streams live apart, pinpointing a failure turns into manually sifting through logs.

Metrics, tracing, and logs as a single loop

Collecting function metrics provides the primary signal: a growing share of cold starts and rising duration on a specific handler is visible before errors start pouring in.

Distributed tracing links calls into a single route, so you can see at which segment of the chain the request loses time. Call logging with a shared ID eliminates the sifting: an engineer opens a single trace and sees the cause instead of assembling it from scattered records.

Threshold alerts instead of "monitoring everything"

Threshold alerts are configured for engineering-significant values: error rate per handler, duration above the baseline, a spike in concurrent invocations, exhaustion of the concurrent-call limit.

A properly set threshold shortens recovery time: the on-call engineer gets a signal with context rather than a stream of noise, and doesn't spend the first minutes figuring out what actually went down.

What this delivers in measurable terms

After the end-to-end loop is in place, the share of calls covered by tracing reaches full coverage of critical routes, and failure localization time drops from hours to minutes — the engineer sees the specific segment of the chain.

A class of failures that used to be found through complaints disappears: silent handler degradation, a rise in cold starts after a configuration change, errors at the boundary between the queue and the function.

Datadog and similar platforms cover tracing and alerts out of the box, but the choice between built-in tools, an in-house stack, and an external platform depends on how many routes are critical and how long data needs to be stored.

How we implement monitoring: stages of work

Implementing serverless monitoring begins not with installing agents, but with an audit of what is already being collected: which functions and queues exist, which metrics the cloud services expose, and where telemetry is lost between invocations.

Without this step, dashboards turn into a set of graphs that cannot be used to localize a failure. Then we build the route so that each stage produces a verifiable result, rather than "we set it up and hope."

1. Telemetry audit. We inventory functions, triggers, queues, and external dependencies, and check which metrics and logs are already being written and which execution paths remain "dark." The output is a coverage map: where every invocation is visible and where it is not.
2. Data collection design. We design how tracing links invocations into a single request, how context is passed between functions and queues, where logs are stored, and at what depth. Separately, we decide what to sample so as not to pay for storing the entire stream.
3. Alert configuration. We tie rules to symptoms rather than to every metric: a rising error rate, a spike in latency, a buildup in the queue, exhaustion of concurrent runs. We configure thresholds and windows so that the on-call engineer receives a signal about real degradation rather than noise.
4. Failure testing. We break the system deliberately — disable a dependency, introduce a delay, crash a function — and see whether the alert fired and whether there is enough data to determine the cause. We rewrite the rules that failed to catch the failure.
5. Documentation handover. We hand over the configured dashboards, alert rules, collection design, and incident triage instructions so that the team can work with the system without us.

Such a route delivers a predictable result: by handover you have invocation coverage with tracing, alerts verified against real failures, and documentation that lets a new engineer triage an incident without verbal hints. Describe your task — we will come back with an estimate and a stage-by-stage plan.

How does function monitoring differ from server monitoring?

Classic server monitoring relies on continuity: an agent lives on the host, captures CPU, memory, disk, and network load, and the on-call engineer watches a 24-hour graph. In a serverless environment, there is no such continuity.

A function instance spins up for a specific invocation, does its work, and is destroyed — often within fractions of a second.

Function metrics are collected not from a machine but from a stream of events: how many invocations, what duration, how many errors, how many "cold" starts. The familiar server metrics are either absent here or misleading.

Ephemeral instances break the very idea of "host state." A function has no long-lived process to connect to, no stable address, no cumulative statistics between invocations.

Instead, each unit of work is a separate invocation context with its own identifier, start time, and end time.

Monitoring is built around request boundaries: from event ingestion to response write, including all calls to the database, queues, and external services within that window.

Hence the key difference: server monitoring answers the question "is the host healthy," while serverless monitoring answers "is a specific processing path healthy." The same code can run normally on 99% of invocations and fail on a rare scenario with a large request body or an external service timeout.

Without tying a metric to the invocation context, such a failure is invisible: averaged graphs blur it out.

In practice, this means three things. Function metrics need to be sliced by dimensions: function name, version, event source, response code. Errors and latencies must be counted at request boundaries, not per host.

And tracing must link a single invocation to all its internal operations, otherwise it's unclear exactly where time is being lost. This set provides what server monitoring does not: an answer to the question of which specific scenario degraded and at which step.

Case: Cut failure root-cause search time by threefold

A serverless service built on dozens of functions: every request passes through an API gateway, a queue, several handlers, and a write to the database.

When errors grew on the storefront, engineers opened the logs of each component separately and manually stitched the chain together by request ID — localizing the cause took on average about two hours, and some incidents were closed "by symptoms" without ever reaching the root.

What it was like before implementation

Logs and metrics lived in different places: function errors — in one service, gateway latencies — in a second, queue states — in a third. The on-call engineer saw a spike in failures but couldn't understand in a single pass exactly where the chain was breaking. Incident triage turned into back-and-forth between teams rather than analysis.

What was changed

End-to-end tracing was enabled: a single identifier now ties a request together along the entire path — from the inbound call to the last write. Unified dashboards were assembled, where a single screen shows the error rate, latencies for each link, and queue depth.

Alerts were configured around the symptoms that actually matter: a growing failure rate and exceeding the response-time threshold, rather than every individual log.

What it became

Root-cause search time was cut threefold, and incident triage no longer depended on who happened to be on call. To achieve this, Datadog was used as a single point for collecting traces, metrics, and logs of serverless functions.

What if the service is already partially covered by monitoring? End-to-end tracing can be enabled gradually, starting with critical paths — the remaining components are connected without stopping operations. What if there are more functions?

The single-request-ID scheme scales along with the service: new handlers pick up tracing automatically.

Describe your task — we'll come back with an estimate and a plan: we'll show what a unified dashboard and end-to-end tracing would look like in your specific service.

Compatibility, access, and rollback: what we discuss before the start

Before the start, three questions usually come up: will monitoring break current processes, what access will be needed, and how to roll back if something goes wrong. We answer them in order — this is the content of the kickoff meeting.

What if we already have log collection and alerts set up? Monitoring is embedded on top of the existing stack rather than replacing it.

Traces and metrics are linked to current dashboards and notification channels, and duplicate rules are disabled after verifying that the new ones cover the same scenarios. Integration with the existing stack is tested on one service before rolling out to the rest.

What cloud access is needed? A read-only role for metrics and logs, a separate account for telemetry collection, with no permissions to modify production resources. We describe access policies explicitly: what we read, what we write, where the boundaries are. All keys are in a secure vault, with rotation and an usage log.

What if monitoring starts slowing down the application? Collection is asynchronous, outside the main request processing thread. If overhead exceeds the agreed threshold, we disable collection on the problematic service with a single setting, without touching the rest.

What does the rollback plan look like? Each change is a separate stage with a return point: saved the configuration, applied it, checked the metrics, and if there is a deviation, restored the previous state. Rollback takes minutes and does not require rebuilding the application.

What the customer receives upon delivery:

• Configuration for collecting metrics, logs, and traces with comments on parameters
• A data flow diagram: from the service to the dashboard and notification channel
• Access policies with role separation and permission boundaries
• A set of dashboards and alerting rules for key scenarios
• An on-call regulation and a rollback plan instruction
• Documentation on integration with the existing stack

Describe your task — we will come back with an estimate and a plan: what access will be needed, what will be included in the first stage, and how rollback will be structured.