MongoDB Atlas monitoring setup: alerts, Prometheus, Grafana

Why MongoDB Atlas failures are noticed late without monitoring

A MongoDB Atlas cluster rarely fails all at once — more often it degrades gradually. Response time for heavy queries grows by tens of percent, some operations no longer fit within the allotted time, while load graphs remain in the usual corridor.

MongoDB Atlas monitoring is needed not to record a failed node, but to catch such a trend days before failure.

There are usually four scenarios of hidden cluster degradation. The number of connections grows slightly faster than traffic, and connection pool exhaustion manifests as a sharp spike in errors during peak load, although the prerequisites had been accumulating for a week.

Replica lag on secondary nodes makes reads from them stale: reports and lists show data that no longer exists. Add gradual memory growth and queries without an index — and the cluster approaches its limit without showing a single red indicator.

The first signal about a problem almost always comes from users: "it won't load," "the data is wrong," "the order disappeared." Then manual log analysis begins, where the cause is no longer distinguishable from the effect, and without a history of metrics over several weeks it is impossible to understand when degradation began and what triggered it.

We explain how this is covered: which metrics to collect from the cluster, which thresholds to consider critical, and how to link alerts with an action runbook.

What continuous load and replication monitoring provides

Monitoring without measurable criteria degenerates into a set of graphs that no one looks at until the first failure.

The checklist below is what is verified on a live MongoDB Atlas installation and in a Prometheus with Grafana setup: each item answers the question "what changed in the numbers," not "how many panels were drawn."

• Incident detection time is minutes, not hours. The signal about growing replica lag, disk journal filling up, and failures in the write queue is collected from every node, so degradation is detected at the input, not after the database becomes unavailable.
• False positive rate is less than one tenth. Threshold alerts are tied to system behavior at the same time of day and to the rate of metric change, rather than to a single static number, so scheduled backups no longer wake the on-call engineer at night.
• No classes of failures remain unnoticed: replica role change, growing read latency from secondary nodes, notification broker failure — each event goes into a separate rule with an explicit recipient.
• It is clear where the bottleneck is: load is broken down by operations, collections, and indexes, and the alert itself contains the metric, deviation value, and node — analysis begins without messaging and calls.

Such a set of rules is tested in a training scenario: we shut down a node, load a collection, and see whether the signal arrived and how long it took. An item that did not trigger is refined to acceptance, not postponed.

Methods of metrics collection: built-in tools, external stack, hybrid

Atlas metrics are available through three channels at once: panels and API of the service itself, push export of metrics to your own endpoint, and polling the cluster with an external exporter.

The built-in stack provides ready alerts and graphs without infrastructure, but keeps history for only a short time and operates with a fixed set of thresholds.

Full-fledged observability begins where metrics enter a common storage — then they can be correlated with application load and network state.

The choice between schemes comes down to three questions: how deep the history needs to be stored, how much composite conditions are needed in alerts, and how much maintenance you are ready to allocate.

In practice, the choice between an external monitoring stack and a hybrid comes down to maintenance effort: an exporter and scrape provide flexibility, but require monitoring access to replicas, metric names, and labels.

In a hybrid scheme, a separate task is not to duplicate alerts and not to split thresholds across two places; otherwise the on-call engineer receives two notifications about one incident and stops responding to both.

We usually leave the built-in stack for fast unavailability signals, and move deep history and composite conditions to the external one.

Setup stages: from schema audit to handover of procedures

We start not with enabling alerts, but with an installation audit: we look at replica and shard topology, collection sizes and load profile, number of active connections, log retention periods, and who has access to the cluster at all.

Without this map, any set of thresholds will be guesswork — it will either stay silent during real degradation or wake the on-call engineer at night because of a scheduled backup. MongoDB Atlas monitoring setup is built so that every signal has an owner and a clear action.

1. Installation audit. We record the cluster composition, index bottlenecks, connection limits, and current out-of-the-box alerts — what is already configured and what is duplicated.
2. Metric schema. We define health indicators: replication lag, operation queue, share of slow queries, disk file growth, authentication errors. We tie each indicator to a business consequence so that the threshold can be justified.
3. Connecting data collection. We deploy a metrics exporter with a separate read-only account, set the polling interval and history retention period. Stack metrics go to common storage — alongside data from the rest of the infrastructure.
4. Alert setup. We separate levels: warning to the work channel, critical to the on-call engineer. We set thresholds by deviation from the baseline, not by absolute values that become outdated as load grows.
5. Alert testing. We run artificial load and controlled failures: shut down a node, fill up a disk, break replication. We see what triggered, what stayed silent, whether the notification reached the on-call engineer, and how long it took.
6. On-call procedure. We describe criticality levels, escalation order, planned maintenance windows, and a list of standard actions for each signal.
7. Handover to the team. We hand over observability dashboards, instructions for standard incidents, and false positive analysis — your engineers run the system without us.

What remains with the team: configurations, dashboards, procedures

We do not leave monitoring as a "service on our side" — we assemble it inside your environment so that access, configurations, and dashboards remain with your team.

Below is the set of artifacts with which the on-call engineer works without us, and a new engineer can deploy collection in a fresh environment.

• Collection configurations — agent parameters and Atlas API integration: polling period, list of databases and collections, exclusions for system namespaces.
• Metric map — a table "indicator → source → threshold → service owner": it shows where the number comes from and whom to wake when it goes out of bounds.
• Observability dashboards — reviews by level: cluster and replication, nodes, connections, operation queues, slow query profile.
• Alert configurations — rules with thresholds, suppression windows, notification routes, and escalation by on-call levels.
• On-call instructions — analysis procedure: what to look at first, standard actions during a failure, escalation point, incident logging.
• Access rules — roles on the dashboard and alerts, separate keys with limited rights and rotation.
• Review procedure — frequency of threshold revision so alerts do not turn into noise that people stop reading.
• Deployment documentation — how to set up collection in a new cluster and what to check after connection.

All of this is stored in your repository and cloud environment: our team leaves, the monitoring's operability does not.

Case study: reduced failure detection time, eliminated false positives

A client came to us with a loaded MongoDB Atlas cluster: alerts were set on CPU and average metric values.

The threshold triggered only after degradation had already occurred, and half of the nighttime notifications turned out to be noise — the on-call engineer woke a developer, and both looked at an ordinary load spike.

What we changed in the rules

We built alerts around signals that predict failure rather than record it: replication lag, oplog window, share of slow operations, queue and held cursors, growth in connection count, driver errors.

Thresholds were tied to deviation from a stable norm on a long window; the short one was left only for hard failures — node unavailability or primary change. Each rule got its own route and a short instruction on what to check first.

Metrics before and after

Detection time dropped from tens of minutes to two or three: the on-call engineer sees the problem, not the user.

The share of false alerts fell from most nighttime triggers to less than one tenth — noise was cut off by windows and derived metrics instead of instantaneous thresholds.

The share of incidents the on-call engineer closes without a developer grew from a quarter to most: they have a dashboard and an action procedure for each rule. Nighttime wake-ups became isolated — for the team, this is the main effect.

A separate class of failures that used to be caught through complaints also disappeared: oplog window exhaustion against the backdrop of replica lag.

Now growing lag is visible long before secondary nodes start serving stale data — there is time margin to intervene according to the procedure.

How built-in alerts differ from an external stack

Built-in Atlas alerts are the first stack enabled when putting a cluster into operation. They monitor project metrics: CPU and disk load, number of active connections, replication lag, volume of read and write operations, node availability.

The threshold is set in the interface, the notification goes to email, a messenger, or an external address — this is enough to catch node failure, disk space exhaustion, or traffic growth.

Then limitations begin. Thresholds are static and exist separately from application load, the averaging window is set by the platform, and metric history depth is limited by its own retention period.

It is no longer possible to correlate load growth with a release from a week ago or with index rebuilding — the data in the needed window is simply not there.

The external stack collects the same metrics independently and stores them in its own time-series database. We determine history depth: a month for operational analysis, a year or more for trends and capacity planning.

Threshold flexibility follows: rules are described with Prometheus queries, can be combined with each other, distinguish "node unavailable" from "node is not exposing metrics," and rely on rate of change, not only on absolute value.

On the dashboards, the cluster sits next to the application server, load balancer, and queue. The incident is visible as a whole: a spike in response time, growth in connection count, and replica lag are read from one graph, not assembled manually from three interfaces.

We explain how this is covered: which metrics to collect first, where to set thresholds, and how to link the external stack with built-in alerts so that signals are not duplicated.

Frequently asked technical questions before starting work

Before starting, clients check three things: whether application code will need to be touched, whether the cluster can withstand connecting observability without downtime, and whether the on-call shift will drown in alerts. We answer each point — this removes most doubts even before estimation.

MongoDB Atlas monitoring is assembled from two streams: service telemetry of the cluster and metrics from the application side. The first shows the state of nodes, replication, queues, and disk I/O; the second shows how a particular service withstands load. Below are the technical details most often clarified at this stage.

The cluster is running under load — how do you connect observability?

Observability is connected on a running cluster and does not require stopping it. The agent reads service statistics and does not interfere with query processing; we limit resource consumption for collection so that it does not compete with the production load. Rolling restart and topology changes are not needed at this stage.

How do you filter out noise from scheduled load?

We build thresholds on deviation from the baseline rather than on absolute values: nighttime reindexing and backup should not look like an incident. Planned maintenance and bulk export windows are marked as expected — alerts for them are suppressed. Only what requires action reaches the on-call shift.

Where and how long are metrics stored?

Collection goes to Prometheus; visualization and dashboards go to Grafana. Fresh detailed data is kept in operational storage, and old data is rolled up into aggregates, so volume grows predictably rather than linearly.

Retention periods are set based on how deeply incidents need to be analyzed and load compared between seasons.

Let's discuss the task: send a diagram — we'll come back with a plan

For the estimate to be substantive rather than a "rough guess," it is enough to attach a few inputs to the conversation — we review them before the call, so the meeting gets to the point: we clarify gaps and immediately move to the setup plan.

• cluster topology: replica set or sharding, number of nodes, deployment regions, fault tolerance mode;
• metrics used for decisions: query latency, collection growth, cache misses, number of connections, replica lag;
• what already works in Atlas alerts: thresholds, recipients, delivery channels, and who handles triggers;
• access — for reading metrics, changing alert rules, at the project and organization level;
• collection and visualization stack: stay inside Atlas or consolidate data into a single metrics collector with dashboards;
• on-call mode and escalation order: who responds to a nighttime trigger and according to which rule.

Separately, we record what is considered normal during peak load hours — without this baseline, any threshold will be arbitrary, and alerts will either stay silent or wake the on-call engineer for no reason.

We will come back with a setup plan: a list of rules and their priorities, a way to test each alert in a real failure, an estimate of work by stage, and an observability handover procedure — so that the on-call shift maintains the system without us. Describe the task in free form; that is enough to start.