Elasticsearch Monitoring and Alert Configuration (Kibana)

We often encounter this situation: an Elasticsearch cluster silently goes red, disk fills to 95%, and the team learns about it from users. Without alerts, an incident turns into an outage. Our experience shows that monitoring configuration is a key step that prevents downtime and data loss. In this article, we'll cover how to set up full monitoring and alerts via Kibana using Metricbeat, Watcher, and Prometheus. Below are battle-tested configs and production tips.

Why Elasticsearch monitoring is critical?

Elasticsearch powers search and analytics in many projects. If the cluster goes down, business processes stop. Monitoring lets you spot degradation early: rising JVM heap, disk filling, increasing search latency. Without it, you hear about problems from users. We offer a ready monitoring scheme with Telegram/Slack alerts so you always know your cluster state.

Which monitoring tool to choose?

For production, we recommend Metricbeat + Stack Monitoring for basic metrics and Watcher for alerts. If you already use Prometheus, integrate elasticsearch_exporter.

How does Stack Monitoring work in Kibana?

Kibana Stack Monitoring is a built-in tool for collecting metrics from Elasticsearch, Logstash, and Kibana. Data can be collected via Metricbeat (recommended) or the built-in agent (deprecated). We strongly advise sending metrics to a separate monitoring cluster — otherwise a primary cluster failure means losing monitoring. We use Metricbeat with X-Pack enabled.

Metricbeat configuration for Elasticsearch metrics:

# metricbeat.yml metricbeat.modules: - module: elasticsearch xpack.enabled: true period: 10s hosts: ["https://localhost:9200"] username: "remote_monitoring_user" password: "${ES_MONITOR_PASSWORD}" ssl.certificate_authorities: ["/etc/elasticsearch/certs/ca.crt"] scope: cluster metricsets: - ccr - cluster_stats - enrich - index - index_recovery - index_summary - ml_job - node - node_stats - pending_tasks - shard output.elasticsearch: hosts: ["https://monitoring-es:9200"] username: "metricbeat_writer" password: "${MONITOR_WRITER_PASSWORD}" 

Key metrics: what to watch

Cluster health — first thing to check: green means all shards assigned, yellow means replicas not assigned (normal for single node, problem for production), red means data unavailable.

JVM heap usage — critical: below 75% normal, 75–85% monitor, >85% degradation, >95% JVM freezes on GC and cluster stops responding.

Disk usage per node — Elasticsearch blocks indexing when disk fills. Flood_stage threshold (95%) makes indices read-only; high_watermark (90%) triggers shard rebalancing; low_watermark (85%) normal.

Setting up alerts via Watcher

Watcher is X-Pack's built-in alerting system. Configured via API or Kibana UI. Here's an example alert for red cluster status:

PUT _watcher/watch/cluster_status_red { "trigger": { "schedule": { "interval": "1m" } }, "input": { "http": { "request": { "host": "localhost", "port": 9200, "path": "/_cluster/health", "auth": { "basic": { "username": "elastic", "password": "{{ctx.metadata.es_password}}" } } } } }, "condition": { "compare": { "ctx.payload.status": { "eq": "red" } } }, "actions": { "send_telegram": { "webhook": { "scheme": "https", "host": "api.telegram.org", "port": 443, "method": "post", "path": "/bot{{ctx.metadata.telegram_token}}/sendMessage", "params": { "chat_id": "{{ctx.metadata.telegram_chat_id}}", "text": "ALERT: Elasticsearch cluster status is RED! Time: {{ctx.execution_time}}" } } } } } 

Alert for disk usage >85%:

PUT _watcher/watch/disk_usage_high { "trigger": { "schedule": { "interval": "5m" } }, "input": { "http": { "request": { "path": "/_nodes/stats/fs", "auth": { "basic": { "username": "elastic", "password": "changeme" } } } } }, "condition": { "script": { "source": """ for (node in ctx.payload.nodes.values()) { def total = node.fs.total.total_in_bytes; def free = node.fs.total.free_in_bytes; def used_pct = (total - free) / total * 100; if (used_pct > 85) return true; } return false; """ } }, "actions": { "log": { "logging": { "level": "warn", "text": "High disk usage detected on Elasticsearch node" } } } } 

How to set up alerts via Kibana UI?

In Kibana 8.x, use Alerts & Actions (Stack Management > Rules). Visual rule builder without writing JSON manually. Ready templates: Elasticsearch cluster health, nodes changed, version mismatch, CPU usage, JVM memory. Notification channels: Email, Slack, PagerDuty, Webhook (Telegram, Teams).

Monitoring via Prometheus and Grafana

If your infrastructure already uses Prometheus, connect elasticsearch_exporter:

docker run -d \ --name elasticsearch_exporter \ -p 9114:9114 \ prometheuscommunity/elasticsearch-exporter:latest \ --es.uri=https://elastic:changeme@localhost:9200 \ --es.ssl-skip-verify \ --es.all \ --es.indices \ --es.shards 

Prometheus scrape_config:

- job_name: 'elasticsearch' static_configs: - targets: ['localhost:9114'] scrape_interval: 30s 

Import Grafana dashboard ID 6483 (Elasticsearch Overview) — a ready dashboard with key metrics. As noted in the Elasticsearch documentation, this simplifies visualization.

What's included in our work

We provide: deployment of Metricbeat and Stack Monitoring dashboards; configuration of alerts via Watcher or Kibana Rules with Telegram/Slack/PagerDuty channels; integration with Prometheus and Grafana (if infrastructure exists); documentation and guidance for your team; 3 months of support and threshold adjustments. Our experience — 5 years in Elasticsearch administration, over 20 projects. We guarantee SLA on response time. Our engineers hold Elastic Certified Engineer certifications.

Timelines

Basic monitoring via Metricbeat and Stack Monitoring — 1 day. Alerts — 1 day. Advanced monitoring with Prometheus+Grafana — 1–2 days. Total from 2 to 4 days. Cost is calculated individually.

Contact us for a consultation — we'll assess your cluster and propose the optimal solution. Get in touch to discuss details.