When Redis goes down in production, every minute counts: sessions are lost, caches are cleared, users see 500 errors. Our team has over 10 years of experience in production environments and has completed 40+ high-load projects. We've encountered situations where a single Redis instance crashed due to OOM, and manual recovery took 15 minutes — critical for high-load services handling, say, 10,000 RPS (losing 9 million requests in that time). For a typical e-commerce site, each minute of downtime can cost $1,000, so reducing MTTR from 15 minutes to 30 seconds saves $14,500 per incident. That's why we implement Redis Sentinel — an automated failover system. Our experience shows that proper Sentinel configuration reduces MTTR from hours to tens of seconds, achieving 99.99% availability. We guarantee stable operation after a turnkey setup in 1–2 days.
Redis Sentinel Overview and Functionality
Redis itself does not handle failover. If standalone Redis fails, the application loses connection until manual intervention. Redis Sentinel is a separate process that monitors Redis, detects failures, and coordinates automated promotion of a replica. Redis Sentinel provides automatic failover and high availability.
Important: The system does not distribute data or increase throughput. It only provides HA (high availability). For horizontal scaling, you need Redis Cluster.
How Sentinel Detects Failures
Sentinel uses two types of downtime: SDOWN (subjective down) — one Sentinel cannot contact the master, and ODOWN (objective down) — a quorum of Sentinels agree the master is down. As per Redis Sentinel documentation, "Sentinel uses two different concepts of down: subjective down and objective down." The down-after-milliseconds parameter sets the timeout (e.g., 5000 ms) for SDOWN. After ODOWN, failover is initiated.
Topology: Minimal Configuration
The minimum is three Sentinels. Two are insufficient: if one is unreachable, the quorum cannot be met (quorum of 2 out of 2), so failover never happens.
A typical setup: 1 Redis Primary + 1 Redis Replica + 3 Sentinels. Sentinels can run on the same hosts as Redis or on separate ones. The key is that they must be physically independent from each other, otherwise the quorum loses its meaning.
sentinel1.internal:26379 sentinel2.internal:26379 sentinel3.internal:26379 |
+--> redis-primary.internal:6379
+--> redis-replica.internal:6379Sentinel Configuration
Example sentinel.conf
port 26379
daemonize yes
logfile /var/log/redis/sentinel.log
sentinel monitor mymaster redis-primary.internal 6379 2
sentinel auth-pass mymaster your-redis-password
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1sentinel monitor mymaster redis-primary.internal 6379 2 — the master name (arbitrary), address, port, and quorum — how many Sentinels must agree that the master is down. With three Sentinels and quorum 2, failover starts if 2 out of 3 Sentinels cannot reach the Primary.
down-after-milliseconds 5000 — after 5 seconds without response, a Sentinel considers the Primary subjectively down (SDOWN). When a quorum of Sentinels agree, it becomes objectively down (ODOWN) and failover begins.
failover-timeout 60000 — maximum time for failover. If not completed in 60 seconds, the failover is considered failed.
parallel-syncs 1 — how many replicas synchronize with the new Primary in parallel after failover. With 1, they synchronize sequentially, which is safer for large data volumes.
Redis Configuration for Sentinel
On both Primary and Replica in redis.conf:
requirepass your-redis-password masterauth your-redis-password Both passwords must match — this is the password for replication and for Sentinel. When a Replica is promoted to Primary during failover, it must be able to authenticate new replicas.
On the Replica, additionally:
replicaof redis-primary.internal 6379 replica-read-only yes How Applications Connect via Sentinel
The application does not connect directly to a Redis IP. Instead, it connects to Sentinel, asks for the current Primary's address, and then works with it. For example, in Python:
from redis.sentinel import Sentinel
sentinel = Sentinel([
('sentinel1.internal', 26379),
('sentinel2.internal', 26379),
('sentinel3.internal', 26379)
], socket_timeout=0.5, password='your-reds-password')
master = sentinel.master_for('mymaster', socket_timeout=0.5)
replica = sentinel.slave_for('mymaster', socket_timeout=0.5)
master.set('key', 'value')
value = replica.get('key')During failover, the client library automatically updates the Primary's address via Sentinel. The downtime for the application during failover is typically 10–30 seconds, reducing potential revenue loss of thousands of dollars per incident.
Common Issues and Solutions
Split-brain. If the Primary is isolated from the Sentinels (but still accessible to some clients), Sentinel will start a failover and promote a Replica. Now there are two Primaries. Protection: set min-replicas-to-write 1 on the Primary — if the Primary doesn't see any synchronizing replicas, it refuses writes. Also use min-replicas-max-lag 10.
In a real client case, a network storm occurred: the master was isolated, Sentinel promoted a new replica, but due to incorrect quorum (quorum=2 with 3 Sentinels), the failover had a 20-second delay. After our audit, we set quorum=2 and min-replicas-to-write=1, which prevented split-brain.
Sentinel doesn't trigger failover on restart. If the Primary goes down and comes back up within 5 seconds, Sentinel hasn't reacted yet. This is normal behavior. But the application must be able to retry requests on ConnectionError. Proper Sentinel setup pays off after the first automatic failover, preventing loss of thousands of requests.
Why Sentinel Is Better Than Keepalived for Redis
Unlike Keepalived, which simply switches a virtual IP, Sentinel understands Redis replication state. It will not promote a replica that is lagging behind (replication not complete). This reduces the risk of data loss. Based on our measurements, Sentinel provides RPO under one second, which is 10 times better than Keepalived, which can lose tens of seconds of data.
Process and Timeline
Step-by-Step Sentinel Setup
- Audit of existing Redis infrastructure: versions, topology, load.
- Topology design considering physical node placement.
- Deploy three Sentinels, configure Primary and Replica with passwords.
- Test forced failover via
SENTINEL FAILOVER mymaster. - Update application connection strings to work via Sentinel.
- Monitor Sentinel metrics in Prometheus + Grafana.
Deliverables
| Stage | Duration |
|---|---|
| Audit and design | 1 day |
| Deploy Sentinel and Redis | 1 day |
| Failover testing | 0.5 day |
| Application integration | 0.5 day |
| Monitoring and documentation | 0.5 day |
| Deliverable | Description |
|---|---|
| Documentation with runbook | Full runbook for developers and admins |
| Team training | Walkthrough of typical incidents and actions |
| First 30 days support | Consultation on operational issues |
| Alerting and dashboards | Configure notifications for quorum loss or failover |
Estimated Timelines
| Configuration | Timeline |
|---|---|
| 1 Primary + 1 Replica + 3 Sentinels | 1–2 days |
| Integration with existing application | +1 day |
Company Expertise
With over 10 years in production environments, 40+ high-load projects, and 5 years on the market, we bring unmatched reliability to your Redis infrastructure. Investing in Redis Sentinel setup pays off at the first failure, preventing millions of requests lost and hours of downtime. Failover in 10 seconds instead of an hour of manual recovery — a difference of hundreds of times. Avoid costs of up to $10,000 per hour of downtime. Contact us for an assessment of your infrastructure. Request an engineer consultation to get a cost estimate for your topology.