DR Site Setup (Hot/Warm/Cold Standby)

DR Site Setup (Hot/Warm/Cold Standby)

A company loses tens of thousands of dollars per hour of downtime — that's the trigger for implementing DR. We design standby sites from scratch: from model selection to drills. Our engineers hold AWS certifications and have experience recovering from outages for clients in fintech and e-commerce. The choice between Hot, Warm, and Cold Standby is not a technical but a financial decision with technical constraints. The faster you need to recover, the more expensive the standby site. The engineer's job is to present the business with realistic options with concrete RTO and cost figures, not just design a "correct" textbook solution.

How to Choose the Replication Model?

The determining factors are acceptable downtime (RTO) and data loss (RPO). According to Wikipedia, RTO and RPO are key DR metrics. The smaller the RTO, the more expensive the infrastructure. We help find a balance: assess service criticality, downtime cost, and budget.

Hot Standby: Real-Time Mirror

Hot Standby is a full copy of the production infrastructure operating in real time. Switching takes seconds or a few minutes. RPO approaches zero with synchronous replication.

Cloud architecture: active/passive cluster in two regions with synchronous database replication and AWS Global Accelerator or Cloudflare for DNS failover.

# hot-standby/rds.tf
resource "aws_db_instance" "primary" {
  identifier = "prod-primary"
  engine = "postgres"
  engine_version = "15.4"
  multi_az = true # within one region
  deletion_protection = true
  backup_retention_period = 7
  tags = {
    Role = "primary"
  }
}

resource "aws_db_instance" "dr_replica" {
  identifier = "prod-dr-replica"
  # Read replica in DR region — on failover promote to master
  replicate_source_db = aws_db_instance.primary.arn
  availability_zone = "eu-west-1a"
  # On promote this replica becomes the new master
  auto_minor_version_upgrade = false
  tags = {
    Role = "dr-hot-standby"
  }
}

Hot Standby cost roughly doubles infrastructure spending. For most startups this is overkill. It is justified for fintech, healthcare, and e-commerce with losses exceeding $10K/hour during downtime.

Warm Standby: Balance of Cost and RTO

Warm Standby means DR infrastructure exists but runs at reduced capacity. Database replicates with a small lag. Application servers are off or run at minimal power. On DR activation, instances must be started, pending configuration changes applied, and DNS switched.

RTO: from 15 minutes to 2 hours depending on complexity. RPO: from a few minutes to 1 hour.

# ansible/dr-warmup.yml — DR site warm-up procedure
---
- name: Warm up DR site
  hosts: dr_site
  become: yes
  tasks:
    - name: Start application servers
      systemd:
        name: "{{ item }}"
        state: started
      loop:
        - app-server
        - worker
        - nginx
    - name: Promote PostgreSQL replica
      command: |
        su - postgres -c "pg_ctl promote -D /var/lib/postgresql/data"
      when: promote_db | bool
    - name: Update DNS to DR endpoint
      community.aws.route53:
        state: present
        zone: company.com
        record: app.company.com
        type: A
        ttl: 60
        value: "{{ dr_lb_ip }}"
        overwrite: yes
    - name: Run health checks
      uri:
        url: "https://app-dr.company.com/health"
        status_code: 200
        retries: 10
        delay: 30

Warm Standby usually costs 20–40% of production infrastructure. It is the most common choice for companies with SLA 99.9%.

Cold Standby: Minimal Cost, Maximum RTO

Cold Standby means DR infrastructure does not really exist. There are backups, documentation for building a new environment, and Terraform configuration. On DR activation, infrastructure is created from scratch, data is restored from backups.

RTO: 4 to 24 hours. RPO: matches backup frequency, typically 1–24 hours.

#!/bin/bash
# dr-coldstart.sh — cold start script
set -euo pipefail

echo "=== DR Cold Start initiated at $(date) ==="

# Step 1: Bring up infrastructure via Terraform
cd /opt/terraform/dr-site
terraform init -backend-config="bucket=company-tf-state-dr"
terraform apply -auto-approve -var="environment=dr"

# Step 2: Restore data from the latest backup
LATEST_BACKUP=$(aws s3 ls s3://company-backups/db/ \
  --recursive | sort | tail -1 | awk '{print $4}')
aws s3 cp "s3://company-backups/$LATEST_BACKUP" /tmp/db-restore.dump

# Step 3: Restore database
pg_restore \
  --host="${DR_DB_HOST}" \
  --username=postgres \
  --dbname=production \
  --jobs=8 \
  /tmp/db-restore.dump

# Step 4: Switch DNS
./scripts/switch-dns-to-dr.sh

echo "=== DR Cold Start completed at $(date) ==="

Cold Standby is suitable for non-critical services or companies with a very limited DR budget. It is important to verify that the Terraform configurations are up to date and terraform apply actually works — otherwise, in a disaster, you may find the state has diverged from reality months ago.

What's Included in DR Site Design?

We provide:

• Audit of current infrastructure and identification of critical services.
• RTO/RPO calculation and Standby model selection.
• DR site architecture with block diagrams.
• Implementation via Terraform and Ansible (code in your repository).
• Data replication configuration (database, files, config).
• Documentation of activation procedures and team roles.
• Conducting the first drill with a report.
• Post-implementation support: consultations and refinements.

All of this — turnkey: from idea to a working DR ready for testing.

Model Selection Checklist

Why Testing Is Critical?

Without drills, a DR site is a theoretical construct. We conduct the first test within a month after deployment. During the drill, we verify: infrastructure startup, data restoration, DNS switchover, response time. We log and fix errors. Our experience shows that even with perfect documentation, the first run reveals 5–10 unforeseen issues.

Design and Implementation Process

We start by capturing RPO/RTO requirements together with the business. Then — audit of current infrastructure: inventory of critical services, dependency analysis, data volume for replication. Based on requirements and budget, we select the Standby model. We design the DR site architecture, describe it in Terraform. We implement data replication. We document activation procedures. We conduct the first test.

Without testing, a DR site is a theoretical construct. The first drill should be conducted within a month of going live.

Timeline Estimates

Cost is calculated individually after an infrastructure audit.

We will assess your project — contact us. We guarantee DR operability subject to annual drills. Over 5 years, we have implemented more than 50 disaster recovery projects.