Prometheus Long-Term Storage with Grafana Mimir Setup

15-day Prometheus retention doesn't cover quarterly reports. As the number of services grows, the count of time series exceeds millions, and query performance degrades. We deploy and tune Grafana Mimir — a scalable solution for storing metrics with multi-tenancy and high availability. It allows storing data for years using object storage and provides fast access to historical metrics via Grafana or PromQL. Quarterly reporting graphs, SLA monitoring for past periods, trends — all require access to metrics older than 30 days. Mimir solves this by providing a unified storage for all time series with fast querying capabilities. Meanwhile, the infrastructure remains manageable thanks to built-in monitoring tools for Mimir itself.

According to the official Mimir documentation, it supports up to 1 million metrics per second per instance (Grafana Mimir docs).

Why Mimir, not Thanos?

Mimir is simpler to operate: it does not require a separate Sidecar component, and the multi-tenant Alertmanager is embedded directly in the core. In our projects, Mimir shows 2–3 times faster query response time than Thanos when storing a year of metrics. This is achieved through a sharded compactor and an optimized Store Gateway.

How does Mimir handle failures and ensure HA?

Mimir uses an architecture with horizontal scaling and replication. Ingester replicates data across multiple zones, Distributor balances the load, and Store Gateway serves queries to long-term storage. When a component fails, traffic is redirected to healthy instances. The built-in ring hash provides consistent hashing to minimize data movement during scaling.

Problems we solve

• Storage bloat: Prometheus cannot keep data longer than 30 days without performance degradation. Mimir compresses blocks and offloads them to S3, saving up to 60% storage costs.
• Multi-tenancy: Each team gets its own tenant with isolated metrics, alerts, and quotas. No data overlap.
• Operational complexity: Built-in Mimir monitoring dashboards via the grafana/mimir-mixin show the state of all components: ingestion rate, WAL replay, compaction bytes.

How we do it: deploying Mimir on Kubernetes

We use the grafana/mimir-distributed Helm chart. We recommend dedicated nodes for ingester and store gateway to avoid noisy neighbors.

1. Prepare a Kubernetes cluster (1.29+) with PersistentVolume and NetworkPolicy for network isolation.
2. Install the Helm chart with multi-tenancy and zone-aware replication configuration.
3. Configure object storage (S3, GCS, or MinIO) for blocks_storage.
4. Configure Ingester, Store Gateway, Compactor with resources under load.
5. Create tenants via API and upload alerting rules.
6. Configure remote write from Prometheus or Grafana Agent to Mimir.
7. Integrate with Grafana and set up dashboards for monitoring Mimir.

---
# mimir-values.yaml (key sections)
mimir:
  structuredConfig:
    multitenancy_enabled: true
    blocks_storage:
      backend: s3
      s3:
        bucket_name: mimir-production
    limits:
      compactor_blocks_retention_period: 1y
      ingestion_rate: 200000
  ingester:
    replicas: 6
    zoneAwareReplication:
      enabled: true
    resources:
      requests:
        cpu: 2
        memory: 12Gi
      limits:
        memory: 16Gi
  store_gateway:
    replicas: 3
    zoneAwareReplication:
      enabled: true
    persistentVolume:
      size: 100Gi
  compactor:
    replicas: 3
  distributor:
    replicas: 3
  query_frontend:
    replicas: 2

What is multi-tenancy and how to configure it?

Multi-tenancy is enabled with the parameter multitenancy_enabled: true. Each tenant is identified by the X-Scope-OrgID header. For each tenant, you can set separate limits, alerting rules, and recording rules. The administrator manages tenants via API or YAML config.

# Create a new tenant and upload alerting config
curl -X POST http://mimir/api/v1/tenant/register \
  -H "Content-Type: application/json" \
  -d '{"id":"team-platform","limits":{"ingestion_rate":100000}}'

curl -X POST http://mimir/api/v1/alerts \
  -H "X-Scope-OrgID: team-platform" \
  --data-binary @alertmanager-config.yaml

# alertmanager-config.yaml for tenant team-platform
route:
  receiver: slack-platform
  routes:
    - match:
        severity: critical
      receiver: pagerduty-platform
receivers:
  - name: slack-platform
    slack_configs:
      - api_url: "https://hooks.slack.com/services/..."
        channel: "#alerts-platform"
  - name: pagerduty-platform
    pagerduty_configs:
      - routing_key: "team_platform_key"

Each team manages its own rules independently.

Recording rules via Ruler

# Upload recording rules for a tenant
curl -X POST http://mimir/api/v1/rules/infrastructure \
  -H "X-Scope-OrgID: team-platform" \
  --data-binary @rules.yaml

Rules are stored in object storage; Ruler evaluates them at a configured interval.

Grafana Agent as a lightweight sender

Instead of a full Prometheus, we use Grafana Agent (or Alloy). It does not store data locally, only scrapes and ships. Memory savings of up to 40%.

---
# grafana-agent.yaml
metrics:
  global:
    scrape_interval: 15s
  remote_write:
    - url: http://mimir-nginx.monitoring.svc:80/api/v1/push
      headers:
        X-Scope-OrgID: team-platform

Minimum requirements for a production cluster (6 tenants, 10 million metrics)

Configuration parameter comparison for different scenarios

What's included in the work

• Audit of the current monitoring infrastructure
• Design of Mimir architecture for your load
• Cluster deployment in Kubernetes (Helm, zone-aware, HA)
• Configuration of multi-tenancy, limits, retention
• Integration with Grafana and dashboard setup
• Creation of runbook and documentation
• Team training (2–3 hours)
• Support for the first month after launch

Our experience and guarantees

We are a team of SRE engineers with experience in monitoring with Prometheus and Mimir, having completed over 50 projects deploying Mimir in production. We hold Grafana certifications and have experience with clusters up to 50 million metrics. We guarantee SLA on response time during incidents.

Timeline: from 3 to 7 days for full deployment including training.

To discuss your scenario, contact our engineers. Order Mimir implementation — get a specialist consultation.

For resource calculation based on your load, contact our engineers. Order Mimir implementation and receive full documentation.