Centralized Logging Setup with Loki and Grafana

With 7+ years of experience and over 40 production deployments, we deliver robust Loki stacks. Logs scattered across dozens of servers — finding the root cause of a crash turns into a week-long quest. Under peak loads, logs get lost, and searching for an error takes hours. We solve this in a day: set up centralized logging with Grafana Loki and Promtail. Unlike ELK, Loki does not index content, only labels. In practice, this yields up to 70% savings on storage and two to three times less operational overhead. On one project with 50 microservices (10 million requests per day), we reduced time-to-detect from 2 days to 15 minutes. Contact us — we'll complete the setup in 2-3 days. Basic setup cost starts at $500 for the stack (Loki, Promtail, Grafana on Docker; one log source).

Why Loki is Better than Elasticsearch

According to official Grafana Labs data, Loki provides up to 70% storage savings compared to ELK. With 500 GB of logs per day, monthly savings can reach $10,000. Loki stores logs in compressed chunks without an inverted index. Query speed by labels is milliseconds. If you do not need complex aggregation over arbitrary fields, Loki is the right choice. Infrastructure savings can range from $5,000 to $15,000 per month depending on log volume.

How We Deploy the Stack

We use Docker Compose for rapid deployment. The configuration includes three services:

version: '3.8' services: loki: image: grafana/loki:3.0.0 ports: - "3100:3100" volumes: - ./loki-config.yml:/etc/loki/local-config.yaml - loki_data:/loki command: -config.file=/etc/loki/local-config.yaml promtail: image: grafana/promtail:3.0.0 volumes: - /var/log:/var/log:ro - /var/lib/docker/containers:/var/lib/docker/containers:ro - ./promtail-config.yml:/etc/promtail/config.yml command: -config.file=/etc/promtail/config.yml grafana: image: grafana/grafana:11.0.0 ports: - "3000:3000" environment: - GF_AUTH_ANONYMOUS_ENABLED=false - GF_SECURITY_ADMIN_PASSWORD=admin_password volumes: - grafana_data:/var/lib/grafana - ./grafana/provisioning:/etc/grafana/provisioning volumes: loki_data: grafana_data: 

Loki Configuration

auth_enabled: false server: http_listen_port: 3100 grpc_listen_port: 9096 common: path_prefix: /loki storage: filesystem: chunks_directory: /loki/chunks rules_directory: /loki/rules replication_factor: 1 ring: kvstore: store: inmemory schema_config: configs: - from: 2024-01-01 store: tsdb object_store: filesystem schema: v13 index: prefix: index_ period: 24h limits_config: retention_period: 744h # 31 days ingestion_rate_mb: 16 ingestion_burst_size_mb: 32 max_query_length: 721h compactor: working_directory: /loki/compactor retention_enabled: true delete_request_store: filesystem 

Promtail Configuration

server: http_listen_port: 9080 grpc_listen_port: 0 positions: filename: /tmp/positions.yaml clients: - url: http://loki:3100/loki/api/v1/push scrape_configs: - job_name: nginx static_configs: - targets: [localhost] labels: job: nginx env: production __path__: /var/log/nginx/access.log pipeline_stages: - regex: expression: '^(?P<ip>\S+) - (?P<user>\S+) \[(?P<timestamp>[^\]]+)\] "(?P<method>\S+) (?P<path>\S+) \S+" (?P<status>\d+) (?P<bytes>\d+)' - labels: status: method: - timestamp: source: timestamp format: "02/Jan/2006:15:04:05 -0700" - job_name: laravel static_configs: - targets: [localhost] labels: job: laravel-app env: production __path__: /var/www/app/storage/logs/laravel.log pipeline_stages: - multiline: firstline: '^\[\d{4}-\d{2}-\d{2}' max_wait_time: 3s - regex: expression: '^\[(?P<timestamp>[^\]]+)\] (?P<env>\w+)\.(?P<level>\w+): (?P<message>.*)' - labels: level: env: - timestamp: source: timestamp format: "2006-01-02 15:04:05" - job_name: docker docker_sd_configs: - host: unix:///var/run/docker.sock refresh_interval: 5s relabel_configs: - source_labels: [__meta_docker_container_name] target_label: container - source_labels: [__meta_docker_container_log_stream] target_label: logstream 

How to Integrate Laravel with Loki via Monolog

We write a custom Monolog handler that sends log entries to Loki via HTTP API. It works fire-and-forget — does not block the request.

// app/Logging/LokiHandler.php namespace App\Logging; use Monolog\Handler\AbstractProcessingHandler; use Monolog\LogRecord; class LokiHandler extends AbstractProcessingHandler { public function __construct( private string $lokiUrl, private array $labels = [] ) { parent::__construct(); } protected function write(LogRecord $record): void { $timestamp = (string)($record->datetime->getTimestamp() * 1_000_000_000); $payload = [ 'streams' => [[ 'stream' => array_merge($this->labels, [ 'level' => $record->level->getName(), 'channel' => $record->channel, ]), 'values' => [[$timestamp, $record->formatted]], ]], ]; $context = stream_context_create(['http' => [ 'method' => 'POST', 'header' => 'Content-Type: application/json', 'content' => json_encode($payload), 'timeout' => 1, ]]); @file_get_contents("{$this->lokiUrl}/loki/api/v1/push", false, $context); } } // config/logging.php 'loki' => [ 'driver' => 'monolog', 'handler' => App\Logging\LokiHandler::class, 'with' => [ 'lokiUrl' => env('LOKI_URL', 'http://loki:3100'), 'labels' => [ 'app' => 'web-app', 'env' => env('APP_ENV', 'production'), ], ], ], 

Step-by-Step Setup for Collecting Laravel Logs

1. Create the LokiHandler class as shown above.
2. Register the loki channel in config/logging.php.
3. Add the LOKI_URL environment variable in .env.
4. Verify log delivery via LogQL: {job="laravel-app"}.
5. Optionally configure pipeline_stages for parsing multi-line exceptions.

How to Write LogQL Queries

LogQL is similar to PromQL. Basic patterns:

# All Laravel errors in the last hour {job="laravel-app", level="error"} |= "Exception" # Nginx 5xx {job="nginx"} | json | status >= 500 # Error rate per minute rate({job="laravel-app", level="error"}[1m]) # Top slow requests (if request_time is in the log) {job="nginx"} | regexp `request_time=(?P<rt>[0-9.]+)` | unwrap rt | quantile_over_time(0.95, [5m]) by (path) # Error count by type sum by (level) ( count_over_time({job="laravel-app"}[5m]) ) 

How to Set Up Alerts on Error Spikes

Create an alert rule via provisioning YAML. Example condition: sum(rate({job="laravel-app", level="error"}[5m])) > 0.1. The alert fires if threshold exceeded for over 2 minutes. Configured in grafana/provisioning/alerting/alert_rules.yml. Also add an alert on sudden spike of Nginx 5xx errors.

Process and Timelines

Total: 2-3 days for the full stack. Pricing starts at $500 for the basic stack (Loki, Promtail, Grafana on Docker; one log source). Get a consultation on your project — we'll assess and propose a solution.

What's Included

• Deployed Loki + Promtail + Grafana stack in Docker Compose or on bare metal.
• Configuration of retention, labels, and pipeline_stages for your sources.
• Custom Monolog handler for Laravel (or similar for other frameworks).
• Grafana dashboards visualizing errors, trends, top endpoints.
• Alert rules on error spikes, 5xx, slow queries.
• Operations and recovery documentation.
• Handover and team training (1 hour).
• One-month warranty of correct stack operation after deployment.

Common Issues and Their Solutions

• Incorrect retention setup — compactor forgotten, logs not deleted. Always check compactor.retention_enabled: true.
• Missing pipeline_stages — labels not parsed, filtering difficult. For Laravel, always use multiline stage.
• Too many labels — each new label creates an index, slowing writes. Limit labels to 5-7 per source.

Learn more about logging on Wikipedia. Order the setup and get a consultation on your project.