Priority queue task prioritization setup

Configuring Task Prioritization in Queue (Priority Queue)

All tasks in one queue are processed in order of arrival. This is fine as long as tasks are homogeneous. As soon as tasks with different requirements appear — password reset emails shouldn't wait in the queue behind reports that take an hour to generate — you need prioritization.

Priority Model

Typical division into three levels:

Implementation in Laravel (Redis)

Laravel supports queue priorities through the order specified in --queue:

php artisan queue:work --queue=critical,default,low

The worker first checks critical, if empty — moves to default, then to low. This is soft priority: if there are tasks in critical, tasks in low aren't processed.

Dispatch to specific queue:

// Critical
SendPasswordResetEmail::dispatch($user)->onQueue('critical');

// Normal priority
SendWelcomeEmail::dispatch($user)->onQueue('default');

// Low priority
GenerateMonthlyReport::dispatch($reportId)->onQueue('low');

Or define the queue inside the Job itself:

class GenerateMonthlyReport implements ShouldQueue
{
    public string $queue = 'low';
    // ...
}

Horizon: Separate Worker Pools for Different Priorities

In Horizon, create separate supervisor pools:

// config/horizon.php
'environments' => [
    'production' => [
        // Pool for critical tasks — always minimum 2 workers
        'critical-supervisor' => [
            'connection'     => 'redis',
            'queue'          => ['critical'],
            'balance'        => 'simple',
            'minProcesses'   => 2,
            'maxProcesses'   => 8,
            'timeout'        => 30,
        ],
        // Pool for standard tasks — autoscaling
        'default-supervisor' => [
            'connection'     => 'redis',
            'queue'          => ['default'],
            'balance'        => 'auto',
            'minProcesses'   => 1,
            'maxProcesses'   => 5,
            'timeout'        => 60,
        ],
        // Pool for heavy tasks — limited
        'low-supervisor' => [
            'connection'     => 'redis',
            'queue'          => ['low'],
            'balance'        => 'simple',
            'processes'      => 2,
            'timeout'        => 3600,
        ],
    ],
],

With this configuration, critical tasks are processed on dedicated workers regardless of other queues' load.

Dynamic Priority Based on Data

Sometimes priority needs to be determined not statically but based on data — for example, paying customers should have higher processing priority:

class ProcessUserExportJob implements ShouldQueue
{
    public function __construct(private int $userId) {}

    public function queue(): string
    {
        $user = User::find($this->userId);
        return match(true) {
            $user?->isPremium() => 'default',
            $user?->isEnterprise() => 'critical',
            default => 'low',
        };
    }
}

However, queue() method as dynamic isn't directly supported in Laravel — priority must be determined at dispatch:

$queue = match(true) {
    $user->isEnterprise() => 'critical',
    $user->isPremium()    => 'default',
    default               => 'low',
};

ProcessUserExportJob::dispatch($user->id)->onQueue($queue);

Priority in BullMQ (Node.js)

BullMQ supports numeric task priority:

import { Queue } from 'bullmq';

const queue = new Queue('tasks', {
    connection: { host: 'localhost', port: 6379 }
});

// Lower number = higher priority (1 = highest)
await queue.add('send-password-reset', { userId: 123 }, { priority: 1 });
await queue.add('send-welcome-email',  { userId: 456 }, { priority: 5 });
await queue.add('generate-report',     { reportId: 789}, { priority: 10 });

BullMQ uses Redis Sorted Set to store tasks with priority — guaranteed sorting by priority within one queue.

The worker simply takes tasks in priority order:

import { Worker } from 'bullmq';

const worker = new Worker('tasks', async (job) => {
    console.log(`Processing ${job.name} with priority ${job.opts.priority}`);
    // ...
}, { connection: { host: 'localhost', port: 6379 } });

Protecting Against Starvation of Low-Priority Tasks

With a large stream of critical tasks, low tasks may never be processed (starvation). Two approaches:

Aging — task increases its priority with wait time. Implemented via scheduled job that periodically reviews the queue:

// Every 15 minutes increase priority of long-waiting tasks
Schedule::call(function () {
    $staleJobs = DB::table('jobs')
        ->where('queue', 'low')
        ->where('created_at', '<', now()->subMinutes(30))
        ->get();

    foreach ($staleJobs as $job) {
        DB::table('jobs')
            ->where('id', $job->id)
            ->update(['queue' => 'default']); // raise to default
    }
})->everyFifteenMinutes();

Dedicated worker for low — one dedicated worker always works only with low queue, not distracted by critical. Guarantees progress even under load:

[program:low-dedicated-worker]
command=php artisan queue:work --queue=low --sleep=5 --timeout=3600
numprocs=1
autostart=true
autorestart=true

Monitoring Queue Depths

It's important to track the length of each queue — if critical accumulates, it's a symptom of insufficient workers:

use Illuminate\Support\Facades\Redis;

$depths = [
    'critical' => Redis::llen('queues:critical'),
    'default'  => Redis::llen('queues:default'),
    'low'      => Redis::llen('queues:low'),
];

// Alert if critical > 50 tasks
if ($depths['critical'] > 50) {
    // notification
}

Horizon shows this in the dashboard without additional code.

Timeline

Setting up three queues, dividing existing tasks by priority, Horizon pool config — 3–5 hours. Anti-starvation logic, queue depth monitoring, alerting — another 2–4 hours.