Scheduled Competitor Price Scraper Bot

Developing a Competitor Price Monitoring Bot (Scheduled)

Competitor price monitoring is the basis for dynamic pricing. The bot collects prices on schedule, stores change history, and can automatically adjust your prices by given rules.

System Architecture

Scheduler → ScrapeCompetitorPrices Job
         → CompetitorScraper (HTTP/Playwright)
         → PriceNormalizer
         → PriceHistoryRepository (INSERT)
         → PriceChangeDetector
             → AlertDispatcher (if change > threshold)
             → RepricingEngine (if autoprice enabled)

Basic Price Scraper

class CompetitorPriceScraper
{
    public function scrapePrice(string $url): ?PriceData
    {
        try {
            $html = $this->fetch($url);
            $crawler = new Crawler($html);

            $priceText = $crawler->filter($this->config['selectors']['price'])
                ->first()->text('');

            $price = $this->extractPrice($priceText);
            if ($price === null) return null;

            $inStock = $crawler->filter($this->config['selectors']['in_stock'])
                ->count() > 0;

            return new PriceData(price: $price, inStock: $inStock);
        } catch (\Exception $e) {
            Log::warning("Price scrape failed: {$e->getMessage()}");
            return null;
        }
    }

    private function extractPrice(string $text): ?float
    {
        $cleaned = preg_replace('/[^\d,.]/', '', str_replace(' ', '', $text));
        $cleaned = str_replace(',', '.', $cleaned);
        return is_numeric($cleaned) ? (float) $cleaned : null;
    }
}

Job with Change Detection

class MonitorCompetitorPrice implements ShouldQueue
{
    public int $tries = 3;

    public function handle(
        CompetitorPriceScraper $scraper,
        PriceChangeDetector $detector,
        RepricingEngine $repricer
    ): void {
        $priceData = $scraper->scrapePrice($this->competitorUrl);
        if (!$priceData) return;

        // Save current price
        $record = CompetitorPrice::updateOrCreate(
            ['product_id' => $this->productId, 'competitor_id' => $this->competitorId],
            ['price' => $priceData->price, 'in_stock' => $priceData->inStock]
        );

        // Change detection
        if ($record->wasChanged('price')) {
            $change = $detector->analyze($record);
            if ($change->isSignificant()) {
                Notification::route('mail', config('monitoring.alert_email'))
                    ->notify(new CompetitorPriceChangedNotification($record, $change));
            }

            if (config('repricing.enabled')) {
                $repricer->recalculate($this->productId);
            }
        }
    }
}

Auto-Repricing Engine

class RepricingEngine
{
    public function recalculate(int $productId): void
    {
        $product = Product::with('competitorPrices', 'repricingRule')->findOrFail($productId);
        $rule = $product->repricingRule;

        if (!$rule || !$rule->is_active) return;

        $competitorPrices = $product->competitorPrices
            ->where('in_stock', true)->pluck('price');

        if ($competitorPrices->isEmpty()) return;

        $newPrice = match ($rule->strategy) {
            'beat_lowest'  => $competitorPrices->min() - $rule->delta,
            'match_lowest' => $competitorPrices->min(),
            'beat_average' => $competitorPrices->avg() - $rule->delta,
            default        => null,
        };

        if (!$newPrice) return;

        $newPrice = max($newPrice, $rule->min_price ?? 0);
        $newPrice = min($newPrice, $rule->max_price ?? PHP_FLOAT_MAX);

        $currentPrice = $product->price;
        if (abs($newPrice - $currentPrice) / $currentPrice < 0.005) return;

        $product->update(['price' => round($newPrice, 2)]);
    }
}

Schedule Configuration

protected function schedule(Schedule $schedule): void
{
    $schedule->command('monitor:prices --priority=high')
        ->everyTwoHours()->withoutOverlapping();

    $schedule->command('monitor:prices --all')
        ->dailyAt('02:00')->withoutOverlapping();
}

Development timeline: monitoring 1 competitor + history + alerts — 4-6 business days. System with auto-repricing and dashboard for 5+ competitors — 10-14 days.