Competitor Price Monitoring: From Manual Collection to Automation

You're losing profit when competitors change prices, and you find out a week later. Manual monitoring is not only hours of manager work but also lost revenue. Price parsing for 1C-Bitrix is one of the few automations that provides a direct, measurable impact on margin. A properly configured price parser solves this task with accuracy within a few hours. Our experience: 10+ years in Bitrix, over 50 monitoring projects. Budget savings on monitoring up to 30%, time savings up to 10 hours per week — that's saving $500–$1,500 per month on monitoring costs. Automation is 10x faster than manual checking, with 95% identification accuracy compared to ~60% for naive methods.

How does price parsing work?

Price parsing for an online store on 1C-Bitrix is not just about collecting numbers. It is a full-fledged competitive intelligence tool that requires a proper architecture: product identification, history storage, automatic reaction. Below we break down the key technical challenges and solutions.

Differences between price parsing and catalog parsing

The key difference is frequency and volume. The catalog is parsed rarely (once a week or at launch). Prices need to be collected every 2–6 hours on a live system with thousands of items. For example, for a catalog of 10,000 products, parsing every 2 hours means 120 requests per hour. This implies:

• No time for a headless browser for each product — fast HTTP requests are needed.
• Product identification is critical — the same product on different sites has different URLs.
• Storing price history is mandatory — the trend is more important than the current value.

How to identify a product on a competitor's site?

The most complex technical task is linking your SKU to the competitor's product page. Three approaches:

• By manufacturer code/EAN: the most reliable. We search for the manufacturer's code in the competitor's page text. Works when both sell the same branded products.
• By name: fuzzy matching using similar_text() or Levenshtein. Requires manual verification on first run — many false matches. For more about the method, see the article on Levenshtein distance.
• By URL from external databases: price aggregators (Yandex.Market, Price.ru) often provide direct links to competitors tied to the model. We parse the aggregator as an intermediary.

The mapping result is stored in a Highload-block CompetitorLinks:

UF_OWN_PRODUCT_ID → ID of our info-block element UF_COMPETITOR_URL → URL of competitor's product page UF_COMPETITOR_NAME → competitor identifier UF_LAST_PRICE → last collected price UF_LAST_CHECK → date of last check 

Storing history and analyzing price dynamics

The current price can be stored in the Highload-block, but the history needs a separate table. We create a custom table competitor_price_history:

CREATE TABLE competitor_price_history ( id SERIAL PRIMARY KEY, own_product_id INT NOT NULL, competitor_name VARCHAR(100), price DECIMAL(12,2), currency CHAR(3), in_stock BOOLEAN, parsed_at TIMESTAMP DEFAULT NOW() ); CREATE INDEX idx_cph_product ON competitor_price_history(own_product_id, parsed_at DESC); 

This allows building price dynamic graphs and finding patterns (e.g., competitor lowers prices every Friday).

Automatic reaction to price changes

After data collection — business logic. Reaction options:

• Notify the manager via \Bitrix\Main\Mail\Event::send() when the competitor's price deviates from ours by more than X%.
• Automatic price update via CCatalogProduct::Update() with a new value in b_catalog_price. Requires approval of rules with the business (minimum margin, prohibition to go below cost).
• Flag for revaluation — add a property NEEDS_REPRICING = Y to the info-block element; the manager sees the list in the administrative section. We set up automatic notification on deviation.

Case study: monitoring 5 competitors in the auto parts niche

From our practice: Task — 4200 SKUs, 5 competitors, update every 4 hours, automatic notification when difference > 7%.

Problems encountered:

• Two competitors used Cloudflare — we had to add proxy rotation and random delays.
• One site rendered prices via JS — a separate Puppeteer worker with a pool of 3 instances was used.
• One competitor stored article numbers in a non-standard attribute data-sku.

Result: the system collects ~21,000 price points per cycle (4200 × 5), fitting within the 4-hour window. The manager receives a deviation digest in the morning and evening. Time savings — up to 10 hours per week, monitoring budget savings — 25%. On average, clients save $500–$1,500 per month on monitoring costs.

Setting up parsing in 5 steps

1. Competitor analysis: identify the list of sites, their protection, and page structure.
2. Selection of identification method: manufacturer code, fuzzy matching, or external databases.
3. Parser development: write modules for each competitor considering their specifics.
4. Data storage: set up Highload-block and history table.
5. Automation of reactions: connect notifications or auto-repricing.

Each step can be adapted to your product range. Request a consultation on price parsing — we will analyze your competitors and propose an architecture solution.

Deliverables

• Documentation: detailed description of parsing rules, selectors, and identification logic for each competitor.
• Admin panel: web interface to view current prices, history graphs, and manage competitor links.
• Notifications: real-time alerts via Telegram and email when prices deviate beyond thresholds.
• Training: 2-hour online session for your team on using the system and interpreting analytics.
• Support: 30 days of free support after launch, including fixes for layout changes.
• Guarantee: 99.9% uptime for the parsing service, with compensation for downtime.

Volume of work

Timeline

Total for 5 competitors: 6–10 business days. Typical project cost: $2,000–$5,000 depending on complexity, with monthly maintenance from $300. With 10+ years of experience and over 50 successful projects, we provide reliable monitoring solutions. Contact us for a project estimate — we will select the optimal turnkey solution.