1C-Bitrix Catalog Auto-Fill System Development

When the product catalog grows, manual updates stop working

Automate your 1C-Bitrix catalog auto-fill system with seamless 1C catalog integration, Bitrix price parser, stock parser, and Bitrix queues. Our system offers comprehensive catalog automation, catalog monitoring, and robust Bitrix development. Based on over 20 projects, we guarantee stable operation with information blocks and commercial catalog via CommerceML. Typical project cost: $12,000–$20,000, with annual savings on manual operations exceeding $80,000. Contact us to discuss your project and get a consultation and project assessment.

System architecture

A proper auto-fill system consists of several layers:

Data collection layer (Collectors) Each source is a separate collector implementing a CollectorInterface::collect(). Collectors know nothing about Bitrix — they only collect and return normalized data structures. This allows independent testing and replacement of sources. Collectors parse prices, stocks, and descriptions from any format.

Transform layer (Transformers) Data normalization: units of measure, date formats, encodings, field mapping. A transformer takes a collector's raw object and returns an array ready for writing to an information block.

Write layer (Writers) Encapsulates work with Bitrix API: CIBlockElement::Add/Update, CCatalogProduct::Update, CFile::SaveFile. A writer knows about the information block structure but not where the data came from.

Orchestration layer (Orchestrator) Manages execution sequences, priorities, error handling, and rollback on failure.

Why is a priority table important?

The key task is to determine which field from which source has priority. A clear priority table is needed:

Implementation: each field has source metadata (SOURCE_* properties). When updating with low-priority data, we check if the field is already filled by a higher-priority source.

How does the system handle load?

For a catalog with 50,000+ items, synchronous processing is impossible. We use queues: Option 1: database queue table

CREATE TABLE parser_queue (
    id SERIAL PRIMARY KEY,
    task_type VARCHAR(50), -- 'price', 'stock', 'description', 'image'
    element_id INT,
    payload JSONB,
    status VARCHAR(20) DEFAULT 'pending', -- pending/processing/done/error
    attempts INT DEFAULT 0,
    created_at TIMESTAMP DEFAULT NOW(),
    processed_at TIMESTAMP
);

Workers pick tasks in batches of 100, process, and update status.

Option 2: Redis Queues — for high-frequency updates (prices, stocks). Faster than a database, but without AOF there is no persistence.

What happens on failure?

A system without monitoring breaks silently. Mandatory metrics:

• Number of processed records in the last cycle
• Number and types of errors
• Last successful run time for each collector
• Queue size (if growing, workers are not keeping up)

Self-healing: tasks with status error and attempts < 3 are automatically returned to pending after an hour — retry logic without manual intervention.

Case from our practice: a wholesale store

Task: 35,000 SKUs, 4 suppliers with different data formats (API, FTP feed, website scraping, email with Excel), price and stock updates every 2 hours.

Architecture:

• 4 collectors (REST API, XML feed via FTP, HTTP parser, Excel reader)
• Single transformer with a field mapping table (configurable in the admin panel)
• Writer via D7 ORM (\Bitrix\Catalog\ProductTable) — documentation
• PostgreSQL queue table with 3 parallel workers
• Prometheus metrics + Grafana dashboard for monitoring

Result: The system has been running for 14 months, average full cycle processing time — 1 hour 40 minutes. Manual interventions in that period — 3 times (twice due to supplier API changes, once due to a change in the scraped site structure). Savings on manual operations exceeded $80,000 per year.

What is included in the work

1. Architecture design for auto-fill considering your sources and catalog
2. Development and configuration of collectors, transformers, and writers
3. Creation of queue system and workers (Redis or database)
4. Monitoring integration (Grafana + alerts)
5. Admin interface for queue viewing and logging
6. Architecture documentation and operator instructions
7. Training for two specialists on the system
8. One month of post-deployment support
9. Project cost: $12,000–$20,000

Timeline

Total: 3–5 weeks — this is a project, not a task. Reducing manual operations by 90% saves substantially in operator salaries.

Our engineers are 1C-Bitrix certified, guaranteeing correct integration with the platform core. "Queues are the only way for catalogs over 50,000 items" — Bitrix expert on community.1c-bitrix.ru forum.

Contact us for a project assessment. Get a consultation on catalog auto-fill architecture. Request a consultation today.