Filling a 1C-Bitrix catalog with products from Yandex.Market is a typical task for online stores. The problem is that Market does not have a public API for bulk card export. The partner Content API only gives sellers access to their own data. For a third-party integrator, the only path is parsing, but with serious technical limitations: JavaScript rendering, captcha, IP blocking, and fingerprinting. Plus legal risks, as it violates the terms of service. Our certified team (over 7 years in Bitrix development and integrations) has implemented more than 30 such projects—from simple catalogs to large marketplaces. Our guaranteed solution includes proxy rotation and captcha handling. This article describes a proven approach that allows stable data collection and import into information blocks with minimal risks.
Parsed Data
A product card on Yandex.Market contains name, description, characteristics (key-value pairs), dynamic seller prices, images (1–15 photos), reviews, rating, and category. For a 1C-Bitrix catalog, you most often need name, description, characteristics, and images—prices are not worth parsing due to high volatility.
Technical Implementation
Yandex.Market is a single-page application. Data is loaded through internal APIs and rendered on the client. A simple HTTP request returns an empty shell. We use two approaches:
- Headless browser (Puppeteer, Playwright)—slow (3–5 seconds per page), but reliable.
- Intercepting internal APIs—10–20 times better (faster) than headless browser for mass extraction. The combined approach is better than using only one method.
Combination: headless for initial analysis and token retrieval, then direct requests for mass extraction.
Bypassing Yandex.Market Protection
Yandex blocks automated requests using SmartCaptcha, fingerprinting, and rate limiting. For stable parsing, you must use residential proxy rotation, random delays (2–10 sec), User-Agent rotation, and captcha handling. Without proxy rotation, a single IP gets blocked within an hour.
Mapping Data to Information Block
Market's structure does not match the information block. A transformation layer is needed:
Mapping Table
| Yandex.Market | Bitrix Information Block | Notes |
|---|---|---|
title |
NAME |
Truncation to 255 characters |
description |
DETAIL_TEXT |
HTML → tag stripping or preservation |
specs[] |
PROPERTY_* |
Map by characteristic name |
images[] |
DETAIL_PICTURE + MORE_PHOTO |
Download and save locally |
categoryPath |
IBLOCK_SECTION_ID |
Mapping via a correspondence table |
modelId |
XML_ID |
Unique identifier for deduplication |
Market characteristics are a flat list, while information block properties are typed. A mapping table is needed: "Weight, g" → PROPERTY_WEIGHT (number), "Color" → PROPERTY_COLOR (list).
Why an Intermediate Layer Is Important
The loading process should go through an intermediate storage (a separate table or JSON files). The import script uses the information block API:
CIBlockElement::Add($arFields); CIBlockElement::SetPropertyValuesEx($elementId, $iblockId, $propertyValues); Direct import from the parser is risky: if the parser breaks mid-way, partially filled cards remain in the catalog. For catalogs over 5,000 products, use \Bitrix\Iblock\ElementTable::add() — D7 API is faster and supports batch operations.
Step-by-Step Implementation Plan
- Analyze target pages: determine data structure and tokens.
- Develop the parser: choose approach (headless + intercept), configure proxies and captcha.
- Create intermediate storage and mapping script.
- Import into 1C-Bitrix information blocks with deduplication.
- Set up incremental updates on a schedule.
Maintaining Freshness
For catalogs up to 1,000 products, a full reimport once a week works (2–4 hours). For 1,000–10,000, incremental crawling daily (4–8 hours). For over 10,000, combine incremental and trigger-based updates.
| Catalog Size | Strategy | Frequency | Time |
|---|---|---|---|
| up to 1,000 | Full reimport | Weekly | 2–4 h |
| 1,000–10,000 | Incremental | Daily | 4–8 h |
| over 10,000 | Incremental + trigger | Scheduled | 8–24 h |
Deliverables and Scope of Work
When ordering Yandex.Market parsing for 1C-Bitrix, we provide:
- Parser development with proxy rotation, captcha handling, and protection against blocks.
- Creation of a characteristic mapping table.
- Scripts for initial import and incremental updates.
- Documentation, accesses, staff training.
- Support during the launch phase (7-day post-launch support).
- Cost estimate and timeline.
Automated parsing saves an average of $1,000–$3,000 per month compared to manual entry. Parsing violates Yandex's terms of service. In practice, claims are rare, but using descriptions and photos "as is" is risky. We recommend rewriting descriptions and checking image licenses.
How to Order
If you need a stable solution for filling a Bitrix catalog with data from Yandex.Market—contact us. We will evaluate the project, choose a strategy, and propose timelines (from 2 to 6 weeks depending on volume). The cost is calculated individually—prices start from $500 for small catalogs, ranging up to $5,000. We have completed over 30 such integrations. Get a consultation—discuss the details.