Product Export from Website to YML Feed

Implementing product export to YML feed

YML (Yandex Market Language) is an XML format that Yandex.Market requires for catalog upload. Properly formed feed ensures error-free indexing, correct offer display and admission to paid placements. Incorrect feed means rejected products, shop penalty points and loss of search visibility.

YML structure and Yandex.Market requirements

Format described in official documentation. Minimal structure:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE yml_catalog SYSTEM "shops.dtd">
<yml_catalog date="2024-01-15 10:00">
  <shop>
    <name>My store</name>
    <company>LLC Company</company>
    <url>https://example.com</url>
    <currencies>
      <currency id="RUR" rate="1"/>
    </currencies>
    <categories>
      <category id="10">Electronics</category>
      <category id="11" parentId="10">Smartphones</category>
    </categories>
    <offers>
      <offer id="12345" available="true">
        <url>https://example.com/product/12345</url>
        <price>29990</price>
        <currencyId>RUR</currencyId>
        <categoryId>11</categoryId>
        <picture>https://example.com/img/12345.jpg</picture>
        <name>Samsung Galaxy S24 Smartphone</name>
        <vendor>Samsung</vendor>
        <description>...</description>
        <param name="Color">Black</param>
        <param name="Memory">256 GB</param>
      </offer>
    </offers>
  </shop>
</yml_catalog>

Critical fields for admission: url, price, currencyId, categoryId, name. Field available affects display — products with false excluded from search but preserved in price history.

Feed generator architecture

For small catalogs (up to 10,000 products) feed can be generated on-the-fly. For larger — only via background task with disk or CDN storage.

Workflow:

Cron/Queue → Builder → XML Writer → Storage (disk/S3) → Public URL

PHP implementation (Laravel):

class YmlFeedBuilder
{
    public function build(string $outputPath): void
    {
        $writer = new XMLWriter();
        $writer->openUri($outputPath);
        $writer->startDocument('1.0', 'UTF-8');
        $writer->writeDtd('yml_catalog', null, 'shops.dtd');

        $writer->startElement('yml_catalog');
        $writer->writeAttribute('date', now()->format('Y-m-d H:i'));

        $this->writeShopInfo($writer);
        $this->writeCurrencies($writer);
        $this->writeCategories($writer);
        $this->writeOffers($writer);

        $writer->endElement();
        $writer->endDocument();
        $writer->flush();
    }

    private function writeOffers(XMLWriter $writer): void
    {
        $writer->startElement('offers');

        Product::with(['category', 'attributes', 'images'])
            ->where('active', true)
            ->where('price', '>', 0)
            ->chunk(500, function ($products) use ($writer) {
                foreach ($products as $product) {
                    $this->writeOffer($writer, $product);
                }
            });

        $writer->endElement();
    }
}

Using chunk() is mandatory — attempting to load 50,000+ products in one query guarantees OOM.

Offer types

Yandex.Market distinguishes several types, each with required fields:

For most online stores vendor.model or default type works.

Feed validation

Before sending to Yandex.Market, feed must be validated locally. Use Feed Validator or shops.dtd schema.

Custom validation during build:

class YmlFeedValidator
{
    public function validate(string $filePath): array
    {
        $errors = [];
        $dom = new DOMDocument();

        if (!$dom->load($filePath)) {
            return ['XML parse error'];
        }

        $offers = $dom->getElementsByTagName('offer');
        foreach ($offers as $offer) {
            $id = $offer->getAttribute('id');

            if (!$offer->getElementsByTagName('price')->length) {
                $errors[] = "Offer {$id}: missing price";
            }
            if (!$offer->getElementsByTagName('url')->length) {
                $errors[] = "Offer {$id}: missing url";
            }
            // Check description length (Yandex truncates after 3000 chars)
            $desc = $offer->getElementsByTagName('description')->item(0);
            if ($desc && mb_strlen($desc->textContent) > 3000) {
                $errors[] = "Offer {$id}: description too long";
            }
        }

        return $errors;
    }
}

Update schedule

Frequency depends on catalog dynamics:

• Prices and stock — change frequently: regenerate every 1–4 hours recommended
• Attributes and descriptions — change slowly: sufficient once daily
• New products — immediately or at next hourly cycle

Example schedule in Laravel (app/Console/Kernel.php):

$schedule->job(new GenerateYmlFeedJob)->everyFourHours();

Feed size optimization

Yandex.Market recommends not exceeding 500 MB per feed. If larger — split by category and register multiple feeds in cabinet.

Reduce size without quality loss:

• Exclude products without price (WHERE price > 0)
• Exclude inactive categories
• Truncate description to 1000–1500 chars
• Compress feed with gzip (Content-Encoding: gzip) — Yandex supports

// Compress ready feed
$source = storage_path('app/public/feed.xml');
$compressed = storage_path('app/public/feed.xml.gz');

$fp = gzopen($compressed, 'w9');
gzwrite($fp, file_get_contents($source));
gzclose($fp);

Timeline

• Basic generator with default type: 1–2 days
• Add typed offers (vendor.model): +0.5 day
• Validator + error logging: +0.5 day
• Cron setup + public URL: +0.5 day

Total typical project: 2–3 working days.