Highload-Block Optimization: Approaches and Tools

Imagine a highload block with an order history of 5 million rows—a filtered query by date takes 40 seconds. After optimization—8 milliseconds. This is achievable with proper indexing, caching, and partitioning. Highload blocks (the highloadblock module) are a Bitrix mechanism for storing arbitrary data in separate tables. Common use cases: event logs, product catalogs with non-standard structure, user profiles, cumulative data (order history, analytics, queues). While rows are under 50–100 thousand, everything works fine. At 1–10 million rows, problems start: the Bitrix ORM generates suboptimal queries, indexes don't cover real selections, JOINs slow down. We have encountered projects where query time was 40 seconds—after optimization it dropped to 8 milliseconds. Savings per query—up to 99.9%.

Why do highload blocks slow down on large data?

Main anti-patterns when working with Highload:

• Missing required indexes. A highload block creates a table with primary key ID and auto-increment. Custom fields like UF_* are not automatically indexed. A getList(['filter' => ['UF_PRODUCT_ID' => 123]]) on a million rows is a table scan.
• SELECT * like queries. By default, the Bitrix ORM selects all fields. If a record has 30 UF fields, including TEXT and FILE, this is an expensive query even with a small result set.
• Unlimited queries without pagination. DataManager::getList() without limit returns all records into PHP memory.
• Linked tables via Reference. If a Highload is linked to another Highload or infoblock via Reference fields—the ORM builds a JOIN that kills performance without proper indexes.
• Frequent UPDATE on fields without an index. Typical for status fields, counters.

How to diagnose bottlenecks?

Enable MySQL slow query log:

[mysqld]
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 1
log_queries_not_using_indexes = 1

How to properly index highload tables?

Add indexes via direct SQL—in an agent during installation or a migration script:

$connection = \Bitrix\Main\Application::getConnection();
$tableName = 'b_hl_product_catalog'; // Example
$connection->queryExecute(
    "CREATE INDEX IF NOT EXISTS idx_product_id ON {$tableName} (UF_PRODUCT_ID)"
);
$connection->queryExecute(
    "CREATE INDEX IF NOT EXISTS idx_status_date ON {$tableName} (UF_STATUS, UF_DATE_CREATE)"
);

What if the ORM still slows down even with indexes?

Explicit SELECT of required fields. Never request select: ['*'] or an empty select array:

$result = ProductCatalogTable::getList([
    'select' => ['ID', 'UF_NAME', 'UF_PRICE', 'UF_ACTIVE'],
    'filter' => ['UF_CATEGORY_ID' => $categoryId, 'UF_ACTIVE' => 1],
    'order' => ['UF_SORT' => 'ASC'],
    'limit' => 50,
    'offset' => ($page - 1) * 50,
]);

Direct SQL for aggregation. For COUNT, SUM, GROUP BY on large tables—direct SQL is 5–7 times faster than ORM. Use Bitrix\Main\Application::getConnection()->query().

Partitioning for chronological data. If a Highload stores logs or events with dates—partitioning by date range drastically speeds up period queries:

ALTER TABLE b_hl_event_log PARTITION BY RANGE (YEAR(UF_DATE_CREATE) * 100 + MONTH(UF_DATE_CREATE)) (
  PARTITION p_jan VALUES LESS THAN (202402),
  PARTITION p_feb VALUES LESS THAN (202403),
  PARTITION p_future VALUES LESS THAN MAXVALUE
);

Caching results. Highload data caches well via Bitrix\Main\Data\Cache with tagging. Example implementation:

class CachedProductCatalog {
    private const CACHE_TAG = 'hl_product_catalog';
    private const CACHE_TTL = 3600;

    public function getByCategory(int $categoryId): array {
        $cache = \Bitrix\Main\Data\Cache::createInstance();
        $cacheKey = 'hl_catalog_cat_' . $categoryId;

        if ($cache->initCache(self::CACHE_TTL, $cacheKey, '/hl/catalog/')) {
            return $cache->getVars();
        }

        $cache->startDataCache();

        // Fetch from DB
        $result = $this->fetchFromDb($categoryId);

        // Tagged cache
        $tagCache = new \Bitrix\Main\Data\TaggedCache();
        $tagCache->startTagCache('/hl/catalog/');
        $tagCache->registerTag(self::CACHE_TAG . '_' . $categoryId);
        $tagCache->endTagCache();

        $cache->endDataCache($result);

        return $result;
    }
}

Benchmarks: what each optimization gives

Step-by-step optimization plan

1. Audit Highload blocks. Collect structure, data volumes, typical queries. Enable slow query log.
2. Analyze bottlenecks. Identify top-5 slowest queries by time.
3. Add indexes. Create single and composite indexes for actual filters.
4. Refactor code. Replace select: ['*'] with explicit list, implement limits and pagination.
5. Implement caching. Use tagged cache for frequently requested data.
6. Partition tables. For chronological tables—split by date.
7. Load testing. A/B comparison of performance before and after.

What's included in the work

• Audit of Highload blocks: structure, data volume, typical queries and slow query log.
• Bottleneck analysis: identification of top-5 slowest queries.
• Index addition: single and composite indexes for real filter patterns.
• Code refactoring: explicit select, limits, pagination, replacement of ORM with direct SQL where it gives substantial benefit.
• Implementation of tagged cache for heavy selections.
• Partitioning of chronological tables (if needed).
• Load testing with A/B comparison before/after.
• Documentation and training for your developer.

Timeline and pricing

Work timeline: audit + indexes + cache — 2–3 weeks. Full optimization with partitioning and refactoring — 4–8 weeks. Cost is calculated individually based on data volume and complexity. Our team with many years of Bitrix experience and over 50 completed optimization projects guarantees a transparent approach and measurable results. Contact us for a preliminary estimate—we will propose a work plan with checkpoints. Order a performance audit and get an engineer consultation.

Learn more about Highload blocks in the official documentation.