Why demand forecasting in retail breaks on promo
The classic retail demand forecasting model is built on sales history: a moving average or exponential smoothing for each item and location. As long as a week resembles the previous one, forecast accuracy remains at an acceptable level.
Once a promo appears in the calendar, the model keeps pulling the trend from the “calm” weeks and misses the peak by a factor of several times.
One-off promotions are the first failure. A discount, pallet display, and newsletter trigger simultaneously, demand grows severalfold, and the procurement system reads this as a new sustainable level.
Then come procurement errors: the next order goes against an inflated forecast, goods settle in the warehouse, expiry dates pass, and the shelf is occupied by what does not sell.
Seasonal peaks and local spikes are structured differently.
Here what matters is not the size of the rise but the date it begins and its depth for each location: in a store by a school, demand for juices rises two weeks before September, while in a neighboring district it does not change at all.
An averaged forecast for the region fails to catch this gap and creates a shortage exactly where demand was real.
New locations are a separate class of tasks: there is no history, nothing to rely on, and for the first weeks purchases go by the network average.
The outcome is the same on both sides: somewhere an empty shelf and a customer lost to a competitor, somewhere excess stock and write-offs. We break down how this is closed — with what data and what checks of forecast accuracy on promo.
Fewer write-offs, more accurate purchasing: measurable effects of the system
Forecasting is built not on the network average but at the “store × item × day” level with a horizon of up to two weeks: the model sees promotions, holidays, weather, and local spikes. The planner receives not abstract demand but a specific quantity to ship for each store.
First measurable effect — write-off share for perishables. While an order is formed by the “not less than shelf” norm, excess settles in the warehouse and goes to disposal.
When the order is calculated by a forecast adjusted for shelf life, write-offs for the category fall from 4–6% to 1.5–2% of turnover, and lost sales decrease by 20–35% — the model raises the order in advance for a local spike.
Second — inventory optimization and warehouse utilization. Excess leaves pallet locations and shelves, category turnover accelerates, and the share of goods sitting longer than the term decreases severalfold. Purchasing stops being insurance against uncertainty and becomes a calculation.
Third — time for manual planning. Daily recalculation and auto-ordering run on schedule in Apache Airflow, and the planner comes to a ready draft and spends time on exceptions rather than reconciling tables: from two days to 40–60 minutes per week per category.
At the same time, the forecast is measured rather than taken on faith — accuracy on the holdout period and bias by store are visible in the report.
Forecasting approaches: statistics, machine learning, hybrid
The choice of approach is determined by the structure of the data, not by fashion: series length, history depth, number of SKUs, and the degree to which demand is distorted by promotions, shelf shortages, and assortment changes. The same method produces different results on stable demand and on an item with a short life cycle.
Below are three options among which companies actually choose, with limitations that appear already in operation. We start comparing forecasting approaches with measurements on your history, not with choosing a favorite method.
| Option | When it fits | Limitations |
|---|---|---|
| Statistical forecasting models | Stable demand, long history, thousands of series, calculation transparency required | Weakly catch promotions and sharp demand shifts, require manual seasonality setup for each group |
| Machine learning for forecasting | Many related factors: price, promotions, stock, weather, holidays; short series | Requires a volume of history and regular retraining; higher risk of overfitting, harder to explain deviation for an item |
| Hybrid approach | Seasonal baseline and promo spikes at the same time; broad and rapidly changing assortment | More expensive to maintain: requires data quality control and monitoring of model aging |
The hybrid covers both extremes: the statistical part is responsible for trend and seasonality, while the model is responsible for adjustments for promo, price, and external events.
We compare approaches on a holdout period: we look at what share of items falls within the acceptable deviation and how many manual edits remain for the category manager. If a simple solution gives the same accuracy, we take it, not a neural network.
Implementation step by step: data audit, setup, validation, handover
Work proceeds in iterations: first we examine what is actually in the data, and only then build models. Sales history almost always has to be cleaned — it contains days without goods on the shelf, revaluations, SKU merges, and promotions recorded retroactively. Until this is disentangled, demand forecasting in retail learns from a distorted picture.
- Data and source audit — extracts from POS systems, the accounting system, and the loyalty program; SKU reconciliation, gap detection, and assessment of history depth by category.
- Feature mart — calendar, promotions, prices, stock, weather; labeling at the “store — item — day” level, accounting for days out of stock on the shelf.
- Baseline as a benchmark: if the model does not outperform a simple seasonal calculation on your data, there is nothing to implement.
- Model setup for the hierarchy — separate loops for perishables and long-shelf-life goods, day and week horizons, bottom-up forecast reconciliation.
- Retrospective validation — rolling window and holdout period, error at the “store — item — day” level, separate analysis of promotions and holidays.
- Model acceptance — agreed accuracy thresholds by category, analysis of the worst items, and a procedure for rollback to the baseline.
- Handover and team training — retraining procedure, data quality control, deviation monitoring, embedding the forecast into purchasing.
Each stage is closed with an artifact: source descriptions, feature mart code, accuracy report, procedure. Acceptance is done on your data — you see where the model errs before the forecast starts affecting purchasing.
What remains in the company: models, data dictionary, procedures
After implementation, the client’s team retains not a black box but a reproducible loop: for each artifact it is clear what it is, where it came from, and how to update it without us.
- Trained demand forecasting model — with fixed accuracy metrics on the holdout period: you compare the forecast with actuals for each product group and location.
- Data dictionary — decoding of all fields: POS sales, stock, supplies, promo calendar, holidays. With source, update frequency, and typical extract errors.
- Model documentation — which features affect what, for which categories the model is applicable, where it produces bias, and under what assumptions seasonality was calculated.
- Retraining procedure — frequency and triggers: opening new locations, assortment changes, demand shifts after promo. Plus the owner and the validation order after recalculation.
- Forecast recalculation configuration — update schedule, rules for filling gaps, and the threshold after which an item goes to manual review.
- Responsibility matrix — who owns the data source, who owns the model, who investigates an accuracy drop during the week.
- Acceptance checklist and tests — run on historical data, restoration from backup, rollback to the previous model version.
Handover takes place in two sessions: your analysts, together with our team, go through updating the data dictionary and retraining on a real sales snapshot. After that, the next assortment change does not require external support.
Case: how a store chain stopped losing on promo and stock
Before the project, a chain of convenience stores with two hundred locations planned supplies in spreadsheets: a category manager once a week manually reconciled sales, the promo calendar, and stock.
Weekly demand forecast accuracy stayed around 60%, so purchasing insured with stock — just so the shelf would not be empty.
The cost of this insurance is visible in two numbers. Write-offs of perishables reached a ninth of category turnover, and during promo weeks 12% of items were still missing: demand jumped, while the order remained at the previous season’s level.
What was changed in the forecasting project
We assembled a sales history mart by store — item — day and connected it to the promo calendar, weather, and local events such as fairs and road repairs.
The forecast is calculated not by category average but for each store and item pair for a horizon from a week to a month; recalculation runs every night, so a demand spike enters the order as early as the next day.
The order is generated automatically, and the manager confirms only exceptions — new items, one-off promotions, obvious anomalies; a category can be returned to manual mode with one setting if its data is incomplete.
Metrics before and after
Weekly forecast accuracy grew from 60 to 88%, perishable write-offs fell threefold, and the share of items missing from the shelf decreased from 12 to 3%.
Working stock decreased by approximately one fifth at the same service level, and preparing a promo plan takes an hour instead of two days.
The effect holds not from a one-time setup but from regular retraining: the model is reviewed against fresh sales, and discrepancies between forecast and actuals are analyzed by category once a month.
What to do if data is fragmented: POS, warehouse, online channel
Fragmented sources are the norm, not the exception: POS systems provide sales by receipt, the warehouse provides movement and stock, and the online channel provides orders, cancellations, and returns.
While they do not have a common axis “item × location × day,” there is nothing on which to train demand forecasting in retail: each system calculates by its own SKU list and its own calendar.
The first step is a unified product master. We match items by barcode, supplier SKU, and normalized name; disputed cases are confirmed by a person, after which the link is maintained automatically.
For each field we set source priority: stock — from the warehouse, prices and discounts — from POS extracts from 1C; conflicts are resolved by a rule, not “by eye.”
The second step is data quality before training.
We bring facts to a single granularity, label returns and write-offs, cut duplicate receipts, and separately mark promotions and days without an extract: if the gaps are not closed, the model will learn integration failures instead of demand.
We fill short history with similar items and category profiles — a forecast for a new item is built on analogues, not on an empty place.
Next, source unification lives in one layer: fact mart, product master, and calendar of locations and promotions. We launch the model on the agreed catalog core, show accuracy on a holdout period, and then expand coverage.
Quality metrics — share of matched items, gaps, master conflicts — remain in monitoring, so data degradation is visible before the forecast diverges from the shelf.
Common technical concerns: data quality and cold start
Cold start is not a reason to wait a year for sales history to accumulate. If an item has several weeks of observations, the forecast is built through analogues and the category as a whole: seasonality, promotions, holidays, local events.
The “category → item” hierarchy provides an estimate even where there is almost no own data, and it is refined as actuals accumulate.
Accuracy validation is done not on a demo but on a holdout period: the model trains on the past, and its forecast is compared with actual sales it has not seen.
We calculate error by store, category, and day of week — it is important to see where the forecast underperforms, not just the average metric.
We have almost no sales history — is forecasting possible?
Yes. We rely on profiles of analogues and category patterns, adding promotions and local events. For the first weeks, the model runs with a wide interval and gradually narrows it as actual sales accumulate.
How can we be sure the forecast is better than current planning?
We run both approaches on the same holdout period: share of items with error within acceptable bounds, systematic overstock or shortage, and number of manual edits by the category manager. We record the difference in a report before launch.
What should be done with incomplete data in the accounting system?
We build a data profile: gaps in receipts, duplicates, stock discrepancies with sales, and different units of measure. Some is cleaned automatically, some by rules on the integration side; items without recoverable history are marked and excluded from the overall metric.
Who supports the model after launch?
Our engineers monitor accuracy drift as assortment, prices, and customer behavior change, retrain the model on schedule and during anomalies, and analyze discrepancies with the client’s team — documentation and knowledge transfer are included in the work.
Let’s discuss the task and come back with a plan and estimate
A substantive conversation about demand forecasting starts with your chain’s data, not with a presentation. The project estimate relies on sales history and process descriptions: without inputs, the timeline range remains a guess, not a calculation.
To discuss the task in substance, it is enough to send what is already in the accounting systems:
- sales history extract by SKUs and locations for 1–2 years — with returns, write-offs, and markdowns;
- product master with categories, hierarchy, and seasonality attributes;
- promo and discount calendar: dates, mechanics, depth, location coverage;
- external factors affecting demand: weather, holidays, local events;
- description of the current purchasing planning process and the roles of those who use it;
- list of reports and metrics by which you currently assess forecast quality;
- constraints: data availability, IT environment, storage and access requirements.
In response, you will receive a phased implementation plan, a scope of work divided into our and your areas of responsibility, a project estimate, and a list of risks related to data quality.
If there is no history in the required dimension, we will propose a scheme for accumulating it and a first forecast loop on what is available right now.