Why demand forecasting breaks on real data
Demand forecasting rarely breaks on the model — it breaks on the input data. Orders, inventory, shipments, and promo plans live in different systems, are exported manually into planning spreadsheets, and consolidated into one workbook where every number is edited by hand.
The planner adjusts a value to the latest conversation with sales rather than to history, so the actual forecast error is not measured at all: it is unclear how far the number in the spreadsheet diverges from what actually happened.
Supply seasonality in such a setup is accounted for “by eye”: an average-over-period aggregate smooths out peaks and dips, while the lag between order and arrival at the warehouse is simply absent from the calculation.
A promotion that doubles demand, and the dip that follows it, live in a separate marketing file and never reach the model — as a result, the system purchases according to an averaged profile that matches no real month.
A separate cause is the gap between departments. Sales, purchasing, and the warehouse calculate their own versions of the plan with different horizons and rounding rules, and each version goes into work as the only correct one.
The error at this junction is not damped but accumulates: an inaccuracy in one product group pulls along orders for adjacent items, and the discrepancy is discovered only when the goods arrive.
Further along the supply chain, the accumulated error turns into two expensive warehouse states. At peak season — stockouts: lost sales and manual expedited purchases. During a downturn — overstocking: frozen working capital, shelf-life issues, and markdowns. Below we break down how this is closed at the level of data, model, and planning process.
Impact of implementation: accuracy, inventory, lost sales
The implementation of demand forecasting is measured not by the “smartness” of the model but by four numbers visible in a report after one or two quarters: forecast accuracy, safety stock volume, warehouse turnover, and the share of lost sales.
We take the first baseline before the start — on historical data — otherwise there is nothing to compare the effect against.
Reducing safety stock without losing service level
Forecast accuracy over a 4–12 week horizon typically rises from 60–70% to 80–90% by weighted error — enough for safety stock on fast-moving items to shrink by 20–35% at the same service level.
The key is moving from a single standard to an interval forecast: where demand is stable, stock is cut, while for volatile items it remains as before. Turnover improves by 15–25%, and write-offs due to shelf life decrease.
How many manual edits leave the plan
The share of manual edits is an indicator of distrust in the forecast, so we measure it before and after: typically 30–40% of rows are edited in the plan, after implementation 5–10% remain, and mostly for promotions and one-off deliveries.
We log every edit with a reason, and those reasons become model features — accuracy grows without manual adjustment.
We take the baseline and seasonality from open-source implementations such as Prophet, and on top — gradient boosting on prices and promotions. We calculate metrics over a rolling window so that a one-off outlier does not distort the picture.
Model formats: statistics, ensembles, neural networks
No single format wins across all categories — the choice is set by the planning horizon, the length of sales history, and the share of slow-moving items. Below are three approaches that are realistically chosen between at the start.
In practice, they are almost always mixed: a statistical model holds the baseline for regular demand, an ensemble of models refines seasonality and promotions, and a neural network model is connected where external factors need to be taken into account.
| Option | When it fits | Limitations |
|---|---|---|
| Statistical model | Stable demand, long history, horizon from a week to a month | Does not see promotions or external events; falls apart on slow-moving items |
| Ensemble of models | Many groups with different seasonality; a stable forecast is needed | Requires regular retraining and promotion labeling |
| Neural network model | Short history, many related factors, short horizon | Requires data volumes; the solution is harder to explain |
How to choose a format by horizon and category
The longer the horizon, the more important stability: for months ahead, statistics and ensembles give a smoother picture, while a neural network — for days and weeks.
The format is then fixed by category: one for fast-moving goods, another for slow-moving ones, otherwise average warehouse accuracy hides failures in individual groups.
What to do with slow-moving and new items
Slow-moving items are forecast not from their own history but through a group of analogues: similar products are combined and a common demand profile is transferred to them.
New items, until data accumulates, follow the launch plan and expert judgment, and the model is connected once enough observations have been collected.
Stages of implementing a supply planning system
The work proceeds sequentially: each stage is closed with a check, and moving to the next is possible only after the previous one is confirmed on the customer’s real data.
This way, errors in data and problem definition are exposed before effort starts being invested in development and tuning.
Data audit and problem definition
- Data audit — we reconcile sales, inventory, and supply history by depth, completeness, and discrepancies between accounting systems; the output is a list of gaps and periods suitable for training.
- Data preparation — we bring series to a single granularity (warehouse, product group, day or week), fill gaps and outliers, and label promotions and supply disruptions as separate features.
- Problem and metric definition — we fix the forecast horizon and target error at the warehouse and product group level, and agree on acceptable quality.
- Model assembly and tuning — comparison of approaches on a holdout sample, feature selection, and separate tuning for seasonality and local demand spikes.
Validation on historical periods and handover
- Model validation — running on historical segments in a mode close to real life: the model sees only the data that was available at that time and does not look ahead.
- Update process — we fix the retraining frequency, the person responsible for supplying fresh data, and the procedure for quality degradation.
- Handover to operation — deployment, monitoring of forecast deviations from actuals, training of the customer’s team, and documentation handover.
What we hand over to the customer’s team: configurations and processes
A demand forecasting project lives long only if it can be developed without its authors.
That is why we hand over not a “working model” but a set of artifacts with which your engineer can retrain the model, connect a new data source, and investigate an incident without exchanging messages with us.
Documentation package and data schemas
- Data schema — sources of sales, inventory, promo, and external factors: granularity, keys, lags, rules for handling gaps and spikes.
- Feature description — what is fed into the model, how lags and rolling windows are calculated, which features are disabled and why.
- Retraining configurations — forecast horizon, rebuild frequency, train/validation split, quality thresholds at which the model goes for additional training.
- Access policy — roles for reading raw data, exporting forecasts, and changing settings; where keys are stored and who issues them.
- Monitoring process — on-duty metrics: deviation of actuals from forecast, share of gaps, drift of input distributions; check frequency and alert threshold.
Team training and knowledge transfer
- Incident review — typical failures: demand shift, incomplete day in the data, change in promo structure; what is checked first.
- Workshops on your data — analysts run retraining themselves and roll out a new forecast version under the supervision of our engineers.
- Decision log — why the current horizons and thresholds were chosen and which alternatives were rejected, so that six months later no one has to guess.
Case study: how accuracy grew after reworking the model
The task came from a retail chain: a four-week forecast for several thousand items was needed not “in general” but for each location, to plan purchasing and shelf space.
Initially, the forecast was calculated as a moving average over past sales — the model did not see promotions, holidays, or assortment changes.
What it was before: discrepancies and manual edits
For key groups, the discrepancy with actuals over a four-week horizon reached 32%. Category managers manually edited about 40% of forecast rows every week — numbers were “finished by hand” to a state they believed in. Emergency reorders went out almost every day, and some items sat on the shelf.
The cause was in the features: the promo calendar and local holidays did not enter the model, and demand spikes were averaged with ordinary days. Seasonal items suffered separately: the average over the past period did not describe current demand.
What it became after: metrics and support mode
The forecast was rebuilt: instead of one model — a set by demand classes (stable, seasonal, promo-dependent, new items), a base layer on Prophet with calendar, promo, and price features.
Quality was checked retrospectively: training on a past segment, comparison with actuals on the next one, without looking ahead.
The discrepancy over the same horizon dropped from 32% to 9–12% by group, the share of manual edits — by about three times, emergency reorders — by half.
The forecast is updated daily, and the error for each group is compared with historical variance: going beyond the bounds triggers a notification and retraining on a fresh window.
Once a month — an accuracy report with a review of groups that declined, so degradation is visible in the first week, not at the end of the quarter.
How does demand forecasting differ from supply planning?
Demand forecasting answers the question “how much will we sell,” while supply planning answers “what and when to purchase so that there is enough.” The first is a statistical task: based on sales history, seasonality, promotions, and external factors, a demand distribution is built by day and point of sale.
The second is an optimization task under constraints: delivery lead times and frequency, minimum order quantity, shelf life, and warehouse capacity.
| Criterion | Demand forecasting | Supply planning |
|---|---|---|
| Forecast horizon | from a day to a quarter, rolling update | from a week to a season, tied to delivery lead time |
| Input data | sales history, promotions, price, inventory, external factors | forecast, delivery lead times, order quantity, warehouse limits |
| Area of responsibility | quality of demand prediction | service level and inventory volume |
| Output | demand distribution, not a single number | reorder point, order quantity, safety stock |
| Quality metric | forecast error by intervals | service level, turnover, stockout share |
The boundary runs exactly at the reorder point. The model provides a demand distribution and its variance, while the replenishment rule decides at what stock level to trigger purchasing and what safety stock to hold in order to maintain the target service level.
These areas must not be mixed: even an accurate forecast will not pull up the service level if the reorder point is calculated by eye, and the growth in inventory will be blamed on the “bad model.”
Therefore, in the project we separate responsibilities explicitly: forecasting owns data, features, and prediction quality control; planning owns rules, constraints, and the service level agreed with the business by product group.
Describe how your purchasing decision is currently made — we will come back with an assessment and a plan for each area.
Technical concerns: data, seasonality, failures
None of the questions below is about the algorithm, but about the data and processes around the model. Five concerns that a customer raises before the start, and how we address them.
Dirty and incomplete data?
Before training, we calculate completeness and desynchronization for each source: gaps, duplicates, lag between shipment and sale. We train on the intersection of clean intervals, and mark a gap with a “no data” feature rather than with the period average.
Anomaly or a real spike?
We do not remove anomalies but label them using robust statistics — the median and deviation from it. The model receives the label and decides for itself how much to trust it: a cut-out outlier takes away sensitivity to genuine peaks.
Promotions and discounts?
Promo is a controlled event, not an outlier: the promotion calendar, discount depth, and media spikes go in as separate features. Promo days are not mixed with ordinary days in training, otherwise the promotion effect is smeared across neighboring weeks.
Overfitting?
We check on a held-out future period, not on a shuffled sample: rolling time-series cross-validation and comparison with the naive “same as last week” forecast. If it does not beat the baseline — it does not go into production.
Quality control after launch?
Daily we calculate weighted average error (WAPE) and bias by item and location; when a threshold is exceeded, a notification arrives. Drift in input data triggers scheduled retraining — the approach from Forecasting: Principles and Practice.
Let’s discuss the task: we will assemble an assessment and implementation plan
To start, we do not need access to all analytics, but a specific minimum: sales history by item and location over a sufficient period, a calendar of promotions and deliveries, and warehouse constraints.
The earlier this data is on the table, the more precise the problem definition and the fewer iterations for rebuilding.
- Problem definition: forecast horizon and step, breakdowns (item, location, channel), acceptable error, and who accepts the result.
- Data export: we agree on the field composition, period, transfer method, and frequency — a one-time export or regular scheduled exchange.
- Data schema: we show how sales, promo, inventory, and external factors are linked to each other, where we fill gaps, and how we cut off outliers.
- Scope of work: stages with the result of each — data preparation, model training and tuning, validation on a holdout period, and implementation in your environment.
- Phasing and implementation plan: the launch sequence by product group or location, so that the first payoff appears earlier than full coverage.
- Timeline estimate: a range tied to data readiness and the speed of approvals, not an average market figure.
- Control points: demonstrations on real data, quality metrics before and after, and the procedure for additional training and rollback to the previous model version.
In response, we send a document with the scope of work, data schema, and stage-by-stage plan. Describe the task — we will come back with a timeline estimate and clarifying questions about the data.
