Why trading strategy optimization breaks on real data
Parameter tuning on history almost always produces a beautiful curve: on the chart of past years the strategy grows confidently, while on a live account the same code behaves differently.
The reason is that trading strategy optimization selects values for a specific past segment, not for market mechanics — when overfitting on history, parameters encode noise that will no longer exist.
The first source of error is the quality of historical data.
Archives contain gaps during sharp moves, prices based on the last trade instead of real order book depth, delisted pairs are simply absent — and such a sample is used to test a strategy that in reality would have traded a different instrument.
The second loss is execution. Trade slippage and commissions eat into profit if you calculate by signal price rather than by actual trade price.
Execution time discrepancy adds a gap between the moment a signal appears and order placement: while data reaches the executor, the price has time to move, and some trades close worse than calculated.
In sum, a gap emerges between history and reality: the selected parameters do not transfer to new conditions, sample size masks overfitting, and execution time discrepancy makes results non-reproducible. Let's break down how this is closed.
What changes in operation: drawdown, response, robustness
We run parameter search not manually, but through Optuna: the same dataset, a fixed generator seed, the same method of calculating the result. Otherwise runs are incomparable, and tuning turns into fitting to one lucky historical segment.
- Drawdown reduction. Maximum drawdown shrinks, but the second point is more important — recovery time compresses: the account returns to previous highs in days, not weeks.
- Market response time. When the regime changes — a volatility spike or a narrowing spread between bid and ask prices — parameters are taken from an already validated region and recalculated in minutes, rather than selected anew.
- Robustness of trading logic. We test on a holdout segment and across several market regimes: logic that holds its result out of sample does not fall apart in the first unusual week.
- Per-trade risk control. Risk is set as a share of capital, position size is calculated from current volatility — a streak of eight losing trades in a row no longer knocks it out of the acceptable level.
- Execution and data gaps. The share of canceled and unexecuted orders falls — entry accounts for liquidity, not just the signal. A missed candle or service restart does not break state: the position is restored from the last snapshot, and duplicates are not opened.
We measure each item before and after on the same dataset, so the report shows which specific change produced the shift.
Strategy tuning modes: comparison of approaches
The tuning mode determines where strategy parameter values come from and how quickly they change. In practice, a choice is made among three circuits: a full run over the trade archive, adjustment directly in live mode, and a combination of both.
We compare them on three things — how long recalculation takes, how the result behaves on a holdout portion of history, and what happens when the market regime changes.
| Variant | When it fits | Limitations |
|---|---|---|
| Parameter selection on history | Recurring market regimes, a long trade archive, and time budget for a run | Fitting to past data: the set holds as long as market statistics have not shifted. One-time recalculation — and a new run is needed again |
| Adaptive adjustment in live mode | Regimes change often, position holding horizon is short | Reaction to noise and parameter "jitter", growth in the number of trades and costs. An error in the recalculation rule is replicated across the entire flow |
| Hybrid execution circuit | Long horizon: both robustness of the base set and response to regime change are needed | Harder to maintain: drift corridors, a change log, and a rollback point to the previous set are required |
We set boundaries explicitly: history provides base values and a corridor within which the live circuit is allowed to move parameters, and crossing the boundary stops trading until manual review. We recalculate on a schedule, not on every tick.
Implementation stages: from data audit to handover
The order of work is the same both for a strategy that is only being assembled on historical data and for an already operating system: first we make sure the data describes reality, then that the order execution model corresponds to it, and only after that do we release it into operation.
- Trade log audit. We reconcile exports with venue reports by time, volume, price, commissions, and partial fills, remove duplicates and gaps, and restore missing intervals. The result is a dataset for which the discrepancy with the broker report is explainable and reproducible.
- Reconstruction of the order execution scheme. We describe how the order entered the order book, where slippage occurred, how rejections were handled, and how much time passed from signal to trade. Without this, the historical result does not transfer to a live account.
- Formalization of strategy rules. Entry, exit, position size, risk limits, and stop conditions are fixed as unambiguous rules — they can be checked line by line, not "by the author's logic".
- Assembly of the validation circuit. The same data, costs, and code for all runs: we version datasets and parameters so that any result can be reproduced again.
- Parameter selection on the training segment with a limit on the number of combinations searched — otherwise the selected values describe noise, not a pattern.
- Validation on an independent segment. The data does not participate in selection; additionally, we test robustness with window shifting and under elevated costs. A strategy that holds only on one interval does not go into operation.
- Deployment and handover. We configure risk control, logging, and deviation notifications, document the shutdown procedure, hand over documentation and configurations to the client's team, and support the first production runs.
Handover package: configurations, policies, procedures
The work ends not with a report, but with a set of artifacts that the client's team then lives with on its own.
The trading circuit configuration is stored in a repository with version control, changes go through review rather than being edited by hand on the server — therefore any version of the strategy can be deployed again and produce a comparable result on the same historical segment.
- Trading circuit configuration: exchange connection points, operating intervals, instrument list, limits on position size and number of simultaneous orders.
- Exchange key access policies: trading keys without withdrawal rights, a separate read-only circuit for data collection, key lifetime and rotation procedure.
- Risk allocation rules: maximum share of capital per position, daily and weekly stop, system behavior when the connection to the exchange is lost.
- Parameter review procedure: what is checked on schedule and what by trigger (drawdown deeper than calculated, volatility regime change), who makes the decision and where it is recorded.
- Strategy documentation: entry and exit logic, list of parameters with acceptable boundaries, known limitations, and conditions for taking the strategy out of operation.
- Measurement logs: saved runs on historical segments, metrics before and after parameter changes, and a note of which configuration version was in production mode.
Such a package makes it possible to transfer support to another team or return to the strategy months later — without digging through logs and reconstructing logic from memory.
Case study: drawdown three times smaller at the same turnover
What it was before
The client traded by a set of rules that lived in notes: entry based on a feeling that "a move is brewing", the stop was moved manually if the price "had to reverse", and position holding was not limited by anything.
On history, this produced a maximum drawdown of 26%, but individual months closed positive — the weak spot did not look like a problem.
The result could not be repeated a second time: entry reasons were not recorded, and manual edits on the live account diverged from the test run.
What we changed
Entry was formalized: the signal is accepted after bar close and confirmation from a higher timeframe, not on touching a level. Stop movement was replaced with volatility-based trailing — the multiplier was selected by price dispersion, not by the desire to keep the position.
Holding was limited by horizon: if the move is not confirmed within a set number of bars, the position is closed by rule, not by mood.
What confirms it
Each run was calculated in Backtrader and entered into the measurement log: parameter set, date, drawdown, profit factor, number of trades, share of exits by the stop rule.
The training and holdout segments were split in advance, so fitting to history was filtered out immediately. Drawdown decreased from 26% to 9% at the same turnover, profit factor rose from 1.1 to 1.4, and about 40% of signals are filtered out by the entry filter.
Manual interventions were removed, and reviewing the set after a market regime change takes hours instead of days.
How does historical validation differ from real time?
Archive calculation uses already closed candles: the trade price is known in advance, the order is executed instantly and in full. In live trading, time passes between signal and trade, the price has time to move, and part of the volume remains unexecuted.
This is where the discrepancy between archive and live trading comes from — and almost always it works against the strategy.
The first source is execution latency. It is made up of segments: signal calculation, order sending, queue placement, trade confirmation.
In a calm market, these fractions of a second are unnoticeable, but during a sharp move, the gap between actual and calculated price becomes systematic rather than random interference — the model enters worse than it expected time after time.
The second source is commissions and slippage. The exchange commission is charged on every trade and is often simply not included in archive calculation.
Slippage depends on order book depth: a large order moves the price itself, and in a thin market the actual trade price is noticeably worse than the one at which the model "made" the trade on the chart.
Discrepancy control is put in place before the strategy is released into operation, not after. The model is run with the same constraints as the live circuit: latency, commission, volume limit per order, cancellation of an unexecuted remainder.
Then the live trade log is reconciled with the archive run on the same interval — we calculate the share of executed orders, the average gap between trade price and calculated price, and the difference in the final result.
As long as the discrepancy stays within tolerance, the strategy operates with limited volume and the tolerance is checked again. If it is violated, parameters and entry rules are recalculated, rather than "tweaking" history to the desired result.
Limitations and risks: data, commissions, slippage
Formally, a strategy may show a robust result on history, but this does not automatically transfer to real trading. Venue archives contain gaps: hours of downtime, breaks in the trade flow, different candle intervals on the same instrument.
If the logic depends on volume and number of trades, such holes distort validation — it sees liquidity that did not exist at that moment.
The second limitation is the variability of the venue's own conditions: fees, minimum price step, rounding and order cancellation rules change, and the previous configuration loses its meaning.
On thin pairs, slippage is added: an order is executed at a price worse than expected, and a wide bid-ask spread eats into part of the profit even before entering the position.
And no historical run guarantees profitability — it is a test of a hypothesis, not a promise of results.
How can you tell that a strategy is overfitted to history?
A sign is that the result drops sharply with a small shift in parameters or on an adjacent candle interval. We check on a segment that did not participate in selection and look at the dispersion, not the best run.
What should be done about gaps in a venue archive?
We mark them as a gap and do not fill them with invented values — otherwise validation will count trades that did not exist. On segments with breaks, the strategy is not evaluated, and the total is calculated over the covered share of the period.
Let's discuss the task: what to send and how we will come back with a plan
For trading strategy parameter optimization to produce a reproducible result rather than fitting to the past, source data for calculation is needed. Below is the minimum set covering most tasks; if something is missing, say so right away — we will suggest what to replace it with.
- Entry and exit rules in the current version: indicators, trigger conditions, stop logic, and profit-taking logic.
- Trade archive for the period: an export from the terminal or exchange in machine-readable form, with timestamps and position volume.
- Risk constraints: acceptable drawdown, maximum share of deposit in one position, limit on the number of trades.
- List of venues and instruments: spot or futures market, available pairs, depth of historical data.
- Execution requirements: manual launch, signals or automated trading, acceptable reaction latency.
- Success criterion: metrics by which you accept the result — return-to-drawdown ratio, share of profitable trades, robustness outside the training sample.
Then the cooperation process is simple: we review what was sent, check the completeness of history and correctness of trade labeling, and if necessary return a list of clarifications.
Then we prepare an estimate and work plan — stages, what we check at each, and which metrics we fix as acceptance criteria.
Describe the task in free form and attach what you have on hand — we will come back with an estimate and work plan and immediately say which data is missing.