Why backtesting a trading strategy lies on live trades
Backtesting a trading strategy on historical data almost always looks more convincing than live trading: the equity curve is smooth, drawdowns are short, and the result reproduces on any segment.
In real trades, the divergence between the run and the trades eats exactly the part of profit for which the strategy was launched.
First reason — overfitting to history. If there are ten configurable entry and exit parameters, brute force on one period finds a combination that perfectly describes the past: it catches specific market moves rather than a recurring pattern.
The check is simple — parameters are fixed on one window, and the result is examined on another that was not involved in the selection.
Second — slippage in the run. Calculation at the candle price assumes the trade will execute exactly at it, whereas in the pool it moves the price along the curve. The larger the order relative to pool reserves, the more the actual entry price differs.
Network fees and the pool's own fee also apply here — in the run they are often not deducted at all.
Third — stale pool data. Liquidity is not constant: the pool shrinks, providers withdraw funds, depth at the moment of the trade changes. A strategy profitable at deep pool executes noticeably worse on a thin one, and an averaged historical price will not show this.
That is why an honest run models pool depth and fees at each point in time, rather than taking a single figure for the entire period.
What a reproducible hypothesis validation loop provides
Reproducibility of a run is when re-running on the same data snapshot gives the same numbers: return, maximum drawdown, moment of exit from a position. We fix the network state snapshot, the fee and slippage model, the order execution sequence, and all sources of randomness.
A divergence between two runs is a sign that the result rests on hidden randomness: such a hypothesis can neither be compared with others nor handed over to production.
Drawdown and liquidation risk are calculated not from the final curve, but across its entire shape. We look at the depth of decline, how long the strategy does not recover losses, and at which price move a leveraged position goes into forced closure.
We separately run episodes of sharp volatility: there it is visible whether the strategy will survive the real market, not a calm stretch.
Return metrics without overfitting are provided by splitting the data: rules are tuned on one segment and tested on another that the strategy has not seen.
If the result holds only on the tuned segment, and the number of parameters exceeds the data volume, it is a rejection before launch, not after losses.
In total, the loop removes a whole class of errors: divergences between runs disappear, unworkable hypotheses are filtered out before the strategy goes into production, and the risk of drawdown and liquidation is visible in advance in clear numbers.
Run modes: history, stream, and stress scenarios
No validation mode replaces the others: a historical run answers the question "did the logic work on past data," streaming replay — "will it survive real latency and event order," stress scenarios — "what happens when the market deviates from normal." We assemble validation as a set of layers and fix what each layer proves.
Comparison of run modes
The key difference is in the data source and whether the market state is reproduced in full.
A historical run goes over saved state snapshots and quickly filters out weak hypotheses; streaming replay mixes in delays, competition for execution, and partial fills; a stress scenario shifts pool depth, execution price, and fees and shows the margin of safety.
| Option | When it fits | Limitations |
|---|---|---|
| Historical run over state snapshots | Initial filtering of hypotheses, comparison of dozens of parameter sets | Does not see delays and competition for execution — the result looks better than real |
| Streaming replay of events | Validating execution logic, reaction to partial fills, and state rebuilding | More expensive in time and infrastructure; requires a complete event log for the period |
| Stress scenarios | Assessing margin of safety before launch, checking risk and stop rules | The scenario is set manually and does not cover an unfamiliar class of failures |
| Liquidity gaps | Strategies dependent on pool depth and execution price | Synthetic data is needed; the result is a stability boundary, not a forecast |
The layers go in increasing order of validation cost: first history filters out most options, then the stream confirms executability, stress sets the boundaries beyond which the strategy stops.
This order saves time, and a reproducible run log shows why the parameters were chosen exactly as they were.
Stages of assembling the testbed: from data to protocol
A testbed for validating a strategy is assembled once and works for years — if the order of steps is fixed. When return calculation, history loading, and trade selection rules live in one script, any edit forces the methodology to be reinvented. We work in a fixed sequence.
-
Data preparation. We collect trade and pool state history, bring it to a single time step, remove duplicates and gaps. We separately mark areas where data is insufficient.
-
Validation scheme. Before launch, we describe the hypothesis and metrics: return-to-drawdown ratio, share of profitable trades, time in position. There too — the conditions under which the strategy is recognized as non-working.
-
Testbed setup. An isolated environment where execution is reproduced: pool fees, slippage, delay between signal and trade. Re-running on the same data gives the same result.
-
Run on the training segment. We look at what the strategy earns on and where it breaks. Tuning parameters to a beautiful curve is not allowed — the next step loses meaning.
-
Validation on an independent segment. A run without changes on a period that did not participate in tuning, including sharp market moves and liquidity failures. Divergence from the training segment is the main sign of overfitting.
-
Re-check protocol. We hand over a document: where data comes from, how the testbed is launched, which metrics we consider working, when to stop the strategy. Using it, the validation is repeated without us.
Thus a one-off calculation turns into a tool used on every strategy edit.
Backtesting: artifacts the team keeps
A report with returns but without data and settings is useless: a month later it can neither be rechecked nor disputed. Therefore, together with run results, we hand over everything they are assembled from — down to the last parameter.
- Datasets — exports for the run window with a network state snapshot: prices, fees, liquidity depth, protocol events. Stored in immutable form with a checksum, so the run is repeatable on the same data.
- Environment configurations — version of the calculation module, simulation step, slippage model, gas limits. Deployed with one command on your test environment without contacting us.
- Run reports — trade log, equity curve, maximum drawdown, position holding time separately for each strategy variant.
- Metric description — what is included in return, how fees and slippage are accounted for, why this evaluation horizon was chosen, and what the metric does not show.
- Re-check protocol — step-by-step order: when and on what data to rerun, which divergences to consider significant, under what condition the strategy goes back for revision.
- Log of rejected hypotheses — what was tested and why it was discarded, so the team does not waste time re-examining the same ideas.
Such a set makes the result reproducible: divergence between runs is explained by a specific parameter, not by "market behavior."
Case: divergence between the run and live trades was reduced severalfold
The client is a team that trades its own strategy in liquidity pools on several networks.
On historical data, the run showed steady growth, but in live trades the picture diverged: entry price moved 2–3% from the model, some trades did not fill at the expected level at all, and the final equity curve behaved differently from the chart.
What was changed in the validation loop
The cause turned out to be not the strategy logic, but the execution model. History was calculated at candle close price — without slippage along the pool curve, without competition for a place in the block, and without gas cost.
On narrow pairs, this produced a divergence an order of magnitude larger than the fee itself, and parameter decisions were made on the data from which those parameters had been selected.
The loop was rebuilt: a run on a network copy via an archive node, substitution of the actual transaction order inside a block, slippage calculation based on current pool depth, and accounting for the delay in sending a signed transaction.
Each hypothesis was run over a rolling window and separately tested on a segment that did not participate in tuning.
The divergence between the run and live trades by entry price decreased from 2.5% to 0.3%.
The share of hypotheses filtered out before market launch grew from about 20% to 80%: previously, an idea that did not work was identified already in live trades — now the run rejects it in hours rather than weeks of observation.
Client CTO's assessment: "Previously we learned about a strategy's unsuitability after the fact, after effort had already been invested. Now filtering happens before trading, and there is no longer any need to argue about whose run is more correct."
How does streaming replay of trades differ from a historical run?
A historical run works with a ready-made array: the engine feeds the strategy bars and trades in batches, executes orders by formal rules, and covers a month of market in minutes.
Streaming replay feeds the same events one at a time, in the original order and with the original delays — from the moment data is received to block recording.
The difference is revealed in look-ahead bias. If a signal relies on the close price of a bar, the run will not notice: the order will execute at a price we did not yet know in real time.
In the stream, the order is sent only after receiving the event, so it is immediately visible how many trades are built on future information. Slippage, competition for pool liquidity, fees, and chain reorganizations that change market state are in the same category.
The problem is not hypothetical: work in Journal of Computational Finance shows that with broad parameter search, the best historical result is almost always random.
Streaming mode is more expensive in time — a month of market takes hours to compute, an event archive and reliable delays are needed — so it is used not for search but for final validation.
The choice of validation mode follows from this. First, a historical run filters out the bulk of hypotheses and roughly outlines the parameters.
Then the same conditions are reproduced by the stream: if the outcome diverges more than the strategy logic allows, it means the result rests on an engine artifact, not on the market — and such a configuration never reaches trading.
Technical questions before starting work
Before starting, the same technical questions almost always come up — and these are the right questions, because the answers determine whether the result of backtesting a trading strategy can be trusted. Below are those that most often determine the scope of work and the accuracy of the final metrics.
How much can historical data be trusted?
We take candles and trades from at least two independent sources and cross-check them against each other.
We fill gaps, mark outliers, and separately highlight intervals with low liquidity: it is precisely there that slippage in validation is underestimated, and returns look better than they would have been in reality.
Does validation transfer to other pools and networks?
We keep the strategy logic separate from integration with a specific pool. During transfer, only execution parameters change — fee level, liquidity depth, price update frequency — not the logic itself.
We run the same set of rules on several pools and compare metrics: if the result holds only on one market, it is visible immediately.
Are network fees and slippage accounted for?
Yes, without this the return in the report is overstated. In the model we include the cost of each transaction, the impact of own volume on execution price based on historical depth, and the costs of entering and exiting a position.
What happens after changes in the strategy or pool?
Every change goes through a repeat run on the same data set and period, with a fixed version of history. We compare metrics before and after — drawdown, share of profitable trades, behavior on volatile segments. This shows whether the edit improved the result or simply fitted it to the past.
What is needed at the input: we will discuss the task and come back with a plan
To reproduce a strategy on history and get numbers that can be trusted, what is needed is not so much code as correct input data and access.
Below is the minimum set we start the conversation with: if something is missing, it is immediately visible, and we honestly say how it changes the accuracy of the result.
- Strategy logic: entry and exit conditions, position size, averaging, stop, and partial close rules — in a form suitable for formalization.
- Access to trade history: an export of trades, orders, and events for the required addresses, or read-only keys for trading accounts.
- Market data for the historical period: candles of the required timeframe, pool metrics, funding rates, liquidity snapshots.
- Cost model: network and protocol fees, slippage, trade impact on the pool — without this, paper returns will not match real ones.
- List of networks, protocols, and contracts involved in the strategy, with each one's roles.
- Reporting requirements: set of metrics, run horizons, export format, and which drawdowns we consider unacceptable.
- A contact person who makes decisions on strategy parameters — so that assumptions are not coordinated for weeks.
We request keys only for read access, without withdrawal rights; where possible, we work with exports rather than a connection to a live account.
If data for part of the period is missing, we record this as an assumption and show it in the report, rather than filling the gap with averaged values.
After discussing the task, we come back with an estimate and work plan: stages, what we validate at each, which metrics we consider sufficient for a decision to launch the strategy.