Building a Backtesting System: Where Portfolio Crypto Strategies Break
Building a backtesting system starts not with code, but with an honest answer to the question: why does a portfolio backtest show growth while the real account shows a drawdown. The typical cause is the execution model.
In crypto strategy testing, a trade is closed at the candle price, while in reality the order passes through several levels of the order book, and on thin pairs, slippage in the backtest eats up the entire profit of the strategy.
If this is not built into the engine, the result of a beautiful curve has nothing to do with trading.
The second group of backtesting errors is fees and delays.
Maker-taker, the funding rate on futures, the cost of withdrawal between exchanges, the lag between the signal and placing the order: each of these items individually looks like a trifle, but on a portfolio of dozens of instruments and a turnover of hundreds of trades per day, they change the sign of profitability.
A portfolio backtest must account for them at the level of each trade, not subtract an average percentage at the end.
The third bottleneck is data. Gaps in history, different timeframes across sources, delistings and token renames, the absence of order book data for earlier periods.
A strategy that "sees" the future price due to candle shifts or gap filling shows a return that is unattainable in reality. That is why a correct portfolio backtest is built on a single data store with an explicit policy for handling gaps and versioning of datasets.
The fourth layer is portfolio logic. Rebalancing, overlapping signals across instruments, limits on correlation and total exposure, capital allocation between strategies.
The error here is not in an individual trade, but in how strategies compete for the same capital and simultaneously open positions in the same direction of the market.
We break down how this is addressed: from engine architecture and the execution model to out-of-sample validation and reconciliation with real trades.
What Correct Development of a Backtesting System for Portfolio Strategies Delivers
Event-Driven vs. Vectorized Approach: Where the Boundary Lies
A vectorized backtest computes returns over an array of prices in a single pass — a run over several years of history fits into seconds, which is convenient for the initial screening of hypotheses.
The limitation shows up on a portfolio: drawdowns, rebalancing, and partial order fills are not reproduced in such a model, so the result is only good for filtering out obviously weak strategies.
An event-driven backtest processes a stream of market events candle by candle or trade by trade, and every order goes through an execution model.
The run takes longer, but it is precisely this approach that reveals the real drawdown and the strategy's behavior during liquidity gaps.
In our practice, a portfolio of dozens of instruments over several years of history is computed in minutes rather than hours — thanks to data preloading and parallelization across instruments.
Reproducibility as an Engineering Requirement
Every run is recorded: the data version, the parameter set, the portfolio state at the start.
Re-running with the same input data produces a bit-for-bit identical report — a discrepancy means a bug in the model, not "market noise." This eliminates an entire class of errors where a strategy looks profitable due to look-ahead bias or incorrect accounting of commissions.
| Approach | When It Fits | Limitations |
|---|---|---|
| Vectorized | Fast hypothesis screening, single instruments | Does not model drawdown or the order queue |
| Event-driven | Portfolio, rebalancing, risk assessment | Longer run, harder to implement |
| Hybrid | Screening on the vector, validation on events | Requires alignment of data models |
Strategy metrics — return, maximum drawdown, Sharpe ratio, turnover — are computed on the same engine for all runs, so the results are comparable to each other.
The backtester architecture is built so that the strategy under test does not know whether it is running on history or on live data: the same code goes into production without rewriting. This approach underlies Backtrader and related frameworks.
How We Build a Backtesting System: Stages of Engineering Work
Developing a backtesting system begins not with code, but with an audit of the data and the rules by which the strategy makes decisions. On the input side — trade exports, quotes from several venues, portfolio rebalancing history.
On the output side — a reproducible pipeline where every figure in the report is explained by a specific source and simulator setting.
- Trading data audit. We reconcile quote sources, check for gaps, duplicates, and discrepancies between venues, and bring time zones and currencies to a unified form. Without this step, any strategy metric is a random number.
- Formalization of the strategy and portfolio rules. We fix the logic of entries and exits, position sizing, instrument limits, and rebalancing rules. We move contentious points into explicit parameters rather than hidden assumptions in the code.
- Execution simulator setup. We set commissions, slippage, execution delay, and the liquidity model. The simulator must reproduce real trading conditions, otherwise the historical results do not carry over to the live market.
- Backtester implementation and historical run. We assemble the calculation pipeline, run the strategy on historical data, and compute returns, drawdown, turnover, and the distribution of trades over time.
- Strategy validation. We check the robustness of the result: splitting the history into periods, tests on different instrument sets, and sensitivity checks to parameters. A strategy that holds up only on one lucky stretch is filtered out here.
- Reconciliation with an independent calculation. We compare key metrics against a manual recalculation on a sample of trades — this is how we catch errors in portfolio logic and cost accounting.
- Handover of the system. We deploy the pipeline in the client's infrastructure, document the parameters, and train the team to run backtests and add new strategies on their own.
Each stage ends with an artifact: a cleaned dataset, a specification of rules, a simulator configuration, a run report. This makes it possible to return to any step without rebuilding the entire pipeline. Describe your task — we will come back with an estimate and a work plan.
How does event-driven backtesting differ from vectorized backtesting, and when should you choose which?
Vectorized backtesting computes the entire history in a single matrix pass: signals, returns, and portfolio weights are multiplied as arrays, and positions are recalculated synchronously on each bar.
This delivers speed — tens of thousands of parameter runs in minutes, which is critical at the initial strategy screening stage.
The limitation is that this approach assumes perfect execution: an order always fills at the closing price, liquidity is infinite, and the order of events within a bar doesn't matter. For strategies on liquid instruments with infrequent rebalancing, this is sufficient.
Event-driven backtesting reproduces the system's operation step by step: a market event arrives, the strategy reacts, the order goes to the execution simulator, and the portfolio is recalculated taking into account commissions, slippage, and partial fills.
Here, stop orders, limit orders, intrabar triggers, and delays between signal and execution are modeled correctly. The trade-off is time: a single run can take minutes instead of seconds, so mass parameter sweeps aren't done with it.
The choice of method is determined by the type of strategy, not by convenience. For momentum portfolios rebalanced weekly or monthly, the vectorized approach gives the same return picture as the event-driven one, but many times faster.
For strategies with frequent entries, stops, and dependence on market microstructure, vectorized backtesting overstates results: it "sees" prices that didn't exist in real execution.
The practice is two-stage. First, a vectorized run filters out obviously weak configurations across a broad parameter grid, then the finalists are verified by an event-driven engine with a realistic cost model.
The discrepancy between the two results is itself a diagnostic signal: if event-driven returns are noticeably lower than vectorized ones, the strategy lives off perfect execution, and this won't be reproduced in real trading.
When developing a backtesting system, we build both engines on a shared data layer so that switching between them doesn't require rewriting the strategy logic.
Case: How a Portfolio Crypto Strategy Stopped Diverging from Reality
The client came with a classic pain point: the portfolio strategy on historical data showed steady growth, but on a live account the result diverged from the backtest by tens of percent.
The analysis revealed three causes — accounting for commissions and slippage at an averaged rate instead of a per-instrument model, rebalancing at the closing price without execution delay, and look-ahead bias when calculating signals.
In other words, the backtest divergence from trading was not a market accident but a defect of the model itself.
We rewrote the run core: candles are loaded per instrument separately, commission and slippage are calculated per trade, the signal is generated only on a closed candle, and execution is shifted to the next bar.
We added a decision log — for each trade you can see which signal generated it and what data was available at that moment. This turned strategy optimization from picking pretty numbers into verifying logic.
What Changed in Numbers
| Metric | Before | After |
|---|---|---|
| Backtest divergence from trading | 30–40% in returns | within 3–5% |
| Portfolio run time | hours per configuration | minutes, parallel runs |
| Logic errors found in the strategy | not tracked | 11 confirmed defects |
| Result reproducibility | depended on run order | deterministic run |
We separately addressed reproducibility: the same dataset and parameters now produce an identical result regardless of run order and machine. This is what a backtesting case is meaningless without — if a run cannot be repeated, there is nothing to compare configurations against.
VectorBT was taken as the foundation for the vectorized run — it provides speed on a portfolio of dozens of instruments, while the wrapper for accounting for commissions and execution delays was written by us, because the default settings did not cover the required trade model.
Compatibility, migration, and access: what to discuss before the start
Developing a backtesting system almost never starts from a clean slate: the client already has exchange keys, historical quotes, internal portfolio reports, and possibly their own calculations in spreadsheets or scripts.
Therefore, before the start, it is important to lock down three things: how the system will sit alongside the current infrastructure, how accumulated data will be transferred, and who gets access to trading keys.
If this is not discussed in advance, surprises surface at the acceptance stage — discrepancies between historical data and previous calculations, missing roles for analysts, or the inability to roll back a strategy update without stopping the entire environment.
What if we already have our own calculations? We don't break them; we reconcile them: we run a reference set of trades through the new system and the old logic, and we resolve discrepancies down to the digit. What if a rollback is needed?
Every version of the strategy configuration and dataset is versioned, so returning to a previous state is a switch to a pinned version, not a restore from a backup. What about access?
Exchange keys are stored in a secure vault, and permissions are assigned by role: an analyst can see data and results but cannot withdraw funds.
Can it be implemented in stages without stopping current operations?
Yes. First, we stand up the environment on historical data in read-only mode, reconcile the results against your reference values, and only after that do we connect real sources and trading keys. Current processes are not affected during this time.
Upon delivery, the client receives:
- a deployed system configuration with a description of all calculation parameters;
- an integration scheme with exchanges and historical data sources;
- an access and role separation policy, including handling of trading keys;
- a migration procedure for accumulated data and reconciliation with previous calculations;
- instructions for rolling back changes and returning to a previous strategy version;
- operations documentation: monitoring, data updates, and incident response.
Describe your task and current stack — we'll come back with an estimate, a work plan, and a list of the data needed at the start.
Timeline for implementing a backtesting system by stages
The timeline depends on three things: how many historical sources need to be connected, how complex the rebalancing logic is, and whether trade execution with slippage and commissions needs to be accounted for.
Below is an approximate breakdown by stages for a typical portfolio strategy. The boundaries of the stages overlap: data preparation runs in parallel with core design so as not to lose weeks waiting.
What pushes the timeline up
The most common source of delays is not code but data: exchange exports arrive with gaps, different time zones, and retroactively rewritten history. If a strategy trades on multiple exchanges, price reconciliation between venues and accounting for discrepancies are added.
The second reason is complex rebalancing logic: rules with thresholds, liquidity constraints, and partial execution require a separate trade simulation layer.
What speeds up the work
When the client immediately provides access to historical data and describes the entry-exit rules formally, the preparation stage is cut in half.
A ready-made layer of metrics and reports is reused across strategies — the second and subsequent models go through the process noticeably faster than the first.
Backtrader is one of the engines we use to build such systems: it provides ready-made infrastructure for running on history, and we add cost accounting and reporting tailored to the task.