Custom Backtesting Framework: Where Non-Standard Strategies Break
Standard testing systems are designed for strategies of the "signal — entry — fixed stop and take" type.
Non-standard strategies are not described this way: entry depends on matching conditions across multiple sources, exit proceeds in stages, and the decision relies on portfolio state.
An off-the-shelf testing tool substitutes its own model for such logic, and distortion of testing results surfaces in live trades.
The first gap is entry conditions: an event requires simultaneous fulfillment of rules across different instruments and periods, while the standard engine reduces them to a single candle close.
In the model, the order is placed not where it would be placed in a real flow, and over a long series of trades the difference accumulates.
The second is multi-stage exit. Partial closing, moving the protective level following price, re-entry on a pullback: the typical template closes the position entirely by one rule.
In the test, the strategy looks profitable precisely because its actual exit logic was not executed.
The third is order execution sequence. On a volatility spike, several orders compete for one price, and which one is processed first changes the result more than the signal itself.
A synthetic flow smooths drawdowns, while rare events — gaps, trading halts, abnormal spreads — are not reproduced at all.
Building a custom backtesting framework means transferring your logic into the model in full, together with the order queue and rare scenarios. Next, we break down how this is addressed stage by stage.
What a Separate Testing Loop Provides: Measurable Effects
Running hypotheses on a live loop is expensive: an error becomes visible only through drawdown, and each check spends real funds.
A separate testing loop moves validation onto historical data — the hypothesis is filtered out before money is put to work, and the run result is compared with previous launches.
Speed of Running a Set of Strategies
On a custom backtesting framework, a set of 200–300 strategy configurations completes overnight, whereas sequential running of the same configurations stretches over a week — an acceleration of 8–12x.
It is achieved by parallelizing across data slices and reusing computed state: when only the exit rule changes, the history is not recalculated from scratch.
Reproducibility and Risk Control
Each run is tied to a fixed data slice and saved configuration, so the result is reproducible both a month later and after code changes: discrepancy between runs is fractions of a percent, and it is visible line by line in the log.
This eliminates an entire class of failures — divergence of results with identical input data and silent substitution of the quote source between runs.
Look-ahead bias, that is, calculation on data that did not yet exist in real time, is cut off by a separate check.
Quotes are verified against the history of oracle prices Chainlink, and the strategy additionally goes through boundary slices: liquidity gap, spike in network fees, data source halt. What survives them goes into the live loop with a pre-understood cost of risk.
Options for Implementing a Backtesting Framework: Comparison of Approaches
The choice between an off-the-shelf product and your own framework determines not launch speed, but which strategies you can test at all. While the logic rests on standard indicators, an off-the-shelf library is enough.
As soon as data from different intervals, an order queue, or a non-standard risk loop appear, the scaffolding around someone else's code begins.
| Option | When It Fits | Limitations |
|---|---|---|
| Off-the-shelf framework as is | Strategies on standard indicators, quick hypothesis testing | The execution model is hardwired into the core; non-standard rules are adjusted to someone else's logic |
| Modifying an off-the-shelf solution | A narrow layer is needed: your own data source, non-standard fee calculation | Changes are lost during updates; core tests do not cover your logic |
| Your own validation layer on top of the off-the-shelf one | Signals are yours, while trade and portfolio accounting is acceptable | Two data models must be synchronized; discrepancy in position calculation becomes visible on a live account |
| Fully custom framework | Unique execution logic, requirements for reproducibility and audit | Slower and more expensive at the start; tests and reference datasets are needed |
Limitations of Off-the-Shelf Solutions
The main limitation is not performance but the hardwired model of the world: event order, data type, method of accounting for slippage and fees.
Everything outside these frames is worked around with crutches, and the divergence between validation and real execution surfaces after launch.
When You Need Your Own Layer
Your own layer is justified when the strategy relies on data outside the typical model: order book, over-the-counter quotes, external signals, manual interventions.
The hybrid option is a compromise: trade and portfolio accounting is handled by the off-the-shelf core, while signal logic remains yours and is tested separately.
Development Stages: From Strategy Review to Acceptance and Handover
Development stages are built around the main risk: a non-standard strategy breaks typical tools, so it must be tested on your logic, not on an averaged one. We start with review, not code, and only after agreeing on the execution model do we move to implementation.
Strategy and Data Review
- Strategy audit. We capture entry and exit logic, the instrument set, rebalancing frequency, leverage handling, and risk management rules. The output is a formalized description that can be unambiguously verified by calculation.
- Data schema. We bring historical series to a single format: time zones, gaps, corporate events, delisting. Separately, we describe how we fill gaps so that this does not create false returns.
- Loop design. We define the execution model: order types, slippage, fees, submission latency, partial fills. This is exactly where testing and real trading most often diverge.
Implementation, Acceptance, and Handover
- Core setup. We build the calculation part for the agreed loop and move strategy parameters into configurations — new logic is connected without rewriting the core.
- Validation. We run control scenarios and compare the result with an independent manual calculation on a short window: this way, errors in the data schema and execution model surface before acceptance.
- Acceptance and handover. A run on your history, comparison of metrics, review of discrepancies, then source code, documentation, data schema, and team training. After handover, you launch new strategies yourselves.
Delivery Scope: Code, Data Schemas, Regulations, and Reports
Delivery under a custom backtesting framework is assembled so that the client's team can further develop and run it without us: code is in your repository, results are reproducible on your loop, and access decisions are made by you. Below is the delivery scope and what secures it.
Documentation and Regulations
- Source code with change history and usage rights, with a build schema — without dependence on our internal services.
- Data schemas: structures of historical quotes and on-chain events, rules for processing corporate actions, points for cleaning and trimming datasets.
- Run regulation: how parameters, data versions, and configurations are fixed so that the result repeats both a month later and after a change in team composition.
- Engine verification reports on reference strategies: where there are discrepancies with the expected result and what explains them.
- Instructions for connecting a new strategy: which modules to edit, how to run regression, and which metrics to watch during acceptance.
Rights and Access to the Loop
- Role matrix: who can change the core, who can only change configurations and data, who can only read run reports.
- Procedure for granting and revoking access, including our technical accounts: after handover, they are disabled at your request.
- Separation of development and run environments, rules for exporting data outside the loop, and logging of actions with it.
Case with Metrics: Validating a Strategy Before and After
The client is a trading team with non-standard entry logic: Uniswap pool liquidity, staged partial closing. Off-the-shelf engines required rewriting the strategy for their candle model, so decisions were made almost by eye.
Task Context
Validating one configuration took about four hours: the engine recalculated the entire historical series for each parameter set, and entry conditions did not fit its model.
They tested three or four variants per day, and rare scenarios — liquidity spikes, cascade liquidations — did not make it into validation. At the end of the quarter, 18% of live trades did not match the backtest.
What Changed in Validation
We built a custom backtesting framework for the team's logic: its own strategy description layer, a separate event store by blocks, and recalculation of only changed sections instead of the entire series.
- Configuration run time — from four hours to 11 minutes; parameter search runs in batches.
- Share of discrepancies with live trades — from 18% to 3%: fees, slippage, and execution delays were added.
- Drawdown in backtest and live converges within two points instead of a threefold divergence.
The first runs showed that some "working" configurations rest on a single lucky period — now this is visible before funds are withdrawn.
What to Do If the Strategy Is Already Being Tested in an Off-the-Shelf Solution?
An off-the-shelf solution and a custom loop do not compete — they answer different questions.
An off-the-shelf solution quickly shows whether there is a signal in the idea; a custom loop shows whether this signal reproduces on our infrastructure, with our fees and our order execution profile. While the answers diverge, it is too early to send the strategy into real trading.
We start with parallel validation, not rewriting. The same historical segment, the same entry and exit rules, the same fees — both systems calculate metrics independently, and we look not at absolute return but at trade-by-trade discrepancy.
The difference in return per bar immediately shows where the models diverge: accounting for slippage, order latency, handling gaps in data, or position calculation order.
Integration into the current process happens on the side and does not stop the team.
The off-the-shelf solution continues running current research, while the custom loop connects to the same data exports and stores results in a separate repository — first on one or two strategies, with daily report comparison. If the loop fails or calculates longer, the research process does not notice.
The transition to a custom loop is not instantaneous but criterion-based.
We consider it complete when, on the selected historical window and across several market regimes, the discrepancy with the off-the-shelf solution falls within the agreed tolerance, and a repeat run produces the same result.
After that, the off-the-shelf solution remains a fast rough filter, while controversial strategies are validated only in the custom backtesting framework.
Common Technical Doubts Before Starting Work
Doubts at the start are usually the same: will the execution model match real trading, how will a custom backtesting framework integrate into already written strategies, and what happens after handover. Below are answers for each point.
How accurate is the execution model?
We calculate not by the candle close price but by how the order will actually enter the order book: order type, submission moment, available depth.
Slippage, latency, and fees are parameters, not hardwired constants: the model is calibrated against your trade history and verified against the actual result.
Will the strategies have to be rewritten?
No. The framework provides an interface: you provide signals and position management rules, while the core handles execution, fee accounting, and metric calculation. Logic from production is transferred point by point and verified on a segment where the actual outcome is known.
Can it calculate on our data and our sources?
Yes. We connect your history — quotes, order book, trades — and, if needed, a real-time stream. If the data is incomplete, we record this in the report: it is better to see where the calculation relies on an assumption than to get a smooth curve without explanation.
What happens after handover — will we be on our own?
We hand over code, configurations, and calculation scenarios with a description: how to launch, reproduce the result, and add a new instrument. Post-handover support is a separate stage: we go through launch with your team and eliminate what is found.
How do we make edits and improvements later?
Through the same loop: a new execution model, metric, or data source is added as a separate module without breaking validated calculations. Configurations are versioned — the result before and after an edit is comparable, and rollback is possible.
Let's Discuss the Task: What Data Is Needed for Assessment and Plan
Assessment of such a project relies not on a general description of the strategy but on its computational profile: how many instruments are in the slice, how often decisions are made, what data is needed at each step.
Therefore, in discussing the task we first collect inputs — from them, the labor intensity is visible, and whether an upgrade of the existing testing environment is enough or a separate loop is needed.
What you send for assessment and work plan:
- Strategy logic: entry and exit rules, filtering conditions, manual exclusions, and cases where a human makes the decision.
- Data: history sources, depth and frequency, file format, whether there are order books and trade feeds, not just bars.
- Execution model: how to account for fees, slippage, submission latency, and partial order fills.
- Robustness validation: how history is split into training and validation segments, by which metrics we reject a result.
- Reporting set: which indicators and charts are needed on output, in what form, and where they are delivered.
- Constraints: access to data, infrastructure requirements, repeat run mode, and rules for reproducibility of the result.
- What already exists: current testing stack, past implementations, bottlenecks that hinder you now.
Send what you have in any working form — a draft specification, scripts, a trade export. We will review it, ask clarifying questions about the execution model, and come back with a timeline assessment and a stage-by-stage work plan.