GAN Generation of Market Scenarios: Why Tests Do Not Catch Risk
The classic failure in projects with synthetic market data looks the same. The adversarial pair is trained on the history of a calm market: there are few gaps, volatility changes gently, and distribution tails are almost absent.
The output is smooth series — they look plausible on a chart and pass basic statistical checks without objections.
Then stress testing of strategies becomes a formality. The model shows a stable profile across thousands of generated trajectories, because all of them are variations of the same calm regime.
The discriminator stops distinguishing synthetic data from real quotes, similarity metrics grow, and this is taken as a sign of quality. In fact, the generator has learned the average regime and has unlearned how to reproduce rare events — exactly those for which it was built.
The cost of the error shows up not in the report but in real trading. The very first liquidity gap or stop cascade produces unrealistic scenarios with the opposite sign: a strategy that passed all checks falls apart on live positions, not on paper.
The collapse in generator quality is almost always explained by three causes: a narrow training data window, no validation on distribution tails, and metrics that measure the smoothness of the series instead of realism.
Plus the lack of control for overfitting to a specific strategy — the generator imperceptibly tunes to it, and the test loses its meaning. We analyze how this is closed.
Measurable Effect: Coverage of Rare Regimes and Strategy Robustness
The effect of generative scenarios is measured not by the beauty of synthetic series but by three groups of numbers: coverage of rare regimes, strategy robustness under regime change, and reproducibility of checks.
Before implementation, the test environment was assembled manually — an analyst cut several episodes from history, usually the last crash and a couple of calm years; other regimes did not make it into the checks.
The generator is trained on exchange archives and quote feeds (Refinitiv and internal sources) and synthesizes exactly what is scarce in the sample: liquidity gaps, correlation changes, prolonged trends with rare pullbacks.
The share of manual scenarios in the validation set falls from about 70% to 10–15%, and coverage of rare events grows from isolated episodes to several dozen per horizon.
Strategy robustness is tested on a regime shift: the same set of rules is run on historical and synthetic windows, and the result is compared by drawdown, share of profitable periods, and behavior when liquidity is absent.
What is visible is not an abstract “the strategy works,” but the specific regime in which it breaks and what buffer is needed to survive it.
Reproducibility rests on fixing the generator initialization parameters, dataset version, training configuration, and scenario selection thresholds.
Any run can be repeated a month later and yields the same set of checks — metric dynamics stop depending on who exactly assembled the tests and how they delineated the history.
Model Training Approaches: Comparing Regimes and Limitations
The training mode determines which scenarios the generator is capable of producing at all, so approaches should be compared by coverage of rare events, not by the plausibility of a single trajectory.
In practice, the modes do not replace each other: training on historical data sets the base distribution, an agent-based market model adds reaction to participants’ actions, and a hybrid approach keeps the economic and regulatory boundaries of the series.
| Approach | When it fits | Limitations |
|---|---|---|
| Training on historical data | There is a long clean history for the instrument, and the market regime is stable | Reproduces only regimes already experienced, rare events are underlearned, tails are smoothed |
| On top of an agent-based market model | A response to participants’ actions and changes in trading rules is needed | Agent calibration error is transferred into scenarios, and computation time grows |
| Hybrid of rules and learning | Part of the dynamics is described by regulation, and part by statistics | The boundary between rule and model must be drawn manually and revised |
| Fine-tuning on client data | There are proprietary trades, quotes, or portfolio, and instrument specifics matter | Little data — risk of overfitting; future leakage into features must be controlled |
We combine sequentially: history as the base, an agent-based model on top, and the final layer is fine-tuning on client data with a frozen part of the weights.
Compatibility is checked not by one metric but by a set: preservation of heavy tails and volatility clustering, absence of future leakage, robustness under regime change.
If a mode fails at least one check, it does not go into production — the generator remains reproducible and explainable.
How Implementation Proceeds: Stages of Engineering Work
A generator trained on raw data produces series that are plausible on a chart and break on validation: connections between assets are lost, and the frequency of sharp moves is underestimated.
Therefore the order of work goes from data and acceptance criteria to the model, not the other way around.
- Data audit and problem statement. We check the history for gaps, breaks, and changes in market regime. We immediately describe what the generator must reproduce: volatility, autocorrelations, heavy tails of returns.
- Generator design. We select an architecture for the number of assets and the amount of history, set the dimensionality of the latent space and the method for accounting for market regime. We agree on the set of inputs — history only or also external factors.
- Training and stabilization. We maintain the balance of generator and discriminator and add regularization against memorizing historical segments: otherwise the model retells the past instead of producing new scenarios.
- Plausibility validation. Statistical checks — distribution shape, autocorrelation, volatility clustering; economic checks — preservation of connections between assets. Separately, we look at a holdout period the model has not seen.
- Model handover. We provide the weights, a generation script with a fixed source of randomness, a description of inputs and outputs, and launch instructions: the result is reproduced on your side without our involvement.
- Result acceptance and policy. We compare metrics with thresholds fixed before the start and analyze discrepancies. Then we define the retraining frequency and the metrics whose deviation triggers an update.
Handover Package: Configurations, Schemas, and Validation Policies
We assemble the handover package so that your team can reproduce training, check quality, and retrain the model without our involvement.
This is not a presentation with charts but a working set in your repository that an engineer can use to repeat the run and obtain a comparable result.
- Training configurations — generator and discriminator parameters, feature set (returns, volumes, macro indicators), history windows, random number generator seeds, stopping criterion. A run using these files is reproducible, not “roughly similar.”
- Data pipeline schema — where quotes and trades come from, how gaps, outliers, and trading breaks are processed, and where fixed snapshots for repeatable checks are stored.
- Validation policy — what we consider a plausible scenario: the shape of the returns distribution, tail events, autocorrelation, behavior on periods the model has not seen. Plus the thresholds at which a scenario is rejected.
- Model documentation — architecture, applicability boundaries, known weak points (underestimation of sharp reversals), and a list of market regimes on which the model has been tested.
- Access control — roles for data, weights, and experiment logs: who changes the configuration, who launches training, and who only reads the results.
- Retraining policy — triggers (distribution shift, growing divergence from real data), frequency, run procedure, and rollback to previous weights.
- Launch instructions — environment deployment, scenario generation, and validation against acceptance criteria.
The package remains with you and does not depend on our infrastructure: you can update configurations, change validation thresholds, and launch retraining on your own, while we step in where refinement is needed.
Case Study: How Coverage of Rare Regimes in Tests Grew
The client is a market position risk control team. The test set for the risk model was assembled manually: analysts cut windows from history and completed rare regimes with edits.
While the market was calm, this worked, but crashes, liquidity gaps, and prolonged sideways markets entered the set as isolated examples — around 8% of scenarios. Checks looked green, while in the live market the model moved into negative forecasts.
We trained a market scenario generator on the history of quotes and orders, labeling the data by regime, and added rejection filtering: a scenario entered the set only after realism checks — preservation of volatility clusters, shape of the returns distribution, absence of degenerate trajectories.
Training stability was maintained by balancing rare regimes in the sample and stopping by distribution divergence; otherwise the generator collapsed into one regime.
| Metric | Before | After |
|---|---|---|
| Coverage of rare regimes in the set | 8% | 74% |
| Share of rejected synthetic scenarios | 61% | 12% |
| Time to assemble the test set | 9 business days | 1 day |
| Training runs that ended in collapse | 4 out of 5 | 1 out of 12 |
A separate effect is on the process. A found defect is captured by a scenario and remains in the Портфель-Стресс set forever; revalidation no longer requires manual market reconstruction.
Rejection filtering stopped being a bottleneck: the engineer reviews only disputed cases, not the entire generation stream.
How Does the Neural Network Approach Differ from Statistical Models?
A statistical model specifies the distribution shape in advance and estimates several parameters — this is how the RiskMetrics methodology is structured, for example.
A trainable generator (GAN is one option) does not specify the shape but reconstructs it from data, so it captures dependencies that cannot be written into a formula: joint movements of assets, asymmetry, heavy tails.
The applicability boundary is determined by three things: the amount of history, the dimensionality of the task, and the requirement for explainability.
On a short series with a small number of signals, statistics is more robust, and its parameters can be read and defended before the risk committee.
When there are dozens of scenarios and they must be consistent with each other, manual tuning of connections turns into a separate project.
A trainable generator is needed where what matters is not averages but joint rare events: a simultaneous rise in volatility and fall in liquidity, correlation breakdown under stress, and a change in market regime.
There is one validation criterion — whether the model reproduces the distributions from crisis episodes in history and remains stable between runs.
Typical selection mistakes are training a generator on a sample that is only enough for two or three parameters; judging quality by the average without checking tails; losing scenario reproducibility after a model change.
The opposite mistake is leaving statistics where multidimensional joint scenarios are needed: trajectories come out smooth, and there are no stress storylines in them.
Technical Concerns: Training Stability and Plausibility Validation
Mode collapse is the main failure during generator training: it finds several successful trajectories and starts replicating them, while the discriminator stops distinguishing them.
The training error at the same time looks healthy, so we monitor not it but the diversity of the sample: returns distribution, share of unique paths, spread of scenarios within one market regime.
As soon as the set collapses, training is restarted with a different initial setting and a penalty for uniformity.
Plausibility validation runs on a holdout segment of history that the model has not seen. Generated scenarios are compared with real ones by risk-relevant statistics: volatility, autocorrelation, heavy tails, behavior during prolonged drawdowns.
Matching averages is not enough — we look at whether the model reproduces the rare events on which stress tests usually break.
We fix the plausibility criteria and applicability boundaries in writing and agree on them with your risk team before training starts. Internal control then proceeds according to a clear checklist, not according to the impression from a showcase demonstration.
How can you tell that the model has overfitted to a narrow segment of history?
We train on one period and validate on another — ideally on a crisis that was not in the training sample. If quality holds on the control period, the model has learned market patterns rather than a specific episode. Limiting network complexity and regularization reduce the risk of fitting to noise.
What should be done if there is little historical data?
We supplement rare history with synthetic data, train on several market regimes at once, and transfer weights from a longer sample on an adjacent instrument. As a result, the generator also works where there are almost no real observations — for example, on deep crisis scenarios.
Let’s Discuss the Task: Where We Start and What We Need from You
To make the task estimate based on facts rather than assumptions, we need a short review: which market scenarios the GAN model must produce, what history it trains on, and how you will validate the result.
Below is what to prepare on the client side; nothing beyond this will be required at the first step.
- Statement and regimes. Which scenarios we generate — trending, sideways, stress; horizon, time step, and what is considered a plausible result.
- Source data. History of quotes or trades: depth, frequency, sources, known gaps and outliers. The sooner we see real extracts, the more accurately we can estimate timelines.
- Access and approvals. Test environment, accounts, rules for working with sensitive data, whether anonymization is needed, and who signs off on access.
- Constraints. Internal and regulatory requirements: what cannot be done with the data and where the result must remain within your environment.
- Acceptance owner. Who accepts the work and by which criteria — comparison with history, statistical indicators, expert assessment by analysts.
- Technical contact. An engineer or analyst who will answer clarifying questions about data, export formats, and your infrastructure.
After that, we will come back with a work plan: stages, timelines, team composition, and points where you see an intermediate result. Project start goes quickly when data and access are ready; if not, their preparation will be the first stage, and we also fix this in advance. Describe the task — we will prepare an estimate and a plan for your case.