Time & Sales tape: where it breaks under load
The Time & Sales tape is a stream of all executed trades: price, volume, side of the initiator.
In a calm market it looks harmless, but it is the first to hit infrastructure limits on a peak order flow, when the execution module emits events faster than they can be delivered and rendered.
Missing trades almost never mean data loss on the exchange side — delivery is lost. If the consumer does not acknowledge processing, and the queue grows faster than it is drained, the buffer overflows and some messages are dropped.
On the tape this looks like a sequence gap: volumes in the order book have already changed, but the client did not see the trades that changed them.
Stream desynchronization occurs at the junction of recovery after a connection break.
The client takes a state snapshot and catches up with it in batches of changes, but while the snapshot is being applied, some changes are already stale or arrived with a different sequence number.
Without strict sequence continuity control, the tape quietly drifts from the real state of the book — outwardly alive, but with incorrect volumes and last trade price.
Response latency and UI freezing are the third point of failure, and they live on the client side.
Rendering each trade as a separate update during a peak keeps the main computation thread busy, accumulated updates merge into a queue, and the tape stops responding to the trader's actions — until the tab is reloaded.
We explain how this is closed: acknowledgment of processing at every stage of the path, sequence continuity control, queue limiting with an explicit overload signal, and batch rendering by frames instead of an update per event.
Implementation effect: responsiveness, completeness, peak resilience
After launching our own tape, it is not the interface design that changes, but the system behavior under load.
The trade stream comes directly from the trading core and is distributed to subscribers, so data completeness and interface responsiveness no longer depend on an external provider and its delays. The implementation effect is visible in measurable indicators, not in general words.
| What we measure | Before implementation | After implementation | Verification method |
|---|---|---|---|
| Data completeness | missing trades on connection break | reconciliation converges after any break | trade counters at source and on tape |
| Interface responsiveness | dips on peak flows | smooth rendering under peak | measure from event to row appearance |
| Peak resilience | queue growth, freezes | burst is absorbed by buffer, stream does not break | replay of recorded real peaks |
| Recovery time | manual restart and backfill | automatic backfill of missing data | time from failure to counter match |
Metrics are taken continuously from two sides — on the distribution service and on the client: monitoring dashboards in Grafana, alerts on counter divergence and on recovery time growth.
A divergence of even one trade becomes a reason for investigation, because the tape participates in reconciliation and audit of trading history.
Peak resilience is checked not by eye, but by replaying a recorded stream: the service must survive a burst without losing rows and without increasing rendering latency.
After a break, the tape catches up with the stream on its own, without operator actions — this is the difference the customer sees in operation.
Tape formats: trade stream, aggregation, snapshots
The tape format determines what exactly the client receives: each trade separately, aggregated intervals, or a slice of history at connection time.
On an exchange these modes coexist, and the choice between them is a trade-off between data completeness, channel load, and delivery latency.
The trade stream goes over a persistent connection: price, volume, initiator side, and sequence ID in each message.
Trade aggregation collects them into intervals — the aggregation window is configurable from fractions of a second to a minute: wider window, fewer messages and higher reaction latency.
History snapshots cover connection and channel break: the client retrieves a slice for the required range and continues from where it stopped, without gaps or duplicates.
| Option | When it fits | Limitations |
|---|---|---|
| Trade stream | Terminals, trading robots, real-time analytics | Channel break requires re-loading |
| Windowed trade aggregation | Charts, mobile clients, monitoring | Detail is lost, exact recalculation is impossible |
| History snapshots | Connection, exports, audit, incident investigation | Not suitable for live trading, volume grows with range |
In the production environment, the modes are combined: a snapshot on connection, then catch-up via the stream by sequence ID.
That is why we agree on the tape format with the customer before starting work — both the storage schema and how the client survives a connection break without losing trades depend on it.
Work order: stream audit, schema, assembly, verification
Work proceeds in stages, and after each one a verifiable artifact remains: a source map, a model specification, a working service, a reconciliation report. This way you see progress on the task and can adjust requirements before they turn into code.
-
Source audit. We inventory venues: trade delivery method, update frequency, stream depth, behavior during peak hours. We note where data diverges from the exchange aggregator and decide in advance how to close these divergences.
-
Data schema. We describe a unified trade model: price, volume, aggressor side, timestamp, venue identifier, cancellation flag. Normalization rules bring different formats to a single contract — the tape stops depending on a specific source.
-
Service assembly. We implement subscription, buffering, deduplication, and reconnection with catch-up of the missed window: a connection break does not leave a hole in the tape. We configure publication so that the latency from trade to record remains predictable.
-
History verification. We run the service on archived data and compare it with the venue's reference tape: we look for gaps, duplicates, out-of-order entries, timestamp divergences. We investigate every divergence down to the cause — in the code or in the source itself.
-
Failure testing. We emulate activity bursts, connection breaks, node restarts, and see whether the tape recovers without loss. Based on the results, we configure signals by which the operator sees degradation before the user does.
-
Work acceptance. We hand over configurations, schema, metrics, and operating instructions, and review typical incidents with your team. After that — support: adjustments for venue changes and investigation of disputed reconciliation cases.
Handover package: services, schemas, regulations, access
Work ends not with a demo, but with a handover: the customer receives not only a working trade stream, but also everything needed for independent operation and development of the tape.
The package is assembled so that the on-call shift can bring up the service without the authors, and a new engineer can understand the architecture from documents, not from chat.
- Documentation package: description of service topology, message formats, and delivery guarantees — including behavior on connection break and reconnection with backfill of missed trades.
- Integration schemas: flow diagrams from the trading core to client subscribers, connection points, routing order, and data normalization rules by instrument.
- Operations regulations: order of planned works and maintenance windows, procedure for rolling out a new version and rolling back to the previous one, criteria for mandatory incident escalation.
- Access policies: roles and rights for production and test environments, key issuance and revocation, separation of access to the trading core and to the historical data mart.
- Instructions for the on-call shift: step-by-step scenarios for typical failures — growth in publication latency, reconnect avalanche, tape divergence from the core, channel degradation.
- Monitoring and alerts: a set of metrics (stream completeness, latency, share of dropped messages), trigger thresholds, and notification recipients.
- Acceptance protocols: results of load and fault-tolerance runs plus a checklist for verification after each release.
The volume of documentation is not "just in case": each artifact covers a specific class of failures that otherwise costs hours of exchange downtime.
We hand over the package in a walkthrough meeting, not as an archive — so that the on-call shift and your engineer can ask questions live.
Case study: peak trade flow and recovery time
An exchange with sharp activity bursts: listings, cascading liquidations, news impulses.
The average trade flow on the Time & Sales tape is steady, but at peak its multiple grew dozens of times — and it was precisely in those minutes that the tape lagged most, when data is needed by the second.
Before us, client reconnection went through a full snapshot and without deduplication by trade ID. The result was either missing trades in history or duplicates, and subscription recovery took minutes.
What we changed
We deliver the trade stream via Kafka: snapshot plus increment, deduplication by trade ID, server-side buffer during reconnection — the subscription does not break, the client catches up on what it missed on its own.
Result
A whole class of failures has been eliminated — divergence of the client tape from actual trades on the exchange.
During peak intervals, delivery latency remained within one second and stopped growing with load, and tape lag no longer requires manual intervention from the on-call shift.
How does the trade stream differ from an aggregated order book?
The trade stream, or Time & Sales tape, is an execution log: price, volume, time, and initiator side in each row. The aggressor flag shows whether the buyer took a limit sell order or the seller hit the bid — a snapshot of the order book does not show this.
The aggregated order book is about intentions, not facts: limit orders are grouped by price levels with total volume at each.
An order can be canceled before execution, so the order book is a state rebuilt from incremental changes, while the trade stream is a sequence of events that cannot be reconstructed from a snapshot.
The difference in approaches shows on the same trade. The order book answers where liquidity density stands and at what level price will meet resistance; the tape — how the volume was accumulated: by one large order or by a series of small executions at different prices.
The task dictates the mode choice. Algorithms that calculate delta and aggressor imbalance need a tape with a side flag; liquidity and slippage assessment is built on order book depth. For arbitrage and execution quality control, both streams are supplied simultaneously.
In the interface, they are carried over different channels: the tape updates on every execution, the order book less often, on accumulated changes.
With a dense flow, records are collapsed by price over a short interval — rendering stops choking, but the breakdown into individual trades is lost.
Technical questions asked before work starts
Before a project starts, customers check approximately the same things: whether the tape will survive a connection break, how trade order is confirmed, and whether the trading platform core will have to be touched for the new stream.
Below are short answers to each point; we discuss details in an architecture session together with your team.
What happens to the tape on a connection break?
The service maintains a monotonic event number and reconciles it with the source.
On a break, the boundary of the last confirmed event is recorded, the connection is re-established, and the missed interval is backfilled from the journal and order book snapshot — no "holes" or duplicated trades remain on the tape. History remains continuous even during peak volatility.
How do you guarantee correct trade order?
Sequence is set by a single time source on the exchange adapter side, not by the browser clock. A packet that arrives out of order goes into a reordering buffer and is published only after the missing one, so rows are not rearranged or "jittering" during rendering.
How difficult is it to integrate this into our platform?
We connect as a separate service to your market data channel and provide a normalized stream under a contract agreed with your team; compatibility with FIX and the broker's internal format is tested on a staging environment before release. We do not change matching and balance modules.
What about support after production launch?
A regulation remains: latency and stream completeness metrics, alerts on divergence from the exchange source, on-call duty, and a clear procedure for rollback to the previous version.
Acceptance is based on criteria — share of closed gaps, publication latency, absence of duplicates in the daily slice.
Let's discuss the task and come back with an estimate and plan
To come back with an estimate and plan, a forty-page document is not needed — input data on the task is enough.
It is important for us to understand which exchanges and trading pairs need the Time & Sales tape, where we take the trade stream from, and how the tape is connected to the order book and chart: as a single stream or separate subscriptions.
The second block is operational requirements.
We need to know the target platforms (web, mobile applications, desktop terminal), expected history depth and acceptable update latency, as well as failure scenarios: what the user sees on a channel break, desynchronization, and missing trades.
- list of exchanges and trading pairs for which the tape is displayed in the interface;
- data sources: proprietary trade calculation core, external channels, history archive;
- connection of the tape to the order book, market depth, and chart — shared stream or separate subscriptions;
- target platforms and display requirements for each device type;
- requirements for history depth and update frequency;
- behavior on connection breaks, desynchronization, and lost trades;
- constraints on infrastructure, access, and update deployment order.
We suggest discussing the task in a free format — the listed data is enough to come back with timelines, scope of work, and a clear start order.
If it is more convenient to start small, we will assemble a tape for one exchange and one pair, show the result in operation, and expand to the remaining venues.