Automatic exchange development: bottlenecks and failures
An exchange runs reliably on the first hundreds of orders, but breaks under growth.
The first bottleneck is manual confirmation: while an operator checks the payment and rate by eye, order flow grows, response time stretches into tens of minutes, and the client goes where they answered faster.
The second is rate desynchronization. The storefront calculates price from one source, execution goes through another, and time passes between quote and trade.
In a calm market this is fractions of a percent; during volatility, it is a direct loss on the quote spread, and it accumulates with every trade.
The third is stuck orders. A connection drop with the exchange, a transaction stuck in the network, partial execution — and the order remains in "processing" status.
The client sees a debit without a credit, support handles it manually, part of the funds is frozen, and trust in the service melts away.
Automated cryptocurrency exchange suffers not from a lack of code, but from a lack of recovery logic.
Therefore, development of an automatic exchange starts not with the interface, but with an analysis of the exchange's bottlenecks: how an order gets an unambiguous status and survives a service restart.
Next — how the rate is fixed at the moment of the trade and how the system itself completes the operation or rolls it back without manual intervention. Below, we break down how this is covered.
What exchange automation delivers: measurable effects
An automatic exchange is not a form for accepting orders, but a pipeline: the order, risk check, liquidity selection, execution, and withdrawal for each asset run without an operator.
Order processing time stops depending on the shift operator and is determined by queue depth and the number of confirmations in the network — on liquid pairs, this is seconds; on rare assets, the limit is set by the chain itself, such as Bitcoin, not by our side.
We measure the median and 95th percentile for each pair separately, so the bottleneck is visible before the client notices it.
What changes in measurable values:
- share of manual operations drops from "every order" to isolated reviews — only disputed and large amounts requiring manual confirmation;
- service availability moves into the range where scheduled maintenance and node restart do not interrupt order intake;
- an entire class of failures disappears: order loss during a connection drop, double execution on resend, a stuck transaction without status;
- rate discrepancy between internal calculation and the liquidity source is closed automatically, not by manual correction.
Exchange fault tolerance rests on three things: idempotency of operations, so a repeated request does not create a second trade; separation of the hot withdrawal circuit and cold storage; automatic pause on a pair when balance or rate discrepancy exceeds a specified threshold.
Every action is written to a log with the order ID, so recovery after a failure is replaying the log, not sorting through messages.
Next — how this is arranged in a specific architecture and which checks cover risks at launch.
Launch formats: ready-made core, custom module, hybrid
The choice of launch format is determined not by business size, but by who is responsible for order routing and how non-standard the matching logic is. A ready-made exchange core covers the standard flow: order intake, liquidity reservation, execution, withdrawal.
A custom matching module is chosen where the pricing rule cannot be expressed by configuration — for example, when the rate depends simultaneously on volume, liquidity source, and reserve level.
A hybrid exchange scheme most often wins: the core is responsible for accounting and fund security, while order routing and matching rules live in a separate service that communicates with the core through an event queue. Below — when each format fits and what you pay for it.
| Format | When it fits | Limitations |
|---|---|---|
| Ready-made exchange core | Fast launch of exchange flow, standard pairs and calculation rules | Non-standard pricing is handled by external scripts, which must be maintained |
| Custom matching module | Matching logic is a competitive advantage, full control over execution is needed | Longer time to production, higher requirements for load testing and failure analysis |
| Hybrid exchange scheme | Fast start and custom order routing are needed at the same time | Requires an explicit contract between the core and the service and event versioning |
Whichever you choose, we fix the order placement contract before work begins — otherwise a format change mid-project turns into rewriting the core.
Implementation stages: from flow audit to handover
Each subsequent iteration relies on the artifact of the previous one: without a routing scheme, reserves cannot be configured; without configured reserves, scenarios cannot be reproducibly tested. Below is the sequence of work in which we run an automatic exchange project.
- Order flow audit. We analyze the current picture: in-demand directions, how orders arrive, where they are lost, what volume the system handles during peak hours. The output is a list of bottlenecks with a load estimate.
- Routing scheme. We fix the order path from entry to execution: rules for choosing a direction, priorities, behavior when quotes diverge between venues. We agree on the scheme before code — disputed points surface on paper, not in production.
- Reserve setup. We determine how much liquidity to hold in each direction, how to align balances, and what to do during a drawdown. The thresholds after which the exchange stops accepting orders instead of executing at a loss are described in advance.
- Scenario testing. We run the system on historical and synthetic data: sharp market movements, partial execution, unavailability of an external venue. Each scenario ends predictably — with a refund or a correct rejection.
- Regulations and monitoring. We write instructions for operators: how to read the monitoring panel, what to do when an alert triggers, how to manually stop a direction. We set alerts for balance deviation and increased execution time.
- Handover to operation. We deploy on your infrastructure, run a control exchange, and hand over access, the scheme, and documentation. We stay in touch during the first weeks — adjustments are based on real order flow.
Exchange configurations, regulations, and documentation
After the exchange launches, the client retains a complete set of materials that can be used to maintain and develop the system without us.
We hand it over with an inventory, versioning, and labels for "what changes when the core is updated" — so a new engineer or contractor does not have to figure out the exchange's behavior by trial and error. Below is what exactly is included in the set.
- Exchange documentation — architecture diagram, data model, order lifecycle from creation to final status, update procedure, and rollback to the previous build.
- Integration diagrams — flows between the exchange core, rate sources, cash module, and payment rails, with failure points and described behavior when each circuit is unavailable.
- Configurations — parameters for each pair: limits, rounding step, thresholds for acceptable rate divergence, fee levels. Each value states why it was chosen and what it affects when changed.
- Access policies — roles and least privileges, separation of operator and administrator duties, mandatory two-factor authentication, procedure for granting and revoking access, action log with history retention.
- Response regulations — incident levels, each person's area of responsibility, procedure for stopping a pair, enabling the backup liquidity route, alert templates, and incident closure criteria.
- Shift instructions — step-by-step scenarios: manual rate adjustment, handling a stuck order, reconciling balances and reserves, checking discrepancies between internal accounting and providers.
- Shift training — operators run these scenarios in a test circuit under our supervision, with an acceptance checklist and review of common mistakes.
The set is updated along with the system: changes to configurations and regulations go through the same version control as code.
Case study: how exchange throughput grew
What it was like before the rework
Orders were accepted through a web form, and the operator started reconciliation of incoming and outgoing payments manually — as flow grew, the queue was processed more slowly than it was replenished.
Any desynchronization of confirmations in the network produced a stuck order: the client saw "processing" status, support saw a ticket. During peak hours, median time reached minutes, and during updates the service stopped accepting orders.
What we changed
The exchange was split into independent stages: order intake, liquidity reservation, incoming transaction confirmation, payout, and final audit.
Each stage has its own handler with an idempotent key — a repeated run does not create a second payout, and a stuck stage goes into retry with escalation to support.
Container orchestration on Kubernetes enabled updates without stopping order intake and predictable scaling for peaks.
Metrics before and after
| Metric | Before | After |
|---|---|---|
| Median order processing time | 4-6 minutes, during peak — more than 20 minutes | 12-18 seconds |
| Share of manual operations | about one third of orders go through an operator | less than 2%, only disputed cases |
| Incidents with stuck orders | 8-12 per month | 0-1, closed by auto-refund |
| Downtime during updates | up to 40 minutes | order intake does not stop |
Address and limit checks now happen before reservation, so rejections at the payout stage have almost disappeared. Idempotency removed the class of double payouts, which previously was caught only by reconciliation at the end of the day.
What to do if the exchange already runs on disparate scripts?
Disparate scripts break not from load, but from state: each has its own order accounting, its own retry method, and its own interpretation of order status.
When one fails mid-exchange, no one knows whether the funds were sent, so reconciliation is done by hand, and the discrepancy surfaces only after a client complaint.
We start with an audit of the current solution: we list all entry and exit points, reconstruct the actual logic for calculating the rate, limits, and fees, and mark places without idempotency and without a unified operations log.
As a rule, migrating exchange logic affects the calculation part and routing rules, while the execution layer is rewritten — status handling, request retries, and accounting of balances and access keys.
Migration without downtime is built on shadow mode: the new system accepts the same orders, calculates the result, and writes a log, but does not send funds.
For several days, its responses are compared with what the scripts do — discrepancies are resolved before the switch, not after.
Then comes a gradual transition by direction: first one currency pair or part of the flow, then the rest.
We keep the reverse switch to the old circuit until the end of observation, and leave the scripts themselves in read-only mode until the operations history and balances reconcile.
As a result, instead of a set of scripts where every step relies on the operator's memory, there is one manageable system with a log and clear handling of disputed operations.
Common technical questions: reserves, access, failures
Engineering doubts about an automatic exchange are almost always the same: behavior under peak load, confirmation delays, key storage, and transaction limits. Below are short answers; the full breakdown with diagrams is in the project documentation.
We do not calculate liquidity reserves "by eye": the service keeps separate accounting for each asset and holds a buffer for daily turnover, while operation limits are set by policy — per pair, per client, per interval.
This dampens anomalous flow without hindering normal exchanges, and observability is built on queue metrics, rejection share, and balance discrepancies.
What happens under peak load?
Orders go into a priority queue, and entry is limited by access key. A spike stretches processing but does not produce partially executed trades: an exchange either completes in full or is canceled.
How are wallet keys stored?
The main volume is in cold storage on hardware modules of the Ledger level; the hot circuit holds only an operational reserve within the limit. Transaction signing is moved to a separate service without access to other data.
What if the network has not confirmed the transaction?
The exchange is not considered complete until the required number of confirmations, and the retry has duplicate protection. If the transaction is stuck longer than the window, it goes into a separate queue and is handled via the log, without manual searching.
How do we learn about failures before the client does?
Alerts are configured for rate deviation, queue downtime, and balance discrepancies, not only for service crashes. The operator on duty receives a signal before the problem becomes visible in orders.
Let's discuss the task: what is needed for an estimate and work plan
For the exchange project estimate to be accurate rather than "by eye," we need a snapshot of the current state: how the exchange is arranged now and what it should become. Below are the inputs that most often determine the scope of work and launch timeline.
If there is no clarity on some point, that is also an answer: we include it as a separate stage in the plan.
- Exchange directions and demand: which pairs and corridors generate the main turnover, how many orders pass per day and during peak hours.
- Fund movement: how assets are accepted and issued, whether a person is involved in confirmation, and where the bottlenecks of manual operations are.
- Liquidity: own reserve, external providers, what happens to the rate and the order if the main source fails.
- Keys and storage: where funds are held, who has access, how the roles of operator, administrator, and support service are separated.
- Checks and rules: customer identification requirements, anti-money laundering controls, limits, and stop lists.
- Integrations: what the exchange must exchange data with — payment acceptance, analytics, support, accounting.
- Launch constraints: jurisdiction and regulator requirements, the date by which the exchange must be operational, who is responsible for acceptance.
Based on this data, we come back with a fund flow diagram, an estimate, a stage-by-stage work plan, and realistic launch timelines — clearly marked as to what is done first and what can be moved to the second phase.
Describe the task in free form: even if some answers are still in question, this is enough to start the discussion and propose a first version of the architecture.