Turnkey logging setup for crypto bot trades with reporting

Why crypto bot logging falls apart on the first failure

The bot sends an order, the exchange does not respond longer than the specified interval, the position remains open — and the log has one line about a connection error.

Then reconstruction from indirect signs begins: balance reconciliation, account statements, matching event times. Such failure diagnostics take hours, and it still does not answer the main question — was there a trade or only an attempt.

The reason is that events are not written as an end-to-end stream but in parts: strategy signal in one place, execution response in another, position state in a third.

When one participant fails, neighboring records lose connection, and loss of trades ceases to be a verifiable fact. Crypto bot logging in this form records the fact of failure but not the chain of decisions that led to it, so every trading bot failure has to be explained anew.

A separate bottleneck of reporting is the record format. Free text is convenient for humans but not amenable to grouping: you cannot calculate the share of rejected orders, the time from signal to confirmation, or the frequency of repeats for one instrument.

Until these values are derived from the log, the argument about whether the market or bot logic is to blame remains an argument about tastes.

Trust in a strategy rests on reproducibility: from the log, a trade should be assembled from signal to position close — with order ID, timestamp, and configuration version. Below we examine how this is addressed.

What changes after implementation: failure analysis and history completeness

Before logging setup, a bot failure looked like this: the order went to the exchange, the response was not parsed, the position remained open, and the logs had one line without an ID.

Analysis of such a case took hours: we had to reconcile exchange statements, wallet balance, and messenger correspondence to restore the order of events.

We move logging to an event model. Each action — signal calculation, order placement, confirmation, partial fill, cancellation, position recalculation — is written as a separate record with an end-to-end correlation ID, timestamp, and snapshot of input parameters. Records go into an immutable log, on top of which trade reporting is built.

Completeness of trade history no longer depends on whether the bot managed to finish writing the file before restart: the record is confirmed before moving to the next step, so no gaps remain in history.

Incident analysis time drops from hours to minutes — by ID, the entire chain from signal to execution is retrieved, including exchange responses and timeout exceedances.

State reproducibility is provided by replaying the log in a test environment: bot behavior on historical data repeats step by step.

This eliminates a whole class of failures — state loss after restart, duplicate orders, and “stuck” positions, which previously were found only by balance discrepancy — and directly increases bot resilience. The monitoring panel in Grafana shows discrepancies in real time, not retrospectively.

Recording modes: full stream, event sampling, errors only

Recording mode determines what can generally be restored during incident analysis. Full stream saves every event — orders, exchange responses, balance and state changes, so storage volume grows linearly with bot activity.

Errors-only recording gives a minimal trace: the exception is visible, but the chain of decisions that led to it is not.

In practice, modes are not chosen one at a time: full stream is enabled on a short window around a release or incident, event selection is kept permanently, and errors are written with extended context — input data and response body.

Such a combination gives analysis without unlimited storage growth. We configure event selection rules for a specific strategy: trades and rejections go into the log, not every intermediate calculation.

Implementation stages: from scheme analysis to handover of procedures

Trade logging in a bot is not outputting lines to a file but an evidence loop: what the bot received, how it calculated, what it sent to the network, and what was confirmed. Without it, a disputed trade can neither be restored nor defended. Work proceeds in stages, and each stage has a verifiable artifact.

1. Bot audit. We analyze the architecture and find points where events are born: order receipt, calculation, transaction sending, confirmation. Output — a flow map and a list of places where records are lost: asynchronous handlers, retries on failures, manual interventions.
2. Logging scheme. We define event composition, order, account, and transaction IDs, mandatory fields. We fix the record format and the rule by which an event is considered complete rather than partial.
3. Configuration. We embed recording into code and environment: buffering, rotation, sending to storage, behavior when the receiver is unavailable — events wait for sending rather than disappear.
4. Record integrity verification. We reconcile the log with actual balances and network history, check sequence continuity and tamper protection. We analyze discrepancies to root cause rather than attributing them to failure.
5. Reporting. We assemble slices from the same records: trades for the period, fees, share of confirmed operations, incidents. Manual reconciliation in spreadsheets is no longer needed.
6. Operational procedures. We hand over instructions: how to read the log, what to do on discrepancy, how to restore storage after failure, and who is responsible for response.

After handover, your team works with the log independently; we remain in touch when the scheme changes along with bot improvements.

Storage schemes, event selection rules, documentation

Logging implementation is not only a working bot but also a set of artifacts by which your team can develop and verify the system without us. Below is what exactly we hand over to the repository and knowledge base at acceptance.

• Log storage scheme — where hot and archive records are stored, by which keys they are laid out, retention period for each event class, rotation and compression rules.
• Event selection rules — which trades, rejections, and order states are written in full, which are aggregated, and which are counters so the log does not grow excessively.
• Field and event dictionary — description of each field, types and allowed values, data source; queries are written from it and reporting is built.
• Access policies — roles and rights to read the log, procedure for issuing and revoking keys, what is recorded in the audit of access to the logs themselves.
• Exports and reporting — composition of summaries, generation frequency, where files are delivered, and how export completeness is verified.
• Discrepancy analysis procedure — what is considered an incident when reconciling the log with trade history, who analyzes it and within what timeframes.
• Bot documentation — configurations, extension points, update and rollback procedures, recovery after failure; plus a set of recorded events for verifying analysis after changes.

We hand over in the form in which the artifacts are used: configurations — next to code, schemes and procedures — in the team knowledge base. With them, a new engineer sets up a stand and repeats analysis of past trades without contacting us — this is the acceptance criterion.

Implementation case study: how discrepancies in reports were eliminated

The client came with a bot on three strategies: in the daily export, the number of trades did not match the exchange statement, and the discrepancy reached 7% of volume.

The cause was not in the strategy — some events were lost between order sending and execution confirmation, and partially filled and canceled orders appeared in the report as fully closed.

Trade reconciliation was done manually from the account export, so the discrepancy was found several days later, already without understanding at which step it arose.

We moved the bot to end-to-end recording of each order event.

The record key is order ID plus execution ID: repeated confirmations and duplicates on reconnection are filtered by this key, and state transitions (created, sent, partially filled, filled, canceled) are stored as separate append-only log entries.

The log is stored in ClickHouse, on top of it hourly reconciliation compares volume and fees with the exchange statement, and discrepancies go into a separate list with trade context — any operation can be returned by key.

Metrics before and after:

• share of trades confirmed by reconciliation: 93% → 100%;
• volume discrepancy per day: up to 7% → zero beyond rounding;
• time to find the cause of discrepancy: hours of manual export → less than a minute via the log;
• correctly reflected partial and canceled orders: about 60% → 100%;
• report: daily slice matches the exchange without manual corrections.

We separately addressed rollback: the log is immutable, so after failure the record is restored from the last confirmed point rather than rebuilt.

Now the bot reports to the owner using the same figures that the exchange sees — the discrepancy is visible at the moment it appears, not at month end.

How does crypto bot logging differ from exchange journal?

The exchange trade journal records only what reached the venue: accepted orders, executions, accruals. It does not answer why a decision was made — signal calculation, limit checks, rejected order, connection break remain off-screen. If the error is in strategy logic, the venue journal will show the consequence but not the cause.

Own history is written on the bot side and preserves the intention before order sending: input data, configuration version, rejection reason, portfolio state at decision time.

Due to this, data reconciliation becomes reproducible: the bot event is matched with execution by order ID and timestamp in a single zone.

Therefore we maintain both streams and link them with regular reconciliation. A discrepancy between the own log and the venue journal is a separate class of incidents: without it, it surfaces only in reporting, when the course of events can no longer be restored.

Record loss, volume growth, and access: common questions

Log volume is the main concern when enabling full logging: we write every event, every order and its execution, so in a month more data accumulates than in the entire rest of the database.

This is solved not by “writing less” but by separating tiers: a hot window with fast access for the last weeks, then a partitioned archive with compression and its own retention policy. Records are marked immutable, so neither volume growth nor cleanup creates gaps in trade history.

Access to records is built by roles: traders and support see their trades, the analyst sees an export without sensitive fields, audit sees the full trace with a note of who read and when.

Private keys and exchange accesses do not leave your infrastructure — we work with the log, not the wallet.

Will logging slow down the bot or become a point of failure?

Recording is asynchronous, into a buffer, and does not block the trading cycle: if the log is unavailable, trades continue to execute, and events are queued for resending. Such a scheme eliminates the class of failures when storage failure stopped trading.

How to verify that records are not tampered with?

Each record is linked to the previous one, and daily slices are fixed with a checksum — retroactive edits are immediately visible. Audit export is delivered together with verification values, so the report can be confirmed rather than simply taken on faith.

Let's discuss the task: we'll assemble a trade recording scheme for your bot

An accurate estimate comes not from “plus or minus” but from analyzing your scenario: which events the bot considers a trade, where position state is stored, who reads reporting and why.

Let's discuss the task at this level — after the conversation you will have a list of work, not general words.

• Event list: opening, partial close, cancellation, fee, transfer — what exactly goes into the log and with what detail.
• Data sources: bot's own logs, trading venue responses, internal database — where we reconcile discrepancies and what we consider truth.
• Storage scheme: table structure, indexes, retention period, separation of fresh and archive records for fast querying.
• Reporting format: slice by days, instruments and pairs, export to table or file, who performs reconciliation and how.
• Behavior on failures: repeated record delivery without duplicates, recovery after connection loss and bot restart.
• Access and security: who sees the log, which fields are masked, where keys are stored and how their rotation occurs.
• Implementation plan: stages, checkpoints, acceptance criteria, and rollout procedure without stopping trading.

Based on the conversation, we will assemble a technical specification that fixes log composition, storage scheme, and report type.

Describe the task — we will return with a work estimate and implementation plan, and also propose a first stage that delivers a working trade log and first reconciliation.