Bitquery API integration: on-chain data and DEX monitoring

Bitquery API integration: where the blockchain data stream breaks

The blockchain data stream looks continuous until you start reconciling it with reality.

An indexer like Bitquery returns transactions, token balances, and trades from liquidity pools in a convenient form, but between “the node wrote a block” and “a trade appeared on the dashboard” there are several layers, and each of them can lose data.

The first break is rate limits. When trade monitoring and portfolio recalculation hit the same access key, some requests are rejected, and the response is not an error but an empty list.

The handler saves the emptiness as fact — and the database permanently keeps a hole instead of a block.

The second is missed blocks and duplicate transactions. During a chain reorganization, the same transfer arrives twice with different hashes, and with cursor pagination some records slip between pages.

If you do not deduplicate by the pair “hash + index in block,” balances start adding up out of thin air.

The third is divergence from the exchange. The price from pool trades comes with a quote update delay, and arbitrage trades are executed within a single block and leave no trace in the usual dashboard.

The dashboard shows one thing, the order book another, and the divergence grows precisely during moments of high volatility.

Below we break down how this is closed: a cursor instead of page offset, re-fetching a block range, deduplication, and latency control.

What direct access to blockchain data provides: measurable effect

Direct access to blockchain data removes the gap between what is recorded on the network and what is visible in reporting. Below are four consequences that the client verifies by metrics, not by a presentation.

Completeness of trade and liquidity history

Indexing proceeds block by block continuously, not over a pre-collected list of addresses: swaps, adding and removing liquidity, and token transfers within a single transaction all land in one sample.

Gaps in history for periods when the aggregator changed its response schema disappear. The metric — the share of transactions confirmed by two independent sources — is brought to full for the analyzed pools.

A unified data format instead of manual assembly

Bitquery returns trades, liquidity, and token transfers in a single structure across all supported networks. An analyst does not need to merge several exports with different field names — discrepancies in reports go away along with manual mapping. The metric — preparing a new data slice is reduced from days to hours.

Anomaly detection speed

Streaming subscriptions deliver an event at the moment it enters a block, not on the next export.

A spike in liquidity withdrawals or a series of trades from one address are visible in the on-call channel within minutes, not by the end of the day — the reaction window is reduced several times over.

Data stream observability

Indexing latency, the share of failed requests, and response completeness are collected into metrics that show that the data has not drifted. The share of manual reconciliation falls: they check not every row, but only discrepancies flagged by the system.

Connection options: direct API, gateway, or your own export

The connection scheme is determined by two factors: how many blockchains need to be covered and how quickly data must reach the interface.

Direct access to Bitquery, a gateway with cache, your own indexer, and batch export solve different tasks and behave differently during failures and when the data schema changes.

The choice is driven by the request quota: if the same queries repeat, a gateway with cache removes load and external limits. An indexer is justified only where the query logic is unique — otherwise its maintenance outweighs the benefit. We fix the data format and failure policy before work starts so the schema does not change midway.

How implementation proceeds: from request audit to handover

We start with an audit, not with connection. We look at where network data currently comes from — a dedicated node, third-party services, manual exports — and which requests go to each channel.

At this step it is clear why reports from different teams diverge and where part of the data is lost.

Next we fix the stream schema: what we pull through Bitquery, what we leave on your side, where results are written, and who reads them. We configure access rules before the first production request — keys, roles, rate limits, and separation of test and production environments.

1. Audit of sources and requests. We analyze where data currently comes from and which requests go to each source. We calculate volume, frequency, and maintenance effort.
2. Data stream schema. We fix what we take from the external service, what remains on your side, where results are written, and how often they are updated.
3. Connection and access rule setup. Keys, roles, request rate limits, environment isolation. Key rotation is described in advance, not “sometime later.”
4. Completeness check on a reference sample. We reconcile transactions, addresses, and pair liquidity against an independent source for a control period. We either close discrepancies or explain the cause.
5. Handover and team training. We provide configurations, the schema, access policies, and a set of ready-made queries for common tasks — support does not depend on us.
6. Ongoing support. We monitor behavior on production traffic and adjust limits and rules based on actual load.

The order is exactly this: first data reliability, then automation. Until the sample reconciles with the reference, it is too early to build reports and alerts on it.

Work result: schemas, configurations, access policies

Integration does not end with working access to data. The client retains a set of artifacts that the internal team can use to deploy the environment from scratch, perform an audit, and make changes without the solution’s authors.

• Integration schema — flows from sources to storage: where responses are buffered, where aggregates are stored, which consumers read the data; it immediately shows where events are lost and where to connect a new stream.
• Working connection configurations — environment parameters, request limits, update intervals, retry rules on failures; keys are stored in a secrets store, not in the repository.
• Access policies and role separation — separate credentials for read and write, minimum necessary rights, key rotation and access revocation procedure when an employee leaves.
• Monitoring and alerting regulations — which metrics we collect (data arrival latency, share of failed requests, remaining limit), which thresholds are considered emergency, and who receives the notification.
• Operations manual — starting and stopping the environment, health checks, handling typical failures, and actions when the limit is exhausted or the response format changes.
• Decision log — why these polling intervals, storage schema, and retry rules were chosen; on the next enhancement, the logic won’t have to be reconstructed from scratch.
• Query registry — which field sets are used, what they mean in terms of DEX monitoring, and which checks confirm data correctness.

We hand over the set with a short recording: what is where, how it is updated, and what breaks with a careless change.

Case study: monitoring decentralized exchanges without data gaps

Monitoring of decentralized exchanges began with periodic polling of public nodes: every few seconds a collector pulled pool state and aggregated it into a database.

On a calm market this was enough, but during bursts of activity requests could not keep up with blocks, some events dropped out — and gaps appeared in the data stream that no one saw.

An anomaly such as a sharp liquidity skew was found manually hours later, when the price had already played out.

What we changed

We switched to Bitquery streaming subscriptions instead of polling nodes: events arrive in a continuous feed, each is deduplicated by transaction hash and goes into a buffer with redelivery if the record is not confirmed.

In parallel, we set up reconciliation with a reference source — a control sample every few minutes is compared by volumes and addresses, and any discrepancy raises a signal before it turns into a gap.

What changed as a result

The share of gaps in the stream decreased from single-digit percentages to tenths of a percent, and anomaly detection time went from hours to several minutes.

During a tenfold burst of activity, the queue smooths the peak: there is no event loss, only a small processing delay, and it is visible in monitoring rather than in a discrepancy of end-of-day totals.

How does a one-time export differ from continuous monitoring?

A one-time export is a single request for a historical slice: operations by address, trades by pair, pool state at the needed moment. History depth is limited only by what the provider returns; update frequency does not matter — one correct pass matters. Infrastructure load is predictable: a peak during the request, then zero.

Continuous monitoring works differently: streaming updates arrive via subscription and are processed around the clock. The latency between an event on the network and its appearance on your side, resilience to connection breaks, and handling of retries are critical.

A missed interval does not “catch up” on its own — it is closed by a separate backfill from history.

The choice is determined by the task, not by data volume. Bitquery provides both modes through one access, so switching is a matter of configuration.

In practice, a hybrid is more often chosen: history is pulled once, then a stream is maintained, and if there is a break, the gap is closed with a history query.

Technical concerns: limits, retries, fault tolerance

Request rate limits are not a surprise but a parameter we build into the architecture in advance. The source has a ceiling of requests per minute and a history depth limit; when exceeded, a limit rejection comes, and a naive client simply loses part of the data.

Therefore requests go through a queue with increasing delay and random jitter — peak load is spread out instead of hitting a wall.

Retries are handled through idempotency: the same request gets its own key, the result is cached briefly, and pagination uses a cursor rather than a page number — if the connection breaks, we continue from the last confirmed position, without duplicates or gaps.

If the source is unavailable, a circuit breaker trips: traffic goes to a backup node or an archival snapshot, while the queue accumulates tasks and replays them after recovery.

We verify data reliability crosswise: we reconcile key values across two independent sources, monitor the continuity of the block chain, and separately handle network rollbacks so that operations from canceled branches do not end up in the report.

How do you make sure the data is not distorted?

Each record is tied to a block hash and confirmation count. Discrepancies between sources are not overwritten but go into a discrepancy log — disputed operations are flagged and investigated separately, before they affect monitoring or calculations.

What about access keys and permissions?

Keys are stored in a secure vault, issued on the principle of least privilege, and separated by environment. Key revocation or rotation does not break collection: replacement happens on the fly, and unfinished requests are retried with the new access.

Let’s discuss the task: what to prepare for the first conversation

An estimate for Bitquery API integration rests not on the amount of code, but on how precisely the data stream and its requirements are described. Below is what removes most questions before the first meeting and allows us to return with a work plan, not with clarifying emails.

• Task statement. Which process we are covering: a one-time wallet history analysis, continuous on-chain monitoring, or enrichment of your own analytics. What is currently done manually and where data is lost.
• List of data sources. Networks and contract addresses, the set of events of interest — trades, adding and removing liquidity, token transfers — and whether history backfill from a certain depth is needed.
• Target monitoring indicators. Latency from block appearance to signal, rule trigger threshold, which addresses and pairs fall under observation, and what we consider a false positive.
• Access for audit. Read-only keys, a test environment, the current storage schema, and the format in which data must arrive to you.
• Where we deliver the result. A monitoring panel, notifications to a messenger or email, writing to your database, regular exports — the data ingestion architecture depends on this.
• Expected timelines and stages. When the first working data stream is needed, whether there are external events with a fixed date, and what must already work in the first iteration.

Such a set closes compatibility, load, and rollback questions in advance, so the plan comes out realistic. Describe the task in free form — that is enough for us to return with an estimate, stages, and scope of work.