dApp Backend Development: Where It Breaks Under Load
The server side of a decentralized application breaks not on business logic, but at the junction with the network.
A typical scenario: a handler reads contract events and applies them to its database, but when the same log is delivered again, the operation is executed twice — state desynchronization occurs.
Cache, database, and network data begin to show different things, and the user sees an incorrect balance where everything has already been confirmed.
The second point is stuck transactions. A transaction with an underpriced fee remains in the mempool, and the next one after it will not go through due to strict ordering by nonce: the service has already responded “sent,” although the operation has not been executed.
Without an explicit lifecycle — sent, confirmed, replaced, canceled — transaction accounting diverges from reality.
At peak load, network communication hits a ceiling: frequent node polling, the absence of a cursor by block number and a cache of confirmed data cause the indexer to lag by minutes.
The resilience of a dApp backend rests on idempotency by event key, a queue with retries, and separation of confirmed and unconfirmed reads.
All three failures do not appear immediately, but at peak — when reprocessing errors accumulate, and network latency adds up with service latency.
We break down how this is addressed in dApp backend development: where the idempotency boundary lies, how to build a transaction lifecycle, and how to limit the frequency of network calls.
The Effect of the Rework: Response Speed and Resilience
Asynchronous processing on the service side removes the main delay: instead of sequential requests to the network node on every call, we maintain a connection pool and a cache of confirmed states.
The dApp read response time drops severalfold — the client receives a response from the cache without waiting for the network. Writes go through a queue with idempotent keys, so the order of operations is preserved under any load.
Service throughput no longer hits a ceiling in a single process: we distribute work among workers, and the queue absorbs peak spikes. At a peak 5–10 times above average, the service does not return errors — only wait time grows, and even that stays within seconds.
Entire classes of failures are addressed separately. Duplicate transactions are eliminated as a class: a deterministic operation identifier and a repeated status check prevent a client retry from creating a second entry on the network.
A connection loss with the node is treated by reconnecting with block catch-up, not by restarting the entire service.
Downtime duration also changes: we roll out updates one instance at a time, state is restored from a snapshot and log, so the interruption is measured in seconds, not minutes.
We build on Node.js — the non-blocking model provides many simultaneous connections without increasing latency. Before implementation, we collect measurements by load profile; after, we compare the same metrics, so the effect is visible in numbers, not in words.
Options for Building a dApp Backend: Comparison
A dApp backend communicates with the network through one of four architectures, and the choice between them determines what breaks first: availability, response speed, or data accuracy.
Below is a comparison across two questions: when the option is appropriate and what limitations must be consciously accepted.
| Option | When It Fits | Limitations |
|---|---|---|
| Own network node | Need control over the version and privacy of requests, predictable load | Maintenance: synchronization, disk space, updates, monitoring lag from the chain head |
| External access provider | Fast start, peak spikes, rare heavy queries | Dependence on third-party limits and availability, some methods are closed, request audit is impossible |
| Hybrid scheme | Main traffic on your own node, backup and heavy queries — on the provider | More complex configuration: two paths, failover logic, the point of failure does not disappear |
| Event indexer | Need historical queries and aggregates instead of scanning blocks | Does not replace network access for sending transactions, requires reindexing after reorganizations |
On networks of the Ethereum level, the working minimum is your own node for writing and an indexer for reading, while an external provider is connected as a backup path.
Such an architecture maintains response time on historical queries and prevents a single configuration error from bringing down the entire service.
Work Stages: From Schema Audit to Project Handover
The order of stages is set by dependencies: each step relies on the artifact of the previous one. As long as there is no data flow map, service configuration turns into guesswork — it is unclear what to cache, which network calls to retry, and which to consider unrecoverable.
- Backend audit. We look at where keys are stored, how operations are signed, what is offloaded to queues, which errors recur in logs. The output is a list of bottlenecks by failure impact.
- Data flow map. We capture the full event path: reception, validation, sending to the network, confirmation, database write, response. It shows where there is no protection against retries and where the client will receive stale state.
- Deployment scheme. We describe environments, service roles, secret storage, limits, and retry rules — the basis both for assembly and for handover to the team.
- Service configuration. Indexing, cache, queues, and observability: metrics, request tracing, alerts on balance discrepancies.
- Load testing. We build the profile from real scenarios: read peak, write spike, network node failure. We look at behavior under degradation, not maximum requests.
- Acceptance. Criteria are agreed in advance: share of confirmed operations, response time at peak, rollback correctness.
- Handover and support. Repository, deployment scheme, update procedure, access — and then we stay in touch for incidents.
Deliverables: Schemas, Configurations, Procedures
A decentralized application backend outlives the team that built it. That is why, along with the code, we hand over a layer of configurations and procedures by which the service can be deployed, updated, and maintained without its authors. Here is what remains in the client’s hands.
- Service configurations — environment templates for each component, connection parameters for network nodes and queues, retry and timeout values, separate profiles for test and production environments.
- Data flow schemas — the path from a client request to an index write: where the transaction is signed, where we wait for confirmation, how a discrepancy during chain reorganization is handled.
- Key access policies — what lives in a hardware module, what lives in secure storage, who is authorized to initiate signing, how rotation and an access log are structured.
- Operational procedures — which metrics the on-call engineer checks first, at what thresholds an alert fires, in what order to act when a node lags or a queue is stuck.
- Upgrade documentation — the order for rolling out contracts and services, preparation and rollback of state migrations, compatibility checks with already deployed versions.
- Acceptance checklist — scenarios that the client runs independently before signing, with the expected result for each item.
Each artifact is versioned together with the code: an edit to an access policy or flow schema goes into the documentation in the same change, rather than as a separate task for later.
Case Study: Eliminated Stuck Transactions and State Desynchronization
The client came with a problem typical for dApps: the service sent transactions and waited for confirmation in the same process where user requests were handled, and the network was Ethereum.
During chain reorganization and retries, operation sequence numbers diverged, sending was duplicated, and some transactions remained pending — up to 7% of operations required manual review, and the team spent days on reconciliation instead of product development.
We reworked the backend on Node.js into three layers: request intake, a sending queue, and a separate confirmation watcher.
Each operation was assigned a key by which a retry does not create a second transaction, and the service state is now assembled from network events at startup — the last processed position is stored in storage, so after a crash, recovery happens automatically, without manual reconciliation.
Next came load testing: we fed a stream of operations several times above peak, with connection drops and node delays, and observed how the queue, retrier, and confirmation logic behaved under pressure.
| Metric | Before the rework | After |
|---|---|---|
| Stuck sends | 6–8% of operations | 0.2% |
| Transaction duplicates | regularly, manually | 0 |
| Recovery after a crash | manual reconciliation, tens of minutes | automatically, less than a minute |
| Service response, 95th percentile | seconds | 6 times faster |
We removed the most unpleasant class of failures — silent stuck operations that you learn about from the user, not from monitoring.
What to Do If the Backend Is Already Deployed by a Contractor?
Someone else’s deployed architecture is always an information deficit: there is code, there is no documentation, and access and keys are scattered across different people.
Therefore, as a first step, we do not rewrite the service but map the system — which processes hold transaction signing, where state is stored, which services collect operation history, and what will happen if the network node fails. Without this map, any contractor replacement turns into a product outage.
Auditing someone else’s solution means testing behavior during failures, not reading source code.
We look at how network rollbacks and repeated confirmations are handled, protection against duplicates on resend, whether the state in the database matches the state on the network after a service crash, and how sending order is arranged.
A separate item is where the signer’s keys are stored, who has access to them, and what will happen if they leak.
The phased transition begins with a parallel architecture in mirror mode: it reads the same events and calculates the same balances, but does not send transactions.
We compare the results with the working solution by checksums; when discrepancies are zero, we switch writing to the new signer and keep the old one as a rollback point. The contractor change happens without a downtime window.
We maintain compatibility with the current architecture at external boundaries: address formats, event schemas, and public interfaces remain the same, so the frontend and partner integrations do not notice the switch.
We migrate storage via dual writes — first we write to both places, then we move reads and disable the old one. Every step is reversible.
Describe what is already deployed and where it hurts — we will come back with a stage-by-stage estimate and a migration plan.
Common Technical Doubts Before Starting Work
Before starting work, technical leaders ask the same questions: where signing keys are stored, what will happen if the Ethereum node is unavailable, how the system survives reprocessing a transaction, and how correctness is verified at each stage. Below are the answers in the form in which we record them in project documentation.
Where are signing keys stored?
Private keys do not end up either in the repository or in the service’s environment variables. Signing is moved to a separate component — a hardware module or an external signing service.
The application keeps only the key identifier, and rotation and revocation are performed without rebuilding or restarting.
What happens if the network node is unavailable?
The backend maintains a pool of providers and switches on timeout or data discrepancy between them. The outgoing transaction queue is written to disk, so after connectivity is restored, sending continues from the same position.
How is transaction reprocessing excluded?
Each operation is assigned an identifier, and the confirmation flag is saved before side effects. The repeated arrival of the same hash is recognized and does not trigger a second credit — this closes double debits on repeated sends and chain reorganization.
How are the work stages and correctness checks structured?
Work proceeds in short stages: flow schema and contracts, signing service and queue, integrations, load tests. Each stage is closed with automated tests, including scenarios of node failure and repeated delivery.
Let’s Discuss the Task: What to Send for an Accurate Estimate
To come back with a timeline estimate, work plan, and team composition rather than general promises, we need a minimum set of inputs.
The more precisely the contract logic and its environment are described, the fewer assumptions remain in the estimate — and the fewer edits are needed already at the integration stage. What to send:
- Business logic: which entities live in the contract, which calls come from the interface, where administrative rights and roles are needed.
- Target networks and mode: mainnet or testnet, whether cross-network transfers are planned and along which routes.
- Data reading model: which events we index, which queries the interface needs, and what history depth it expects.
- Access scheme: how the user confirms login — wallet connection, message signature, or linking to an existing account.
- Expected load: how many writes to the chain and read requests per minute, whether sending queues and retries are needed.
- Key requirements: where signing keys are stored, who initiates the operation, whether a multi-confirmation architecture is needed.
- Infrastructure inputs: where the backend and database are hosted, whether there are requirements for logs, backups, and service monitoring.
If some inputs are not yet available, that does not block the start: we clarify half in conversation, and record the rest in the work plan as open questions.
Describe the task — we will discuss it and come back with a timeline estimate, a breakdown by stages, and a team composition for the specific scope.







