Cosmos SDK Development: Where the Prototype Fails Under Load
An application network prototype is usually assembled on several validators and a synthetic transaction stream.
In this mode, Cosmos SDK produces steady blocks and predictable confirmation—and creates the impression that all that remains before mainnet is to change the configuration.
When the number of validators grows and the stream becomes real, behavior changes qualitatively, not quantitatively.
The first bottleneck is consensus. A round requires an exchange of proposals and votes among all participants, so as the validator set grows, round time increases rather than remaining constant.
Blocks fill to the limit, the transaction queue does not clear between blocks, confirmation latency moves from seconds to tens of seconds, and fees become unpredictable for the user.
The second is the lack of failure isolation. A heavy handler in one module consumes an entire block time: other transactions wait, state grows, lagging nodes cannot keep up.
Version desynchronization on some validators or an error in module logic halts block production—network downtime requires a coordinated restart and manual actions by operators.
The third is operational limits: state growth, synchronization time for a new node, lack of an update plan without downtime. This manifests after launch, when the configuration is fixed. We discuss how this is addressed at the design stage.
What a dedicated network provides: predictable load and failure isolation
An application network on Cosmos SDK is a separate blockchain for a single task, not a section within a shared one. Resources, rules, and update cadence are tuned to a specific scenario, so a spike in one service does not affect neighboring ones. Below are four effects that are visible in operations, not in a presentation.
Failure isolation
A node crash or overload of one service stops only that service: the rest of the network continues accepting transactions, and recovery is done by replacing the node from a prepared set.
We separate critical circuits across different validator sets, so the failure class of “one overload—the whole product” is eliminated.
Predictable throughput
Block space is not shared with other applications, so confirmation latency can be calculated in advance. We set the block limit and operation cost according to the load profile, measure the share of rejected transactions at peak, and keep it within an agreed threshold.
Independent validators
Block confirmation is distributed among independent validators: there is no single point that can stop the network or silently rewrite history.
The consensus threshold and set composition are fixed in the configuration—this also becomes the basis for audit and regulatory requirements.
Network update time
Logic updates go through a governance proposal and are applied at an agreed block: the timeline is measured in days, not quarters, and does not require network downtime. The rollback scenario is also described in advance—returning to the previous state via a proven procedure.
Network architecture options: isolated, application-specific, shared
Architecture determines who pays for the load, how logic is updated, and how open your assets are. The three options below are not an abstraction but real forks when launching a network on the current Cosmos SDK branch.
| Option | When it fits | Limitations |
|---|---|---|
| Isolated network | Closed circuit: asset accounting and settlements within a single product | External smart contracts are unavailable, integrations only through your own gateway |
| Application network | You need your own logic, fees, and transaction inclusion order | Validators, updates, and monitoring remain on your side |
| Shared network | Fast start, liquidity, ready-made tools | Transactions share a common queue, logic is limited by smart contract capabilities |
Selection criteria
We calculate three values: peak load, frequency of rule changes, and need for external smart contracts. If rules change monthly and you do not control validators, a shared network saves months at launch—at the cost of managing fees and transaction ordering.
What should not be moved to your network
Personal data, signing keys, and heavy computations remain outside the blockchain. Only what requires coordination between participants goes into shared state: rights, balances, settlement results. Everything else is cheaper and safer to keep in conventional infrastructure.
Development stages: from network design to handover to operations
Appchain development is not “writing modules” but a sequence of stages where we verify the result of each before moving to the next. Below is the workflow our team follows: from requirements analysis to handover of the network to operations along with training for your team.
Workflow
- Requirements analysis. We determine which operations really need to live on the blockchain, who acts as a validator, and what data goes to external systems. The output is a list of modules and acceptance criteria by which we later hand over the network.
- Network design. We design the topology and roles: validator nodes, archive and public nodes, access, and state backups. We agree on the design before the first line of code—changing it later is noticeably more expensive.
- Module configuration. We assemble the application from ready-made modules and write custom ones: governance parameters, operation rights, and fee logic. Everything that can change is moved to governance rather than code.
- Test network assembly. We stand up a local circuit for fast iterations and a public testnet where scenarios are run on real validators. We catch configuration errors here, not on the production network.
- Load testing. We run a transaction profile close to the expected one and observe behavior at peak: block inclusion time, state growth, node lag, and recovery after restart.
- Production rollout and handover. We launch in stages—from a limited circle of validators to the full set. We hand over update and recovery procedures, configure alerts, and train your team.
What remains with the customer: configurations, documentation, procedures
The deliverable package is not an archive with source code but a set of artifacts that your team can use to continue working without us. We hand them over along with a walkthrough: we discuss each decision on a call and document the reason the parameter is set exactly that way.
- Network configurations — startup files, node parameters, and values changed from defaults, with an explanation of what will happen if they are edited.
- Network design — topology of validators, full and public nodes, connection points, failure zones, and recovery procedure if a segment is lost.
- Roles and keys — a registry: which key belongs to whom, where it is stored, who has signing authority, and how rotation is performed.
- Access policies — least-privilege access separation, separation of network administrator keys and application keys, and hardware storage of critical keys.
- Update procedure — the path of change from testnet to mainnet: who prepares, who votes, the maintenance window, and rollback criteria to the previous state.
- Instructions for the on-call team — what to monitor, which thresholds to treat as alarming, and what to do if blocks stop, a node desynchronizes, or a service goes down.
- Integration documentation — description of external interfaces, request formats, and load limits.
These artifacts are the acceptance criterion: if an on-call engineer can use them to bring up the network and perform an update without us, the work is accepted. We go through the first scheduled update together with your team; after that, it follows the procedure.
Case study: bringing an application network to production in two iterations
An application network operator came with a working prototype on Cosmos SDK: everything matched on the test circuit, but under a real stream transactions accumulated in the queue, and validators diverged in block height.
The task was stated as follows—keep response time, reduce the share of rejected transactions, and make the network resilient to the failure of an individual node. Below are before-and-after measurements and what had to be reworked along the way.
What before-and-after measurements showed
Initially, transaction confirmation time at peak reached 12 seconds, 4–6% of the stream was rejected, and stopping one validator delayed processing by minutes.
We separated application logic from the state coordination layer, added priorities for urgent operations, and limited resource consumption per transaction.
After re-measurement, confirmation time decreased threefold, the share of rejected transactions fell below 0.5%, and removing a node from the active set stopped affecting request processing—validator set reconfiguration takes seconds instead of minutes.
What was reworked along the way
The first iteration relied on a shared state storage scheme: as the number of accounts grew, writes began to slow down reads, and this showed up specifically during peak hours.
We reworked the key model, separated hot and archive data, and added indexes for the most frequent queries. The state migration was done step by step—without network downtime or loss of operation history.
What to do if part of the logic already runs on another network?
Short answer: migrating all logic wholesale is rarely necessary. The fork depends on how broad your logic’s interface with the other network is—if it is a single call at the entry point, that is what gets migrated, not the entire product.
Below are three scenarios that teams realistically choose between at the start, and what each means for module and data compatibility.
We check compatibility across two layers. Module layer: whether interfaces between modules match, library versions in the Cosmos SDK dependency tree, and initialization order at network start.
Data layer: state storage schema, keys, serialization format, and the genesis file from which the new blockchain is brought up. A discrepancy here does not break the build—it produces a different state after launch and surfaces during reconciliation.
| Scenario | When it fits | Limitations |
|---|---|---|
| Leave as is | The logic works; you only need your own external perimeter: validators, communication channel, access rules | Update rules and fees of another network are outside your control |
| Hybrid scheme | Some modules are tested and stable, while others require custom rules | Two states need to be synchronized; a confirmation layer between networks is required |
| Full migration | You need your own consensus rules and independent network launch | All modules are rewritten plus migration of accumulated state |
In practice, we more often use a hybrid scheme: we migrate module by module, run each on a test network, and compare behavior with the original. We keep the old network running until confirmation—this is the rollback point if reconciliation shows a state discrepancy.
Technical questions asked before network launch
Before mainnet launch, technical doubts recur: will it withstand peak load, can logic be refined after launch, and how is access to keys protected. We answer briefly—details are discussed in a technical session on your architecture.
How will the network behave as load grows?
Scaling uses two levers: block parameters—gas limit, block size, and block production rate—and adding validators with traffic distribution.
We run a model of your activity in advance and record the headroom before degradation, while monitoring reveals the bottleneck before users notice it.
Can a module be added or rules changed after launch?
Yes. Application logic on Cosmos SDK is assembled from modules, and a new module is connected through a scheduled update: nodes transition to the new version in a coordinated way at a specified block height, and accumulated state is preserved.
Rules are changed through governance—parameters take effect based on voting results, not by manually editing a server.
Where are keys stored and how is access restored?
We keep validator and treasury keys in isolated hardware-type storage, and signatures are assembled using a multi-confirmer scheme.
Recovery is based on secret sharing among responsible parties: losing one medium does not block access, and the sequence of actions is fixed in the procedure.
Let’s discuss your task: we’ll come back with an estimate and work plan
For the task estimate to be based on facts rather than guesses, we need context about the network and its scenario. Send us the inputs in free form—Cosmos SDK development rarely starts from a blank slate, and existing work significantly shortens the path to a working prototype.
- Network goal and scenario: which operations are recorded on the blockchain, who the participants are, and what must be stored in state.
- Module and tokenomics logic: token issuance and distribution, fees, administrator rights, and the procedure for updating network parameters.
- Load and topology: how many nodes and validators are expected, fault-tolerance requirements, and whether circuit isolation is needed.
- Integrations: wallets, exchange and payment services, cross-network transfers, external interfaces, and analytics.
- Current work: repository, prototype, test network, data schemas, and results of previous tests.
- Constraints and readiness criteria: timelines, regulatory requirements, infrastructure access, and who on the customer side makes product decisions.
On our side, we will come back with a stage-by-stage work plan—prototype, module assembly, test network launch, mainnet preparation—along with team composition and role allocation: who leads modules, who handles infrastructure and release.
We will specify timelines and the points where your decision is needed so the project does not stall.
Describe your task—we’ll come back with an estimate and work plan.