Why data distribution degrades without its own IPFS infrastructure
While files are distributed via public gateways and a random set of nodes, the scheme looks workable — exactly until the first traffic spike.
Under peak load, the request queue grows faster than throughput, and data delivery degradation begins: some requests fail by timeout, while others arrive with latency several times higher than usual.
The root cause is a bottleneck on the delivery side.
When blocks are pinned on two or three nodes at one provider, they share a channel and a shared limit on simultaneous connections; losing one node forces clients to wait for it longer than others, and routing leads to the slowest peer. Gateway cache helps only for popular objects and does almost nothing for the long tail.
For business, this is visible not in graphs but in audience behavior. A player, gallery, or file loads in fits and starts, some users leave before waiting for the content, repeat visits drop, and support receives complaints about "slow and not opening."
Own IPFS infrastructure removes exactly this set of problems: pinning blocks on several independent nodes, own delivery points, and cache warm-up before peak. Next, we break down how this is covered — from auditing current distribution to launch under load.
What changes after implementation: speed, delivery, resilience
After switching to a dedicated IPFS node, four measurable values change: data delivery time, share of available copies, behavior when an individual node fails, and load on the primary channel. They need to be measured before and after; otherwise, the implementation effect remains just words.
Storage fault tolerance is built on replication: each content unit is pinned on at least several distributed storage nodes, and addressing is by content, not by server address.
If a node goes offline, a neighboring copy serves the request — for the user, this is a normal download, without errors or retries. Recovery after a failure comes down to returning the node to the set, not manually re-uploading data.
| Option | When it fits | Limitations |
|---|---|---|
| Single node with a gateway | small volumes, one region, predictable audience | the node is a point of failure, all delivery goes through one channel |
| Dedicated node with gateway cache | repeated requests, stable content core | cold requests go to the origin, cache needs warm-up |
| Distributed storage on several nodes | media, catalogs, archives where availability matters | requires a replication policy and control over file pinning |
| Own nodes plus a public gateway at peaks | sharp traffic spikes, campaigns, launches | part of traffic goes outside your control |
Another effect is reduced load on the channel: heavy static delivery moves to dedicated nodes and cache, while the primary channel remains for the application and API.
The share of available copies stays above the target threshold, so delivery time stops depending on whether a particular server is alive, and applications get predictable response under load.
IPFS infrastructure designs: comparing options
The choice of design is determined by who the data consumer is and how critical delivery continuity is. A public gateway gives a quick start, an own node gives control over storage, a hybrid covers both external and internal consumers at once, and a cluster handles load. Below is what each option provides and where it stops working.
| Option | When it fits | Limitations |
|---|---|---|
| Public gateway | Prototype, internal tasks, rare requests, when own infrastructure is excessive | Delivery speed is not controlled, request limits apply, part of the data is effectively stored on someone else's side |
| Own IPFS node | Need control over role-based access, volume, and data pinning policy | Requires maintenance: updates, monitoring, disk space, on-call during failures |
| Hybrid scheme | Public distribution outward plus a closed environment for critical data | More complex routing and configuration, requires a consistent pinning policy in both environments |
| Local cluster | Large volumes, high read load, fault-tolerance requirements | Many resources for maintenance, requires data replication between nodes and automatic recovery |
In any scheme except a one-off prototype, data pinning becomes decisive: without explicit policies, the garbage collector eventually deletes blocks, and links stop opening.
Data replication on several nodes removes the single point of failure — losing one node does not interrupt delivery, and bringing a new one online restores prior redundancy within minutes.
At the task analysis stage, we check volume, read profile, and access requirements, after which we propose one scheme or a combination with a transition plan.
How we deploy: from scheme audit to handover
Work proceeds in stages, and each closes with a verifiable result — without it, we do not move to the next step. This makes it clear at what stage the project is at any moment, and rework does not surface at the end.
- Audit of the current scheme. We analyze what is already stored, who writes and reads data, what nodes exist and how they are addressed. The output is an environment map and a list of bottlenecks: duplicates, lost pins, unaccounted traffic sources.
- Design. We define the topology: how many nodes, where they are located, how many replicas per object, what we serve through the gateway and what we keep in a private network. We fix the addressing scheme and pinning rules.
- Node setup. We deploy and configure nodes: limit the garbage collector so that needed data is not purged, configure access, replication, and monitoring with alerts on pin failures.
- Data availability verification. We run scenarios: cold request response time, behavior when a node is disconnected, replica recovery. We verify integrity by checksums.
- Commissioning. We migrate real traffic, collect baseline metrics, and compare them with the calculated ones. We record a point we can return to in case of rollback.
- Maintenance procedure. We hand over documentation, the update, rotation, and backup procedure, as well as escalation contacts.
After handover, the customer receives a working environment and a clear procedure — after that, the support team runs it without developer involvement.
Delivery package: nodes, gateways, access policies, and procedures
We hand over not a "working cluster" but a kit that your team can use to operate, expand, and restore the network without us.
Before handover, configurations are taken from actually running machines, not written from memory, so there are no discrepancies between the document and the actual state of the nodes.
- Node configurations — configuration files for all roles: entry points, storage nodes, connection limits, disk quotas, garbage collection rules. Versioned in your repository, not left in chat.
- Keys and access — list of peer keys, role permission matrix, distribution and data pinning policies, rotation procedure. We hand over private keys to your storage media, and we keep no copies.
- Network diagrams — topology with addressing, external gateways, replication paths, and failure points; separately described is what happens when each segment fails and how traffic flows to the reserve.
- Administrator instructions — typical operations step by step: connecting a new node, decommissioning, re-pinning data, diagnosing slow delivery.
- Maintenance procedure — frequency of disk and replica checks, alert thresholds, update procedure, recovery plan with target time.
- Monitoring dashboards — metric sets for availability, volume of pinned data, and delivery speed, with ready alert rules for the on-call engineer.
- Acceptance protocol — running failure and recovery scenarios in your environment, with recorded results and a list of known limitations.
Such a kit removes dependence on the contractor: any new engineer gets up to speed on operations using documents, not verbal explanations.
Case: data delivery under peak load before and after
A media catalog with previews and heavy source files was served by one node — that was enough exactly until the first spikes.
When the collection went out in a newsletter, incoming traffic grew sevenfold: preview delivery went beyond two seconds, some requests were cut off by timeout.
A failed node was brought up manually — up to half an hour, and all that time some users saw empty cards instead of content.
What changed
We assembled a cluster of IPFS nodes and pinned content addresses on several independent locations so that a copy would not depend on one point.
In front of the cluster, we placed a gateway layer with cache: repeated requests for a popular preview are served from the edge and do not reach the disk. The pinning policy was separated by tiers — hot previews are kept as replicas on all locations, archival source files on two.
Node health checks run continuously; if one fails, the request goes to a live copy without on-call involvement.
What became
At peak, where timeouts used to begin, the infrastructure sustains the flow without degradation: successful delivery share grew from 92% to 99.9%, and the 95th percentile of preview delivery dropped from 2.1 to 0.2 seconds.
The failure class "node went down — content unavailable" disappeared from the known issues: independent copies and automatic failover close it entirely.
Recovery stopped being a separate operation — instead of half an hour of manual work, it takes seconds to switch traffic.
How a dedicated node differs from a public gateway
A public gateway is someone else's node that receives a request by your content identifier and decides for itself where to get blocks: from its cache, from peers, or from your node. You influence neither request priority nor delivery geography.
The consequence: time to first byte is unpredictable and depends on the load of someone else's service.
A dedicated node keeps data with you: blocks are pinned to your instance, cache is warmed for your traffic, and delivery goes directly to clients and peers. Delivery speed becomes a function of your network and disk, not a queue on the gateway provider's side.
A caching layer can be placed in front of the node and serve popular objects without accessing the network.
Access restriction is the second difference. A public gateway is open: whoever knows the content identifier gets the file, and access cannot be revoked.
On a dedicated node, access is closed at the network and application level: a separate network with a shared key, request signature verification, allowlists, and links with a limited lifetime.
Caching on a public gateway works in the service's interests: eviction rules, TTLs, and volumes are someone else's, and some requests bypass the cache.
On your own node, you set these parameters — what to store and for how long, what to warm up in advance, what to remove after it loses relevance.
Finally, dependence on an external service. Public gateways limit request rates and change rules, and a gateway failure means the entire delivery fails — you find out about it from users.
A dedicated node can also fail, but updates, backups, and availability metrics remain under your control.
Common technical doubts before starting work
Doubts before starting are usually the same: what will happen to files if a node drops, will anyone outside see the content, and will the current environment have to be rebuilt for this. Below are short engineering answers you can use to check your requirements before work begins.
What happens to data if a node goes offline?
Each object is pinned and replicated to several nodes. If one drops, the content remains available at the same address, and the scheduler restores the required redundancy on live nodes.
Can content be distributed privately?
Yes. The cluster is launched with a private network key: nodes see only each other, and content does not enter the public routing table. External access goes through your own gateway with authorization.
Will this integrate into our current environment?
The environment does not need to be rewritten: a layer compatible with object storage is placed on top, and applications continue to work through familiar calls. Deployment is possible in a closed network without external requests.
How do you make sure data is available?
Addresses are traversed on a schedule: the system requests objects from several replicas and records delivery time. Deviations go to monitoring, so degradation is visible before the user notices it.
What about backup?
We copy not only files but also the address map: without it, the restored array will not assemble into the same links. Content and metadata are exported to object storage, and recovery is tested on a staging environment.
Let's discuss the task and come back with an estimate and work plan
Estimating a project in this area does not come down to a single number: it is important to understand the data profile, access scenarios, and what is already working in the current environment.
Therefore, first — a discussion of the task: 40–60 minutes with an engineer who will lead the implementation, not a sales manager.
For a substantive estimate, send, if you have them at hand:
- data profile: file types, average and maximum size, expected monthly growth;
- access scenarios: who reads, from where, whether public distribution is needed or closed access by keys;
- privacy requirements: which datasets must not be released to the public network and whether an encrypted wrapper is needed;
- load: peak number of simultaneous requests and acceptable response time;
- current environment: servers, object storage, static content delivery, domains — what already exists and what we will reuse;
- constraints: storage regulations, what cannot be changed in a running infrastructure, hard deadlines;
- acceptance criteria: by which metrics we will consider the task complete.
After the analysis, we come back with a solution design: where data lives, how content is addressed, how replication between nodes is arranged, and what happens when an individual node fails.
The implementation plan will include stages, responsible parties, and timelines for each — with points where you can stop and verify the result. Describe the task in a convenient form, and we will come back with an estimate and plan.