Pinata integration for IPFS: uploading and managing files

Why files in IPFS disappear without pinning

IPFS addresses data by content: a file has no path on a specific server; it has a fingerprint that uniquely describes the bytes. A node that accepts an object places it in local storage and serves it to other network participants as long as there is demand for the content.

Demand disappears — the object turns into ordinary cache, and any node has the right to clear cache.

This is where “file uploaded” and “file available” diverge. IPFS pinning is an explicit mark “do not delete, I am responsible for availability”: it removes the object from automatic cleanup and keeps it in node storage permanently.

Without such a mark, after some time the node starts garbage collection in distributed storage and frees space for new blocks — this is not a failure, but normal network behavior.

The most unpleasant part: the link remains valid. The fingerprint in the smart contract, token metadata, image address in the web interface — everything is in place and syntactically correct. But on request, no node finds any copy and returns emptiness.

This is the loss of links to content: the code works, but files are unavailable in IPFS entirely — from the image to the JSON description.

At scale it looks like this: a collection loses all graphics at once, a marketplace cannot render cards, and recovery rests on source files that the team may no longer have.

Let us further examine how pinning works: where it is set, how it differs from one-time upload, and how to verify that an object is really alive.

What managed IPFS pinning gives in real work

IPFS addresses a file by content: instead of a path — a hash computed from the data itself.

As long as a node holds the blocks and cleanup of unpinned data has not occurred, the object is served on request; after restart or garbage collection it disappears from the network, and access at the same address returns an error.

It is not storage that breaks, but the link in a contract or metadata — the one that can no longer be changed.

Pinning fixes the address and keeps blocks with the storage provider — in our case, Pinata with a pinning policy for each uploaded artifact.

The share of unavailable objects stops being a random variable: a record either is in the pinning registry or it is not, and this is visible before release, not from a user complaint about a broken image.

The second effect is the speed of artifact assembly.

An immutable address allows fetching data from the nearest gateway cache instead of re-uploading it entirely on each build; rebuilding the same version yields the same address, so the result is reproducible and does not depend on the state of a particular engineer's machine.

The third is reduction of manual reconciliations.

The pinning registry becomes the source of truth: the report builds discrepancies between the artifact table and the actual network state, and the engineer looks only at error rows instead of manually going through the entire list before each rollout.

In sum, these are three parameters we monitor: share of unavailable objects, build time, and number of manual checks per release.

The first tends to zero, the second stops depending on the size of the dependency tree, the third is counted in rows instead of a full traversal — and it is on these metrics that one can see whether pinning paid off.

Options for pinning files: node, gateway, service

A file address in IPFS does not store the file itself — blocks live exactly as long as at least one node holds them and announces them in the distributed table.

Pinning is not “saving a copy” but a permanent obligation: the garbage collector does not touch marked blocks while the rule is in effect. Hence three fundamentally different ways to fulfill this obligation.

Key property: the file identifier itself does not depend on where it is pinned.

Therefore the options do not exclude each other but combine — the working contour is usually: pinning on your own node, a second copy in an external service, a gateway in front of them as a single entry point for applications. Migration between options does not break links.

The choice comes down to which class of failures you are ready to accept: unavailability of your own node, a change of provider terms, or a gateway bottleneck at peak. Then we look at volume, retention period and access requirements — the configuration depends on them.

Implementation stages: from audit to handover of regulations

Pinning regulations are not “uploaded the file and forgot”: each object has an address by content, and the node has a list of what it must store.

If the list is maintained manually and without accounting, some files become unpinned during cleanup, and links in the interface or token start returning emptiness. Therefore work proceeds in a fixed order — from inventory to documentation handover.

1. Audit of storages. We collect the list of already placed objects, their volumes, duplicates and owners. We count how many really unique sets there are and which of them are critical for the product.
2. Design of the pinning scheme. We mark sets by groups, determine what is stored permanently, what on demand, and how copies are distributed between nodes. The scheme is fixed before the first write, not drawn up along the way.
3. Access rules setup. Separate keys for write, read and administration, binding to groups, scope restriction and regular rotation. A leak of the publication key no longer means loss of the entire storage.
4. Upload pipeline assembly. The checksum is calculated on the service side before sending, retry on network failure goes to the same address, so duplicates do not multiply, and an interrupted upload continues from the same place.
5. File integrity verification. Periodic reconciliation of the index with the actual contents of nodes: the discrepancy is visible before the user reports it. Lost objects are restored from backup or a second node.
6. Monitoring and alerts. We monitor node availability, volume growth and pinning counters. An alert comes on unpinning or space exhaustion, not after a complaint.
7. Handover of regulations. We hand over the addressing scheme, description of rights, recovery instructions and a checklist for the on-duty engineer. Then the customer team maintains the storage itself.

Each stage ends with an artifact: a list of objects, a scheme, a set of keys, reconciliation logs. From them it is visible at which step something deviated — and this is cheaper than dealing with the consequences of unpinning critical files.

What we hand over: configurations, schemes, policies

We hand over not a “working service” but a set of artifacts by which the customer team itself changes pinning rules, switches provider and handles incidents.

We build the integration with Pinata so that all logic is described, not “locked in the engineer's head” — below is what physically remains in the repository and on the wiki after launch.

• Integration scheme — the file path from upload to pinning: where the content address is computed, at what moment the pin is set, what happens at each step on failure and how to repeat the operation idempotently.
• Node and gateway configurations — pinning parameters, timeouts, limits, access keys in secure secret storage plus instructions for their rotation.
• Pinning regulations — which objects we keep permanently, which we remove by term, who approves a rule change and how it is recorded in history.
• Access policies — roles, key scopes, upload and read restrictions, procedure for granting and revoking access for employees and services.
• Storage documentation — where metadata is stored, how replication is arranged, how to estimate storage volume and growth without contacting us.
• Monitoring and alerts — checks of gateway availability, pinning speed and share of failed operations with assignment to owners and trigger thresholds.
• Recovery plan — procedure for when the pinning provider is unavailable, switching to a backup node and verifying integrity of objects by content addresses.
• Instructions for engineers — environment launch, migrations, typical operations and release acceptance checklist.

Separately we hand over the decision log: why such pinning terms, replication levels and failure points were chosen. This is what usually gets lost when the contractor changes.

Case: file registry without losses and manual reconciliations

Before: pinning without replication

The document registry lived on one distributed storage node: files were added by script, IPFS pinning was without redundancy, and some pins were lost when the worker node restarted.

The link by CID continued to open, but the content behind it was no longer served — a silent failure noticed only by user request. Reconciliation of the registry with actually available files was done manually once a week, and it took about four hours.

What changed

Pinning was moved to Pinata: each file gets pins on several provider nodes, and the registry state is reconciled with the provider via a programmatic interface.

Availability is checked by automatic traversal — it requests content for each CID and marks a discrepancy before the user sees it. Retries and re-pinning on failure are moved into a separate policy, and the registry change history is stored separately from the files themselves.

What became

File availability rose to 99.9% and is maintained at this level without manual intervention. Recovery after node failure takes minutes: the registry itself shows which pins need to be re-raised.

Manual reconciliations were removed completely — instead of a half-day task there remains a report read in a couple of minutes, and the failure class “link exists, file missing” no longer occurs in the registry.

For the customer this means the registry describes the actual storage state, not what it was at upload time. Next we examine how such a contour is maintained in working mode.

How pinning differs from backup

In the IPFS network, pinning is not a copy but an obligation of a specific node not to delete a data block during garbage collection. As long as the node is alive and holds the file in its storage, the content is available at the address computed from the data itself.

A backup is arranged differently: it is an independent instance in another place, with versions and a retention policy.

Hence different failure scenarios. The pinning node may be turned off, lose its disk or disappear together with the provider — the address remains valid, but no one answers at it. A copy survives such a loss: it lies separately and does not depend on the life of one node. Therefore pinning and backup cover different risks and do not replace each other.

The retention period is also arranged differently. Pinning acts while someone holds it — formally indefinite, but without obligations: the promise lives exactly as long as the node lives.

For a copy, the term is set by policy — how many previous versions to keep, how often rotation occurs, what is deleted first. This is a managed parameter, not a side effect, and it determines what will remain of the data in a year.

File recovery relies on copies, not pinning. The working scheme: pinning with two independent providers, regular checksum reconciliation and export of an archival copy to cold storage with versioning.

Then failure of any one node or a provider change does not turn the data into unavailable, and the share of lost objects remains zero.

Technical doubts: terms, integrity, access

Before the start, the engineering team usually checks four things: in what time a file is actually pinned, how to make sure the content was not substituted, who of external recipients gets access and what will happen to the already accumulated registry.

Below are answers substantively, without promising “works out of the box”, including for pinning services like Pinata.

Pinning is not a single call but a state: while a file has active pins with several independent keepers, it is available even if one of them fails.

Integrity is ensured by content addressing itself: the file hash is its address, any edit changes the address, and substitution becomes visible on the first read. Access is restricted at the gateway level and by signed links with expiration, not by the “secrecy” of storage.

Migration of an existing registry goes as a separate stage: count the volume, remove duplicates, transfer, verify checksums — and only then turn off the old contour.

How to check that a file was not substituted?

Verification runs by content address on every read, not once at upload. If the checksum diverges, the read returns an error and the file is marked as corrupted — substitution does not pass silently.

What will happen to the already accumulated registry?

We transfer without stopping operation: the old contour continues to respond until the new one is reconciled across all records. Rollback is returning the switch to the previous source; re-upload is not required.

Who can access files during transfer?

Links are issued signed and time-limited, rights are checked at the gateway, and internal keeper addresses are not exposed externally. During transfer, old links are not revoked until their term expires.

Let's discuss the task: file pinning for your contour

So that the estimate is based on numbers, not guesses, we need a snapshot of the storage: what we pin in IPFS, who reads it and how long objects live.

Below is the minimal set of inputs sufficient to calculate volumes, assemble an implementation plan and avoid redoing the access scheme midway. You can send it in free form: a list, a table or an export from the current registry.

• File registry composition — object types (media, documents, user exports), average and peak size, daily growth, current volume. Upload throughput and pinning strategy are calculated from this.
• Access scenarios — who reads and from where: public links, internal services, mobile applications. We separately record whether a gateway or direct content addressing is needed.
• Retention periods — what lives indefinitely, what goes to archive, what is deleted by regulation. Cleanup policies and rules for transition between storage tiers are assembled from this.
• Access requirements — who issues keys, how read and write rights are separated, whether isolation by contours and logging of object access is needed.
• Integration points — existing backend, queues, processing services and where the identifier of the pinned object is returned.
• Your contour constraints — closed perimeter, data placement requirements, backup scheme and tested recovery scenario.
• Current state — which pinning provider you use, how addressing is arranged, on which operations you already hit the ceiling.

With this set we return with an estimate by storage volume, a phased implementation plan and a description of the registry that will survive load growth. Describe the task — we will assemble a proposal and show what exactly goes into work.