Configuring Git LFS: storing large files and accelerating CI/CD

Why Git LFS is needed: repository grows, builds slow down

Git stores each version of a file as a separate object, so commit history grows with the number of edits. While a repository contains only source code, this is almost unnoticeable — delta compression absorbs the difference between adjacent versions.

As soon as binary artifacts get in — media, layouts, models, test databases, built distributions — each edit adds a full new copy of the file to history, not a difference.

Then the cascade kicks in. Cloning the repository stretches to tens of minutes because the client downloads all versions of the file, including long-unneeded ones.

The build agent spends more time warming up the working copy than on the build itself, parallel jobs hit disk and network limits, and the cache is constantly reset when it overflows.

The more active the team, the more the history bloats — and the more expensive each next run becomes.

At the same time, publishing hits a wall: hosting refuses to accept changes when the size of objects being sent exceeds the limit, and artifact uploads fail on timeout.

Treating this by rewriting history retroactively is risky — branches, commit references, and signatures break, and all developers have to re-clone the project.

At the same time, large files in the repository remain common practice because "that's how it has always been."

Git LFS changes the storage model itself: large objects move to separate storage, while pointers remain in the repository — they weigh bytes and do not grow from version to version.

We break down how this is addressed in practice: from choosing a storage location and migrating already accumulated history to rules that prevent new binary artifacts from leaking past LFS.

What changes after moving large files out of the repository

Large files — binary builds, media, layouts, model weights — hold repository history hostage: each edit adds a new full copy of the object, and previous ones remain in history forever.

In Git LFS, content is replaced by a pointer, and the data itself moves to separate object storage. The repository again stores text changes, not artifacts.

First measurable effect — clone time. A developer and build machine pull only current versions of large files, not their entire history.

The working copy stops bloating: instead of gigabytes of binaries — kilobytes of pointers, switching between branches fits into seconds even on a narrow channel.

Second — faster builds. Steps that previously waited for artifact upload from history get them on demand, and the intermediate cache is not rebuilt entirely on every commit.

At the same time, a whole class of failures goes away: storage limit overflow, interruption while uploading a large object, task failure due to lack of space.

Build stability here is not about speed, but about predictability: a task fails on an error in code, not on infrastructure. The trade-off is environment discipline: an LFS client in the build machine image, retention rules, and backups of object storage. History remains intact — rollback to any previous release remains possible.

Options for storing large files: comparison of approaches

Choosing a way to store large files determines how quickly the repository clones, how much history takes up, and what happens if an external service becomes unavailable. Below are four options that teams actually choose between on projects.

In practice, a combination most often works: Git LFS covers what everyone needs in the build flow, external storage covers large media and artifacts, and local storage covers drafts.

We look at which files participate in the build, which change often, and which are needed only temporarily, and on that basis we distribute them across storages.

How implementation proceeds: from audit to handover of regulations

Implementing Git LFS is not installing an extension, but changing the storage scheme for all heavy artifacts in the repository. The work order is structured so that by the time of switchover, no revision is lost and teams are not idle. Below is how the process is structured from audit to handover of regulations.

1. Repository audit. We measure volume and history: which extensions account for the main weight, how many revisions are affected, who commits large files and how often. The output is a list of paths to move to external storage and a size estimate after migration.
2. Migration plan. We determine whether we rewrite history entirely or migrate only new commits, and agree on a window when edits to the main branch are paused. We fix the rollback procedure if validation fails.
3. Storage setup. We deploy or connect object storage, set retention rules: lifetime of old versions, limits on object size, role-based access separation. We write filters and rules in the repository configuration.
4. Transition of teams and builds. We update settings for developers and build agents so heavy files are downloaded on demand, not in every clone. For a repository of tens of gigabytes, this cuts working copy retrieval time several times over.
5. Integrity check. We compare object checksums before and after migration, build the project from scratch on a clean clone, and check availability of old revisions and tags. Only after that do we close the migration window.
6. Handover to operation. We provide regulations and instructions, set up alerts for storage growth, train the team, and assign an owner responsible for the rules.

Pointer rules, access, and project documentation

Implementation is considered complete not when large files stop slowing down cloning, but when the team has a set of artifacts that allows changing rules without us. Below is what the customer receives as a result and what we go through with their engineers during handover.

• Mapping rules file — which extensions and paths go to LFS, and which remain in the regular index. A new category of binary files is added by editing one line, not by rebuilding the repository.
• Pointer and storage configuration — object storage address, lock mode for "non-mergeable" files, limits on parallel transfers. The same setting on work machines and build agents so behavior does not diverge.
• Storage scheme — where objects physically reside, how the cache on agents is organized, what cleanup rules apply: how many revisions we keep, what is deleted first, how an object is restored after accidental deletion.
• Access policies — which accounts and tokens can read and write objects, how rights are issued to contractors and external teams, where build secrets are stored and how they are rotated.
• Instructions for the team — how to properly clone a repository with large files, what to do if a transfer is interrupted, how to free up space in the working copy, how to lock a file that cannot be merged automatically.
• Operating regulations — the procedure for editing the rules file: who approves moving a new category of files to LFS, how it is verified that a pointer has not been left without its object, who is responsible for storage growth.
• Checks in the build — automatic control of "pointer without object" and size limit, so the problem is detected at the build stage, not when a new developer clones.

This set is enough for your team to change the rules: expand the list of tracked files, onboard new people and external repositories, without coming back to us for every setting. Describe your task — we will come back with an estimate and a handover plan.

Case: how we unloaded the repository and sped up the build

The product repository was growing not because of code: next to the sources were interface layouts, test videos, and datasets for validating exports.

Cloning on a clean machine took about 40 minutes, the working copy was fourteen times heavier than the code itself, and out of 30 minutes of build time, 12 went to downloading history. Exports failed in about every fifth attempt — usually on timeout when retrieving attachments.

We separated the flows: code remained in the main history, heavy files were moved to Git LFS with rules in .gitattributes by type masks, and rarely changing test datasets were moved to separate storage with link-based versioning.

We added preloading of required objects to the builder image so steps would not pull them one by one over the network, limited write permissions, and enabled a check for files over the threshold — a commit can no longer unnoticeably weigh down history.

After the move, cloning fit into 4 minutes, the build was reduced to 11, and the developer's working copy became fourteen times smaller. Failures on export per quarter — a few instead of every fifth run, and the share of unsuccessful deliveries fell below one percent.

The repository now grows predictably: the retention policy limits the service storage size, and rollback to the previous version of assets remains available.

How Git LFS differs from regular storage in the repository

With regular storage, a binary artifact — image, dataset, video — is committed to the same object database as code. Delta compression, which keeps the history of text files compact, does not work on binaries: each version is stored as a full copy.

Deleting from the current branch frees nothing — the object remains in history, and a clone pulls all its revisions.

Git LFS changes the scheme: a text pointer enters the repository — object type, hash, and size — while the bytes themselves move to separate storage. Commit history, branches, and tags remain in place, but the version tree weighs as much as the pointer lines occupy.

The difference between approaches shows not in developer commands, but in what exactly is pulled over the network during clone and build.

Restoring the working tree also differs. A regular clone downloads the entire history of large files, including versions deleted many releases ago; pointer mode delivers only the objects needed by the current tree.

The trade-off is network dependency: without access to storage, text placeholders remain in place of files, and filters must be configured before the first clone.

When branching, identical files are deduplicated by hash: one copy in the release and working branches is stored once, whereas in the regular scheme the object lives in all branches and slows down history traversal.

Storage limitations do not disappear, but move: a separate server appears, quotas on volume and traffic, its own lifecycle policy and backup not tied to the repository.

In the build pipeline, this gives selective loading of required paths instead of full history — environment preparation speeds up, but the object download step is configured explicitly, otherwise the build will silently receive pointers instead of files.

Concerns about storage, access, and file recovery

Implementing LFS rarely ends with a single tracking command. Binary objects already lie in history as regular files, so a migration plan is needed: without it, a fresh clone pulls old branches entirely.

The second group of concerns is operations: where objects physically reside and who has access to them.

Separately, we check storage availability at build time: the build agent either stops with an explicit error or takes objects from the local cache — this is defined in advance, not at the moment of failure.

How do we give access to external contractors?

We issue a separate account to the contractor: read-only and only the required object space. They can retrieve large files but cannot overwrite others' files, and access is revoked after the work is accepted.

How do we restore large files from a backup?

A copy of the repository is useless — it contains pointers, not the data itself. We add object storage backup to the regulations and verify restoration on a test clone so that an incomplete copy is not discovered at the moment of an incident.

Let's discuss the task: storage scheme and migration procedure

For the discussion, the customer brings a list of repositories and an understanding of which files get into them: binary builds, models, media, exports from external systems.

Current tracking rules, the size of developers' working copies, and complaints about clone or build time are useful — from them it is immediately clear where the bottleneck is and what exactly we are speeding up.

From this, a work plan is assembled: storage scheme (where the LFS storage is located, which file masks go into tracking, what limits we set), implementation order by repository, and timeline estimate.

We do not start with mass migration — first a pilot on one repository to check clone, build, and rollback on a live pipeline.

• inventory of repositories and files to be moved to LFS;
• storage scheme: storage placement, tracking rules by masks, size limits;
• implementation order: pilot repository, then the rest in waves;
• migration of history and cleanup of already committed large objects;
• changes to the build pipeline, dependency cache, and access rules for agents;
• roles and rights: who reads and writes to storage, how keys are issued;
• rollback plan if clone or build degrades after the transition.

We will come back with a timeline and scope estimate: stages, what we do, what is needed from the customer's side, acceptance criteria — clone time, working copy size, build duration. Describe the task — we will come back with an estimate and a plan.