Git LFS: Sensibly Versioning Large Files

1. Why Git struggles with large binary files

Git's storage model is optimized for source text. On every commit, Git internally computes deltas between text lines and stores objects highly compressed, because textual changes compare well line by line. For a PSD file, a video, or a ZIP archive, this diffing logic barely applies: even a single changed pixel row or a freshly recalculated compression header causes the entire binary file to be written into history as a brand new, complete object. There is no meaningful line structure for Git to work with.

The consequences add up quickly: fifty versions of a 100-megabyte design file mean five gigabytes of pure history data, even though the working directory only shows the current version. Because Git history is immutable by default, every version ever committed stays in the repository forever, even after the file has been deleted from the current state. Every git clone downloads this entire history, which noticeably slows down onboarding new teammates, CI pipelines, and even simple backups, while needlessly straining the available storage on Git hosting platforms.

2. How Git LFS works under the hood: pointer files and smudge/clean filters

Git LFS (Large File Storage) solves this by no longer storing the actual binary content in Git's object database, replacing it instead with a tiny pointer file. This text file sits at the exact same place in Git history where the binary used to be, but is only a few bytes in size and contains just a version marker, the SHA-256 hash of the content (oid), and its file size. The actual binary content is stored on a separate LFS server, provided by GitHub, GitLab, or a self-hosted instance.

To keep this substitution transparent, Git LFS registers two Git filters: the clean filter, which replaces a binary file with its pointer representation on git add/git commit and uploads the content into a local LFS cache and, on push, to the server, and the smudge filter, which does the reverse on checkout, automatically swapping the pointer file back for the real binary content. For developers this mechanism is invisible: the working directory always shows the full file, while Git itself only ever tracks the lightweight pointer version.

# What actually gets stored in the Git history instead of the binary blob:
cat assets/hero-banner.psd

# version https://git-lfs.github.com/spec/v1
# oid sha256:4d7a2c8e9f1b3a6d5c8e0f2a1b4c7d9e6f3a2b5c8d1e4f7a0b3c6d9e2f5a8b1c
# size 157286400

# Locally, "git lfs" resolves the pointer back into the real file
git lfs pointer --file=assets/hero-banner.psd

3. Installing and setting up Git LFS

Git LFS is not part of Git itself; it is a separate client tool that must be installed additionally, for example via apt install git-lfs, brew install git-lfs, or the official installer package. After installation, git lfs install activates the necessary Git filters globally in the user profile, so that every newly cloned or initialized repository on that machine automatically recognizes and applies the filter=lfs directives from .gitattributes. This step only needs to run once per machine, not once per repository.

For cases where the global filters should deliberately not be set, for example on a CI runner with restricted permissions, git lfs install --local registers the filters only in the current repository's local .git/config. git lfs version confirms that both the LFS extension and the underlying Git version are compatible, which regularly causes hard-to-diagnose errors on older CI images when the extension is silently missing.

# Install the git-lfs client (once per operating system / package manager)
sudo apt-get install git-lfs        # Debian/Ubuntu
brew install git-lfs                # macOS

# Register the Git filters globally for this user account, once per machine
git lfs install

# Register the filters only for the current repository (e.g. restricted CI runner)
git lfs install --local

# Verify the installed client and Git compatibility
git lfs version

4. Tracking file types with git lfs track and .gitattributes

Which files get managed through LFS is defined by git lfs track. Running git lfs track "*.psd" writes a corresponding line with the attributes filter=lfs diff=lfs merge=lfs -text into the .gitattributes file at the repository root. These attributes tell Git to use the LFS filters for matching files and to treat them as binary, without normalizing line endings. Patterns can be combined freely, for example scoping to entire directories with assets/videos/**/*.mp4, or tracking several file types with separate track calls.

The critical, often overlooked step: the .gitattributes file itself must be versioned like any other file, with git add .gitattributes and git commit. Only then do the same tracking rules apply for every teammate who clones the repository. If the file is forgotten, the original author's files get correctly stored via LFS, but every other clone treats new binary files of the same type as ordinary blobs stored directly in history again.

# .gitattributes - generated and extended via "git lfs track"
*.psd filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.mp4 filter=lfs diff=lfs merge=lfs -text
assets/videos/**/*.mov filter=lfs diff=lfs merge=lfs -text

# List every pattern currently tracked by Git LFS in this repository
# git lfs track

5. The everyday workflow: commit, push, clone with LFS

After the initial setup, the familiar routine with git add, git commit, and git push doesn't visibly change. Behind the scenes, Git automatically recognizes which files fall under LFS based on the .gitattributes rules, replaces them with pointers on commit, and uploads the actual content separately to the LFS endpoint on push. git lfs ls-files reliably shows which files in the current commit are actually managed as LFS objects, while git lfs status additionally shows whether local LFS files have already been uploaded or are still pending.

On clone, Git downloads only the pointer files immediately by default; the associated binary content is only fetched during checkout via the smudge filter, which noticeably speeds up the initial git clone. Important in practice: GitHub and GitLab meter LFS storage and bandwidth separately from the regular repository quota, with comparatively small free allowances that get exhausted quickly on active design or video repositories, at which point paid add-ons become necessary.

# Clone downloads pointers immediately, binary content follows on checkout
git clone https://git.example.com/team/shop.git

# List which files in the current commit are actually managed as LFS objects
git lfs ls-files
# 4d7a2c8e * assets/hero-banner.psd
# 9b3f1e0a * assets/videos/teaser.mp4

# Show which LFS files are staged, modified, or still pending upload
git lfs status

6. Migrating existing large files retroactively with git lfs migrate

Tracking rules set via git lfs track only apply to future commits. Binary files already checked into history before LFS was introduced remain untouched and keep bloating the repository. For that case, git lfs migrate import walks the entire commit history, extracts every matching blob, replaces it with a pointer file, and rewrites the affected commits. With --include="*.psd" --everything, the migration can be scoped to specific file types across every branch.

Because every affected commit ends up with a new checksum, this effectively changes the entire subsequent history, and a plain git push no longer works afterward, it requires git push --force-with-lease. A full backup of the repository is mandatory before running such a migration, since the operation is destructive and cannot easily be undone. Team coordination is just as important: after the force push, everyone must re-clone their local copies or explicitly rebase their branches onto the new history, otherwise duplicate, conflicting histories emerge.

# Always create a full backup/mirror before rewriting history
git clone --mirror https://git.example.com/team/shop.git shop-backup.git

# Rewrite history: replace already-committed .psd files with LFS pointers
git lfs migrate import --include="*.psd" --everything

# Push the rewritten history; regular push is rejected due to changed hashes
git push --force-with-lease origin --all
git push --force-with-lease origin --tags

# Every teammate must re-clone or reset onto the rewritten history
# git fetch origin && git reset --hard origin/main

7. Limits and pitfalls of Git LFS

Git LFS shifts the storage problem rather than fully resolving it: on common hosting platforms, LFS storage and bandwidth are billed separately and are often tight even in the base plan. For large, frequently changing asset libraries, these costs can climb quickly, especially because every new version of a binary file is stored as an entirely new LFS object with no delta compression between versions. On top of that, LFS only helps where it has been actively set up: without a retroactive migration, binary files already committed in the past remain sitting unchanged in the regular Git object store.

Teams that don't want to rely on a large hosting provider for LFS can run their own LFS server; many self-hosted Git platforms such as GitLab CE or Gitea already ship with built-in LFS support. As a lightweight complement when a full LFS setup isn't wanted, partial clone (git clone --filter=blob:none) is also worth considering, fetching large blobs only on actual access, without requiring a separate LFS server at all.

8. Alternatives and complements to Git LFS

git-annex follows a similar basic idea to Git LFS but is considerably more flexible and more complex: it supports arbitrary storage backends, encrypted remotes, and fine-grained control over which files are actually present locally. For teams primarily looking for a simple, widely adopted tool, the extra learning curve usually isn't justified. Another complement is Git submodules, which offload large asset collections into a dedicated repository that's only pulled in when needed, plus DVC (Data Version Control), purpose-built for ML datasets and pipelines, combining versioning with experiment tracking.

Just as important is recognizing when LFS is the wrong answer altogether. Generated or imported media, such as Magento product images under pub/media/, generally shouldn't live in a Git repository at all, regardless of whether LFS is involved. Such assets belong in database imports, S3 buckets, or CDN deployments, while Git stays reserved for source code and clearly scoped, genuinely version-worthy design assets. Git LFS is a tool for files that should actually be part of the development history, not a generic substitute for asset storage.

9. Git LFS compared: patterns applied right vs. wrong

Most Git LFS problems trace back to a handful of recurring scenarios that are fully avoidable with the right approach. The table below lines up typical missteps against the recommended strategy.

The table shows a consistent pattern: Git LFS works reliably once tracking rules, history, and team communication are all consistently aligned. Most problems don't come from the tool itself, but from a half-finished setup, such as a forgotten .gitattributes commit or a migration run without prior agreement across the team.

10. Summary

Git LFS solves a specific structural problem: binary files don't fit Git's line-based delta compression and needlessly bloat both the repository and clone times. Pointer files with a version marker, SHA-256 hash, and file size replace the actual content in Git history, while smudge and clean filters handle the substitution transparently in the background on checkout and commit. git lfs track plus a committed .gitattributes define which file types are affected, and the familiar commit-push workflow stays unchanged for developers.

For repositories that already exist, git lfs migrate import is the only way to actually remove historical binary files from the past, though only with a full backup and a coordinated force push across the whole team. Teams that apply these steps consistently, while also knowing when assets shouldn't be in the repository at all, such as generated Magento media, keep their repository lean and performant over the long run.

11. FAQ: Git LFS