Using Shallow Clones Deliberately in CI Pipelines

1. What a shallow clone technically means

A normal git clone transfers a repository's complete commit history, every commit ever created, every blob, and every tree, regardless of whether that historical state is actually needed. For a repository actively maintained for years, that alone can easily add up to several hundred megabytes or more of pure history, even when the current working state is much smaller.

A shallow clone caps exactly that transfer through the --depth parameter. With --depth=1, Git transfers only the latest commit of every requested branch along with its corresponding file tree, without any prior history. Internally, Git marks the boundary of this limited history with a so called shallow boundary, stored in the .git/shallow file, which Git takes into account for all further operations.

# Clone only the latest commit of the default branch
git clone --depth=1 https://example.com/project.git

# Shallow clone a specific branch
git clone --depth=1 --branch feature/checkout-optimization \
  https://example.com/project.git

2. Why CI pipelines benefit especially from shallow clones

A CI job generally only needs the current state of a commit to run tests, build a Docker image, or deploy static files. Transferring the entire history in full, even though it offers zero value for that particular job run, is pure waste of checkout time and network bandwidth, especially for pipelines that run again several times a day against the same commit or closely related ones.

For a repository with ten years of history, the difference between a full clone and a shallow clone with depth one can cut the checkout step from several minutes down to a few seconds, an effect that multiplies with every single pipeline run and adds up to significant CI minutes, and therefore cost, saved.

3. Where shallow clones hit their limits

The missing history is not a purely cosmetic detail, it concretely affects several common Git operations. git log in a shallow clone only shows the commits actually transferred, git blame can no longer determine authorship for older changes, and git describe, which normally derives a version label from tags, may fail to find a matching tag depending on the depth and error out.

Merge operations and a direct comparison against a distant branch can fail too, because Git simply has no shared prior history from which to compute a merge base. For jobs that depend on exactly that kind of information, say a release script deriving the latest version number from a tag, or a coverage comparison against an older commit, a shallow clone is therefore unsuitable or must be deliberately extended.

4. Finer control: shallow-since and shallow-exclude

Besides a fixed commit count, Git also allows a time or reference based limitation of history. --shallow-since transfers all commits from a given date onward, handy for pipelines that need, say, the last two weeks of changes for a changelog job without loading the entire history. --shallow-exclude instead limits history relative to a specific tag or branch and fits cases where exactly the difference to a known reference point matters.

These finer variants are especially useful when a job does not need complete history but still more than just the last commit, for example an automated listing of every commit since the last release tag.

# Only load commits from the last 14 days
git clone --shallow-since="14 days ago" https://example.com/project.git

# Limit history relative to a known tag
git fetch --shallow-exclude=v2.4.0

5. Loading full history afterward

If it turns out mid pipeline run that a single job actually needs the full history, say because an analysis tool needs access to old commits, the entire checkout does not necessarily have to be repeated. git fetch --unshallow fetches the missing history for the already existing shallow clone and then fully removes the shallow boundary, after which the repository behaves like a normal, complete clone.

That after the fact step naturally costs time and bandwidth again, often even more than a complete clone from the start, because extra protocol round trips are required. It only pays off when the full history remains genuinely the exception and most jobs get by fine with the shallow state.

# Fetch the full history for an existing shallow clone
git fetch --unshallow

6. Configuration in GitLab CI and GitHub Actions

GitLab CI controls checkout depth centrally through the GIT_DEPTH variable, settable at project or job level and defaulting to 20, a compromise between speed and enough context for most standard jobs. A value of 1 minimizes checkout time further but can, depending on the job, cause the problems already described, for example when a job relies on tags or merge base calculations.

GitHub Actions already defaults the official checkout action to fetch-depth: 1, so jobs needing more history must explicitly set it higher, or to 0 for full history. In both systems it pays to configure depth per job rather than globally, so fast standard jobs benefit from the reduction while jobs with special requirements request more history deliberately.

# GitLab CI: override the default depth per job
build:
  variables:
    GIT_DEPTH: "1"
  script:
    - echo "Fast checkout for the build job"

release:
  variables:
    GIT_DEPTH: "0"
  script:
    - echo "Full history for tag based version detection"

7. Case study: shallow clone in a Magento deployment pipeline

In a typical Magento deployment pipeline that installs Composer dependencies, builds static files, and then rolls them out to a target server via rsync or Deployer, the complete Git history is not actually needed for a single step, since Composer works exclusively with the current state of composer.json and composer.lock, without inspecting Git history at all.

A GIT_DEPTH: 1 setting for the build and deploy job reduces checkout here to a few seconds, while a separate, less frequently run job for changelogs or release notes that depends on tags and commit history deliberately works with full history or a targeted --shallow-since. This split by actual need, rather than a blanket setting for the whole pipeline, is the most robust approach in practice.

8. Troubleshooting: common shallow clone problems in practice

A typical symptom is the error message fatal: reference is not a tree when a job tries to check out a commit that falls outside the loaded shallow boundary, say because a deployment script wants to step back two commits but only the latest commit was transferred. The fix is either a larger checkout depth or a targeted git fetch --deepen=, which extends the history by a fixed number of additional commits without fully unshallowing.

A second common problem involves tools that silently assume a complete repository, say code analysis tools computing change statistics over longer time windows. In a shallow clone, such tools often produce no error at all, just wrong or incomplete results, which makes it worth explicitly checking whether a newly introduced analysis job actually works correctly with limited history.

# Extend the history by 50 additional commits without fully unshallowing
git fetch --deepen=50

9. When shallow clones should be avoided

For semantic release tools that automatically derive the next version number from commit history and existing tags, a shallow clone is generally unsuitable, because exactly the tags and the history between them needed for that are missing. The same applies to blame based analyses, say code ownership reports, which produce wrong or incomplete results without full history.

Caution is also warranted when combining shallow clones with a sparse checkout in very large monorepos: the two mechanisms independently limit different dimensions, history for shallow clones, file scope for sparse checkout, and combining them can produce unexpected failures when a tool in the pipeline implicitly assumes a complete working tree.

10. Summary

11. FAQ: Shallow Clones in CI at a Glance