BuildKit Remote Cache: Sharing Build Cache via Registry and GitHub Actions

1. Why local cache is not enough in CI environments

The Docker build cache normally relies on locally available image layers: if a layer was already built once with identical content and identical previous layers, it gets reused on the next build instead of being re-executed. On a developer's laptop this works reliably because the Docker daemon persists between builds and all previous layers remain on disk. In CI environments, however, reality looks different, since many systems such as GitHub Actions or GitLab CI spin up a fresh, isolated runner for every job with no Docker history whatsoever.

Without a countermeasure, this means every CI build starts from zero, even if nothing changed in composer.json or package.json since the last run and the most expensive layer, the dependency installation, could in principle be reused unchanged. On large projects with several minutes of install time per build, this quickly adds up to considerable wasted compute time and longer wait times for developers across many daily pipeline runs. This is exactly where BuildKit remote cache comes in, storing the cache not locally but in an externally reachable registry or a CI-native cache backend.

2. How --cache-from and --cache-to work

BuildKit distinguishes between reading and writing external cache via two separate flags: --cache-from specifies one or more sources from which existing cache layers are loaded before the build, while --cache-to specifies where the cache layers produced during the current build should be exported to after completion. Both flags accept a type parameter that determines which backend is used, commonly type=registry for any OCI-compatible registry or type=gha for the GitHub Actions cache.

It is important to understand the difference between the classic inline cache, which embeds cache metadata directly in the image manifest layers, and the long-recommended standalone or max mode, which with mode=max caches every intermediate step of a multi-stage build definition, not just the final layers of the last stage. Especially with multi-stage Dockerfiles that have separate build and runtime stages, mode=max is crucial, because without it the layers of, say, a pure compiler or test stage are never exported at all and must be rebuilt from scratch on every run.

# Read cache from a registry and write it back there after the build
docker buildx build \
  --cache-from type=registry,ref=ghcr.io/mironsoft/app:buildcache \
  --cache-to type=registry,ref=ghcr.io/mironsoft/app:buildcache,mode=max \
  -t ghcr.io/mironsoft/app:latest \
  --push .

3. Setting up registry cache in practice

The registry cache stores cache layers as a separate manifest, usually tagged distinctly like :buildcache, in the same registry the actual image gets pushed to. This has the advantage that no additional infrastructure is needed, since any registry that supports OCI manifests, such as GHCR, ECR, GCR, or a private Harbor instance, can serve as a cache backend. The cache layer is treated like a regular image and is subject to the same push and pull permissions as the actual application image.

In practice it is advisable to clearly separate the cache tag from the actual version tags, for example app:buildcache instead of app:latest, so an accidental docker run of the cache image is ruled out and the registry's garbage collection can manage the cache tag independently of production tags. For multi-branch setups it is also worth keeping a separate cache tag per branch, for example app:buildcache-main and app:buildcache-feature-x, to prevent parallel feature branches from overwriting each other's cache and thereby reducing cache hits.

# Separate cache tag per branch, with fallback to the main cache
BRANCH_TAG=$(echo "$CI_COMMIT_REF_SLUG" | tr '/' '-')

docker buildx build \
  --cache-from type=registry,ref=ghcr.io/mironsoft/app:buildcache-${BRANCH_TAG} \
  --cache-from type=registry,ref=ghcr.io/mironsoft/app:buildcache-main \
  --cache-to type=registry,ref=ghcr.io/mironsoft/app:buildcache-${BRANCH_TAG},mode=max \
  -t ghcr.io/mironsoft/app:${CI_COMMIT_SHA} \
  --push .

4. GitHub Actions cache with type=gha

For projects that build exclusively in GitHub Actions, type=gha is an appealing alternative to the registry cache, because it uses the cache service built into GitHub Actions instead of pushing cache layers to an external registry. This not only saves additional registry traffic but is also directly usable in workflows via the Actions Cache API without a separate registry login, which significantly simplifies setup, especially in private repositories without a public registry.

An important difference from the registry cache is that the GitHub Actions cache has a limited total volume per repository and is automatically cleared after a certain period of inactivity, which is why it is especially suited to frequently running pipelines but not intended as permanent cache storage for rarely built images. In practice, type=gha is usually configured via docker/build-push-action, which automatically passes the required cache parameters through to buildx.

# .github/workflows/build.yml (excerpt)
- name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v3

- name: Build and push
  uses: docker/build-push-action@v6
  with:
    context: .
    push: true
    tags: ghcr.io/mironsoft/app:${{ github.sha }}
    cache-from: type=gha
    cache-to: type=gha,mode=max

5. Multi-stage builds and the difference between mode=min and mode=max

Without an explicit setting, BuildKit uses mode=min for --cache-to, which only exports the layers of the final image stage. In a typical multi-stage Dockerfile with separate stages for Composer install, a Node build, and a lean PHP-FPM runtime, this means only the last runtime stage gets cached, while the expensive earlier build stages have to be re-executed completely on every CI run, even though their inputs never changed.

With mode=max, BuildKit instead exports cache layers for all intermediate stages, so an unchanged composer install stage, for example, gets pulled from cache even if a later stage, say the Tailwind build, genuinely needs to re-run. The trade-off is a larger cache export with correspondingly more storage need and a longer push time for the cache itself, which in practice almost always pays off as soon as a Dockerfile has more than one stage.

6. How cache invalidation works with remote caches

Cache invalidation for remote caches follows the same rules as the local layer cache: if the content of a COPY instruction or a preceding layer in the Dockerfile changes, the cache hit for that layer and all subsequent ones becomes invalid. A common mistake is copying composer.json and the rest of the application code in a single COPY command, which means every code change automatically invalidates the composer install layer too, even if the dependencies never changed.

The solution is the well-known ordering optimization: copy composer.json and composer.lock first and separately, then run composer install, and only afterward copy the rest of the application code. Combined with remote cache, this means dependency installation is served from cache across dozens of CI runs as long as composer.lock stays unchanged, and only needs to re-run on actual dependency updates.

# syntax=docker/dockerfile:1
FROM composer:2 AS vendor
WORKDIR /app

# Copy only the lock file so code changes don't invalidate the cache
COPY composer.json composer.lock ./
RUN composer install --no-dev --no-scripts --no-interaction --prefer-dist

# Application code copied only afterward, in its own layer
COPY . .

7. Measured effects in practice

In our own measurements on a mid-sized Magento project with separate Composer and Node build stages, the average build time of a CI run with no actual dependency changes dropped from roughly nine minutes to under two minutes once registry cache with mode=max was enabled, because both the PHP and the Node dependency stages were served entirely from cache and only the actual application code needed to be re-copied and the final stage assembled.

Even with genuine code changes and no dependency updates, the savings remained substantial, since only the last one or two layers had to be rebuilt while all preceding, more expensive stages came from cache. The additional time spent pushing the cache itself was, in testing, a few seconds up to just under half a minute, which is negligible given the overall savings.

8. Troubleshooting: when the cache does not hit

The most common reason for an ineffective remote cache is a missing or misconfigured --cache-to in the previous run, since without a successful export there is simply nothing to load. Equally common is a typo or inconsistency in the cache reference between --cache-from and --cache-to, for example different registry paths or tags between two pipeline configurations that are supposed to share the same cache.

Another typical pitfall is missing push access to the cache registry reference in pull-request pipelines, which for security reasons often only have read but not write permissions on the registry. In this case, --cache-to should be deliberately omitted for PR builds or redirected to a separate, less privileged cache reference, while --cache-from still points at the main cache, so PR builds at least benefit from the existing cache without polluting it.

9. Best practices and a backend comparison

As a general recommendation: type=registry suits heterogeneous CI landscapes with multiple runners or even multiple CI systems, because any environment with registry access can use the same cache, while type=gha is the simpler choice for pure GitHub Actions setups, since no additional registry configuration is needed. For both backends, mode=max is almost always the right choice for multi-stage Dockerfiles, as long as the additional storage requirement is acceptable.

It is also advisable to refresh the cache regularly, at least rewriting the main-branch cache on every successful merge, so feature branches always benefit from an up-to-date base cache. The table below compares the key properties of both backends.

10. Summary

11. FAQ: BuildKit Remote Cache: Key Takeaways