GitLab Cache vs. Artifacts: Handling Composer, npm, and Vendor Correctly

1. The Core Difference Between Cache and Artifacts

The conceptual difference between GitLab cache and artifacts is misunderstood or ignored in many pipelines, with measurable consequences for build times and reproducibility. Cache is persistent storage between pipeline runs that holds data which is expensive to fetch but not critical to the correctness of the build. Composer packages that have already been downloaded from Packagist once do not need to be downloaded again on the next build, they can be read from the cache instead, which eliminates network latency and reduces build time.

Artifacts, on the other hand, are files a job produces as the result of its work and that get consumed by subsequent jobs in the same pipeline. When the build job populates vendor/ through composer install, that populated directory has to be passed to the test job and the deploy job as an artifact. A cache would be the wrong tool here, because the cache comes from an earlier build and may not match the current composer.lock version.

The most important rule: whatever can be deterministically built from the source code belongs in an artifact. Whatever only shortens fetch time without affecting the result belongs in the cache. A vendor directory without a matching lock file is a security problem and a reproducibility problem. A Composer download cache without a valid vendor directory is just an empty holding area, harmless and useful.

2. Configuring the Composer Download Cache Correctly

Composer has two different caches: the download cache, which stores downloaded packages as ZIP archives, and the repository cache, which stores Packagist metadata. For GitLab pipelines the download cache is the most valuable, because it prevents the same packages from being downloaded again on every build. This cache typically lives under ~/.composer/cache or a configured directory.

The correct pattern is to configure the Composer cache directory as a GitLab cache with a key based on the branch name or the hash of composer.lock. When the lock file changes, a new cache key is used and Composer downloads the new packages, while the old packages remain available for earlier builds. After the download through composer install, the finished vendor/ directory is declared as an artifact so that subsequent jobs can use it without running Composer again.

# build.yml: Composer cache and artifact configuration
build:composer:
  stage: build
  image: php:8.4-cli
  variables:
    COMPOSER_CACHE_DIR: "${CI_PROJECT_DIR}/.cache/composer"
    COMPOSER_HOME: "${CI_PROJECT_DIR}/.cache/composer"
  cache:
    key:
      files:
        - composer.lock  # Cache invalidates when lock file changes
    paths:
      - .cache/composer/  # Cache the downloaded ZIPs, not vendor/
    policy: pull-push     # Download existing cache, push updated cache
  script:
    - composer install --no-dev --prefer-dist --no-interaction --optimize-autoloader
  artifacts:
    paths:
      - vendor/           # Built result passed to subsequent jobs
    expire_in: 2 hours    # Short TTL: artifacts are only needed within pipeline
  only:
    - main
    - tags
    - merge_requests

An important detail is the expire_in setting for artifacts. Vendor directories are typically very large (Magento can reach 400 to 600 MB) and are only needed within a single pipeline. A retention period of two hours is sufficient, but it saves considerable storage space on the GitLab server compared to a default of 30 days or no limit at all.

3. Passing the Vendor Directory Between Jobs as an Artifact

In a multi-stage Magento pipeline, Composer dependencies are installed in the build job and are then needed in test jobs, the DI compile job, and the deploy job. The correct way to pass on the populated vendor directory is GitLab's artifact system. Jobs that need the vendor directory declare it as a dependency through needs, or receive it automatically when they depend on a job that produces artifacts.

A mistake that shows up often in pipelines is running composer install again in every job that needs vendor. That doubles or triples the runtime, because every job computes the same result that would already be available if artifacts were configured correctly. The rule is: composer install runs once in the build job, and the result is stored as an artifact for all subsequent jobs.

The vendor directory of a Magento project is large, but GitLab compresses artifacts before uploading and unpacks them before use. In practice, a 400 MB vendor directory as an artifact is available noticeably faster than running composer install again over the network, even with the download cache in place. Transferring an artifact within the GitLab system is considerably faster than an external package download.

4. npm and Tailwind: Cache vs. node_modules as an Artifact

The same principle that applies to Composer applies to the frontend build with npm and Tailwind CSS. The npm download cache, by default under ~/.npm or a configured directory, stores downloaded packages as tarballs. This cache belongs in the GitLab cache configuration with a key based on package-lock.json. The built node_modules/ directory and the CSS build output under pub/static/ or the Tailwind output directory are artifacts.

A common optimization for npm heavy pipelines is npm ci instead of npm install. npm ci installs exactly the versions from the lock file, always deletes the existing node_modules/ first, and is typically faster than npm install in CI environments. Combined with an npm cache that holds the downloaded tarballs, npm ci is the best combination for reproducibility and speed.

# build.yml: npm cache and Tailwind build artifact
build:frontend:
  stage: build
  image: node:22-alpine
  variables:
    NPM_CONFIG_CACHE: "${CI_PROJECT_DIR}/.cache/npm"
  cache:
    key:
      files:
        - package-lock.json  # Cache invalidates on lock file change
    paths:
      - .cache/npm/          # Cache downloaded npm tarballs
    policy: pull-push
  script:
    # npm ci: deterministic install from lock file
    - npm ci --prefix app/design/frontend/Mironsoft/default/web/tailwind
    # Build Tailwind CSS output
    - npm run build --prefix app/design/frontend/Mironsoft/default/web/tailwind
  artifacts:
    paths:
      # Pass built CSS to deploy job, not node_modules
      - app/design/frontend/Mironsoft/default/web/tailwind/css/
      - pub/static/frontend/
    expire_in: 2 hours
  needs:
    - job: build:composer  # Requires vendor/ artifact first

5. generated/ and pub/static/ as Build Artifacts

In Magento, setup:di:compile and setup:static-content:deploy produce two more directories that are build artifacts: generated/ contains the generated DI code (interceptors, factories, proxies), and pub/static/ contains all static files for the frontend. These directories are created in the build job and must be passed to the deploy job as artifacts.

setup:static-content:deploy is often run as a server-side step after deployment. That has downsides: it extends downtime during deployment, loads the production server with the build process, and makes the deployment less reproducible, because the build result then depends on the production server's environment. It is better to generate static content in the pipeline's build job and deploy the result as an artifact.

6. Cache Keys: Reproducibility Through Lock File Hashing

Cache keys determine whether an existing cache can be used for a given build. The simplest key is a branch name, but that means the same cache applies to every commit on a branch, even if dependencies have changed. A better approach is hashing the lock file into the cache key. GitLab supports this through the key.files configuration, which automatically computes a SHA-based key from the contents of the named files.

When composer.lock changes, the cache key changes, and GitLab either uses an existing cache with the same key (if the lock file is unchanged on another branch) or creates a new, empty cache. The old cache object remains available under the old key. This configuration guarantees that Composer never works with a stale cache that belongs to a different lock file.

# Cache key strategy for Magento projects
# Combines branch name with lock file hash for isolation + reuse

.composer_cache: &composer_cache
  cache:
    key:
      prefix: "composer-v1"
      files:
        - composer.lock
    paths:
      - .cache/composer/
    policy: pull-push

.npm_cache: &npm_cache
  cache:
    key:
      prefix: "npm-v1"
      files:
        - app/design/frontend/Mironsoft/default/web/tailwind/package-lock.json
    paths:
      - .cache/npm/
    policy: pull-push

# Job using both caches
build:all:
  stage: build
  <<: *composer_cache
  # Note: only one cache block per job; combine manually or use extends
  script:
    - composer install --no-dev --prefer-dist --optimize-autoloader
    - npm ci --prefix app/design/frontend/Mironsoft/default/web/tailwind
    - bin/magento setup:di:compile
    - bin/magento setup:static-content:deploy de_DE -f

7. Comparison: What Goes in Cache, What Goes in Artifacts

The table below shows the correct mapping of the most important Magento build components to cache and artifacts. These decisions have a direct impact on build time, storage consumption, and the reproducibility of the pipeline.

The table makes it clear: caches speed up the fetching step, artifacts carry the result. A vendor directory in the cache would mean that different builds on the same branch use the same vendor state, regardless of whether composer.lock has changed. That is a reproducibility problem and a security problem. As an artifact with a lock file based key, it is guaranteed that every build contains exactly the dependencies the current lock file specifies.

8. Pitfalls with Cache and Artifacts in Magento Pipelines

The most common pitfall is a cache that does not get invalidated when dependencies change. Teams that set the cache key to the branch name run into sporadic build failures when a developer adds packages and the next build finds a stale cache. The solution is always a lock file based cache key. The second common pitfall is a forgotten cache: policy: pull in jobs that should only read the cache and not update it. By default, every job downloads the cache and pushes it back afterward, even if nothing has changed. policy: pull reduces unnecessary cache writes.

A third pitfall involves the size of artifacts. If node_modules/ is configured as an artifact (instead of just the build result), artifacts can grow to several gigabytes. That slows down every job that has to download those artifacts and significantly increases storage consumption on the GitLab server. The rule is: only pass on the minimal set as an artifact that subsequent jobs actually need. For Tailwind builds that is the CSS output, not the full node_modules/ directory.

9. Optimizing Build Time Through Targeted Cache Use

A well configured cache strategy can reduce the build time of a Magento pipeline from 15 to 20 minutes down to 5 to 8 minutes. The biggest gain comes from caching the Composer download cache: a typical Magento project with a Hyva theme downloads 200 to 300 packages from Packagist on the first build. With a warm cache, only changed or new packages get downloaded, which reduces the network phase from 3 to 5 minutes down to under 30 seconds.

For teams working in parallel across multiple branches, cache isolation through branch specific prefixes is worthwhile. A feature branch can then benefit from the main branch cache (a fallback key) without overwriting it. GitLab supports fallback keys through the key configuration: if no cache is found for the current key, a fallback key is tried. That makes it possible to use the stable main branch cache as a warmup cache for new feature branches.

# Advanced cache strategy with fallback keys
build:composer:optimized:
  stage: build
  image: php:8.4-cli
  variables:
    COMPOSER_CACHE_DIR: "${CI_PROJECT_DIR}/.cache/composer"
  cache:
    # Primary key: exact lock file match
    - key:
        prefix: "composer-${CI_COMMIT_REF_SLUG}"
        files:
          - composer.lock
      paths:
        - .cache/composer/
      policy: pull-push
    # Fallback key: main branch cache as warmup
    - key: "composer-main-fallback"
      paths:
        - .cache/composer/
      policy: pull  # Only read from fallback, never write to it
  script:
    - composer install --no-dev --prefer-dist --optimize-autoloader
  artifacts:
    paths:
      - vendor/
    expire_in: 3 hours

10. Summary

The correct separation of GitLab cache and artifacts for Composer, npm, and the vendor directory is one of the most important optimizations in Magento CI pipelines. Cache speeds up the fetching step by reusing downloaded packages. Artifacts carry built results between jobs within a pipeline. Vendor directories and generated code belong as artifacts, not in the cache, because they have to match a specific lock file version.

Lock file based cache keys ensure that stale caches are never used for new dependency states. Short artifact TTLs save storage space, since build artifacts are only needed within a single pipeline. policy: pull for read-only jobs prevents unnecessary cache writes. With this configuration, build time goes down and pipeline reproducibility goes up, because every build stands exactly on what the lock files specify.

11. FAQ: GitLab Cache vs. Artifacts for Composer, npm, and Vendor