Git Secrets Scanning: Preventing Accidental Leaks

1. How secrets end up in Git history despite good intentions

Almost no developer intentionally commits a password. Yet secrets end up in Git repositories constantly, usually through three recurring paths. The first is the classic forgotten .env commit: a local .env file with real database credentials is accidentally missing from .gitignore, a quick git add . picks it up, and the commit is out before anyone notices. The second path is the hardcoded API key in a configuration file, often meant as a quick stopgap during development that never got removed because the file kept working and nobody touched it again.

The third and most underestimated path is the debug commit: to chase down a bug, a developer temporarily adds a var_dump($apiKey) or logs an active access token, accidentally commits it, and "fixes" it in the next commit. This case is especially deceptive, because the developer assumes the problem is solved once the line is removed again. In reality, the secret remains in history and stays fully retrievable for anyone who clones the repository later.

2. Why deleting the file in a later commit does not remove the secret

Git stores every commit as a complete, immutable snapshot. A new commit that deletes a file or replaces a secret with a placeholder simply produces another snapshot in which the secret no longer appears. The old snapshot with the real value stays intact as its own blob object under .git/objects and remains fully visible through git log -p, git show <commit>, or simply checking out an older commit. For Git, deletion is not an overwrite, it is just one more state in a chain that preserves every previous state.

In practical terms: once a commit with a secret has been pushed, a single git fetch by any teammate, fork, CI runner, or automated mirror is enough to permanently duplicate that secret. Actually removing it requires rewriting history with tools like git filter-repo or BFG Repo-Cleaner, followed by a force push and a mandatory re-clone for every developer. That is disruptive for the whole team, and it still does not solve the underlying problem that the value has already been exposed.

3. gitleaks: installation and use as a pre-commit hook

gitleaks is a fast secret scanner written in Go that can search either the full history of a repository or only the content that is currently staged. Installation happens via Homebrew, a prebuilt release binary, or go install, and it runs from the command line without further dependencies. The command gitleaks detect scans an entire repository's history and reports every match with its commit hash, file path, and a redacted preview of the found value.

For everyday workflow, gitleaks protect --staged is the decisive command: it checks only what has just been staged with git add for the next commit, exactly the moment where intervention is still possible without a history cleanup. Wired into .git/hooks/pre-commit as a native git hook, gitleaks blocks any commit containing a detected secret before it is even committed locally, let alone pushed.

# Install gitleaks (macOS/Linux via Homebrew)
brew install gitleaks

# Or download the prebuilt binary directly
curl -sSL https://github.com/gitleaks/gitleaks/releases/latest/download/gitleaks_8.18.4_linux_x64.tar.gz | tar -xz gitleaks
sudo mv gitleaks /usr/local/bin/

# Scan the full commit history of the current repository
gitleaks detect --source . --verbose

# Scan only what is currently staged, before a commit is created
gitleaks protect --staged --verbose

# Install gitleaks as a native git pre-commit hook
cat > .git/hooks/pre-commit << 'EOF'
#!/usr/bin/env bash
set -euo pipefail
gitleaks protect --staged --redact --verbose
EOF
chmod +x .git/hooks/pre-commit

4. .gitleaks.toml: custom rules, entropy, and regex detection

gitleaks ships with an extensive set of default rules that reliably catch known formats such as AWS access keys, Stripe tokens, or private SSH keys through regex-based detection. Regex rules are precise because they check for a known prefix or format, for example AKIA followed by 16 characters for an AWS key. They fail, however, on project-specific secrets with no fixed format, such as a self-generated internal API token. This is exactly where entropy-based detection comes in: it evaluates the statistical randomness of a string and flags values that look too "chaotic" to be natural text or code.

Custom rules, entropy thresholds, and an allowlist for known exceptions live in a .gitleaks.toml file at the project root. With [extend] useDefault = true, the default rules stay active while project-specific patterns, for example Magento database passwords inside env.php, are added on top. The allowlist prevents test fixtures, image files, or vendor directories from being falsely reported as findings.

# .gitleaks.toml: custom rules, allowlist and severity tuning
title = "mironsoft gitleaks config"

[extend]
# Start from the built-in default ruleset, then extend it
useDefault = true

[[rules]]
id = "magento-db-password"
description = "Hardcoded Magento database password in env.php"
regex = '''(?i)'password'\s*=>\s*'[^']{8,}'''
tags = ["magento", "credentials"]

[[rules]]
id = "generic-high-entropy-token"
description = "Generic high-entropy string, catches unknown token formats"
regex = '''[A-Za-z0-9_\-]{32,}'''
entropy = 4.5
secretGroup = 0

[allowlist]
description = "Known false positives"
paths = [
  '''(.*?)(png|jpg|jpeg|gif|svg|lock)$''',
  '''vendor/.*''',
]
regexes = [
  '''EXAMPLE_[A-Z_]+''',
]

5. git-secrets: an AWS-focused scanner with git hook integration

git-secrets from AWS Labs takes a deliberately leaner approach than gitleaks: it works exclusively with explicit regex patterns, without an entropy heuristic, and ships with prebuilt patterns specifically for AWS credentials. After installing via make install, git secrets --install registers the checks directly as git hooks in the local repository, while git secrets --register-aws activates the AWS-specific patterns for access keys and secret keys.

Project-specific patterns can be added with git secrets --add, for example for internal token formats or Magento Marketplace credentials. Known, non-critical matches, such as a dummy key in the README, are excluded explicitly with git secrets --add --allowed. The command git secrets --scan-history additionally checks the full existing history, which matters especially when introducing git-secrets into an already grown repository, to surface legacy findings.

# Install git-secrets (AWS Labs)
git clone https://github.com/awslabs/git-secrets.git
cd git-secrets && sudo make install

# Register AWS-specific patterns and install as git hooks in this repo
cd /path/to/repo
git secrets --install
git secrets --register-aws

# Add a project-specific regex pattern, e.g. a Magento authorization key
git secrets --add '[0-9a-f]{32}'
git secrets --add --literal 'MAGENTO_MARKETPLACE_TOKEN'

# Scan the entire history, not just the working tree
git secrets --scan-history

# Allow a known-safe match (e.g. a documented dummy key in README)
git secrets --add --allowed 'EXAMPLE_API_KEY_00000000000000000000000000000000'

6. The pre-commit framework: centrally managed hooks

Instead of maintaining hook scripts manually per repository under .git/hooks, which gets lost on every clone, the pre-commit framework centralizes management through a versioned .pre-commit-config.yaml. It is installed once per machine via pip install pre-commit or brew install pre-commit, and activated per repository with pre-commit install. The decisive advantage: the configuration lives inside the repository itself, is part of the codebase, and is automatically available to every teammate as soon as the hook has been installed once.

gitleaks ships an official pre-commit hook definition that can be wired in with just a few lines and automatically checks only staged changes. It is important to understand that a local hook is still not complete protection. It can be deliberately bypassed with git commit --no-verify, and on a freshly cloned repository it does nothing until pre-commit install has explicitly been run. This exact gap is what makes a second, server-side check indispensable.

# Install the pre-commit framework once per machine
#   pip install pre-commit
#   brew install pre-commit
#
# Then activate it inside this repository:
#   pre-commit install

repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.4
    hooks:
      - id: gitleaks

  - repo: local
    hooks:
      - id: block-env-files
        name: Block committing real .env files
        entry: sh -c '! git diff --cached --name-only | grep -E "^\.env$"'
        language: system
        stages: [commit]

7. CI-based scanning as a second line of defense

Pre-commit hooks are valuable but optional: they can be bypassed with --no-verify, they do not exist on a freshly cloned repository without pre-commit install, and they do not protect against commits made directly through the GitHub web interface or a CI-generated script. A CI job that runs gitleaks on every push and every pull request update closes exactly this gap server-side, regardless of whether or how an individual developer has configured their local environment.

What matters is that the CI job fails the pipeline with a non-zero exit code as soon as gitleaks reports a finding, and that the checkout fetches full history (fetch-depth: 0) instead of just the last commit. That way a merge into the main branch is blocked before a secret lands there, even if the local hook did not fire for whatever reason.

# .github/workflows/secrets-scan.yml
name: Secrets Scan

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

jobs:
  gitleaks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0   # full history, not just the last commit

      - name: Run gitleaks
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        # Action exits non-zero on any finding, failing the pipeline

8. A secret has leaked: rotate first, clean history second

If a real secret is discovered despite every precaution, the order of the next steps determines the actual risk. The first and non-negotiable step is immediately rotating the credential at the source: the API key gets revoked with the provider and replaced with a new one, the database password gets changed, the token gets invalidated. Only after that comes cleaning the Git history with git filter-repo or BFG Repo-Cleaner.

The reason for this order is simple: once a commit has been pushed, the value was already visible to everyone with access, teammates, CI runners, caches, forks, and local clones. Cleaning history without rotating first removes the secret from the repository but changes nothing about the fact that it was already compromised and remains valid. CDN caches, independent forks, and clones downloaded long ago retain the old history regardless of how thoroughly the force push is executed in the origin repository. Only an invalidated credential makes the secret worthless to anyone who has already seen it.

9. False positives, secret managers, and organizational prevention

Entropy-based rules tend to produce more false positives than plain regex patterns, for example on hash values, base64-encoded test fixtures, or lock files full of long, randomly-looking character strings. Instead of disabling rules wholesale, known exceptions are maintained in a .gitleaksignore file, where every allowed finding is referenced by its unique fingerprint. That keeps the underlying rule strict without wearing the team down with recurring false alarms.

Technical scanning does not replace organizational prevention. Secrets fundamentally do not belong in Git at all, they belong in a dedicated secret manager such as HashiCorp Vault or AWS Secrets Manager, loaded at runtime instead. For local development, the established pattern is a checked-in .env.example with placeholder values alongside a consistently ignored real .env. The table below shows how much actual protection different scanning stages really provide.

10. Summary

Git secrets scanning solves a problem that discipline alone cannot reliably solve: secrets end up in history despite good intentions, through forgotten .env files, hardcoded configuration values, and debug commits, and deleting the file in a later commit does not remove the value from history. gitleaks and git-secrets catch these findings locally as a pre-commit hook, and combining regex- and entropy-based detection covers both known formats and project-specific tokens. Because local hooks can be bypassed with --no-verify and are missing on fresh clones, a second, CI-based scan on every push and every pull request is indispensable.

In an actual leak, the order of operations decides the outcome: revoke and rotate the credential at the source first, clean the Git history second. Organizationally, a centralized secret manager combined with the .env.example pattern reduces how often a real secret ever comes within reach of a git add in the first place.

11. FAQ: Git Secrets Scanning