Measuring and Improving Code Coverage for Bash Scripts

1. Why code coverage matters for shell scripts too

In many other programming languages, code coverage is a natural part of the testing infrastructure, whereas in the Bash world the metric is frequently ignored. Yet exactly the same principle applies to shell scripts as to any other language: a test suite that only checks the success case but never exercises the error path with invalid arguments or missing files leaves blind spots that become visible exactly when a script hits an unexpected state in production.

Code coverage for Bash scripts answers a simple but important question: which lines of the script were actually executed during a test run, and which never were? A backup script with error handling for the case that the target disk is full may exist in the code, but if not a single test case simulates that state, that line stays untested and its actual behavior remains unknown until the error occurs live for the first time.

The tool kcov brings code coverage to the Bash world by running a script in an instrumented fashion and logging which lines were actually traversed. Combined with an existing bats test suite, this creates a complete picture: not just whether tests are green, but also how much of the actual code these tests even touch. The following sections show the complete workflow from installation to enforcing coverage thresholds in the CI pipeline.

2. Installing kcov and running a first measurement

kcov is a standalone command line tool that measures code coverage for compiled binaries and scripting languages, without requiring the application under measurement itself to be modified. For Bash, that means: kcov attaches to a script's execution and logs at the operating system level which lines were actually executed, without the script itself needing any instrumentation.

Installation on Debian and Ubuntu is done via the package manager, alternatively you build kcov from source for newer versions with additional features. The basic invocation kcov output-directory ./script.sh runs the script perfectly normally while simultaneously writing a complete HTML coverage report to the given output directory, which can then be opened in a browser.

# Install kcov on Debian/Ubuntu
sudo apt-get install kcov

# Or build the latest version from source for full Bash support
git clone https://github.com/SimonKagstrom/kcov.git
cd kcov && mkdir build && cd build
cmake .. && make && sudo make install

# Run a script under kcov coverage measurement
kcov --include-path=. ./coverage-output ./deploy.sh

# Open the generated HTML report
xdg-open ./coverage-output/deploy.sh/index.html

# Coverage summary is also printed to stdout, e.g.:
# Percent covered: 73.500

An important detail: without the --include-path flag, kcov also measures all included libraries from the system, for example internal functions from /usr/lib/bash, which unnecessarily bloats the report and dilutes the actually interesting code coverage of your own script. With --include-path=., the measurement is limited to files in the current project directory, which is the right setting for most use cases.

3. Reading coverage reports: lines, branches, functions

The HTML report generated by kcov shows the script's source code color coded line by line: green for lines that were executed at least once, red for lines that were never reached. This simple visual representation makes code coverage for Bash immediately tangible, without having to interpret raw numbers: a glance at the report directly shows which error handling blocks, which case branches, or which functions were never reached by a test case.

Besides pure line coverage, kcov also provides information on branch coverage, that is, whether both sides of a condition, both the true and the false branch of an if, were actually traversed. A line can be considered "executed" even though only one of two possible branches was ever tested, a detail that pure line coverage obscures but that is decisive for actual test quality.

# Example: a function with a branch that coverage should reveal as untested

validate_backup_size() {
  local size_bytes="$1"
  local min_size=$((10 * 1024 * 1024))  # 10 MB minimum

  if [[ "$size_bytes" -lt "$min_size" ]]; then
    echo "[ERROR] Backup suspiciously small: ${size_bytes} bytes" >&2
    return 1
  else
    echo "[OK] Backup size looks reasonable: ${size_bytes} bytes"
    return 0
  fi
}

# If tests only ever call validate_backup_size with a large value,
# the kcov report will show the "if" branch (line 6-7) as never
# covered — a real gap, since the error path was never verified.

The textual summary report at the end of a kcov run additionally shows an aggregated percentage per file and for the whole project. That number alone, however, says nothing yet about whether the tested lines were actually meaningfully checked, a point elaborated on in the section about the limits of code coverage.

4. Combining coverage with bats test suites

The real value of code coverage for Bash emerges when kcov instruments not a single script, but a complete bats test suite. Since bats-core itself runs a Bash script, kcov can wrap the bats invocation directly, aggregating the coverage measurement across all test cases instead of measuring each individual test case separately.

This combination answers the actually relevant question: which lines of the library under test are covered by the entire test suite, across all test cases? A single test case might only cover a small part of a function, but together with further test cases exercising other code paths, a more complete picture of actual test coverage emerges.

#!/usr/bin/env bash
# scripts/run-coverage.sh — run the full bats suite under kcov
set -euo pipefail

readonly COVERAGE_DIR="coverage-report"
readonly BATS_BIN="$(command -v bats)"

# kcov wraps the bats binary itself, aggregating coverage
# across every single @test case in the suite
kcov \
  --include-path="$(pwd)/lib" \
  --exclude-pattern="/test_helper/,/.bats/" \
  "$COVERAGE_DIR" \
  "$BATS_BIN" test/

echo "Coverage report generated at: ${COVERAGE_DIR}/index.html"

# Extract the aggregated percentage for a quick console summary
grep -o 'covered: [0-9.]*' "${COVERAGE_DIR}/index.html" | head -1

The --exclude-pattern parameter is crucial here: without it, kcov also measures the bats framework's internal logic and the test helper libraries themselves, which distorts the coverage number of your own project code. With targeted inclusion and exclusion of paths, the report stays limited to exactly the files that are actually part of your own library, not the testing framework itself.

5. Identifying and closing untested code paths

Once a coverage report is available, the next step is to systematically go through the lines marked red and decide whether it is a real testing gap or code that deliberately does not need to be tested, for example pure debug output. For every real gap, a new test case is written that creates exactly the state that leads to executing that line.

A proven approach: prioritize gaps by risk, not by quantity. Untested error handling in a backup script that silently continues in case of failure weighs more heavily than an untested debug output. Code coverage provides the map, but the prioritization of which gaps to close first remains a human decision based on the actual criticality of the respective code path.

# lib/backup.sh — the function under scrutiny after a coverage report

backup_database() {
  local db_name="$1"
  local backup_dir="${2:-/var/backups}"

  if [[ ! -d "$backup_dir" ]]; then
    echo "[ERROR] Backup directory does not exist: $backup_dir" >&2
    return 2
  fi

  if ! mysqldump "$db_name" > "${backup_dir}/${db_name}.sql" 2>/dev/null; then
    echo "[ERROR] mysqldump failed for database: $db_name" >&2
    return 3
  fi

  echo "[OK] Backup created: ${backup_dir}/${db_name}.sql"
  return 0
}

#!/usr/bin/env bats
load "test_helper/bats-assert/load"

setup() {
  load "../lib/backup.sh"
}

# This test case was added specifically because coverage showed
# the "directory does not exist" branch was never exercised
@test "backup_database fails with exit code 2 for missing backup dir" {
  run backup_database "shop" "/nonexistent/path"
  assert_equal "$status" 2
  assert_output --partial "does not exist"
}

# This test case closes the second gap the coverage report revealed
@test "backup_database fails with exit code 3 when mysqldump fails" {
  mysqldump() { return 1; }
  export -f mysqldump

  run backup_database "shop" "/tmp"
  assert_equal "$status" 3
  assert_output --partial "mysqldump failed"
}

After adding these two test cases, a rerun of kcov shows both previously red lines as green, and the aggregated coverage percentage rises accordingly. It matters that the new tests do not just execute the line, but also check the correct exit code and the correct error message, as described in the earlier article on unit tests for Bash functions.

6. Enforcing coverage thresholds in CI pipelines

A one time coverage measurement is only a snapshot. To prevent code coverage from declining over time because new functions get added without accompanying tests, a minimum threshold can be enforced directly in the CI pipeline. If measured coverage falls below that value, the build fails before the code can be merged.

kcov itself does not come with a built in threshold flag, but the summary embedded in the HTML report can be extracted with simple shell tools and checked against a limit. This pattern can be integrated directly into a GitLab CI or GitHub Actions pipeline, so a declining coverage value automatically blocks the merge request instead of only being noticed manually later.

#!/usr/bin/env bash
# scripts/check-coverage-threshold.sh — enforce a minimum coverage percentage
set -euo pipefail

readonly MIN_COVERAGE=80
readonly COVERAGE_DIR="coverage-report"

kcov --include-path="$(pwd)/lib" "$COVERAGE_DIR" bats test/ > /dev/null

# Extract the aggregated percentage from kcov's summary output
actual_coverage=$(kcov --include-path="$(pwd)/lib" "$COVERAGE_DIR" bats test/ 2>&1 \
  | grep -oP 'Percent covered: \K[0-9.]+' | head -1)

echo "Measured coverage: ${actual_coverage}%"
echo "Required threshold: ${MIN_COVERAGE}%"

if (( $(echo "$actual_coverage < $MIN_COVERAGE" | bc -l) )); then
  echo "[ERROR] Coverage ${actual_coverage}% is below the required ${MIN_COVERAGE}% threshold" >&2
  exit 1
fi

echo "[OK] Coverage threshold met"

In practice, it is advisable not to set the threshold immediately to a high value like 90 percent, but to raise it gradually: first establish the current status quo as a baseline so that no existing coverage is allowed to decline, then manually raise the threshold with every improvement of the test suite. That avoids frustration from an unreachable target set from the start, while the metric still steadily moves in the right direction.

7. bashcov as an alternative to kcov

bashcov is a Ruby based alternative specifically for Bash scripts, built on the established SimpleCov framework from the Ruby ecosystem. While kcov is a generic, cross language tool that works via ptrace based instrumentation, bashcov is specifically designed for Bash scripts and brings a simpler installation as well as a more familiar HTML report look for teams already familiar with SimpleCov from other projects.

The downside of bashcov: it requires a Ruby environment as a dependency, which means an extra installation step in pure Bash or container environments without Ruby. kcov, on the other hand, is a native C++ binary without runtime dependencies, which often makes it more practical for lean CI containers. Both tools, however, deliver essentially the same core function, line coverage measurement for Bash scripts without changing the code of the script under measurement.

# Install bashcov via RubyGems (requires Ruby to be installed)
gem install bashcov

# Run a script or test suite under bashcov measurement
bashcov ./deploy.sh

# Run an entire bats test suite under bashcov
bashcov -- bats test/

# bashcov generates a SimpleCov-style HTML report in ./coverage/
xdg-open ./coverage/index.html

For teams that already use Ruby tooling, for example for other testing frameworks or deployment scripts, bashcov can be the more pragmatic choice. For pure Bash or Go heavy environments without an existing Ruby dependency, kcov usually remains the leaner option, especially in minimal Docker images for CI pipelines.

8. Limits of code coverage: high number, bad tests

A high code coverage number is no guarantee of good tests. A test case that calls a function but contains not a single assertion on the result technically produces one hundred percent line coverage for that function, without actually checking whether the behavior is correct. Coverage measures which code was executed, not whether the result of that execution was actually validated.

A second problem: coverage says nothing about the quality of the tested input values. A function called ten times with the same input reaches the same line coverage as a function tested with ten different, carefully chosen edge cases, even though the second test suite is considerably more valuable. For this reason, code coverage should always be understood as a necessary but not sufficient condition for good tests, complemented by manual reviews of the actual assertions in every test case.

9. Coverage tools compared

Depending on the project environment and already existing tooling, kcov, bashcov and a purely manual coverage analysis are suited to different degrees.

In practice, kcov is the most pragmatic choice for most Bash projects, since it needs no additional runtime environment and integrates easily into existing CI containers. bashcov pays off especially where Ruby is already part of the toolchain anyway. ShellCheck does not replace coverage measurement, but complements it sensibly, since it statically finds issues such as unquoted variables even before test execution.

10. Summary

Code coverage for Bash scripts makes visible what plain test results conceal: which lines were actually executed and which have never gone through a single test run since the code was written. kcov instruments Bash scripts without code changes, produces color coded HTML reports, and can be combined directly with existing bats test suites to aggregate coverage across all test cases.

Coverage thresholds in the CI pipeline prevent a gradual decline in test quality, while prioritized addition of test cases for the riskiest untested code paths delivers the greatest safety gain. What remains important is the insight that a high coverage number alone does not guarantee a good test suite, as long as the actual assertions in every test case are not also critically checked.

11. FAQ: Code Coverage for Bash Scripts