Using BATS for Bash Tests in CI

1. Why shell scripts need to be tested

Shell scripts form the backbone of many automation infrastructures: deployments, backups, data migration, system configuration. Yet while mature test frameworks exist for Python, Go, or PHP, shell scripts are rarely tested systematically in practice. Instead, the prevailing attitude is "it runs until it doesn't", and when it stops running, that usually happens in production. BATS, the Bash Automated Testing System, closes this gap with a test framework that fits naturally into the shell world.

The problem with untested shell scripts isn't just the risk of production failures. Refactoring becomes dangerous because nobody can be sure whether a change has unintentionally altered behavior. New developers extending an existing script have no contract to rely on. Regressions creep in because the intended functionality was never formally described. BATS tests solve all of this: they document expected behavior, catch regressions automatically, and make shell scripts safe to refactor.

Another common argument against testing shell scripts is that they're "too small" or "too trivial" to justify tests. Practice regularly disproves that. A fifty-line backup script often has half a dozen decision paths, some of which are only reachable under specific system conditions. BATS makes it possible to verify those paths in a controlled test environment without touching production at all.

2. Installing and setting up BATS

The modern bats-core project is the actively maintained successor to the original BATS by Sam Stephenson. The recommended installation method is a git submodule inside the project, so that every developer and the CI pipeline use exactly the same version. Alternatively, BATS is available through most package managers: brew install bats-core on macOS, apt install bats on Debian/Ubuntu, npm install -g bats across platforms. For CI environments, though, the git submodule pattern is more reliable because it introduces no external runtime dependency.

The recommended project layout keeps tests clearly separate from production code. The test/ directory at the project root holds all BATS test files with the .bats extension, a test/fixtures/ subdirectory holds test data, and test/helpers/ holds reusable helper functions. The three commonly used helper libraries, bats-assert, bats-support, and bats-mock, are also added as submodules. This layout lets you initialize the entire test stack with a single git clone --recurse-submodules command.

#!/usr/bin/env bash
# setup-test-environment.sh: install BATS and helper libraries as git submodules

set -euo pipefail

# Add bats-core and helper libraries as submodules
git submodule add https://github.com/bats-core/bats-core.git test/bats
git submodule add https://github.com/bats-core/bats-support.git test/test_helper/bats-support
git submodule add https://github.com/bats-core/bats-assert.git test/test_helper/bats-assert
git submodule add https://github.com/bats-core/bats-mock.git test/test_helper/bats-mock
git submodule update --init --recursive

# Create directory structure
mkdir -p test/fixtures test/helpers

# Add bats runner script
cat > test/run_tests.sh << 'EOF'
#!/usr/bin/env bash
# Run all BATS tests with TAP output for CI
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "${SCRIPT_DIR}/bats/bin/bats" \
  --formatter tap \
  --report-formatter junit \
  --output "${SCRIPT_DIR}/../reports/" \
  "${SCRIPT_DIR}"/*.bats
EOF
chmod +x test/run_tests.sh

echo "BATS test environment initialized"

3. Writing your first BATS test

At first glance, a BATS test file resembles a regular Bash script, but it uses special syntax for test cases. Each test is a function introduced by the @test keyword, followed by a descriptive name in quotes. The assertion happens via the run command, which executes the command under test and exposes the exit code and output through the $status and $output variables. Checking the exit code and output then happens with plain Bash conditions or with the assertions from bats-assert.

The most important principle when writing BATS tests is that every test must be isolated and idempotent. Tests must not rely on side effects from other tests, and they must restore system state after they run. For this, BATS provides the setup and teardown hooks, which run before and after each individual test, respectively. Since bats-core 1.2, setup_file and teardown_file also provide file-level hooks that run once per test file, ideal for expensive resources such as Docker containers or test databases.

#!/usr/bin/env bats
# test/validate_config.bats: tests for the config validation script

load 'test_helper/bats-support/load'
load 'test_helper/bats-assert/load'

# Script under test
SUT="${BATS_TEST_DIRNAME}/../bin/validate_config.sh"

setup() {
  # Create a temporary directory for each test
  export TEST_TMPDIR
  TEST_TMPDIR="$(mktemp -d)"
}

teardown() {
  # Clean up after each test, always runs, even on failure
  rm -rf "${TEST_TMPDIR}"
}

@test "validate_config exits 0 for valid config file" {
  cp "${BATS_TEST_DIRNAME}/fixtures/valid_config.env" "${TEST_TMPDIR}/app.env"
  run "${SUT}" "${TEST_TMPDIR}/app.env"
  assert_success
}

@test "validate_config exits 1 when config file is missing" {
  run "${SUT}" "${TEST_TMPDIR}/nonexistent.env"
  assert_failure
  assert_output --partial "Config file not found"
}

@test "validate_config reports missing required keys" {
  cp "${BATS_TEST_DIRNAME}/fixtures/incomplete_config.env" "${TEST_TMPDIR}/app.env"
  run "${SUT}" "${TEST_TMPDIR}/app.env"
  assert_failure
  assert_output --partial "DB_HOST"
  assert_output --partial "Required variable missing"
}

@test "validate_config accepts optional keys as absent" {
  cp "${BATS_TEST_DIRNAME}/fixtures/minimal_config.env" "${TEST_TMPDIR}/app.env"
  run "${SUT}" "${TEST_TMPDIR}/app.env"
  assert_success
}

4. bats-assert and bats-support for readable assertions

The bats-core library on its own only gives you run, $status, and $output. Working with raw Bash conditions such as [ "$status" -eq 0 ] is possible, but uninformative on failure: you only see that the test failed, not what was expected and what actually happened. bats-assert adds expressive assertion functions that print a structured failure message and show the expected value next to the actual one.

The most important assertions from bats-assert are assert_success and assert_failure for exit codes, assert_output and assert_output --partial for standard output, and assert_line for individual output lines. refute_output and refute_line let you verify that certain content does not appear in the output, useful for security tests that need to confirm sensitive data isn't being logged. Together, bats-support and bats-assert produce readable, self-documenting BATS tests.

5. Organizing fixtures and test data

Fixtures are immutable test data that represent a known system state for tests. In the BATS world, fixtures are typically configuration files, input data, or directory structures stored in the test/fixtures/ directory. BATS provides the $BATS_TEST_DIRNAME variable for this purpose, which points to the directory of the currently running test file, so fixtures can be referenced reliably with a relative path regardless of where BATS is invoked from.

For tests that need directory structures, a setup hook that copies the fixture data into a temporary directory before the test runs is recommended. The temporary directory is created with mktemp -d, stored in an exported variable, and removed again with rm -rf in the teardown hook. This keeps tests fully isolated: every test starts from a clean, defined filesystem state. The $BATS_TMPDIR variable points to a BATS-managed temporary directory that is cleaned up automatically at the end of the test run.

#!/usr/bin/env bats
# test/backup_script.bats: integration tests for backup.sh using fixtures

load 'test_helper/bats-support/load'
load 'test_helper/bats-assert/load'
load 'helpers/mock_helpers'

SUT="${BATS_TEST_DIRNAME}/../bin/backup.sh"
FIXTURES="${BATS_TEST_DIRNAME}/fixtures"

setup_file() {
  # One-time setup: create fixture directory tree (runs once per file)
  export FIXTURE_DATA_DIR="${BATS_TMPDIR}/fixture_data"
  mkdir -p "${FIXTURE_DATA_DIR}"/{docs,logs,config}
  echo "important document" > "${FIXTURE_DATA_DIR}/docs/report.txt"
  echo "error entry"       > "${FIXTURE_DATA_DIR}/logs/app.log"
  echo "DB_HOST=localhost"  > "${FIXTURE_DATA_DIR}/config/app.env"
}

setup() {
  # Per-test setup: fresh backup target directory
  export BACKUP_TARGET
  BACKUP_TARGET="$(mktemp -d)"
  export BACKUP_SOURCE="${FIXTURE_DATA_DIR}"
}

teardown() {
  rm -rf "${BACKUP_TARGET}"
}

@test "backup creates archive with correct name pattern" {
  run "${SUT}" --source "${BACKUP_SOURCE}" --target "${BACKUP_TARGET}"
  assert_success
  # Verify archive naming: backup-YYYY-MM-DD.tar.gz
  run bash -c "ls '${BACKUP_TARGET}' | grep -E '^backup-[0-9]{4}-[0-9]{2}-[0-9]{2}\.tar\.gz$'"
  assert_success
}

@test "backup preserves all source files in archive" {
  run "${SUT}" --source "${BACKUP_SOURCE}" --target "${BACKUP_TARGET}"
  assert_success
  local archive
  archive="$(ls "${BACKUP_TARGET}"/*.tar.gz | head -1)"
  run tar -tzf "${archive}"
  assert_output --partial "docs/report.txt"
  assert_output --partial "logs/app.log"
  assert_output --partial "config/app.env"
}

6. Mocking with bats-mock and stub functions

The biggest obstacle when testing shell scripts is external dependencies: database commands, cloud CLIs, mail programs, systemd commands. A test that actually sends an email or creates a database is no longer a unit test, it's an integration test with unwanted side effects. bats-mock solves this problem with a stub mechanism that replaces external commands with controlled fakes returning predefined exit codes and output.

A simpler alternative to bats-mock is defining shell functions that override external commands of the same name within the test scope. Since Bash looks up functions before external commands, defining mail() { echo "mock: $*"; } inside the setup function routes every mail call in the tested script to the mock function. This technique works reliably for simple cases. For more complex scenarios, such as returning different results depending on the call count or verifying how often a command was invoked, bats-mock offers a more expressive approach with stub and unstub.

#!/usr/bin/env bats
# test/notify.bats: tests with mocked external commands

load 'test_helper/bats-support/load'
load 'test_helper/bats-assert/load'
load 'test_helper/bats-mock/stub'

SUT="${BATS_TEST_DIRNAME}/../bin/notify.sh"

setup() {
  # Stub 'curl' to simulate HTTP API call without network access
  stub curl \
    '--silent --fail -X POST * : echo "mock curl: notification sent"; exit 0'
  # Stub 'mail' for email notifications
  stub mail \
    '-s * * : echo "mock mail: subject=$2 recipient=$3"; exit 0'
}

teardown() {
  unstub curl
  unstub mail
}

@test "notify sends HTTP request with correct payload on failure" {
  run "${SUT}" --event "deploy_failed" --environment "production" --notify http
  assert_success
  assert_output --partial "mock curl: notification sent"
}

@test "notify sends email when mail transport is selected" {
  run "${SUT}" --event "backup_done" --environment "staging" --notify mail
  assert_success
  assert_output --partial "mock mail"
}

@test "notify exits 1 when unknown transport is specified" {
  run "${SUT}" --event "test" --notify "unknown_transport"
  assert_failure
  assert_output --partial "Unknown notification transport"
}

7. setup, teardown, and test organization

The setup and teardown hooks are the core of clean BATS test organization. setup runs before every individual test and prepares the state the test needs. teardown runs after every test, even if that test failed, which is essential so that temporary resources get cleaned up even on failure. Without teardown, temporary files and processes accumulate and can affect later tests.

For larger test suites with many files, a shared test/helpers/ library that consolidates recurring setup logic is a good idea. This helper file is included in every test file via load 'helpers/common'. The BATS command skip marks individual tests as skipped when certain preconditions are missing, for example when Docker isn't installed or a particular port isn't reachable. BATS clearly flags skipped tests in the output without turning the overall status into a failure.

The --filter flag lets you run specific tests or groups of tests by name when invoking BATS. That's useful during development, letting you run only the relevant tests while working on a feature instead of waiting for the entire suite. For the CI pipeline, however, the full suite should always run, ideally with the --timing flag added, which shows the runtime of every test and helps identify slow ones.

8. BATS helper libraries compared

The BATS ecosystem offers several helper libraries for different testing needs. Choosing the right combination depends on the complexity of the scripts under test.

The combination of bats-core, bats-support, and bats-assert covers most testing needs. bats-mock becomes relevant once the script under test calls external processes such as cloud CLIs, mailers, or database clients. bats-file rounds out the suite when scripts primarily perform filesystem operations and the test needs to explicitly verify that specific files were created, modified, or deleted.

9. Integrating BATS into CI pipelines

Integrating BATS into CI pipelines is straightforward because BATS supports two universal output formats: TAP (Test Anything Protocol) and JUnit XML. TAP is understood by nearly every CI system, and JUnit XML is the standard format for test results in GitHub Actions, GitLab CI, and Jenkins. The --formatter tap or --formatter junit flag selects the desired format. The JUnit format lets BATS test results appear directly as a test report in the CI UI, making failed tests easy to spot at a glance.

For GitHub Actions, using the official BATS action or a simple run: ./test/bats/bin/bats test/ step is recommended. Submodules need to be checked out with submodules: recursive in the checkout step. The same pattern works in the script block of a GitLab CI job. Important: place the BATS step in its own job stage that runs after linting (ShellCheck) but before deployment. This turns the CI pipeline into a clear quality gate: no deploy without passing BATS tests.

# .github/workflows/shell-tests.yml: GitHub Actions workflow for BATS tests

name: Shell Script Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  shellcheck:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: recursive
      - name: Run ShellCheck on all scripts
        run: |
          find bin/ -type f -name "*.sh" -print0 \
            | xargs -0 shellcheck -S warning

  bats-tests:
    runs-on: ubuntu-latest
    needs: shellcheck
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: recursive
      - name: Run BATS test suite
        run: |
          mkdir -p reports
          ./test/bats/bin/bats \
            --formatter junit \
            --output reports/ \
            test/*.bats
      - name: Upload test results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: bats-test-results
          path: reports/
      - name: Publish Test Results
        uses: EnricoMi/publish-unit-test-result-action@v2
        if: always()
        with:
          files: reports/*.xml

10. Summary

The Bash Automated Testing System brings structured, automated testing to the shell world, closing one of the biggest gaps in modern automation infrastructures. BATS tests document the expected behavior of shell scripts, catch regressions automatically, and make refactoring predictable. The combination of bats-core, bats-assert, and bats-mock fully covers unit tests, assertions, and isolating external dependencies. Fixtures together with the setup/teardown hooks provide test isolation and reproducible results.

Integrating BATS into CI pipelines takes only a few lines of YAML and produces JUnit XML reports that every common CI system understands. The payoff shows up quickly: every hour invested in BATS tests pays for itself many times over through prevented production failures and safer refactoring. Shell scripts without BATS tests are no longer an acceptable standard for production-ready automation.

11. FAQ: Using BATS for Bash Tests in CI