Getting stdin, stdout, and Interactive Tools Right

1. File descriptors: stdin, stdout, and stderr

Every process on Unix inherits three open file descriptors: stdin (file descriptor 0) for input, stdout (file descriptor 1) for normal output, and stderr (file descriptor 2) for error messages. These three streams are interchangeable with files, devices, or other processes, which is the foundation of the Unix philosophy of "small tools that communicate through pipes." In an interactive shell, all three are connected to the terminal. In a script running inside a pipe, stdin is bound to the previous process and stdout to the next one. In a CI pipeline, all three are often connected to log aggregators.

The different ways tools behave depending on the stdin/stdout context is a common source of bugs. curl shows a progress bar on stderr in a terminal, but not in a pipe. grep highlights matches in color in a terminal, but not without one. less only works with a real terminal as stdout. read without options reads from stdin: if stdin is not a pipe, it waits for keyboard input and blocks a non-interactive script. Understanding and correctly handling this context dependency is essential for scripts meant to run in more than one context.

Bash's redirection syntax is rich: > file redirects stdout to a file (overwriting it), >> file appends, 2> file redirects stderr, and &> file redirects both stdout and stderr. 2>&1 connects stderr to the current target of stdout. The order of redirections matters: cmd > file 2>&1 is correct (stderr follows stdout into the file), while cmd 2>&1 > file is wrong (stderr goes to the terminal, stdout to the file). This ordering rule regularly trips up even experienced shell users.

2. /dev/null: discarding output correctly

/dev/null is the "black hole" of a Unix system: everything written to it is discarded immediately. For stdin/stdout handling in scripts, /dev/null has two main uses: discarding output that is irrelevant in the current context, and serving as an empty input source. The pattern command > /dev/null 2>&1 discards both stdout and stderr, useful for commands invoked in scripts only for their exit code. The shorthand command &> /dev/null is equivalent and shorter.

As an input source, /dev/null used as stdin opens an input stream that is closed immediately: command < /dev/null. This is especially useful for commands that read from stdin and would otherwise block if stdin is connected to a terminal. SSH with -n or an explicit < /dev/null prevents SSH from consuming the script's stdin stream, a classic bug in scripts that call SSH inside a loop: the first SSH command consumes the rest of the loop's input from stdin, and every subsequent iteration gets empty input.

#!/usr/bin/env bash
# stdin-stdout-patterns.sh: correct /dev/null and redirection usage
set -euo pipefail
IFS=$'\n\t'

# Discard stdout and stderr, we only care about the exit code
check_connectivity() {
  local host="$1"
  ping -c1 -W2 "$host" &>/dev/null
}

# /dev/null as stdin: prevent SSH from consuming script's stdin in loops
deploy_to_hosts() {
  local -a hosts=("$@")
  local host
  for host in "${hosts[@]}"; do
    # Without </dev/null, SSH would consume remaining stdin iterations
    ssh -n -o BatchMode=yes "$host" 'bash /opt/deploy.sh' < /dev/null
  done
}

# Separate stdout (data) from stderr (status messages)
# Callers can capture stdout cleanly without status noise
process_and_report() {
  local input_file="$1"
  echo "Processing $input_file..." >&2   # Status -> stderr
  grep -c '^ERROR' "$input_file"         # Result count -> stdout
}

# Capture only stdout, let stderr pass through to terminal
error_count="$(process_and_report /var/log/app.log)"
echo "Found $error_count errors"

# Redirect stdout to log while preserving exit status
log_output() {
  local logfile="$1"
  shift
  exec > >(tee -a "$logfile") 2>&1
  "$@"
}

3. /dev/tty: addressing the terminal directly

/dev/tty is the controlling terminal of the current process, regardless of where stdin and stdout have been redirected. When a script runs inside a pipe but still needs to read something from the terminal (for example a password or an interactive confirmation), /dev/tty is the solution. read -p "Confirm? " -r answer < /dev/tty reads directly from the terminal even when stdin comes from a file or a pipe. This is the pattern tools like git commit, sudo, and ssh use internally to read passwords securely without disturbing the normal data flow.

For output, writing to /dev/tty works the same way: echo "Warning" > /dev/tty always prints the message to the terminal, even when stdout is redirected to a file. This is useful for progress indicators or warnings that the user should see but that must not appear in the piped output. Important: /dev/tty is only accessible when the process has a controlling terminal. In cron jobs, systemd services, and CI pipelines there is no controlling terminal, so trying to open /dev/tty fails with "No such device or address." Always check whether a terminal is present before using /dev/tty.

4. isatty: detecting an interactive context

The ability to distinguish between an interactive and a non-interactive context is essential for scripts and tools that need to behave correctly in both environments. In Bash, the test [[ -t 0 ]] checks whether stdin (FD 0) is connected to a terminal, which corresponds to the C standard call isatty(0). Likewise, [[ -t 1 ]] checks stdout and [[ -t 2 ]] checks stderr. These tests are the correct way to adapt behavior to context: colored output only in a terminal, no progress indicators in pipes, and no interactive prompts in CI environments.

The pattern for context-dependent behavior in shell tools: detect the context at the start of the script and store it in a variable. Every subsequent output function then uses that variable to enable or disable terminal-specific features. The result is a tool that is colorful and informative in a terminal, clean and machine-readable in a pipe, and free of extraneous output in CI, all from the same code, without needing manual reconfiguration. This technique is used consistently by professional CLI tools such as git, docker, and modern Bash frameworks.

#!/usr/bin/env bash
# isatty-context.sh: context-aware output based on terminal detection
set -euo pipefail
IFS=$'\n\t'

# Detect interactive context once at startup
readonly IS_INTERACTIVE_STDIN=$( [[ -t 0 ]] && echo 1 || echo 0 )
readonly IS_INTERACTIVE_STDOUT=$( [[ -t 1 ]] && echo 1 || echo 0 )
readonly HAS_COLORS=$( [[ -t 1 ]] && tput colors &>/dev/null && echo 1 || echo 0 )

# ANSI colors only when stdout is a terminal
if (( HAS_COLORS )); then
  RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; RESET='\033[0m'
else
  RED=''; GREEN=''; YELLOW=''; RESET=''
fi

log_ok()   { printf "${GREEN}[OK]${RESET}   %s\n" "$*" >&2; }
log_warn() { printf "${YELLOW}[WARN]${RESET} %s\n" "$*" >&2; }
log_err()  { printf "${RED}[ERR]${RESET}  %s\n" "$*" >&2; }

# Interactive prompt only when stdin is a terminal
confirm() {
  local prompt="$1"
  if (( IS_INTERACTIVE_STDIN )); then
    printf '%s [y/N] ' "$prompt" > /dev/tty
    read -r -n1 answer < /dev/tty
    printf '\n' > /dev/tty
    [[ "$answer" =~ ^[Yy]$ ]]
  else
    # Non-interactive: default to 'no' unless CI_CONFIRM=1 is set
    [[ "${CI_CONFIRM:-0}" == "1" ]]
  fi
}

# Progress indicator only in terminal
show_progress() {
  local label="$1"
  if (( IS_INTERACTIVE_STDOUT )); then
    printf '\r%s...' "$label" > /dev/tty
  fi
}

confirm "Deploy to production?" && log_ok "Proceeding" || { log_warn "Aborted"; exit 0; }

5. Pipe composition and process substitution

Pipes (|) and process substitution (<(), >()) are the most powerful stdin/stdout tools in Bash. A pipe connects the stdout of the command on the left to the stdin of the command on the right. The crucial difference from sequential commands: piped commands run concurrently, not one after another. The left command produces data while the right one consumes it immediately, without the entire output of the left command being buffered first. This matters a great deal for large volumes of data: grep 'ERROR' large.log | wc -l never needs more than a single buffer chunk of memory, no matter how large the log file is.

Process substitution extends pipes with an important capability: a command expression becomes usable like a file. <(command) creates a virtual file path (typically /dev/fd/63) that can be passed as a file argument. This makes it possible to compare two command outputs directly (diff <(sort a.txt) <(sort b.txt)) without creating temporary files. >(command) works as an output target: tee >(gzip -9 > backup.gz) >(sha256sum > backup.sha256) forwards stdout to two processes at once. These compositions are cleaner and more efficient than managing temporary files explicitly.

6. tee: writing to a file and a pipe simultaneously

The tool tee is the link between monitoring a pipeline and continuing to process it: it reads stdin, writes the data to a file, and simultaneously passes it on to stdout. In scripts that need to log deployment output without removing it from the pipeline, tee is indispensable. The pattern exec > >(tee -a "$LOGFILE") 2>&1 at the start of a script sends all output to the terminal (or the CI console) and to the log file at the same time, without adjusting a single echo call.

A common mistake when using tee: the exit code of the primary command gets lost. command | tee logfile returns the exit code of tee, not that of command. With set -o pipefail, the exit code of the failed left-hand command becomes available again, but only the "worst" exit code in the pipe, not necessarily that of the specific command that failed. The Bash array ${PIPESTATUS[@]} holds the exit codes of every segment of the pipeline right after the last command runs, letting you check specifically for a failure in the first command of cmd | tee without enabling pipefail for the whole pipeline.

#!/usr/bin/env bash
# tee-process-sub.sh: tee with PIPESTATUS and process substitution
set -euo pipefail
IFS=$'\n\t'

readonly LOG_FILE="/var/log/deploy/$(date +%Y%m%d-%H%M%S).log"
mkdir -p "$(dirname "$LOG_FILE")"

# Redirect all output to log AND terminal simultaneously
exec > >(tee -a "$LOG_FILE") 2>&1

echo "=== Deployment started at $(date) ==="

# Process substitution: split stream to two consumers simultaneously
# tee with process substitution: compress and checksum in one pass
backup_database() {
  local output_base="$1"
  mysqldump --single-transaction --quick mydb \
    | tee \
      >(gzip -9 > "${output_base}.sql.gz") \
      >(sha256sum > "${output_base}.sha256") \
    > /dev/null
  # Verify that both outputs exist
  [[ -f "${output_base}.sql.gz" ]] || { echo "Backup failed" >&2; return 1; }
}

# PIPESTATUS: capture exit codes from each pipe segment
check_pipe_status() {
  set +o pipefail  # Manage ourselves
  long_running_cmd | grep 'SUCCESS'
  local pipe_codes=("${PIPESTATUS[@]}")
  set -o pipefail
  if (( pipe_codes[0] != 0 )); then
    echo "[ERROR] long_running_cmd failed with ${pipe_codes[0]}" >&2
    return "${pipe_codes[0]}"
  fi
}

echo "=== Deployment finished at $(date) ==="

7. Non-interactive mode: configuring tools correctly

Many tools behave differently once they detect that they are not running in an interactive terminal. That is desirable, but only when the tools detect the context correctly. Problems arise with tools that hardcode interactive features (--progress, pagers, confirmation dialogs) without checking the stdin/stdout context. For stdin/stdout composition in scripts, it is important to know and consistently use the correct non-interactive flags for every tool involved.

The most important non-interactive flags for commonly used tools: curl with -s (silent) or --no-progress-meter suppresses the progress bar on stderr. git with -q (quiet) or GIT_TERMINAL_PROMPT=0 as an environment variable prevents interactive password prompts. apt-get with -y and DEBIAN_FRONTEND=noninteractive answers every dialog automatically. rsync with --no-progress. docker with --no-ansi. Consistently setting these flags is an essential part of cleanly composable stdin/stdout pipelines in scripts.

8. Here-documents and here-strings for stdin

Here-documents (<<EOF ... EOF) and here-strings (<<< "string") are inline stdin sources in Bash. They let you pass multi-line content or strings directly to a command as stdin, without temporary files or pipes. Here-documents are especially useful for configuration files, SQL queries, and remote commands in SSH connections: ssh server 'bash -s' << 'EOF' ... EOF sends the entire script as stdin to the remote bash process, without needing to copy the script beforehand.

The variant <<'EOF' (with quotes around the delimiter) suppresses variable expansion inside the here-document, which matters when the content contains shell variables that should be evaluated on the remote system rather than locally. <<-EOF ignores leading tabs (not spaces) in the here-document's indentation, which improves code readability in deeply nested contexts. Here-strings with <<< are more compact for single-line stdin input: wc -w <<< "text" counts words without a subshell or a pipe. One subtle detail: here-strings add a trailing newline, while printf '%s' "$var" | cmd does not.

9. Redirection variants compared

For stdin/stdout handling in Bash, there are usually several solutions for every requirement, each with different properties. Choosing the right variant affects readability, performance, and correctness.

Mastering stdin/stdout redirection in Bash is not a mere exercise, it is a prerequisite for scripts that need to work reliably in different contexts. Combining isatty detection, non-interactive flags, /dev/null for empty inputs, and tee with process substitution for multi-output pipelines covers most practical requirements.

10. Summary

Correctly combining stdin, stdout, and interactive tools in Bash requires understanding the execution context. [[ -t 0 ]] and [[ -t 1 ]] reliably detect whether stdin/stdout are connected to a terminal. /dev/null used as stdin prevents SSH and other interactive tools from consuming the script's stdin stream. /dev/tty allows direct terminal access even when stdin/stdout are redirected. Non-interactive flags for external tools eliminate blocking prompts and progress indicators inside pipes.

Pipe compositions with tee and process substitution enable elegant multi-output pipelines without temporary files. ${PIPESTATUS[@]} gives access to the exit codes of every pipe segment. Here-documents with <<'EOF' send scripts as stdin to remote shells without copying anything beforehand. Combining these techniques produces scripts that work consistently and correctly in an interactive terminal, in pipes, as cron jobs, and in CI pipelines, without needing manual reconfiguration.

11. FAQ: Getting stdin, stdout, and interactive tools right