Using Bash Coprocesses in Practice

1. What a coprocess is and when it pays off

A coprocess is an asynchronously started background process that the calling shell can communicate with bidirectionally over a pair of file descriptors, instead of merely collecting its output. The crucial difference from an ordinary background job started with & is that Bash automatically provides one descriptor for writing to the process's standard input and one for reading from its standard output. That lets a program started once be reused for the entire runtime of the script.

The classic use case for a coprocess is a loop that sends hundreds or thousands of small requests to the same external tool, for example a database CLI, a calculator like bc, or a REPL. Without a coprocess, every single request would have to spawn a new process, which feels fast for small tools but adds up measurably over a large number of iterations. A coprocess starts the tool once and keeps the connection open, eliminating the overhead of repeated fork and exec.

Not every problem justifies a coprocess. For simple, one-off calls the added complexity isn't worth it, and for very high parallelism a single coprocess is inherently a serial bottleneck, since only one request can be answered at a time. The following sections show how a coprocess is declared syntactically, how the bidirectional communication works, and where the practical limits lie.

2. The coproc syntax: declaration and file descriptors

The coproc builtin was introduced in Bash 4.0 and starts a named or unnamed coprocess. The simplest form is coproc followed by the command, which makes Bash automatically create an array named COPROC: COPROC[0] is the file descriptor for reading the process's output, COPROC[1] the file descriptor for writing to its input. These two numbers are indices into the shell's file descriptor table and can be used directly with redirection operators such as <& and >&.

If the coprocess is declared with its own name, for example coproc DB { sqlite3 app.db; }, the arrays are called DB instead of COPROC, and multiple coprocesses can exist in parallel without overwriting each other. This named form is almost always preferable in practice, because the unnamed COPROC array gets overwritten on a second coproc call, otherwise losing access to the first coprocess.

#!/usr/bin/env bash
set -euo pipefail

# Named coprocess: keeps its own file descriptor array
coproc CALC { bc -l; }

echo "Read FD:  ${CALC[0]}"
echo "Write FD: ${CALC[1]}"

# Write an expression into the coprocess's stdin
echo "22/7" >&"${CALC[1]}"

# Read exactly one line back from the coprocess's stdout
read -r pi_approx <&"${CALC[0]}"
echo "22/7 = $pi_approx"

# Send another expression on the same still-running process
echo "sqrt(2)" >&"${CALC[1]}"
read -r sqrt2 <&"${CALC[0]}"
echo "sqrt(2) = $sqrt2"

# Terminate the coprocess explicitly when done
exec {CALC[1]}>&-
wait "$CALC_PID"

Worth noting is the automatically created variable with the _PID suffix, CALC_PID in the example. It holds the process ID of the coprocess child and is needed to terminate it deliberately with kill later or to wait for its exit code. Without this variable the PID would have to be looked up manually via ps, which is error prone and unnecessarily complicated.

3. Bidirectional communication through arrays

The real value of a coprocess lies in bidirectional communication, something a simple pipe does not offer. A pipe like command_a | command_b connects two processes in one direction only, while a coprocess lets the shell itself write and read alternately, in a classic request response pattern. That is exactly what makes a coprocess the right choice for interactive tools that answer every input with exactly one output line.

What matters for the robustness of this pattern is that exactly one line comes back per request, otherwise read either blocks forever because no further line arrives, or accidentally reads the answer meant for a later request. Tools that produce multi-line or unpredictable output are therefore poor candidates for a coprocess, unless you can force a unique delimiter at the end of every answer, for example a prompt pattern or a sentinel line.

#!/usr/bin/env bash
set -euo pipefail

coproc JQFILTER { jq --unbuffered -c '.value * 2'; }

# Send several independent JSON lines to the same running coprocess
for n in 3 7 21; do
  echo "{\"value\": $n}" >&"${JQFILTER[1]}"
  read -r result <&"${JQFILTER[0]}"
  echo "doubled: $result"
done

exec {JQFILTER[1]}>&-
wait "$JQFILTER_PID"

4. Practical example: an SQLite session as a coprocess

A realistic example from everyday administration: a script needs to run a single lookup in an SQLite database for each of thousands of CSV rows. Without a coprocess, every row would start a new sqlite3 process, complete with database connection, opening the file and terminating the process, repeated thousands of times. With a coprocess, sqlite3 is started exactly once and stays open for all lookups.

The trick with sqlite3 as a coprocess is configuring the output mode so every answer ends in exactly one line, and printing a unique sentinel line after each query so the script knows where an answer ends. This pattern transfers to any REPL-style tool that processes input line by line in batch mode.

#!/usr/bin/env bash
set -euo pipefail

coproc DB { sqlite3 -batch -noheader app.db; }

query_db() {
  local sql="$1"
  # Emit a unique sentinel after each query result to mark its end
  echo "${sql}" >&"${DB[1]}"
  echo "SELECT '###END###';" >&"${DB[1]}"

  local line result=""
  while IFS= read -r line <&"${DB[0]}"; do
    [[ "$line" == "###END###" ]] && break
    result+="${line}"$'\n'
  done
  printf '%s' "$result"
}

while IFS=, read -r customer_id order_total; do
  balance=$(query_db "SELECT balance FROM accounts WHERE id = ${customer_id};")
  echo "Customer ${customer_id}: balance ${balance}, order ${order_total}"
done < orders.csv

exec {DB[1]}>&-
wait "$DB_PID"

This example shows the full benefit of a coprocess: a single database connection serves thousands of lookups, while the shell keeps using familiar control structures like while loops and CSV processing. In benchmarks with several thousand rows, the speed gain over repeated process startup is typically a factor of ten to twenty, depending on the startup cost of the tool in question.

5. Coprocesses vs. named pipes vs. process substitution

Named pipes, created with mkfifo, solve a problem similar to a coprocess, but require two separate file system entries and more manual opening and closing. A coprocess fully encapsulates setting up the connection inside a Bash builtin, without any temporary files becoming visible in the file system, which is also a small security advantage in multi-user environments.

Process substitution with <(command) and >(command), on the other hand, is unidirectional: it is excellent for comparing the outputs of two commands or duplicating one output into several sinks, but not for a request response pattern inside a loop. A coprocess is the only pure Bash built-in solution that allows true bidirectional, interactive communication with a running process without reaching for external tools such as socat.

6. Error handling and timeouts with coprocesses

A coprocess that crashes unexpectedly leaves behind an invalid file descriptor, and the next read call does not fail with an error but often blocks forever if no timeout safeguard exists. read -t seconds gives read a timeout and returns a non-zero exit code if no line arrives in that time, letting the script detect the failure of the coprocess and react instead of hanging indefinitely.

In addition, before writing to a coprocess you should check whether the process is still alive, for example with kill -0 "$DB_PID" 2>/dev/null. If that check fails, the coprocess has already ended, and a write attempt would trigger SIGPIPE. Anyone who consistently combines these two safeguards, a read timeout and an existence check before writing, avoids the most common hangs in production scripts using coprocesses.

#!/usr/bin/env bash
set -euo pipefail

coproc WORKER { ./slow-lookup-tool.sh; }

safe_query() {
  local input="$1"
  # Guard: is the coprocess still alive before writing?
  if ! kill -0 "$WORKER_PID" 2>/dev/null; then
    echo "[ERROR] Coprocess is no longer running" >&2
    return 1
  fi

  echo "$input" >&"${WORKER[1]}"

  local answer
  # Guard: don't block forever if the coprocess hangs
  if ! read -r -t 5 answer <&"${WORKER[0]}"; then
    echo "[ERROR] Coprocess timed out after 5s" >&2
    return 1
  fi
  printf '%s\n' "$answer"
}

7. Lifecycle: shutting down coprocesses cleanly

A coprocess must be shut down actively, otherwise the child process stays alive even after the parent script ends, since it is not automatically terminated with the parent shell. The correct way is to first close the write descriptor with exec {ARRAY[1]}>&-, which signals EOF on the coprocess's standard input. Many REPL-style tools then shut themselves down cleanly, similar to pressing Ctrl-D in an interactive session.

If the tool does not react to EOF, an explicit kill "$ARRAY_PID" is needed, followed by wait to avoid zombie processes and collect the final exit code. In production scripts, shutting down a coprocess belongs in a trap cleanup function, so that even after an error or signal in the middle of the script, no orphaned process is left behind.

#!/usr/bin/env bash
set -euo pipefail

coproc SESSION { ./interactive-tool.sh; }

cleanup() {
  # Signal EOF to the coprocess, then force-terminate if needed
  exec {SESSION[1]}>&- 2>/dev/null || true
  if kill -0 "$SESSION_PID" 2>/dev/null; then
    kill "$SESSION_PID" 2>/dev/null || true
  fi
  wait "$SESSION_PID" 2>/dev/null || true
}
trap cleanup EXIT

8. Limits of coprocesses in Bash

Bash effectively supports only one unnamed coprocess per shell at a time; named coprocesses work around this limitation but bring extra complexity along. For true parallel processing with several coprocesses working simultaneously, the code quickly becomes hard to follow, since every combination of read and write descriptor has to be managed manually. Beyond a certain complexity, a coprocess in Bash is no longer the right tool.

A second edge case involves tools with buffered output. Many programs buffer their output when connected to a pipe instead of a terminal, which makes a coprocess appear to hang even though the process has long since answered internally. Options such as --unbuffered or the stdbuf tool help force or disable buffering in such cases. Without this adjustment, every coprocess wrapping a buffered tool falsely looks frozen.

9. Coprocess approaches compared

The table below compares the different ways of process communication in Bash and shows when a coprocess is the right choice.

A coprocess is thus the most compact pure Bash solution for bidirectional, persistent communication, while named pipes offer more control over file system visibility but also require more boilerplate. For most batch processing cases in administration scripts, a coprocess is entirely sufficient.

10. Summary

A coprocess is Bash's built-in tool for bidirectional, persistent communication with a background process. coproc automatically creates an array with read and write descriptors plus a PID variable, through which the shell can talk to the process in a request response pattern. The biggest gain shows up in loops with many iterations against the same external tool, where repeated process startup would otherwise cost noticeable time.

Robust coprocesses additionally need a read timeout safeguard, an existence check before writing, and a clean cleanup function that closes the write descriptor and terminates the process if necessary. Anyone who consistently combines these three building blocks uses a coprocess without the typical hangs that quickly appear with naive usage.

11. FAQ: Bash Coprocesses