Named Pipes (FIFOs) in Bash: Connecting Processes Without a Temp File

1. What a FIFO is and how it differs from a plain pipe

A named pipe (short: FIFO, first in first out) is a special file type that the filesystem tracks but never actually backs with bytes on disk. Unlike the anonymous pipe a shell creates with | between two commands, which only exists within a single process family, a FIFO has a visible name on the filesystem and can be opened by completely unrelated processes, as long as they have read and write permissions on that path.

That makes a FIFO the right tool whenever two scripts need to talk to each other outside a single pipe chain, for example a daemon writing log lines and a separately started script consuming them live. A temp file would technically work too, but brings unnecessary disk I/O, race conditions between concurrent readers and writers, and the problem of eventually cleaning up stale data. A named pipe solves all three at once because it lives entirely in a kernel buffer.

2. mkfifo: creating a named pipe and understanding its lifetime

The mkfifo command creates a FIFO at any filesystem path. An ls -l shows it with the p file-type letter at the start of the line, and its size stays at zero forever because no data is ever persisted. The FIFO's lifetime as a filesystem entry is independent of the data flow through it: it keeps existing until explicitly removed with rm, even while no process is currently reading or writing.

A common beginner mistake is creating a named pipe in a temp directory and forgetting it might already exist from a previous run. A robust Bash pattern checks whether the path is already a FIFO before calling mkfifo blindly and aborting on error.

#!/usr/bin/env bash
set -euo pipefail

readonly PIPE_PATH="/tmp/mironsoft-log-pipe.$$"

# Idempotent creation: only create if it does not already exist as a FIFO
if [[ ! -p "$PIPE_PATH" ]]; then
  mkfifo -m 600 "$PIPE_PATH"
fi

# Always clean up, no matter how the script exits
trap 'rm -f "$PIPE_PATH"' EXIT

ls -l "$PIPE_PATH"
# prow------- 1 deploy deploy 0 Aug  6 10:00 /tmp/mironsoft-log-pipe.12345

3. Connecting processes through a FIFO: decoupling writer and reader

Once the FIFO exists, any process can write into it and any other process can read from it, as long as both know the same path. That decouples writer and reader completely: they do not need to start at the same time, share a parent process, or even be written in the same language. A Python daemon can write into the same named pipe a Bash script reads from.

The data flow itself behaves exactly like an anonymous pipe: whatever the writer sends with > arrives at the reader with < in the same order, up to the kernel's buffer size (typically 64 KB on Linux). If the writer exceeds that limit before the reader catches up, the write call blocks until the buffer has room again. That is expected backpressure behavior for a FIFO, not a bug.

#!/usr/bin/env bash
set -euo pipefail

readonly PIPE_PATH="/tmp/mironsoft-events.pipe"
[[ -p "$PIPE_PATH" ]] || mkfifo "$PIPE_PATH"

# Reader: runs in the background, processes every line as it arrives
process_events() {
  while IFS= read -r line; do
    echo "[EVENT] $line"
  done < "$PIPE_PATH"
}
process_events &
reader_pid=$!

# Writer: an unrelated process, started independently, feeds the FIFO
{
  echo "deploy.started"
  sleep 1
  echo "deploy.finished"
} > "$PIPE_PATH"

wait "$reader_pid"
rm -f "$PIPE_PATH"

4. Blocking behavior: why a plain open() on a FIFO hangs

The most important difference from a regular file shows up at open time: a process that opens a FIFO for reading only (< pipe) blocks until at least one other process opens it for writing, and vice versa. Anyone who accidentally writes a script that reads from a FIFO without a writer ever starting sees an apparently frozen script, even though nothing is actually wrong. This is the single most common source of confusion for beginners.

A proven Bash pattern for sidestepping this block is opening the FIFO for both reading and writing on the same file descriptor (exec 3<>"$PIPE_PATH"). Bash opens the descriptor in read-write mode, which makes the kernel return immediately because, from its perspective, a writer (the process itself) is already present. This pattern is especially useful for heartbeat scripts that want to keep a FIFO open without waiting for an external writer.

#!/usr/bin/env bash
set -euo pipefail

readonly PIPE_PATH="/tmp/mironsoft-heartbeat.pipe"
[[ -p "$PIPE_PATH" ]] || mkfifo "$PIPE_PATH"

# Open for BOTH read and write on fd 3 -- avoids the "wait for a writer" block
exec 3<>"$PIPE_PATH"

# Now a plain read never blocks forever, because fd 3 counts as a writer too
read -t 2 -u 3 line && echo "Received: $line" || echo "No data within timeout"

exec 3>&-  # close fd 3
rm -f "$PIPE_PATH"

5. Using FIFOs for logging pipelines and multiplexing

One productive use case for named pipes is multiplexing several log sources into a single processing script. Instead of every script in a deployment pipeline writing its own log file, all scripts write into the same FIFO, and a central log collector reads from it, tags each line with a timestamp and source, and forwards it to a monitoring system.

This architecture scales fine as long as it is clear that several concurrent writers to the same FIFO can interleave their lines once a single write exceeds the atomic write buffer of PIPE_BUF (4096 bytes on Linux). For normal log lines that is rarely an issue, but anyone writing very long, single-line JSON events should keep it in mind, or switch to a real message-queue system once line length regularly crosses that boundary.

6. FIFOs vs. process substitution: which tool for which job

Bash offers process substitution (<(command) and >(command)) as syntactic sugar that, under the hood, in many cases builds a construct very similar to an anonymous FIFO, on Linux actually implemented through /dev/fd/N. The difference: process substitution is tied to the lifetime of the calling command and needs no explicit mkfifo call, while a real named pipe exists independently of a single command invocation and can be used by completely independently started processes.

As a rule of thumb: process substitution fits when writer and reader are created within the same script, in the same execution, for example comparing two command outputs with diff <(sort a) <(sort b). A real FIFO created with mkfifo is required when writer and reader run in separate process invocations, separate cron jobs, or even separate containers and only find each other through a shared, known filesystem path.

7. Error handling and cleanup: trap, race conditions, avoiding deadlocks

Because a FIFO stays on the filesystem as an entry until it is removed, trap 'rm -f "$PIPE_PATH"' EXIT belongs in every script that uses mkfifo, exactly like with temp files. Without that trap, orphaned FIFOs accumulate in the temp directory, especially when the script is killed by a signal or aborts before reaching its normal cleanup logic.

A deadlock happens when a script opens a named pipe for writing while it is itself the only process that could ever read it, but sequentially only plans to read after the write. Since opening for writing blocks until a reader exists, the script ends up waiting on itself. The fix is either to start the reader as a background job (&) in parallel, or, as shown in the previous section, to open the FIFO in combined read-write mode to avoid the block altogether.

8. Security considerations: FIFOs in shared directories

If a FIFO is created in a world-writable directory like /tmp, another local user can preemptively claim the same path with their own mkfifo call (a preallocation attack), causing the real script to unknowingly work with a stranger's data instead of its own. The mkfifo call itself fails if the path already exists, but a script that ignores that failure and just proceeds with whatever is there opens a security hole.

The safe Bash pattern combines a guaranteed-unique path from mktemp -u with restrictive permissions set at creation time (mkfifo -m 600), so only the owning user can read or write. For production deployment scripts, the FIFO should also live in a directory owned exclusively by the executing service account, not in world-writable /tmp.

9. FIFOs compared to other IPC mechanisms

Named pipes are just one of several ways independent processes exchange data on Linux. The right mechanism depends on whether data only needs to flow once or should persist, whether multiple readers need concurrent access, and how much data is being transferred.

10. Summary

11. FAQ: Named Pipes (FIFOs) in Bash: The Essentials at a Glance