Controlling ulimit and Resource Limits in Bash Scripts

1. What ulimit is: soft limits, hard limits, and how they interact

ulimit is a Bash built-in that reads and sets the per-process resource limits enforced by the kernel, such as the maximum number of open files or the maximum process count for a user. Every one of these limits always exists in Bash as a pair: a soft limit, the currently enforced value, and a hard limit, marking the absolute ceiling up to which an unprivileged process is allowed to raise its own soft limit.

A process with no special privileges can lower its own soft limit at any time and raise it again up to the hard limit, but never beyond it, and the hard limit itself can only be lowered, never raised, without root privileges. That interplay explains why a deployment script attempting to set ulimit -n 100000 fails with an error if the system-wide hard limit is lower, even when the script runs with full root privileges, as long as no system-wide limit has been raised.

2. ulimit -n: open file descriptors and the cause of too many open files

ulimit -n caps the number of file descriptors a single process may hold open at once, where every opened file, socket, and pipe each consumes one descriptor. Once a process exceeds that limit, the kernel refuses any further open() or socket() call with the EMFILE error, which typically shows up in application logs as "Too many open files".

In deployment scripts, this error hits web servers or database processes especially often, since they hold many concurrent connections open and run into a soft limit that is too low, frequently inherited from an outdated distribution default of 1024. The short-term fix inside the script itself is ulimit -n 65536 before actually starting the service, provided the hard limit allows that value; otherwise the hard limit must first be raised system-wide.

# Check current limits before starting the service
echo "Soft limit (open files): $(ulimit -Sn)"
echo "Hard limit (open files): $(ulimit -Hn)"

# Raise the soft limit up to the hard limit for this process
ulimit -n 65536 || {
  echo "Could not raise file descriptor limit" >&2
  exit 1
}

exec ./my-webserver

3. ulimit -u: capping the maximum number of processes per user

ulimit -u determines how many processes and threads a user may start simultaneously, and on Linux the kernel counts threads as separate entries, which makes this limit hit far sooner in heavily multithreaded applications than the plain process name might suggest. Once the limit is reached, every further fork() call fails, and a deployment script can no longer start any additional background processes itself.

This limit is especially treacherous when set independently for a service account that launches several parallel worker processes: an overly low ulimit -u does not cause an immediate, clear failure during deployment itself, but rather a seemingly random failure of individual workers once the account's total running thread count crosses the limit, often only visible under production load.

echo "Current process limit: $(ulimit -u)"

if (( $(ulimit -u) < 4096 )); then
  echo "WARNING: process limit below 4096, workers may fail" >&2
fi

4. ulimit -v: capping virtual memory per process

ulimit -v caps the maximum reservable virtual memory of a process in kilobytes, not the physical memory actually used. Because modern applications, especially those on a Java or Node.js runtime, frequently reserve far more virtual address space than they actually use physically, an overly tight ulimit -v often crashes the application right at startup, long before any real memory pressure appears.

For deployment scripts that want to deliberately protect a process from excessive memory consumption, ulimit -v is therefore usually the wrong tool, since it caps virtual rather than physical memory. A more precise tool for that goal is a cgroup-based memory limit through systemd, while ulimit -v in practice remains mainly useful where a deliberately tight sandbox for small, known scripts is desired.

# Deliberately restrictive sandbox for a small trusted script
( ulimit -v 262144  # 256 MB virtual memory
  exec ./small-batch-job.sh )

5. Setting limits only for a script's runtime: the subshell pattern

ulimit calls inside a running Bash script default to affecting the current process and get inherited by every subsequent child process, but stay entirely scoped to the current shell session and vanish once the script ends. Anyone wanting to change a limit for just a single step of a larger script should deliberately wrap that step in a subshell using parentheses.

Limits set inside the subshell apply exclusively to that subshell and the child processes it launches; the surrounding script and every step that follows keep their original limits unchanged. This pattern is especially useful when a deployment script wants to deliberately cap one potentially memory-hungry build step, without affecting limits for the rest of the deployment pipeline.

echo "Before the subshell: $(ulimit -v)"

(
  ulimit -v 1048576   # only inside this subshell: 1 GB
  ./memory-intensive-build.sh
)

echo "After the subshell: $(ulimit -v)"   # unchanged

6. Setting limits permanently: limits.conf versus systemd unit files

A limit set with ulimit inside a script only ever applies to the current process tree instance and is lost once the script ends. For permanent, system-wide, or user-specific limits meant to apply automatically at every new login or service start, /etc/security/limits.conf is the classic place from which PAM-based logins pull their limits, configurable per user, group, or with a wildcard entry for everyone.

For services started through systemd, however, limits.conf frequently does not apply, since systemd services bypass PAM login sessions. The more reliable approach here is setting LimitNOFILE, LimitNPROC, or LimitAS directly in the service's systemd unit file, because systemd enforces these directives itself when starting the service, independently of limits.conf, and thus applies consistently on every restart, regardless of who triggers the service.

7. Typical problems in deployment scripts caused by too-low limits

The most common symptom of a too-low limit in production deployment scripts is a service that starts fine locally during development but crashes sporadically in production with "Too many open files" or "Resource temporarily unavailable", because the production environment ships a more restrictive default configuration than the local development machine, which often runs on more generous distribution defaults.

A second common problem arises when a deployment script raises limits via ulimit in its own process, but the actual service starts through systemd or a process manager that builds its own, independent environment with its own limits. The ulimit values set in the deployment script then simply get ignored, since they only applied to the short-lived deployment process itself, not to the actually long-lived service process.

8. Checking and diagnosing limits: ulimit -a and /proc/[pid]/limits

ulimit -a prints a full overview of every limit currently in effect for the running shell, handy for a quick manual check while developing a deployment script. For diagnosing an already-running, unrelated process, /proc/[pid]/limits is the right source instead, since it reports both soft and hard limits separately for every resource, regardless of which shell, or no shell at all, originally started the process.

A diagnostic script that needs to verify whether an already-running production service actually started with the expected limits should therefore read /proc/[pid]/limits rather than relying on the limits of its own diagnosing shell, which typically has entirely different values than the monitored target service.

# Full overview in the current shell
ulimit -a

# Limits of a specific running process
pid=$(pgrep -f "my-webserver" | head -n1)
grep -E "open files|processes" "/proc/$pid/limits"

9. Best practices for production deployment scripts

A clean deployment script checks required limits explicitly before actually starting the service and aborts with a clear error message, instead of blindly starting the service with insufficient limits and only letting the failure surface in production. That check should always account for both the soft and the hard limit, since only the hard limit reliably shows how far the soft limit can even be raised without extra privileges.

For long-lived, permanently running services, limits belong consistently in the systemd unit file rather than in a one-time deployment script, because systemd reliably re-enforces those limits on every service restart, regardless of when or how the deployment script last ran. ulimit inside the script therefore remains the right tool mainly for short-lived build and batch steps.

10. Summary

11. FAQ: ulimit in Bash: The Essentials at a Glance