nohup command: keep a process running after you log out

Short answer: nohup COMMAND & starts COMMAND in the background with the hangup signal (SIGHUP) ignored, so it keeps running after you close the terminal or your SSH connection drops. Give it a log file of your own, or its output lands in nohup.out in the current directory: nohup ./job.sh > job.log 2>&1 & is the line to remember. Watch it with tail -f job.log and stop it with kill PID.

nohup ./job.sh > job.log 2>&1 &    # start it, output and errors to job.log
echo $!                             # its PID (the shell also prints it as [1] 48213)
tail -f job.log                     # follow the output; Ctrl+C stops tail, not the job
kill 48213                          # stop the job

The examples below were run on Debian 12 with GNU coreutils 9.1 and bash 5.2.15, and the messages are copied from those runs. The logout case was reproduced by cutting the pseudo-terminal under an interactive bash, which is what sshd does when a connection drops. The notes on sudo, setsid and systemd-run rest on their documentation.

nohup cheat sheet

Why a dropped SSH session kills your jobs: SIGHUP

When the terminal goes away (you close the window, or the SSH connection drops or times out), the kernel sends SIGHUP, signal 1, to the shell that owns the terminal. Bash then sends the same signal to every job it started, running or stopped, before it exits; the bash manual describes exactly this. The default action for SIGHUP is to terminate, so a long rsync, an import or a compile started with a plain & dies with your session.

nohup sets SIGHUP to “ignored” and then replaces itself with your command. An ignored signal stays ignored across that switch, so the program never notices the hangup. You can see it in the process’s ignored-signal mask, where the lowest bit stands for signal 1:

nohup sleep 600 > /dev/null 2>&1 &
grep SigIgn /proc/$!/status

[1] 48213
SigIgn:	0000000000000001

Here is what survived the simulated dropped connection:

The last row surprises people. Typing exit does not send SIGHUP to background jobs (bash only does that when the huponexit option is set and a login shell exits), so the job keeps going; it only runs into trouble if it writes to the terminal that is no longer there (more on that under disown below). Closing the window or losing the connection is a hangup, and that does kill it.

nohup.out: where the output goes

$ nohup ./backup.sh &
[1] 48213
nohup: ignoring input and appending output to 'nohup.out'

nohup only redirects what is still connected to the terminal. The rules, from the GNU coreutils manual:

• Standard output is a terminal: output is appended to nohup.out in the current directory. If that is not writable it goes to $HOME/nohup.out, and if neither works the command is not run at all.
• Standard error is a terminal: it goes wherever standard output went, so errors land in the same file.
• Standard input is a terminal: it is replaced by an unreadable file descriptor, so a program that tries to read from the keyboard gets an error instead of waiting forever.

Two details matter on a server. nohup appends: run the same job twice from one directory and both runs end up in one file, which grows until someone deletes it. A forgotten nohup.out is a classic way to end up with No space left on device. And a nohup.out that nohup creates is readable and writable by you only (mode 600, whatever your umask), so another user or the web server cannot read it.

Redirecting nohup output to your own log

nohup ./import.sh > import.log 2>&1 &     # start a fresh import.log each run
nohup ./import.sh >> import.log 2>&1 &    # keep adding to it
nohup ./import.sh > /dev/null 2>&1 &      # keep nothing

> import.log sends standard output to the file and 2>&1 then points standard error at the same place. Write the file first and 2>&1 after it: for almost every other command, 2>&1 > import.log leaves the errors on your screen. nohup happens to forgive it (it sees that standard error is still the terminal and points it at the file itself, which is the “redirecting stderr to stdout” message), but the habit pays off everywhere else. To get rid of the nohup: ignoring input line at the top of the log, add < /dev/null: with nothing left on the terminal, nohup prints nothing.

The log stays empty while the job runs

That is usually output buffering, not nohup. A program that writes to a file instead of a terminal collects its output and writes it in blocks. Checked with Python 3.11 on Debian 12: a script started with nohup python3 job.py > job.log 2>&1 & left the log empty until it exited, while the same script under python3 -u wrote each line at once.

nohup python3 -u job.py > job.log 2>&1 &                    # Python: unbuffered
PYTHONUNBUFFERED=1 nohup python3 job.py > job.log 2>&1 &    # the same, by environment
nohup stdbuf -oL ./tool > tool.log 2>&1 &                   # C programs: line-buffered

stdbuf only changes programs that use the C library’s default buffering; one that sets up its own buffering, as Python does, ignores it.

Pipelines, several commands and root

nohup protects one program: the one named right after it. In nohup cmd1 | cmd2 & and nohup cmd1 && cmd2 & only cmd1 ignores the hangup, so the second half dies at logout. Wrap the whole line in a shell instead:

nohup bash -c 'tar -cf - /var/www | gzip > www.tar.gz' > tar.log 2>&1 &
nohup bash -c './import.sh; echo "exit status: $?"' > import.log 2>&1 &

The second line is worth copying for long jobs. Once you have logged out and back in, nothing waits for the job any more, so nobody sees its exit code unless it is written to the log.

For a job that needs root, start a root shell first (sudo -i) and use nohup there, or hand the job to systemd with systemd-run (see the comparison below). Prefixing only the command, as in sudo nohup cmd > job.log &, still opens job.log as your own user, because your shell sets up the redirection before sudo starts, and a password prompt cannot be answered once the job is in the background.

Checking on a nohup job and stopping it

$ nohup ./import.sh > import.log 2>&1 &
[1] 51877
$ echo $! > import.pid       # keep the PID for later
$ jobs -l
[1]+ 51877 Running                 nohup ./import.sh > import.log 2>&1 &

jobs only knows the jobs of the shell you are typing in. After you log back in it is empty, so find the job by name or by the PID you saved:

pgrep -af import.sh                              # PID and full command line
ps -o pid,etime,stat,cmd -p "$(cat import.pid)"  # how long it has been running
tail -f import.log                               # what it is doing now

51877 /bin/bash ./import.sh
    PID     ELAPSED STAT CMD
  51877       42:16 S    /bin/bash ./import.sh

To stop it, ask first and force only if it ignores you:

kill 51877                            # SIGTERM: exit cleanly
kill -0 51877 && echo "still running"  # check after a few seconds
kill -9 51877                         # last resort

kill -HUP does nothing to a job started with nohup, unless the program installs its own handler for it: that is the whole point. Checked: after kill -HUP, a nohup’d sleep was still running while a plain one exited with Hangup. Signals and when to escalate are covered in how to kill a process in Linux.

disown: the shell’s own way

disown is a bash builtin that removes a job from the shell’s job table, so bash does not pass SIGHUP on to it. disown -h keeps the job in the table, so jobs still lists it, but marks it to be skipped. Both survived the dropped connection in the table above. What disown does not do is touch the job’s output, and that matters:

Forgot nohup? Ctrl+Z, bg, disown -h

$ ./import.sh
^Z
[1]+  Stopped                 ./import.sh
$ bg
[1]+ ./import.sh &
$ disown -h %1

Ctrl+Z pauses the foreground job, bg resumes it in the background and disown -h %1 keeps bash from sending it SIGHUP. That sequence survived a hangup on Debian 12. Its output is still wired to your terminal, though, so the warning above applies: a job that prints progress will probably die at its next line once you disconnect. Treat it as a rescue for quiet jobs, not a habit.

setsid: no terminal at all

setsid -f from util-linux starts the command in a new session with no controlling terminal, so there is no hangup to receive in the first place. It does nothing about output either, so redirect all three streams:

setsid -f ./job.sh > job.log 2>&1 < /dev/null

nohup vs tmux, screen and a systemd service

If you might need to type into the job or see its screen again, use tmux instead. If it has to keep running across reboots and come back after a crash, write a unit file: how to create a systemd service walks through it. nohup is for the space in between, and systemd-run is the quick version of a service for a single run:

systemd-run --unit=import /usr/local/bin/import.sh
journalctl -u import -f      # its output
systemctl status import      # still running?

Common nohup problems

• The job still died when I disconnected. It was part of a pipeline or an && chain (only the first program is protected), logind is set to KillUserProcesses=yes, or it was disowned without redirecting its output.
• It died hours later. nohup only covers the hangup. Check whether the kernel killed it for memory with journalctl -k | grep -i "killed process" (see OOM killed process), or whether the server rebooted with uptime -s.
• nohup.out is empty. Output buffering (above), or the output went to the log you named instead. ls -la nohup.out *.log shows which file is growing.
• nohup.out keeps growing. Redirect to /dev/null if you never read it, or to a named log you clean up. truncate -s 0 nohup.out empties it without stopping the job: nohup opens the file for appending (the coreutils manual says so), so the job simply carries on writing at the new end of the file.
• failed to run command. Exit code 127 means the name is not a program on disk (a typo, a missing PATH entry, or a builtin like cd); 126 means the file exists but is not executable.

Frequently asked