Create a systemd service: run any script or app at boot and restart it on crash

To run a script or an app as a systemd service, write a unit file to /etc/systemd/system/myapp.service, run systemctl daemon-reload, then systemctl enable --now myapp. That starts it immediately, starts it again at every boot and, with Restart=on-failure, restarts it when it crashes. This minimal unit covers most apps, bots and workers:

[Unit]
Description=My app
# start once the network is actually up, not just configured
After=network-online.target
Wants=network-online.target

[Service]
# run as an unprivileged account, never as root without a reason
User=myapp
WorkingDirectory=/opt/myapp
# KEY=value lines, readable by root only
EnvironmentFile=/etc/myapp/myapp.env
# absolute path to the program, then its arguments
ExecStart=/usr/bin/node /opt/myapp/server.js
# restart after a crash, five seconds later
Restart=on-failure
RestartSec=5

[Install]
# start at boot, in the normal multi-user system state
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now myapp
systemctl status myapp

Comments go on their own lines: systemd does not accept a # after a value. The commands work the same on Ubuntu, Debian, Rocky and Alma. The rest of this guide covers the parts that go wrong: paths, users, secrets, crash loops, logs and the error codes systemd prints when a service will not start.

What goes in each section of a unit file?

• [Unit] describes the service and how it relates to others. After= only orders the start; Wants= or Requires= actually pulls the other unit in. That is why network-online.target appears in both.
• [Service] says what to run and how: the program, the user, the working directory, the environment, the restart policy and any limits.
• [Install] is only read by systemctl enable. WantedBy=multi-user.target hooks the service into a normal boot. Without an [Install] section, enable has nothing to do and the service never starts at boot.

Your own units belong in /etc/systemd/system/. Units installed by packages live in /usr/lib/systemd/system/ (on Debian and Ubuntu also reachable as /lib/systemd/system/); do not edit those, because the next package update overwrites them. To change one, use a drop-in, as shown further down. Before you start a new unit, let systemd check it:

sudo systemd-analyze verify /etc/systemd/system/myapp.service

Which Type= do I need?

• simple (the default) — the program stays in the foreground and runs until it is stopped. Node, Python, Go and Java apps, bots and workers.
• exec — like simple, but systemd only reports the start as successful once the program has actually been launched. A wrong path then makes systemctl start itself fail instead of failing a moment later. A good choice for new units.
• forking — for old-style daemons that put themselves into the background and exit the parent process. Add PIDFile= if the daemon writes one. If the program has a “foreground” or “no daemon” option, use that with simple instead.
• oneshot — a script that does a job and exits: a migration, a backup, a firewall setup at boot. Add RemainAfterExit=yes if the service should count as active after the script has finished.

There is also notify, for programs that tell systemd themselves when they are ready. Use it only when the program’s documentation says so.

Why must ExecStart use absolute paths?

Because a service does not run in your shell. systemd reads no .bashrc or .profile, so whatever you added to PATH there does not exist for the service. A bare program name in ExecStart= is looked up only in a fixed list of system directories, which never includes nvm, pyenv, a virtualenv or anything in a home directory. Find the real path in your shell and paste it in:

command -v node        # e.g. /usr/bin/node
command -v python3     # e.g. /usr/bin/python3
readlink -f "$(command -v node)"   # follows symlinks to the real file

ExecStart= is not a shell either. Pipes, redirects, && and ~ do not work in it. If you need shell syntax, run a shell explicitly, or better, put the logic into a script with a shebang line and run that:

ExecStart=/bin/sh -c 'exec /opt/myapp/bin/worker >>/var/log/myapp/worker.log 2>&1'

When a command inside a script cannot be found, the service fails with exit code 127, and Environment=PATH= is the fix; our guide to exit code 127 covers that case.

How do I run a systemd service as a non-root user?

Create a system account for it and set User=. A system account has no password and no login shell, so nobody can log in as it, and a bug in your app can only touch what that account can touch:

sudo useradd --system --create-home --home-dir /opt/myapp --shell /usr/sbin/nologin myapp
sudo chown -R myapp:myapp /opt/myapp

The account must be able to read the program and its files and write wherever the app writes. Two details trip people up. On current Ubuntu and Debian releases, home directories are not readable by other accounts, so an app or a runtime installed under /home/you (nvm is the usual one) fails for the service user. Install the runtime system-wide or move the app to /opt. And an unprivileged account cannot listen on ports below 1024. Add the one capability it needs instead of running as root:

AmbientCapabilities=CAP_NET_BIND_SERVICE

The usual alternative is to let the app listen on a high port such as 3000 and put nginx in front of it.

A few cheap hardening options

These lines lock the service down further and rarely break a normal app. systemd-analyze security myapp rates a unit and lists what else you could turn on.

NoNewPrivileges=yes
PrivateTmp=yes
# /usr, /boot and /etc become read-only for the service
ProtectSystem=full
# home directories become invisible; leave this out if the app lives in /home
ProtectHome=yes

An app that writes below /etc or /usr then fails with Read-only file system; ReadWritePaths= with that one directory opens it again.

How do I pass environment variables and secrets?

Short, harmless settings can go straight into the unit as Environment=NODE_ENV=production. Tokens, passwords and API keys should not. Unit files are readable by every account on the server, and systemctl show prints Environment= values to any user who asks. Put secrets in a separate file that only root can read:

sudo mkdir -p /etc/myapp
sudo tee /etc/myapp/myapp.env >/dev/null <<'EOF'
NODE_ENV=production
PORT=3000
API_TOKEN=paste-the-real-token-here
EOF
sudo chmod 600 /etc/myapp/myapp.env
sudo chown root:root /etc/myapp/myapp.env

The file can stay owned by root even though the app runs as myapp: systemd reads it before it switches users and hands the values to the process. The format is plain KEY=value lines without export; lines starting with # are comments. If the file is optional, write EnvironmentFile=-/etc/myapp/myapp.env with a minus, or a missing file stops the service from starting.

Restart=always or Restart=on-failure?

on-failure restarts the service when it exits with an error, is killed by a signal such as SIGKILL or SIGSEGV, or times out. always also restarts it when it exits cleanly with code 0. Neither restarts a service you stopped yourself with systemctl stop.

• Use on-failure as the default. A clean exit usually means the program meant to stop.
• Use always for things that must never stay down, even when they quit politely, such as a bot that exits after losing its connection.

Then think about the crash loop. By default systemd gives up when a unit starts more than 5 times within 10 seconds. With RestartSec=5 that limit is never reached, so a broken app is restarted every five seconds, forever, filling the journal. Set a limit that fits, in the [Unit] section:

StartLimitIntervalSec=300
StartLimitBurst=5

Once the limit is hit, the unit shows start-limit-hit and stays down. Fix the cause, then run sudo systemctl reset-failed myapp and start it again. On systemd 254 and newer, RestartSteps= and RestartMaxDelaySec= can also stretch the delay between attempts instead of retrying at a fixed pace.

How do I read the logs of a systemd service?

Everything the program writes to stdout and stderr ends up in the journal, tagged with the unit name:

journalctl -u myapp -f                    # follow live, like tail -f
journalctl -u myapp -n 100 --no-pager     # the last 100 lines
journalctl -u myapp --since "1 hour ago"
journalctl -u myapp --since today -p warning   # warnings and worse only
journalctl -u myapp -b                    # since the last boot

• Nothing shows up from a Python app? Python buffers output when it is not writing to a terminal. Add Environment=PYTHONUNBUFFERED=1 to the unit.
• “No journal files were opened” or only part of the log? Run journalctl with sudo, or add your user to the adm or systemd-journal group.
• Logs from before the last reboot are missing? Then the journal is kept in memory only, which is what happens when /var/log/journal does not exist. sudo mkdir -p /var/log/journal and sudo systemctl restart systemd-journald make it persistent.

What do status=203/EXEC and 217/USER mean?

When systemd cannot even start the program, systemctl status shows a code between 200 and 250 instead of the app’s own exit code. The journal line just above it names the reason. The common ones:

• 203/EXEC — the program in ExecStart= could not be executed. “No such file or directory” means the path is wrong, or a script’s shebang names an interpreter that does not exist. That includes a script saved with Windows line endings, whose first line then ends in an invisible \r; the fix is in bad interpreter: No such file or directory. “Permission denied” means the execute bit is missing (chmod +x) or the file sits on a filesystem mounted noexec. “Exec format error” means a script without a shebang line.
• 217/USER — the account in User= does not exist. Check the spelling, and create the account first.
• 200/CHDIR — the WorkingDirectory= does not exist, or the service user cannot enter it.
• status=1, 2 or anything else below 200 — systemd started the program fine, and the program itself failed. The answer is in journalctl -u myapp.

cd /opt/myapp && sudo -u myapp env -i /usr/bin/node /opt/myapp/server.js

That runs the program as the service user, in its working directory, with an empty environment, which is close to what systemd does. If it fails here, it is not a systemd problem.

How do I change a systemd unit safely?

For your own unit in /etc/systemd/system, edit the file and run sudo systemctl daemon-reload, then restart the service. Forget the reload and systemd keeps running the old version and warns you that the file changed on disk.

For a unit that came with a package, use a drop-in. It is a small file that only holds your changes and survives package updates:

sudo systemctl edit nginx          # opens /etc/systemd/system/nginx.service.d/override.conf
systemctl cat nginx                # the unit plus every drop-in, in the order they apply
sudo systemctl restart nginx

systemctl edit reloads systemd when you save. One trap: settings that can appear more than once, such as ExecStart=, are added to rather than replaced. To replace the command, empty it first:

[Service]
ExecStart=
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;' -c /etc/nginx/custom.conf

How do I limit memory or CPU for a service?

systemd puts every service in its own control group, so limits apply to the service and all its child processes:

# start reclaiming memory here
MemoryHigh=400M
# hard cap; above it, only this service is OOM-killed
MemoryMax=512M
# at most half of one CPU core; 200% would be two cores
CPUQuota=50%
# stop a fork bomb or a thread leak
TasksMax=200

sudo systemctl set-property myapp MemoryMax=512M applies a limit to a running service and saves it as a drop-in. systemd-cgtop shows what each service uses right now. When a capped service gets killed for memory, the log reads like the kernel’s out-of-memory killer, because that is what it is; the details are in why Linux kills a process when memory runs out.

Can a systemd timer replace cron?

Yes, and for jobs that matter it is often the better choice: the job gets the journal, the restart and limit options and the same environment as any service, and a missed run can be caught up after a reboot. A timer is two units with the same name, a oneshot service that does the work and a timer that starts it:

[Unit]
Description=Nightly backup

[Service]
Type=oneshot
User=backup
ExecStart=/usr/local/bin/backup.sh

[Unit]
Description=Run the nightly backup

[Timer]
OnCalendar=*-*-* 03:00:00
# run at the next boot if the server was off at 03:00
Persistent=true
RandomizedDelaySec=10min

[Install]
WantedBy=timers.target

sudo systemctl daemon-reload
sudo systemctl enable --now backup.timer
systemctl list-timers backup.timer        # next and last run
systemd-analyze calendar "*-*-* 03:00:00" # check what an expression means
sudo systemctl start backup.service       # run the job once, right now

OnCalendar=hourly, daily, weekly and Mon..Fri *-*-* 08:00 all work too. Cron is still fine for quick one-liners; a timer pays off when you need to see whether the job ran and what it printed. If you stay with cron and a job silently does nothing, see cron job not running.

Can a normal user run a service without sudo?

Yes, with a user service. Put the unit in ~/.config/systemd/user/, leave out User=, use WantedBy=default.target, and manage it with systemctl --user. A user service normally stops when the user’s last session ends. To keep it running after you log out and start it at boot, enable lingering for that account once, as root. Log in to that account directly over SSH for the --user commands; after su or sudo -u they usually fail with “Failed to connect to bus”.

mkdir -p ~/.config/systemd/user
nano ~/.config/systemd/user/myapp.service
systemctl --user daemon-reload
systemctl --user enable --now myapp
journalctl --user -u myapp -f

# once, as root: keep this user's services running without a login
sudo loginctl enable-linger alice

What do real service files look like?

A Node.js app

[Unit]
Description=Node.js web app
After=network-online.target
Wants=network-online.target

[Service]
Type=exec
User=webapp
WorkingDirectory=/opt/webapp
Environment=NODE_ENV=production
EnvironmentFile=/etc/webapp/webapp.env
ExecStart=/usr/bin/node /opt/webapp/server.js
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

A Python app in a virtualenv

There is no need to “activate” the virtualenv. Calling the Python binary inside it is enough, and the same goes for tools installed into it, such as gunicorn.

[Unit]
Description=Python API
After=network-online.target
Wants=network-online.target

[Service]
Type=exec
User=api
WorkingDirectory=/opt/api
Environment=PYTHONUNBUFFERED=1
EnvironmentFile=/etc/api/api.env
ExecStart=/opt/api/venv/bin/gunicorn --workers 2 --bind 127.0.0.1:8000 app:app
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

A Discord bot

A bot should come back on its own after any exit, so this one uses Restart=always with a crash-loop limit. The token stays in the environment file, never in the code or the unit. The full bot setup, from the developer portal to updates, is in hosting a Discord bot on a VPS.

[Unit]
Description=Discord bot
After=network-online.target
Wants=network-online.target
StartLimitIntervalSec=300
StartLimitBurst=5

[Service]
Type=exec
User=discordbot
WorkingDirectory=/opt/discord-bot
Environment=PYTHONUNBUFFERED=1
# DISCORD_TOKEN=... lives here, chmod 600
EnvironmentFile=/etc/discord-bot/bot.env
ExecStart=/opt/discord-bot/venv/bin/python bot.py
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

A game server works the same way, with one extra concern: it should save before it stops. The Minecraft server guide shows a unit with a clean stop command.

What if systemctl says “command not found”?

Then the system does not run systemd, and nothing on this page applies as written. That is normal inside Docker containers, on Alpine Linux and on some container-based VPS platforms. cat /proc/1/comm tells you what runs as PID 1; systemctl: command not found has the alternatives for each case.

Frequently asked