Short answer: “Segmentation fault” means the program touched memory it is not allowed to touch, so the kernel killed it with signal 11 (SIGSEGV); the shell reports exit code 139 (128 + 11), and adds “(core dumped)” when a core file was written. Find out where it crashed before you change anything:
sudo dmesg -T | grep -i segfault | tail -5 # which binary or library crashed
sudo journalctl -k -g segfault --since today # same lines from the journal
ulimit -c # 0 means no core file is written
cat /proc/sys/kernel/core_pattern # where a core file would goIf the crash is in a program you run from a package, the kernel line usually tells you which library to look at. If it is your own code or a Python script, a backtrace (gdb, faulthandler or AddressSanitizer, all below) points at the exact line.
What the error looks like in bash, sh, zsh and containers
The message is not printed by the program: it dies before it can print anything. The shell that started it sees the signal and prints its own text, so the wording depends on the shell:
| Where | What you see | Exit status |
|---|---|---|
| bash, interactive | Segmentation fault, or Segmentation fault (core dumped) when a core file was written | 139 |
| bash -c or a script | bash: line 1: 3017049 Segmentation fault python3 ... | 139 |
| sh (dash on Debian and Ubuntu) | Segmentation fault | 139 |
| zsh | zsh: segmentation fault ./program | 139 |
| Docker, Kubernetes | container exits with exit code 139 | 139 |
| systemd service | status=11/SEGV in systemctl status | unit fails |
To show the real thing without a buggy program, Python can crash itself on purpose by reading address 0 through ctypes. This is what bash 5.2 and dash printed on our Debian 12 host, run in a scratch folder:
$ bash -c 'python3 -c "import ctypes; ctypes.string_at(0)"; echo "exit=$?"'
bash: line 1: 3017049 Segmentation fault python3 -c "import ctypes; ctypes.string_at(0)"
exit=139
$ sh -c 'python3 -c "import ctypes; ctypes.string_at(0)"; echo "exit=$?"'
Segmentation fault
exit=139
$ kill -l 139
SEGVNo “(core dumped)” here: ulimit -c is 0 on that host, so no core file was written. The zsh wording above comes from the zsh source code (its signal message table and job report), which also sets $? to 128 plus the signal number; we did not run zsh. A similar looking status, 126, means the file could not be started at all, which is an exec format error and a different problem.
What SIGSEGV and exit code 139 mean
The signal(7) man page lists SIGSEGV as signal 11 on x86 and ARM, described as “Invalid memory reference”, with the default action “Core”: terminate the process and write a core dump. The GNU bash manual says a command killed by fatal signal N exits with 128+N, so SIGSEGV gives 139. A program can only cause it, never ignore it safely: typical triggers are dereferencing a NULL or freed pointer, writing past the end of a buffer, writing to read-only memory, and running out of stack.
Do not confuse it with running out of memory. The OOM killer sends SIGKILL, the shell says Killed and the exit status is 137 (128 + 9). If that is what you see, read why Linux killed your process instead.
Read the kernel's segfault line
A segfault in a process without its own signal handler normally leaves one line in the kernel log (the kernel rate-limits them when many arrive at once). These are the lines from our test runs, read with dmesg (the same lines are in journalctl -k):
python3[3017049]: segfault at 0 ip 00007ff53dac6ad8 sp 00007ffe3fa717a8 error 4 in libc.so.6[7ff53d985000+156000] likely on CPU 1 (core 1, socket 0)
python3[3017681]: segfault at 0 ip 00007f0e5444cad8 sp 00007ffc1fd8ed38 error 4 in libc.so.6[7f0e5430b000+156000] likely on CPU 5 (core 5, socket 0)
crash[3017998]: segfault at 0 ip 00007fb4e6d24a32 sp 00007fffae23a258 error 6 in libc.so.6[7fb4e6be3000+156000] likely on CPU 7 (core 7, socket 0)
deep[3020410]: segfault at 7ffea69b2c6c ip 000055ed6f4a6144 sp 00007ffea69b2c60 error 6 in deep[55ed6f4a6000+1000] likely on CPU 5 (core 5, socket 0)python3[3017049]: the program and its PID.segfault at 0: the bad address. 0 or a small number is a NULL pointer. An address within a few bytes ofsp(the stack pointer), as in the fourth line, is a stack overflow.error 4orerror 6: the x86 page fault code. 4 is a read from user space, 6 a write.in libc.so.6[...]: the file the crashing instruction lives in. A crash inside libc almost always means the caller passed it a bad pointer; libc is the place, not the cause. A crash in a library you recently upgraded or copied in by hand is a strong lead.
Common causes and how to tell them apart
| Cause | How it shows up | Fix |
|---|---|---|
| Bug in your own C, C++ or Rust unsafe code | Same address and line every run | Backtrace with gdb or AddressSanitizer, fix the pointer |
| Mismatched shared library after an upgrade | Crash in a .so from /usr/local/lib or LD_LIBRARY_PATH | Check ldd ./program, remove the stray copy |
| Broken Python C extension or wheel | faulthandler stops in one import or package call | Reinstall that package in a fresh venv |
| Stack overflow (deep recursion) | Fault address next to sp | Fix the recursion; raise ulimit -s only as a test |
| Faulty RAM (rare) | Different programs crash at random addresses | Memory test on bare metal; on a VPS ask the provider |
After an upgrade: check which libraries are loaded
A program built against one version of a library can crash when it gets an incompatible one with the same file name. That happens with a hand-copied .so in /usr/local/lib, an LD_LIBRARY_PATH left in a profile, or an application bundle that ships its own copies. Compare what the program resolves with what the package manager installed:
ldd ./program # every library and the file it resolves to
echo "$LD_LIBRARY_PATH" # should usually be empty
env -u LD_LIBRARY_PATH ./program # does it still crash without it?If a library is missing instead of wrong, the program does not even start and you get a different message, covered in error while loading shared libraries.
Python scripts that segfault
Python code by itself should not segfault, its errors become exceptions. When a Python process dies with SIGSEGV, the crash is almost always in compiled code: a C extension (numpy, a database driver, an image library), a wheel built for a different version, or ctypes. The other route is very deep recursion after sys.setrecursionlimit() was raised, when every level passes through C code (a __repr__ that calls repr() on itself, for example): on our Python 3.11 that overflowed the C stack and segfaulted, while plain Python recursion ended in RecursionError. Python 3.12 added a separate guard for such C level recursion. Turn on the faulthandler module, which the Python documentation enables with -X faulthandler or PYTHONFAULTHANDLER=1, and Python prints its own traceback just before it dies. Python 3.11 on our host:
$ python3 -X faulthandler -c "import ctypes; ctypes.string_at(0)"
Fatal Python error: Segmentation fault
Current thread 0x00007f3410e26040 (most recent call first):
File "/usr/lib/python3.11/ctypes/__init__.py", line 519 in string_at
File "<string>", line 1 in <module>
/bin/bash: line 1: 3017676 Segmentation fault python3 -X faulthandler -c "import ctypes; ctypes.string_at(0)"The top frames name the package whose compiled part crashed. Recreate the virtual environment and reinstall that package so pip fetches a wheel that matches your Python; on Debian 12 and Ubuntu 24.04 that has to happen in a venv anyway, see externally managed environment.
Enable core dumps
A core file is a snapshot of the process memory at the moment of the crash. The core(5) man page names the two settings that matter most for whether one is written: the RLIMIT_CORE limit (what ulimit -c shows) and /proc/sys/kernel/core_pattern. Raise the limit for the current shell only, then reproduce the crash:
ulimit -c unlimited # this shell and its children only
cat /proc/sys/kernel/core_pattern
./program # crash again, now "(core dumped)"What core_pattern contains decides where the file goes:
core(the kernel default): a file namedcorein the current directory of the crashed process. With the limit raised in a scratch folder, our test printedSegmentation fault (core dumped)and left a 5 MBcorethatfile coreidentified asELF 64-bit LSB core file ... from 'python3 -c import ctypes...'.- A line starting with
|andsystemd-coredump: the core is stored by systemd. List and inspect it withcoredumpctl. - A line with
apport: Ubuntu's crash reporter takes it and writes a report to/var/crash.
coredumpctl list # recent crashes: time, PID, signal, executable
coredumpctl info 3017049 # details and a short stack trace
coredumpctl dump 3017049 -o core # write the core to a file
coredumpctl debug 3017049 # open it in gdb directlyGet a backtrace with gdb
A backtrace lists the chain of function calls that led to the crash. With gdb installed (sudo apt install gdb), either load a core file or run the program under gdb. The flags are from the GDB manual: -batch exits after the commands, -ex runs one command, --args passes the rest of the line to the program:
gdb -batch -ex bt ./program core # from a core file
gdb -batch -ex run -ex bt --args ./program --its-flags # run until it crashes
gdb -batch -ex "thread apply all bt" ./program core # every threadFrames that show ?? instead of function names lack debug symbols. For packaged programs, Debian and Ubuntu serve them over debuginfod: with DEBUGINFOD_URLS set, an interactive gdb asks whether to use it, while a -batch run needs -iex "set debuginfod enabled on". For your own code, rebuild with -g.
C and C++: build with AddressSanitizer
For code you compile yourself, GCC and Clang can add checks that catch the bad access and print where it happened, before or instead of a plain segfault. Our test program passes a NULL pointer to strcpy. Built with GCC 12 on the same host, the plain build only says Segmentation fault, the -fsanitize=address build names the line:
$ gcc -g -fsanitize=address -o crash-asan crash.c && ./crash-asan
AddressSanitizer:DEADLYSIGNAL
==3018005==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f5d98985a32 bp 0x7ffe1f04b110 sp 0x7ffe1f04b0f8 T0)
==3018005==The signal is caused by a WRITE memory access.
==3018005==Hint: address points to the zero page.
#0 0x7f5d98985a32 (/lib/x86_64-linux-gnu/libc.so.6+0x167a32)
#1 0x55c46aa9e1ab in copy_name .../crash.c:5
#2 0x55c46aa9e1d4 in main .../crash.c:10The same build caught our deep recursion test as AddressSanitizer: stack-overflow ... deep.c:2 in depth. Use it on a test or staging build: it makes the program slower and uses more memory. A sanitizer build exits with status 1, not 139.
On a VPS or dedicated server
Before you suspect hardware, read the segfault line. If the same program crashes in the same library at the same offset every time, it is software, whatever machine it runs on. The offset is ip minus the start address in the brackets: for both python3 lines from our runs it is 0x141ad8 in libc.so.6, although the raw addresses differ because of address space randomisation. On a VPS the guest has no access to the host's memory, so faulty RAM is the provider's to test; report it only when unrelated programs crash at random addresses. On a dedicated server, that random pattern (and the CPU number at the end of the line, if it keeps repeating) is the point to run a memory test during a maintenance window.
Frequently asked
What does Segmentation fault (core dumped) mean?
The program accessed memory it was not allowed to, so the kernel killed it with signal 11 (SIGSEGV). The shell prints the message because the program dies before it can print anything. "(core dumped)" means a core file with the process memory was also written, which happens only when ulimit -c allows it or a crash handler such as systemd-coredump collects it.
Why is the exit code 139?
Shells report a process killed by signal N as exit status 128 plus N. SIGSEGV is signal 11, so a segfault gives 139, and kill -l 139 prints SEGV. Docker and Kubernetes report the same 139 for a container whose main process segfaulted.
How do I find out where a program segfaulted?
Start with sudo dmesg -T | grep segfault, which names the program and the library it crashed in. For the exact function, enable core dumps with ulimit -c unlimited and run gdb -batch -ex bt ./program core, or use coredumpctl debug where systemd-coredump is active. Python scripts show their own traceback with python3 -X faulthandler.
Is a segmentation fault caused by bad RAM?
Rarely. If the same program crashes in the same library at the same offset every time, it is a software bug or a mismatched library. Faulty memory shows up as different programs crashing at random addresses; on a dedicated server that is the time for a memory test, on a VPS the host hardware is the provider's to check.