I am trying to attach to process sshd (from openssh) and call fprintf function.
It works fine on one computer, but on another causes SIGSEGV.
Could anyone of you tell what could be the reason SIGSEGV, please?
Test on Computer 1 - everything works fine:
Reading symbols from /usr/sbin/sshd...
(No debugging symbols found in /usr/sbin/sshd)
Reading symbols from /lib/x86_64-linux-gnu/libwrap.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libwrap.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libaudit.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libaudit.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libpam.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libpam.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libsystemd.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libsystemd.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.3...
(No debugging symbols found in /lib/x86_64-linux-gnu/libcrypto.so.3)
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libz.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libcrypt.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libselinux.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libselinux.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libgssapi_krb5.so.2...
(No debugging symbols found in /lib/x86_64-linux-gnu/libgssapi_krb5.so.2)
Reading symbols from /lib/x86_64-linux-gnu/libkrb5.so.3...
(No debugging symbols found in /lib/x86_64-linux-gnu/libkrb5.so.3)
Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...
(No debugging symbols found in /lib/x86_64-linux-gnu/libcom_err.so.2)
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...
Reading symbols from /usr/lib/debug/.build-id/96/2015aa9d133c6cbcfb31ec300596d7f44d3348.debug...
Reading symbols from /lib/x86_64-linux-gnu/libnsl.so.2...
(No debugging symbols found in /lib/x86_64-linux-gnu/libnsl.so.2)
Reading symbols from /lib/x86_64-linux-gnu/libcap-ng.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libcap-ng.so.0)
Reading symbols from /lib/x86_64-linux-gnu/liblzma.so.5...
(No debugging symbols found in /lib/x86_64-linux-gnu/liblzma.so.5)
Reading symbols from /lib/x86_64-linux-gnu/libzstd.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libzstd.so.1)
Reading symbols from /lib/x86_64-linux-gnu/liblz4.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/liblz4.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libcap.so.2...
(No debugging symbols found in /lib/x86_64-linux-gnu/libcap.so.2)
Reading symbols from /lib/x86_64-linux-gnu/libgcrypt.so.20...
(No debugging symbols found in /lib/x86_64-linux-gnu/libgcrypt.so.20)
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/.build-id/24/6ac0d8deba5a40c63e9a1a87d4d779d8eb589f.debug...
Reading symbols from /lib/x86_64-linux-gnu/libpcre2-8.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libpcre2-8.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libk5crypto.so.3...
(No debugging symbols found in /lib/x86_64-linux-gnu/libk5crypto.so.3)
Reading symbols from /lib/x86_64-linux-gnu/libkrb5support.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libkrb5support.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libkeyutils.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...
Reading symbols from /usr/lib/debug/.build-id/01/c8e0c8389941a2ff675b7e41799362d24cd30b.debug...
Reading symbols from /lib/x86_64-linux-gnu/libtirpc.so.3...
(No debugging symbols found in /lib/x86_64-linux-gnu/libtirpc.so.3)
Reading symbols from /lib/x86_64-linux-gnu/libgpg-error.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libgpg-error.so.0)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x000074157c3274f7 in __libc_connect (fd=5, addr=addr@entry=..., len=len@entry=110) at ../sysdeps/unix/sysv/linux/connect.c:26
26 ../sysdeps/unix/sysv/linux/connect.c: No such file or directory.
(gdb) call fprintf(stderr, "aaah\n")
$1 = 5
sshd version:
root@zauek:~# file /usr/sbin/sshd
/usr/sbin/sshd: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=957213c191d0ab833656c6390749a1270d54d018, for GNU/Linux 3.2.0, stripped
gdb version:
gbajson@zauek:~ (main)$ gdb -v
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Test on Computer 2 - SIGSEGV
Attaching to process 1
Reading symbols from /usr/sbin/sshd...
(No debugging symbols found in /usr/sbin/sshd)
Reading symbols from /usr/lib/x86_64-linux-gnu/libwrap.so.0...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libwrap.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libaudit.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libaudit.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libpam.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libpam.so.0)
Reading symbols from /usr/lib/x86_64-linux-gnu/libsystemd.so.0...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libsystemd.so.0)
Reading symbols from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1)
Reading symbols from /lib/x86_64-linux-gnu/libutil.so.1...
Reading symbols from /usr/lib/debug/.build-id/16/1124b8378f9e197f17726e406f5437fd5a33dd.debug...
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libz.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libcrypt.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libselinux.so.1...
--Type <RET> for more, q to quit, c to continue without paging--
(No debugging symbols found in /lib/x86_64-linux-gnu/libselinux.so.1)
Reading symbols from /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2)
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5.so.3...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libkrb5.so.3)
Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...
(No debugging symbols found in /lib/x86_64-linux-gnu/libcom_err.so.2)
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...
Reading symbols from /usr/lib/debug/.build-id/09/0256e8388c10b0a5a367e269cc0ed39d15d20b.debug...
Reading symbols from /usr/lib/x86_64-linux-gnu/libnsl.so.2...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libnsl.so.2)
Reading symbols from /lib/x86_64-linux-gnu/libcap-ng.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libcap-ng.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...
Reading symbols from /usr/lib/debug/.build-id/25/5e355c207aba91a59ae1f808e3b4da443abf0c.debug...
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...
Reading symbols from /usr/lib/debug/.build-id/46/b3bf3f9b9eb092a5c0cf5575e89092f768054c.debug...
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...
Reading symbols from /usr/lib/debug/.build-id/02/fef90b340c553239e4bf4b2213cc4cb49787eb.debug...
Reading symbols from /lib/x86_64-linux-gnu/liblzma.so.5...
(No debugging symbols found in /lib/x86_64-linux-gnu/liblzma.so.5)
Reading symbols from /usr/lib/x86_64-linux-gnu/libzstd.so.1...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libzstd.so.1)
Reading symbols from /usr/lib/x86_64-linux-gnu/liblz4.so.1...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/liblz4.so.1)
Reading symbols from /usr/lib/x86_64-linux-gnu/libgcrypt.so.20...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libgcrypt.so.20)
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/.build-id/1b/3277a419c3fa42b199e5a170ea215b32689793.debug...
Reading symbols from /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0)
Reading symbols from /usr/lib/x86_64-linux-gnu/libk5crypto.so.3...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libk5crypto.so.3)
--Type <RET> for more, q to quit, c to continue without paging--
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5support.so.0...
(No debugging symbols found in /usr/lib/x86_64-linux-gnu/libkrb5support.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...
(No debugging symbols found in /lib/x86_64-linux-gnu/libkeyutils.so.1)
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...
Reading symbols from /usr/lib/debug/.build-id/82/845af78df2c2866f440f3cae5a8103bd3b5acb.debug...
Reading symbols from /lib/x86_64-linux-gnu/libtirpc.so.3...
(No debugging symbols found in /lib/x86_64-linux-gnu/libtirpc.so.3)
Reading symbols from /lib/x86_64-linux-gnu/libgpg-error.so.0...
(No debugging symbols found in /lib/x86_64-linux-gnu/libgpg-error.so.0)
Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...
Reading symbols from /usr/lib/debug/.build-id/ba/b4b71665bcc7f3f9b142804534c6de15b6e824.debug...
0x00007efcff386dd6 in setdomainname () at ../sysdeps/unix/syscall-template.S:120
120 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb)
(gdb)
(gdb) call fprintf(stderr, "aaah\n")
Program received signal SIGSEGV, Segmentation fault.
0x00007efcffa45425 in malloc (n=6) at dl-minimal.c:50
50 dl-minimal.c: No such file or directory.
warning: Unable to restore previously selected frame.
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on".
Evaluation of the expression containing the function
(malloc) will be abandoned.
When the function is done executing, GDB will silently stop.
sshd version:
root ➜ /devcontainer $ file /usr/sbin/sshd
/usr/sbin/sshd: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=d71ef5505229d585281cb949a536e6d1c1749a77, for GNU/Linux 3.2.0, stripped
gdb version:
root ➜ /devcontainer $ gdb -v
GNU gdb (Debian 10.1-1.7) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
root ➜ /devcontainer $ objdump -T /lib/x86_64-linux-gnu/libc.so.6 | grep fprintf
000000000004df90 g DF .text 0000000000000007 GLIBC_2.2.5 _IO_vfprintf
0000000000053c30 w DF .text 00000000000000b3 GLIBC_2.2.5 _IO_fprintf
0000000000109c60 g DF .text 00000000000000bc GLIBC_2.3.4 __fprintf_chk
0000000000053c30 g DF .text 00000000000000b3 GLIBC_2.2.5 fprintf
0000000000109d40 g DF .text 0000000000000018 GLIBC_2.3.4 __vfprintf_chk
000000000004df90 g DF .text 0000000000000007 GLIBC_2.2.5 vfprintf
root ➜ /devcontainer $ objdump -T /usr/sbin/sshd | grep fprintf
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.3.4 __fprintf_chk
With @Employed Russian suggestion, I also ran it with write, here are results:
(gdb) call write(2, "aaah\n", 5)
Program received signal SIGSEGV, Segmentation fault.
0x00007f2209c3a425 in malloc (n=6) at dl-minimal.c:50
50 in dl-minimal.c
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on".
Evaluation of the expression containing the function
(malloc) will be abandoned.
When the function is done executing, GDB will silently stop.
Calling malloc causes this problem, but I noticed it only in sshd. By contract, in sleep I can call it without issues.
Calling malloc in sshd:
(gdb) call (int)malloc(5)
Program received signal SIGSEGV, Segmentation fault.
0x00007f774e038425 in malloc (n=5) at dl-minimal.c:50
50 dl-minimal.c: No such file or directory.
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on".
Evaluation of the expression containing the function
(malloc) will be abandoned.
When the function is done executing, GDB will silently stop.
(gdb) bt
#0 0x00007f774e038425 in malloc (n=5) at dl-minimal.c:50
#1 0xcc007f774ddd7225 in ?? ()
#2 0x000000006e08d23a in ?? ()
#3 0x000000007de7ef00 in ?? ()
#4 0x00007ffe47eea6c0 in ?? ()
#5 0x000000000000a890 in ?? ()
#6 0x00007f774da56b80 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#7 0x0000000000000008 in ?? ()
#8 0x0000556378985770 in ?? ()
#9 0x0000000000000000 in ?? ()
(gdb)
Calling malloc in program sleep works without any problems:
Reading symbols from /bin/sleep...
(No debugging symbols found in /bin/sleep)
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...
Reading symbols from /usr/lib/debug/.build-id/f3/f3ff7a8641c7e365d35db4423c1422b1ab154b.debug...
Reading symbols from /lib64/ld-linux-x86-64.so.2...
Reading symbols from /usr/lib/debug/.build-id/1b/3277a419c3fa42b199e5a170ea215b32689793.debug...
0x00007f59355d8943 in __libc_pause () at ../sysdeps/unix/sysv/linux/pause.c:29
29 ../sysdeps/unix/sysv/linux/pause.c: No such file or directory.
(gdb) call (int)malloc(5)
$1 = 896430032
Backtrace after SIGSEGV:
(gdb) call (int)fprintf(1, "aah\n")
Program received signal SIGSEGV, Segmentation fault.
0x00007fbf34f9ed91 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
The program being debugged was signaled while in a function called from GDB.
GDB remains in the frame where the signal was received.
To change this behavior use "set unwindonsignal on".
Evaluation of the expression containing the function
(fprintf) will be abandoned.
When the function is done executing, GDB will silently stop.
(gdb) bt
#0 0x00007fbf34f9ed91 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007fbf34f8bcc6 in fprintf () from /lib/x86_64-linux-gnu/libc.so.6
#2 <function called from gdb>
#3 0x00007fbf35029dd6 in select () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x000055ad64061574 in ?? ()
#5 0x00007fbf34f5bd0a in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#6 0x000055ad640631ba in ?? ()
Finding:
Everything worked fine with gdb version 12.1.
Program received signal SIGSEGV, Segmentation fault.
0x00007efcffa45425 in malloc (n=6) at dl-minimal.c:50
This looks like a bug in GLIBC -- calling fprintf should not use malloc from dl-minimal.cc (it should use a different malloc).
That said, calling fprintf at arbitrary time is a bad idea(TM). Here you can achieve the exact same result by calling write(2, "aaah\n", 5).