I was curious what would happen if I put db 0x41 in the .text section, instead of in .data where it would normally belong. It segfaults, but why exactly?
The bits of code below were compiled, linked and executed in Mint 19.1 with nasm and ld.
No segfault:
global _start
section .data
db 0x41
section .text
_start:
mov rax, 60 ; Exit(0) syscall
xor rdi, rdi
syscall
Segfault:
global _start
section .text
_start:
db 0x41
mov rax, 60 ; Exit(0) syscall
xor rdi, rdi
syscall
I'm using the following commands to assemble, link and run it:
nasm -felf64 main.s -o main.o
ld main.o -o main
./main
If you tell the assembler to assemble arbitrary bytes somewhere, it will. db is a pseudo-instruction that emits bytes, so mov eax, 60 and db 0xb8, 0x3c, 0, 0, 0 are exactly equivalent as far as NASM is concerned. Either one will emit those 5 bytes into the output at the current position.
If you don't want your data decoded as (part of) instructions, don't put it where it will be reached by execution. (For example, put it in section .rodata or section .data before or after your code, like in the question. You can switch back and forth between section .text and other sections, if you want your asm source to define static/global data near each function that uses some.)
Since you're using NASM1, it optimizes mov rax,60 into mov eax,60, so the instruction doesn't have the REX prefix you'd expect from the source.
Your manually-encoded REX prefix for mov changes it into a mov to R8D instead of EAX:
41 b8 3c 00 00 00 mov r8d,0x3c
(I checked with objdump -drwC -Mintel instead of looking up which bit is which in the REX prefix. I only remember that REX.W is 0x48. But 0x41 is a REX.B prefix in x86-64).
So instead of making a sys_exit system call, your code runs syscall with EAX=0, which is __NR_read. (The Linux kernel zeros all the registers other than RSP before process startup, and in a statically-linked executable, _start is the true entry point with no dynamic linker code running first. So RAX is still zero).
$ strace ./rex
execve("./rex", ["./rex"], 0x7fffbbadad60 /* 54 vars */) = 0
read(0, NULL, 0) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV (core dumped) +++
And then execution falls through into whatever is after syscall, which in this case is 00 00 bytes that decode as add [rax], al, and thus segfault. You would have seen this if you'd run your code inside GDB.
Footnote 1: If you'd used YASM which doesn't optimize to 32-bit operand size:
Intel's manuals say that it's illegal to have 2 REX prefixes on one instruction. I expected an illegal-instruction fault (#UD machine exception -> kernel delivers SIGILL), but my Skylake CPU ignores the first REX prefix and decodes it as mov rax, sign_extended_imm32.
Single-stepping, it's treated as one long instructions, so I guess Skylake chooses to handle it like other cases of multiple prefixes, where only the last one of a type has an effect. (But remember this is not future-proof, other x86 CPUs could handle it differently.)
Related / same bug in other situations: