My problem is very similar but not the same with the this one.
I run the same example of exploit_notesearch.c in the book: Hacking, the Art of Exploitation on my 64-bit OS, Archlinux and it doesn't work.
From the above link I learnt that it just can't work on most 64-bit systems. But I still can't understand why the programme have to do this: ret = (unsigned int)&i - offset. Why can't I just do this: ret = (unsigned)shellcode so that I can replace the vulnerable program's return address with shellcode's beginning address?
The ret = (unsigned)shellcode will make the ret equals to the address of the shellcode array in your program. But that address is not the address of your malicious code in the target program(notesearch.c). The target process will put its searchstring on stack, so that your malicious code will be also put onto the stack of the target process.
In old days, the memory layout of processes was typically highly deterministic - the location of the stack buffer could usually be predicted quite well by the attacker (particularly if they knew exactly which version of the target software was being attacked). So it will be very easy to know what is the exact address of the searchstring and your shellcode.
However, today, many operating system will perform ASLR. So attackers trying to execute shellcode injected on the stack have to find the stack first. The system obscures related memory-addresses from the attackers. These values have to be guessed, and a mistaken guess is not usually recoverable due to the application crashing(Segmentation Fault).
To improve the chances of success when there was some guesswork involved, the active shellcode would often be preceeded by a large quantity of executable machine code that performed no useful operation - called a "NOP sled" or "NOP slide".
So even the ret = (unsigned int)&i - offset can not make sure your shellcode will be executed succesfully.