I've been trying forever to put a value into a passed by address argument. It's on the stack somewhere.
The function is called through a C++ program linked to the assembly program. It passes in 8 arguments, one of which is a float. First argument is in RDI and the 2nd to last is in r9. The final argument, the one I'm trying to get is somewhere on the stack.
I've tried every possible offset it seems, and none seem to work. I've managed to change the value of some other variable passed in, yet not the one I want! The professor wrote [rbp + 16] next to it, implying the address can be accessed through that, but it doesn't change the value at all when I try to change the value:
movss [rbp+16], xmm0
At the moment, I'm pushing RBP at the start of the function, moving 100 into eax, putting eax into xmm0 (cvtsi2ss), and then moving xmm0 into rbp+whateverOffset.
Is there something I'm missing? Any insight is appreciated, thanks!
EDIT:
This is done on Ubuntu. Here's the Assembly Code:
global Error
Error:
push rbp
mov eax, 100
cvtsi2ss xmm0, eax
movss [rbp+16], xmm0```
ret
This is called using this C++ Code:
Error(array, length, PrValue, &One, &Two, &Three, &Four, &Five);
The variable I want to access is "Five", I want to move a value into the memory address that's moved in. All values are floats except for the second which is an integer, array is float array.
extern "C" void Error(float[], unsigned int, float,
float *, float *, float *, float *, float *);
On function entry, the first stack arg is at [rsp + 8].
(The first 6 integer/pointer args and the first 8 FP args are passed in registers in the x86-64 System V calling convention.)
Your function does push rbp to save the caller's RBP, but you didn't do mov rbp, rsp to make RBP a frame pointer. You also don't pop rbp at the end.
After setting up RBP as a traditional frame pointer, the first stack arg will be at [rbp + 16].
movss [rbp+16], xmm0 stores xmm0 somewhere into your caller's stack frame, if your caller was compiled with -fno-omit-frame-pointer (the default at -O0). i.e. it uses whatever value your caller happened to leave in RBP, not your own frame pointer.
If you wanted to use a pointer arg that was passed on the stack, you need to first load it into an integer register and then dereference it. You don't want to just over-write the arg like argfive = float, you want *argfive = float.