assemblyx86-64masm64

Trying to learn x86 Assembly, getting odd results in simple test program


I'm attempting to learn the basics of x64 assembly language from the 2nd Edition of Modern x86 Assembly Language Programming.

The fifth example program in chapter 2 introduces how to perform simple integer division using various width parameters, and provides a simple example test routine to showcase the result. The problem I'm getting is that I seem to be returning an erroneous result, and cannot for the life of me understand why.

The provided C++ code for the routine is as follows:

void UnsignedIntegerDiv(void)
{
    uint8_t a = 12;
    uint16_t b = 17;
    uint32_t c = 71000000;
    uint64_t d = 90000000000;
    uint8_t e = 101;
    uint16_t f = 37;
    uint32_t g = 25;
    uint64_t h = 5;
    uint64_t quo1, rem1;
    uint64_t quo2, rem2;

    quo1 = (a + b + c + d) / (e + f + g + h);
    rem1 = (a + b + c + d) % (e + f + g + h);
    UnsignedIntegerDiv_(a, b, c, d, e, f, g, h, &quo2, &rem2);

    cout << "\nResults for UnsignedIntegerDiv\n";
    cout << "a = " << (unsigned)a << ", b = " << b << ", c = " << c << ' ';
    cout << "d = " << d << ", e = " << (unsigned)e << ", f = " << f << ' ';
    cout << "g = " << g << ", h = " << h << '\n';
    cout << "quo1 = " << quo1 << ", rem1 = " << rem1 << '\n';
    cout << "quo2 = " << quo2 << ", rem2 = " << rem2 << '\n';
}

and the corresponding assembly routine is:

UnsignedIntegerDiv_ proc

; Calculate a + b + c + d
        movzx rax,cl                        ;rax = zero_extend(a)
        movzx rdx,dx                        ;rdx = zero_extend(b)
        add rax,rdx                         ;rax = a + b
        mov r8d,r8d                         ;r8 = zero_extend(c)
        add r8,r9                           ;r8 = c + d
        add rax,r8                          ;rax = a + b + c + d
        xor rdx,rdx                         ;rdx:rax = a + b + c + d

; Calculate e + f + g + h
        movzx r8,byte ptr [rsp+40]          ;r8 = zero_extend(e)
        movzx r9,word ptr [rsp+48]          ;r9 = zero_extend(f)
        add r8,r9                           ;r8 = e + f
        mov r10d,[rsp+56]                   ;r10 = zero_extend(g)
        add r10,[rsp+64]                    ;r10 = g + h;
        add r8,r10                          ;r8 = e + f + g + h
        jnz DivOK                           ;jump if divisor is not zero

        xor eax,eax                         ;set error return code
        jmp done

; Calculate (a + b + c + d) / (e + f + g + h)

DivOK:  div r8                              ;unsigned divide rdx:rax / r8
        mov rcx,[rsp+72]
        mov [rcx],rax                       ;save quotient
        mov rcx,[rsp+80]
        mov [rcx],rdx                       ;save remainder

        mov eax,1                           ;set success return code

Done:   ret

Whilst the code seems to make sense, I'm getting differing output from the C++ implementation of the routine compared to the assembly implementation (the C++ implementation is giving the expected result for quo1, rem1 - it's quo2, rem2 that are incorrect):

Results for UnsignedIntegerDiv
a = 12, b = 17, c = 71000000 d = 90000000000, e = 101, f = 37 g = 25, h = 5
quo1 = 536136904, rem1 = 157
quo2 = 0, rem2 = 90071000029

Which has left me scratching my head as to why. My guess is that there's a typo that I'm missing somewhere (although I get the same result from the downloadable code from the books GitHub page) leading to an overflow or truncation somewhere within the ASM portion.

Any help from the community would be greatly appreciated as I know that I don't know enough to fully troubleshoot this on my own.


Solution

  • Looks like I've been suffering code blindness :)

    The prototype for the UnsignedIntegerDiv_ function had an incorrect datatype for one of the parameters (h in the definition below):

    extern "C" int UnsignedIntegerDiv_(uint8_t a, uint16_t b, uint32_t c, uint64_t d, uint8_t e, uint16_t f, uint32_t g, uint32_t h, uint64_t* quo, uint64_t* rem);
    

    instead of

    extern "C" int UnsignedIntegerDiv_(uint8_t a, uint16_t b, uint32_t c, uint64_t d, uint8_t e, uint16_t f, uint32_t g, uint64_t h, uint64_t* quo, uint64_t* rem);
    

    Thanks to @Peter Cordes for pointing me in the right direction.