assemblyx86-64infinite-loopcalling-conventionmethod-call

How to avoid infinite loops in a function call in assembly?


I am writing a program in assembly for x86-64 microprocessors in 64-bit mode, and that calls a function f in a loop 5 times which in turn calls puts in a loop 3 times to display "hello world".

To assemble the code, I am using gcc myfile.s -o myfile and then I run it using ./myfile.

This is the code:

.section .rodata
.LC0:
    .ascii "hello world\0"

.text
.global main
.type main, @function
main:
    pushq %rbp
    movq %rsp, %rbp
    movl $0, %ecx

main_loop:
    cmpl $5, %ecx
    jge main_end

    call f
    incl %ecx
    jmp main_loop

main_end:
    movq %rbp, %rsp
    popq %rbp
    ret

.global f
.type f, @function
f:
    pushq %rbp
    movq %rsp, %rbp
    movl $0, %edx

f_loop:
    cmpl $3, %edx
    jge f_end

    leaq .LC0(%rip), %rdi
    call puts
    incl %edx
    jmp f_loop

f_end:
    movq %rbp, %rsp
    popq %rbp
    ret

The problem is that I am entering an infinite loop and "hello world" is being printed infinitely. What could have gotten wrong? Are the registers for loop counters becoming zero somewhere or is it something else?
I am pretty new to assembly in general and function calls specifically, so any help is appreciated.


Solution

  • Your RCX and RDX registers are not preserved across calls! See What registers are preserved through a linux x86-64 function call.

    You could use RBX instead of RDX:

    f:
        pushq %rbp
        movq  %rsp, %rbp
        subq  $16, %rsp            ; Space to preserve RBX plus keep stack 16-byte aligned
        movq  %rbx, -8(%rbp)       ; Preserve RBX
        movl  $3, %ebx
    
    f_loop:
        leaq  .LC0(%rip), %rdi
        call  puts
        decl  %ebx
        jnz   f_loop
    
        movq  -8(%rbp), %rbx       ; Restore RBX
        movq  %rbp, %rsp
        popq  %rbp
        ret
    

    Modify the main in a similar way!


    Some alternatives as mentioned by @PeterCordes in a comment:

    With prologue/epilogue

    f:
        pushq %rbp
        movq  %rsp, %rbp
        pushq %rbx                 ; (1) Keep stack 16-byte aligned
        pushq %rbx                 ; (2) Preserve RBX
        movl  $3, %ebx
    
    f_loop:
        leaq  .LC0(%rip), %rdi
        call  puts
        decl  %ebx
        jnz   f_loop
    
        popq  %rbx                 ; (2) Restore RBX
        movq  %rbp, %rsp           ; (1) ???
        popq  %rbp
        ret
    

    Without prologue/epilogue

    f:
        pushq %rbx                 ; Preserve RBX
        movl  $3, %ebx
    
    f_loop:
        leaq  .LC0(%rip), %rdi
        call  puts
        decl  %ebx
        jnz   f_loop
    
        popq  %rbx                 ; Restore RBX
        ret