cx86kernel-moduleinline-assembly

Ubuntu freezes completely on executing this inline asm


I am trying to check if the VMX hardware extensions are supported by the processor using inline assembly. I have tried the following two ways of doing it:

Method 1:

int vmx_support(void) {
    int get_vmx_support, vmx_bit;
    asm volatile ("mov $1, %eax");
    asm volatile ("mov $0, %ecx");
    asm volatile ("cpuid");
    asm volatile ("mov %%ecx, %0\n\t":"=r" (get_vmx_support): : "memory"); 

    vmx_bit = (get_vmx_support >> 5) & 1;
    if (vmx_bit == 1) {
        return 1;
    } else {
        return 0;
    }
}

Method 2:

int vmx_support(void) {
    unsigned int eax, ebx, ecx, edx;
    eax = 1;
    ecx = 0;
    asm volatile("cpuid"
        : "=a" (eax),
          "=b" (ebx),
          "=c" (ecx),
          "=d" (edx)
        : "0" (eax), "2" (ecx)
        : "memory");
    vmx_bit = (ecx >> 5) & 1;
    if (vmx_bit == 1) {
        return 1;
    } else {
        return 0;
    }
}

When I try to execute vmx_support() from Method 1 inside a kernel module, Ubuntu freezes completely when I do insmod vmx.ko and I have to restart it to get it back. When I try to execute vmx_support() from Method 2 inside the kernel module, it executes and shows [VMX] vmx is supported. on dmesg | tail.

Also, when I try to run vmx_support() from Method 1 as a userspace program, it executes and prints [VMX] vmx is supported. as output to the console.

Question: Why does code from Method 1 freeze Ubuntu whereas code from Method 2 does not? Also, is there a safer way to test and debug code that uses inline assembly? (that is, avoid freezes for example)

Links to Makefile, kernel module and userspace program can be found here:

Makefile

vmx.c (kernel module. The code from method 2 is commented inside, uncomment it and comment the code from method 1 to see how it works)

vmx_sup.c (userspace program)


Solution

  • Method 1 has several problems, but the one that is causing it to hang is undoubtedly that it changes ebx without telling the compiler. In your user mode program, probably ebx doesn’t happen to have anything important in it, but in the kernel module, it apparently contains something critical.