linux-kernelgdbu-boot

How should I apply add-symbol-file command during u-boot linux boot debug?


I'm following linux bootloading using u-boot (using SPL falcon mode where u-boot-spl launches linux directly) on a qemu virtual machine. Now the code jumped to linux kernel and because I have done add-symbol-file vmlinux 0x80081000 I can follow the kernel code step by step using gdb connected to the virtual machine. Actually I loaded the kernel image to 0x80080000 but I had to set the address to 0x80081000 to make the source code appear on the gdb correctly according to the PC value(I don't know why this difference of 0x1000 is needed).
Later I found the kernel sets up the page table (identity mapping and swap table) and jumps to __primary_switched and this is where pure kernel virtual address is used first time for the PC. This is where the call is made at the end of the head.S file.

ldr x8, =__primary_switched
adrp    x0, __PHYS_OFFSET
br  x8

In the symbol file (vmlinux, an elf file), the symbols before __primary_switched are all mapped at virtual addresses (starting with 0xffffffc0..... high addresses) but the gdb could follow the source even when the PC value was using physical address. (The PC was initially loaded with physical address of the kernel start and PC relative jumps were being used until it jumps to __primary_switched, mmu disabled or using identity mapping) So does this mean, in doing add-symbol-file only the offset of the symbols from the start of text matters?
Another quetion : I can follow the kernel source with gdb but after __primary_switched, I cannot see the source. The debugger doesn't show the correct source location according to the now kernel virtual PC value. Should I tell the debugger to use correct offset using add-symbol-file again? if so how?

ADD (8:32 AM Wednesday, January 12, 2022, UTC)
I found from gdb manual,

"add-symbol-file filename [ -readnow | -readnever ] [ -o offset ] [ textaddress ] [ -s section address ... ] The add-symbol-file command reads additional symbol table information from the file filename. You would use this command when filename has been dynamically loaded (by some other means) into the program that is running. The textaddress parameter gives the memory address at which the file's text section has been loaded. You can additionally specify the base address of other sections using an arbitrary number of '-s section address' pairs. If a section is omitted, gdb will use its default addresses as found in filename. Any address or textaddress can be given as an expression. ..."

I changed my program a little bit to fix a problem. The readelf shows the .text section starting at ffffffc010080800. So I adjusted the command to "add-symbol-file vmlinux 0x80000800" and gdb shows the kernel source correct after jump to linux. Still it doesn't show me the source code after __primary_switched.

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .head.text        PROGBITS         ffffffc010080000  00010000
       0000000000000040  0000000000000000  AX       0     0     4
  [ 2] .text             PROGBITS         ffffffc010080800  00010800
       0000000000304370  0000000000000000  AX       0     0     2048
  [ 3] .rodata           PROGBITS         ffffffc010390000  00320000
   .... (skip) ...
  [12] .notes            NOTE             ffffffc01045be18  003ebe18
       000000000000003c  0000000000000000   A       0     0     4
  [13] .init.text        PROGBITS         ffffffc010470000  003f0000
       0000000000027ec8  0000000000000000  AX       0     0     4
  [14] .exit.text        PROGBITS         ffffffc010497ec8  00417ec8
       000000000000046c  0000000000000000  AX       0     0     4

Since '__primary_switched' resides in section .init.text, I tried adding "-s .init.text 0xffffffc010470000" or "-s .init_text 0x803ef800"(physcial address) to the add-symbol-file command to no avail. Is my command wrong? Or could this be from page table (virtual -> Physical) problem because I see synchronous exception right after I enter __primary_switched (I see PC value has become 0x200. If the exception vector is located in 0x0, this is the vector entry for synch exception like undefined instruction. I should also check the vector base address has not been set correctly.)


Solution

  • I found my kernel load address was wrong (__PHYS_OFFSET was below physical ddr address start). After fixing it, the PC increments normally with kernel virtual address and I should just apply the add-symbol-file command using the virtual address.
    This was the new section addresses.

            Section Headers:
              [Nr] Name              Type             Address           Offset
                   Size              EntSize          Flags  Link  Info  Align
              [ 0]                   NULL             0000000000000000  00000000
                   0000000000000000  0000000000000000           0     0     0
              [ 1] .head.text        PROGBITS         ffffffc010080000  00010000
                   0000000000000040  0000000000000000  AX       0     0     4
              [ 2] .text             PROGBITS         ffffffc010080800  00010800
                   0000000000304370  0000000000000000  AX       0     0     2048
              [ 3] .rodata           PROGBITS         ffffffc010390000  00320000
                   00000000000a6385  0000000000000000  WA       0     0     4096
              [ 4] .modinfo          PROGBITS         ffffffc010436385  003c6385
                   00000000000018ff  0000000000000000   A       0     0     1
              [ 5] .pci_fixup        PROGBITS         ffffffc010437c90  003c7c90
                   00000000000020f0  0000000000000000   A       0     0     16
              [ 6] __ksymtab         PROGBITS         ffffffc010439d80  003c9d80
                   0000000000006d20  0000000000000000   A       0     0     4
              [ 7] __ksymtab_gpl     PROGBITS         ffffffc010440aa0  003d0aa0
                   0000000000005808  0000000000000000   A       0     0     4
              [ 8] __ksymtab_strings PROGBITS         ffffffc0104462a8  003d62a8
                   00000000000134f2  0000000000000000   A       0     0     1
              [ 9] __param           PROGBITS         ffffffc0104597a0  003e97a0
                   0000000000000b68  0000000000000000   A       0     0     8
              [10] __modver          PROGBITS         ffffffc01045a308  003ea308
                   0000000000000cf8  0000000000000000   A       0     0     8
              [11] __ex_table        PROGBITS         ffffffc01045b000  003eb000
                   0000000000000e18  0000000000000000   A       0     0     8
              [12] .notes            NOTE             ffffffc01045be18  003ebe18
                   000000000000003c  0000000000000000   A       0     0     4
              [13] .init.text        PROGBITS         ffffffc010470000  003f0000
                   0000000000027ec8  0000000000000000  AX       0     0     4
      [14] .exit.text        PROGBITS         ffffffc010497ec8  00417ec8
    

    The final kernel image is loaded at 0x80080000. Then __PHYS_OFFSET becomes 0x80000000. (TEXT_OFFSET is 0x80000 by default). Now I can debug the kernel source before __primary_switch using this command.

    add-symbol-file images/vmlinux 0x80080800 -s .head.text 0x80080000 -s .init.text 0x803f7800
    

    And after the kernel entered __primary_switched (now kernel virtual address is used), I added this command to see the source and I can follow code using qemu and gdb step-by-step.

    add-symbol-file images/vmlinux 0xffffffc010080800 -s .head.text 0xffffffc010080000 -s .init.text 0xffffffc010470000 Hope this helps someone later.
    

    But after some days, I think I could just use add-symbol-file images/vmlinux 0xffffffc010080800 (applying all the section info).