linux-device-driverxilinxpci-e

Address of a struct used to set MSI Base address, how does it work? (in Xilinx PCIe RC driver)


Usually in PCIe RC side, the S/W should set MSI (message signaled interrupt) Base register address in the circuit right in front of the PCIe core I guess so that the PCIe core (or bridge connected to it) can extract the software interrupt message and pass it to the interrupt controller (which then converts it to LPI in arm64 case). I was following Xilinx PCIe driver for RC side. In this code

/* Setup MSI */
if (IS_ENABLED(CONFIG_PCI_MSI)) {
    phys_addr_t pa = ALIGN_DOWN(virt_to_phys(port), SZ_4K);

    ret = xilinx_allocate_msi_domains(port);
    if (ret)
        return ret;

    pcie_write(port, upper_32_bits(pa), XILINX_PCIE_REG_MSIBASE1);
    pcie_write(port, lower_32_bits(pa), XILINX_PCIE_REG_MSIBASE2);
}

Here, the physical address of the struct port is obtained and used to set the MSIBASE registers. The Xilinx PCIe bridge manual says for this register:

MemWr TLPs to addresses in this range are interpreted as MSI interrupts. MSI TLPs are interpreted based on the address programmed in this register.

I thought the MSI base address should be hardware register address (like arm gic-v3 ITS's translate register), but how is this address of a struct is set in the bridge to divert the MWr TLP to the interrupt controller?


Solution

  • MSI is identified by the PCIe RC when you write a TLP to a specific memory region.

    MSI base address is the base address of this memory region (this base address is stored in a HW register of the RC).

    The next section is more architecture/system/configuration dependent:

    This memory region is usually mapped to an address space inside the interrupt controller. So whenever something is written to this region, the RC identifies it, and sends it to the interrupt controller internal address space.

    The interrupt controller reads the written TLPs, parses the data written in the TLP, and assigns (according to the data of the TLP, and maybe extra routing tables or configurations) the interrupt to the relevant core, cores, or whatever unit which needs to handle it.

    Assigning the interrupt usually means writing it to a cyclic buffer in memory, and increment the write pointer of this buffer. This increment is followed by an interrupt to the relevant unit.

    Again, the last section is very architecture/system/configuration specific. Some architectures don't even have an interrupt controller, or routing tables, or multiple cores, or anything else I didn't think of.

    You can find a reasonably good example of it in this PPT (start from page 8):

    https://community.qnx.com/sf/sfmain/do/downloadAttachment/projects.community/discussion.community.topc21944/post93964;jsessionid=C7505E29A4E991B7F42ED649A13F90FC?id=atch11922

    and another basic description here:

    http://fundasbykrishna.blogspot.com/2013/05/message-signaled-interrupt.html

    Hopes it convers what you wanted to know.