In linux, the function dma_alloc_coherent() has this form (declared in include/linux/dma-mapping.h)
static inline void *dma_alloc_coherent(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t gfp)
The returned address is the kernel address for the allocated buffer and the dma_handle is filled by the kernel with DMA address seen for the device. The document says, in some cases the returned dma_handle is just the physical address but when there is an IOMMU connected to the device, it is another virtual address which should be converted to physical address.
I recently wrote a driver (an interim, or test driver) for a device supposed to be used with IOMMU but because our hardware is not equipped with it, I put the (user space virtul address) to (physical address) conversion in the driver so that the device can just use the "addresses" as is. The data structures passed to the device contained some 'addresses'. Later this address conversion will be removed and instead the IOMMU driver will be added so that the "some virtual" to "physical" address conversion is done by the IOMMU.
My question is, in what case is the dma_handle physical address? In my test driver, I have used functions like get_user_pages(), kmap() and virt_to_phys(), or mmaped huge pages in some cases, but it came to my mind that I could have just used dma_alloc_coherent() and use the dma_handle if the value is physical address in some cases. (Actually the small buffers are pointed by some data, I think I should use dma_pool_alloc() which is for small buffers.)
This is purely my reading comprehension with some code browsing. It looks like whatever returns to your dma_handle should be given to the device as is and it will be correct. Do not do any of your own address translation.
I was reading the kernel doc you pointed out.
The first section seems to indicate that kmalloc() + dma_map_single() == dma_alloc_coherent()
In VA = kmalloc(), you get the VA and of course kernel knows it’s PA. Then you pass the VA to dma_map_single() and you get the dma VA (or dma PA if no IOMMU) which this page refers to as Bus Address Space
I browse into the code of dma_alloc_coherent(), and found some clues:
// no IOMMU: this seems to be returning physical
if (dma_alloc_direct(dev, ops))
cpu_addr = dma_direct_alloc(dev, size, dma_handle, flag, attrs);
// IOMMU or possibly some other dma_map_ops ??? not sure why we need an abstraction for DMA device
else if (ops->alloc)
cpu_addr = ops->alloc(dev, size, dma_handle, flag, attrs);
ops->alloc() can be iommu_dma_alloc() in drivers/iommu/dma-iommu.c:iommu_dma_ops
Keep following this path, indeed you’ll find either of the following:
return iommu_dma_alloc_remap(dev, size, handle, gfp,
dma_pgprot(dev, PAGE_KERNEL, attrs), attrs);
*handle = __iommu_dma_map(dev, page_to_phys(page), size, ioprot,
dev->coherent_dma_mask);
Going back to dma_map_single().
You also find a similar bifurcation here with direct vs ops (PA vs IOMMU):
if (dma_map_direct(dev, ops) ||
arch_dma_map_page_direct(dev, page_to_phys(page) + offset + size))
addr = dma_direct_map_page(dev, page, offset, size, dir, attrs);
else
addr = ops->map_page(dev, page, offset, size, dir, attrs);
ops->map_page() has one possibility being iommu_dma_map_page() now we are back at drivers/iommu/dma-iommu.c
And finally this gives you the address the device should use in the bus address space
iova = __iommu_dma_map(dev, phys, size, prot, dma_mask);