We are running DPDK v.19.11.14 on Ubuntu 22.04 LTS. Our goal is to use DPDK to accelerate UDP communication on a Broadcom 57504 network controller.
We run dpdk-devbind.py to bind the first device of the 57504 controller to DPDK. dpdk-devbind.py reports:
Network devices using DPDK-compatible driver
============================================
0000:43:00.0 'BCM57504 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb
Ethernet 1751' drv=igb_uio unused=bnxt_en,vfio-pci
but when our application calls rte_eth_dev_count_avail() the return value is 0. This means our program does not detect the bound device.
The rte_eal_init arguments are:
-c 0x40000001 -n 4 -m 4000
EAL reports:
EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: 1024 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
EAL: Probing VFIO support...
EAL: VFIO support initialized
The linked shared DPDK libraries are (CMake):
target_link_libraries(${_lib_name}
PRIVATE ${_usr_lib_64_path}libdl.so.2
PRIVATE ${_usr_lib_64_path}libutil.so.1
#PRIVATE ${_usr_lib_64_path}libm.so
PRIVATE ${_dpdk_lib_path}librte_flow_classify.so
-Wl,--whole-archive
PRIVATE ${_dpdk_lib_path}librte_pipeline.so
PRIVATE ${_dpdk_lib_path}librte_table.so
PRIVATE ${_dpdk_lib_path}librte_port.so
-Wl,--no-whole-archive
PRIVATE ${_dpdk_lib_path}librte_pdump.so
PRIVATE ${_dpdk_lib_path}librte_ip_frag.so
PRIVATE ${_dpdk_lib_path}librte_meter.so
PRIVATE ${_dpdk_lib_path}librte_lpm.so
PRIVATE ${_dpdk_lib_path}librte_acl.so
PRIVATE ${_dpdk_lib_path}librte_power.so
PRIVATE ${_dpdk_lib_path}librte_kni.so
PRIVATE ${_dpdk_lib_path}librte_efd.so
PRIVATE ${_dpdk_lib_path}librte_bpf.so
-Wl,--whole-archive
PRIVATE ${_dpdk_lib_path}librte_cfgfile.so
PRIVATE ${_dpdk_lib_path}librte_gro.so
PRIVATE ${_dpdk_lib_path}librte_gso.so
PRIVATE ${_dpdk_lib_path}librte_hash.so
PRIVATE ${_dpdk_lib_path}librte_member.so
PRIVATE ${_dpdk_lib_path}librte_vhost.so
PRIVATE ${_dpdk_lib_path}librte_kvargs.so
PRIVATE ${_dpdk_lib_path}librte_mbuf.so
PRIVATE ${_dpdk_lib_path}librte_net.so
PRIVATE ${_dpdk_lib_path}librte_ethdev.so
PRIVATE ${_dpdk_lib_path}librte_cryptodev.so
PRIVATE ${_dpdk_lib_path}librte_security.so
PRIVATE ${_dpdk_lib_path}librte_compressdev.so
PRIVATE ${_dpdk_lib_path}librte_eventdev.so
PRIVATE ${_dpdk_lib_path}librte_timer.so
PRIVATE ${_dpdk_lib_path}librte_mempool.so
PRIVATE ${_dpdk_drv_path}librte_mempool_ring.so
PRIVATE ${_dpdk_lib_path}librte_ring.so
PRIVATE ${_dpdk_lib_path}librte_pci.so
PRIVATE ${_dpdk_lib_path}librte_eal.so
PRIVATE ${_dpdk_lib_path}librte_cmdline.so
PRIVATE ${_dpdk_lib_path}librte_reorder.so
PRIVATE ${_dpdk_lib_path}librte_sched.so
PRIVATE ${_dpdk_drv_path}librte_bus_pci.so
PRIVATE ${_dpdk_drv_path}librte_bus_vdev.so
PRIVATE ${_dpdk_drv_path}librte_mempool_bucket.so
PRIVATE ${_dpdk_drv_path}librte_mempool_stack.so
PRIVATE ${_dpdk_drv_path}librte_pmd_af_packet.so
PRIVATE ${_dpdk_drv_path}librte_pmd_i40e.so
PRIVATE ${_dpdk_drv_path}librte_pmd_null.so
PRIVATE ${_dpdk_drv_path}librte_pmd_ring.so
PRIVATE ${_dpdk_drv_path}librte_pmd_tap.so
PRIVATE ${_dpdk_drv_path}librte_pmd_virtio.so
PRIVATE ${_dpdk_drv_path}librte_bus_vmbus.so
PRIVATE ${_dpdk_drv_path}librte_pmd_null_crypto.so
PRIVATE ${_dpdk_drv_path}librte_pmd_crypto_scheduler.so
PRIVATE ${_dpdk_drv_path}librte_pmd_virtio_crypto.so
PRIVATE ${_dpdk_drv_path}librte_pmd_skeleton_event.so
PRIVATE ${_dpdk_drv_path}librte_pmd_sw_event.so
PRIVATE ${_dpdk_lib_path}librte_eal.so
PRIVATE ${_dpdk_drv_path}librte_pmd_bnxt.so
-Wl,--no-whole-archive
PRIVATE Threads::Threads
-lrt -lm -lnuma -ldl
${_usr_lib_64_path}libconfig.so
-L${DpdkPath}/examples/l2fwd/build/lib
-L${DpdkPath}/lib
)
EAL log:
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 19 on socket 0
EAL: Detected lcore 2 as core 8 on socket 0
EAL: Detected lcore 3 as core 27 on socket 0
EAL: Detected lcore 4 as core 3 on socket 0
EAL: Detected lcore 5 as core 24 on socket 0
EAL: Detected lcore 6 as core 12 on socket 0
EAL: Detected lcore 7 as core 29 on socket 0
EAL: Detected lcore 8 as core 1 on socket 0
EAL: Detected lcore 9 as core 18 on socket 0
EAL: Detected lcore 10 as core 9 on socket 0
EAL: Detected lcore 11 as core 31 on socket 0
EAL: Detected lcore 12 as core 4 on socket 0
EAL: Detected lcore 13 as core 21 on socket 0
EAL: Detected lcore 14 as core 13 on socket 0
EAL: Detected lcore 15 as core 26 on socket 0
EAL: Detected lcore 16 as core 5 on socket 0
EAL: Detected lcore 17 as core 23 on socket 0
EAL: Detected lcore 18 as core 10 on socket 0
EAL: Detected lcore 19 as core 28 on socket 0
EAL: Detected lcore 20 as core 2 on socket 0
EAL: Detected lcore 21 as core 17 on socket 0
EAL: Detected lcore 22 as core 14 on socket 0
EAL: Detected lcore 23 as core 30 on socket 0
EAL: Detected lcore 24 as core 6 on socket 0
EAL: Detected lcore 25 as core 20 on socket 0
EAL: Detected lcore 26 as core 11 on socket 0
EAL: Detected lcore 27 as core 25 on socket 0
EAL: Detected lcore 28 as core 7 on socket 0
EAL: Detected lcore 29 as core 16 on socket 0
EAL: Detected lcore 30 as core 15 on socket 0
EAL: Detected lcore 31 as core 22 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Ask a virtual area of 0x7000 bytes
EAL: Virtual area found at 0x100000000 (size = 0x7000)
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Buses did not request a specific IOVA mode.
EAL: KNI is loaded, selecting IOVA as PA mode for better KNI performance.
EAL: Selected IOVA mode 'PA'
EAL: 1024 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
EAL: Probing VFIO support...
EAL: IOMMU type 1 (Type 1) is supported
EAL: IOMMU type 7 (sPAPR) is not supported
EAL: IOMMU type 8 (No-IOMMU) is not supported
EAL: VFIO support initialized
EAL: Ask a virtual area of 0x2e000 bytes
EAL: Virtual area found at 0x100007000 (size = 0x2e000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 1048576
EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
EAL: Creating 2 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x100035000 (size = 0x1000)
EAL: Memseg list allocated: 0x100000kB at socket 0
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x140000000 (size = 0x800000000)
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x940000000 (size = 0x1000)
EAL: Memseg list allocated: 0x100000kB at socket 0
EAL: Ask a virtual area of 0x800000000 bytes
EAL: Virtual area found at 0x980000000 (size = 0x800000000)
EAL: Allocating 4 pages of size 1024M on socket 0
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Restoring previous memory policy: 0
EAL: Added 4096M to heap on socket 0
EAL: TSC frequency is ~2000000 KHz
EAL: Master lcore 0 is ready (tid=7f030bfb6c40;cpuset=[0])
EAL: lcore 30 is ready (tid=7f02bd9fc640;cpuset=[30])
What are we missing?
igb_uio with vfio-pci. Will doing so make a difference?-w option to specify the device to pick, -w 43:00.0.sudo privilege.It is also worth seeing to the EAL: 1024 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size issue. There is a helper to get that mountpoint working.
If none of above suggestions proves helpful, consider to run dpdk-testpmd application, probably with full log enabled, too. That should help to see the difference with what the original program in question does. Will dpdk-testpmd be able to detect the device successfully?