How to load balance by way of round-robin in DPDK using RTE_FLOW？

in my problem RSS did not have a good load balance between CPU cores case the rx packets has been modified by insert tags between mac and ip so the dpdk could not recognize it.assume I want to load balance by way of round-robin, Multiple rx queues have been set up. in this question answer: How to disable RSS but still using multiple RX queues in DPDK? it says its possible to load balance in round robin fashion by using RTE_FLOW. what is the right way to do it grammatically ? I would like to know the API or structs for setting up the round-robin method here is my runtime environment infomation: 1）dpdk version: 19.11.9 2）nic PMD :ixgbe 3) fireware:825999 and XXV710 4) os version: ubuntu 16.04 kernel:4.4.0-186

As per the question

RSS did not have a good load balance between CPU cores case the rx packets has been modified by insert tags between mac and ip

There are couple of items which needs to be clarified. so let me explain

1. Certain Physical and virtual NIC exposes RSS via DPDK RX offload for fixed tuples like IP, Protocol, TCP|UDP|SCTP port number.
2. Certain NIC allows to configure the hash reta algorithm to better suit to needs (example when the source destination IP address is fixed we can skip and use others).
3. As I recollect from DPDK 18.11, RTE_FLOW is been introduced to support RSS on selected RX queues (example Q1,Q2,Q3 can be RSS for TCP packets while Q4,Q5 can be used for UDP). But again this is based on the either Inner or Outer IP+PORT number
4. For DPDK version 19.11 onwards RTE_FLOW is been enhanced to support RAW Pattern. The intend of this feature is support Special protocol which by default the NIC does not understand like (VXLAN, GENEVE, RTP and other protocols).
5. For NIC like Fortville & Columbiaville (from Intel) allows loading of special firmware via DDP (Dynamic Device Personation) to configure special fabric headers or MPLS like headers (between ethernet and ip) to be parsed, lookup and used as seed for RSS (allowing better distribution).
6. There are NIC which do support L2 layer but these would be limited SMAC, DMAC, VLAN1,VLAn2, MPLS only and not custom header.

Hence depending upon NIC, vendor, RSS support for L2 and firmware the ability calculating RSS on fields between varies in port init or RTE_FLOW specific configuration. For example RSS ETH supported on

1. I40E is I40E_INSET_DMAC | I40E_INSET_SMAC
2. DPAA2 is NH_FLD_ETH_TYPE and NET_PROT_ETH
3. CNXK is RSS_DMAC_INDEX
4. OCTEONX2 is FLOW_KEY_TYPE_VLAN and FLOW_KEY_TYPE_CH_LEN_90B

Hence for NIC ixgbe and XXV710 there is no ready support for the custom header between ethernet and IP.

Alternatives:

1. Use a smart NIC or FPGA: that is programmed to parse and RSS on your specific headers to RSS on multiple RX queue
2. Work with Intel using XXV710 (Fortville): to create DDP which can parse your specific headers as RSS on multiple RX queue.
3. Identify DPDK NIC: which can parse RAW header as defined 12.2.6.2. Once the support is added by the vendor you can create a simple traffic spread tool which will ensure traffic distribution across the desired RX queues in round robin fashion.
4. Use SW to support the missing hardware.

note: I am not recommending use of HW based static Round Robin as it will create 2 fold problem

• if it is pure DPDK BOND Round ROBIN you will not have any flow pinning
• If you use hash based pinning, there are chances elephant flow can pushed 1 or a few queues causing performance drops in cpu processing.
• My recommendation is to use EVENTDEV model with atomic mode, which ensure better cache locality (at a given instance same flows will fall onto worker thread) and almost linear performance performance. sample app

For option 4 (Software Model):

1. Disable RSS in port_init function
2. Use single RX queue to receive all the packet either custom rx thread.
3. calculate the hash based on the desired header and update mbuf hash field.
4. Use rte_distributor library to spread traffic based on the custom.
5. or use rte_eventdev with atomic model to spread the work load on multiple worker.

[Clarification from Comments]:

1. I have asked relevant practitioners， they said modifying the pmd driver can solves my problem, its the only way?

[ANSWER] Since you are using custom header and not generic VLAN|IP|Port this suggestion is not correct. As you have clarified in question and comments you want to distribute like RSS for custom header.

2. I haven't written any code about rte_flow distribution yet, i read the rte_flow example, don't see the code to configure round-robin

[ANSWER] as explained above not all nic support RSS and RAW. Since your current NIC is ixgbe and i40e the function of parsing and executing RSS for custom header is unlikely. you can try option 2 (work with intel create new ddp) for i40e to achieve the same or implement in SW as suggested in option 4.

3. im not ask for solution, I just want to know how to set up round robin by 'RTE_FLOW' Can you give me a few API

[ANSWER] normally one updates with code snippets or steps used for reproducing the error. But the current question is more like clarification. Please refer above.