I am having trouble understanding the context from http://www.redbooks.ibm.com/redbooks/pdfs/sg247351.pdf
It is describing how the transport layer is implemented but every info I have read only explains the features not how the transport layer is implemented.
InfiniBand architecture offers a significant improvement for the transport layer over other technologies such as Ethernet: all functions are implemented in the hardware.
Is there a way to find out the basic idea of how the functions are implemented in hardware for Infiniband transport layer? For example, is it using the logical controls and registers to run all the necessary transport layer functions?
I have figured out and confirmed with the Infiniband providers, although the document explains that transport layer is built on hardware, it is only referring to the the queue pairs in HCA. Accordingly, most of the logics are just working in the background process from the Infiniband driver.
Generic Process
1) Application is written to use InfiniBand.
2) Interfaces in the Mellanox driver (Like MPI) allow the application, through the driver, to write directly to memory space (posting a work request) on the server (now we are into Transport layer).
3) Memory space details between source and destination servers are exchanged, so that communication will now flow between these RAM memory regions. This is a key benefit to InfiniBand, because it is bypassing the kernel/CPU and exchanging data in hardware (between memory regions using queue pairs between the HCAs (adapters). When we say data is exchanged in hardware, we are talking about server memory, and the HCA.
4) The HCA will segment packets according to MTU and push data out onto the wire. At the other end, it will be reassembled. This is still "Transport Layer".
5) As the bits get pushed onto the wire – this is physical layer.
Conclusion
The driver is in the background, telling the OS how to interoperate with the HCA (deciding transport protocol InfiniBand, vs, Ethernet, MTU, CM vs, RD queue pair exchange mode, etc.). Additionally, it supplies the InfiniBand programming interfaces and additional utilities for troubleshooting etc. I guess you could say there is some logic here, provided by the OFED HCA driver.