I have been working on a program which uses a TUNTAP interface (in TUN mode) on a routing device which runs on top of the Linux kernel. It is a multicast tunneling protocol, and I am trying to send MLD joins to the kernel through my application so it can be recieved elsewhere. However, even though I have quadruple-checked my packets being sent on the interface, the linux kernel is dropping the packet before it gets passed on.
Tediously I have been tracing the path of the packet through the linuxkernel trying to figure out why it is being dropped and I think I have figured out to some extent why it is not being processed. The Hop-by-Hop options (containing the Router-Alert option which is necessary for MLD) is being parsed in net/ipv6/ip6_input.c in the ipv6_rcv function, but instead of continuing to process the packet in ip6_rcv_finish, the packet is dropped since the NF_HOOK at the end of the ipv6_rcv function is somehow interpreting the packet as being processed by something else. (NF_STOLEN instead of NF_ACCEPT)
Once the ipv6_rcv function finishes executing, something else executes ip6_mc_input, (in net/ipv6/ip6_input.c still) but from here the Hop-by-Hop options are not processed, which means when the kernel ends up processing the Layer-4 protocol, it has nothing to handle the protocol since the Hop-by-Hop options were meant to be processed beforehand. This means that the kernel drops the packet due to an unknown protocol.
What I am trying to figure out is what is calling ip6_mc_input. I have looked quite a bit on elixir for what could call it, but there are so many possibilities since it is called from a pointer in a rt6_info struct which is difficult to trace since so many things use it. Does anyone know anything that could help me in my search?
IGMP joins work fine, but the IPv4 stuff is probably quite similar so information from that context would probably be helpful too.
For reference, the linux kernel version in use is v4.4.6
I figured out what was going on.
Using a macro that printed out the file location of the caller of the ip6_mc_input, i found that the packet came from my ipt_netmap.c file. It looks like the packet was being taken by the IPTables, which were not programmed to handle the hop options. It turns out I had a configuration option set which didn't need to be set though, so disabling that fixed the issue for me.