Message corruption with hao and route2 XFRM rules [SOLVED]

I had some serious problems with destination option XFRM processing on newer Linux kernels (3.3 and 3.4). The sent Binding Update (BU) message was corrupt, because the Destination Option header overwrote the beginning of the MH part.



I have done the following tests to find the issue:

First case:
No XFRM policies and states.
Sending MH messages without destopt header.
In this case the message format is OK, I have tested it with tcpdump and wireshark.



Second case:
Adding destopt XFRM policy and state:

ip -6 xfrm policy add src 2001:470:7210:10::11 dst 2001:470:7210:10::1000 proto 135 type 5 dir out priority 2 ptype sub tmpl src 2001:470:7210:10::11 dst 2001:470:7210:10::1000 proto hao reqid 0 mode ro level use
ip -6 xfrm state add src 2001:470:7210:10::11 dst 2001:470:7210:10::1000 proto hao reqid 0 mode ro replay-window 0 coa 2001:470:7210:11:20c:29ff:fe46:a0e3 sel src 2001:470:7210:10::11 dst 2001:470:7210:10::1000

In this case, the message format was corrupted:



As you can see, the IPv6 header is OK. Next, the destination option header is OK. Finally, the following part of the packet isn't OK. If you compare the two dumps carefully, you will see, that the last 8 bytes are identical. The mip6_destopt_output function adds the destination option header correctly, but overwrites the existing MH header, and doesn't shift it after the destopt header.

Third case:
I have created ESP TUNNEL XFRM rules manually
In this case the message format is OK again.


Solution:

I have added a lot of debug messages to the kernel source and finally found the problem. When the kernel creates the skb from iovec (ip6_append_data) it sets the pointer of the network header to a wrong position. It will be shifted with 24 bytes (it is the length of the HAO dest. opt. header with paddings).

After this point, the message will be corrupted, the beginning (the first 24 bytes) of the MH part will be truncated. Later, when the kernel adds the dest. opt. header itself, there isn't any issue.

So, back to the wrong network header pointer. It is shifted by exthdrlen (= 24) by the skb_set_network_header() function. This exthdrlen comes from rt->rt6i_nfheader_len, which comes from the dst_entry chain. (More about dst_entry chain here and here). This nfheader_len value comes from the header_len of the desired xfrm type (in this case hao dest opt):



I have run a fast grep on the kernel tree, and this XFRM_TYPE_NON_FRAGMENT does not have any effect, just sets (or not) nfheader_len here. So, the following patch solves the issue:



My thread on the netdev Linux list:

• http://marc.info/?l=linux-netdev&m=134031152618901&w=2
• http://marc.info/?l=linux-netdev&m=134253671029669&w=2

Posted in: fix,issue,kernel,linux,mipv6