What's the correct way to process all the payload of a sk_buff packet in Linux

Question

I'm currently trying to debug an ATM encapsulation layer that runs on-top of Ethernet. Basically the ATM cells are stored in order after the ethernet header. However I suspect the drivers naive approach to the sk_buffs is broken.

The driver blindly assumes that skb->data can be iterated through but looking at the kernel code for virtio_net.c:page_to_skb I see the following behaviour:

memcpy(hdr, p, hdr_len);
len -= hdr_len;
p += offset;
copy = len;

if (copy > skb_tailroom(skb))
        copy = skb_tailroom(skb);

memcpy(skb_put(skb, copy), p, copy);

Then further on:

while (len) {
        set_skb_frag(skb, page, offset, &len);
    page = (struct page *)page->private;
    offset = 0;
}

Which seems to show the buffer is fragmented with only the first portion being directly accessible from skb->data.

What should I be using to get at the underlying data. Ideally I want to peek a few bytes at arbitrary offsets into the ethernet packet before memcpy'ing chunks into a reassembly buffer I maintain. What should I be using to do this?

Answer 1

The implementation of socket buffers are comprised of a linear data buffer and one or more page buffers.

The existence of paged data in a socket buffer is indicated by skb->data_len member being non-zero.

bool skb_is_nonlinear(const struct sk_buff *skb) defined in /include/linux/skbuff.h is used to test this.

The amount of non-paged data at skb->data can be calculated as skb->len - skb->data_len. unsigned int skb_headlen(const struct sk_buff *skb) defined in /include/linux/skbuff.h is used to test this.

The skb->data pointer only points to the non paged data, which what the driver you described possibly relies on.

void *skb_header_pointer(const struct sk_buff *skb, int offset, int len, void *buffer) is defined in /include/linux/skbuff.h . It takes the socket buffer, byte offset and byte length your want and a local data buffer which is only used if the data is in one of the page buffers.

It returns a pointer to either the data in the linear data buffer from skb->data or the pointer to the local data buffer supplied, or NULL if your offset and length were incorrect.

For pieces of data larger than protocol headers you either want to use

int skb_copy_bits(const struct sk_buff *skb, int offset,void *to, int len);

to copy from a given socket buffer ,byte offset and byte length to a given kernel buffer.

or

int skb_copy_datagram_iovec(const struct sk_buff *from, int offset, struct iovec *to, int size);

to to copy from a given socket buffer ,byte offset and byte length to a given iovec structure in user space.

Examples of usage can been seen in netfilter code and in other Ethernet drivers.

For further information

• Socket Buffer Introduction by David S. Miller
• Socket Buffer Data Layout by David S. Miller
• Socket Buffer Tutorial by Roberto Innocente, O.S.Adewale