The same driver for multiple network cards -- performance bottleneck?

Question

I'm using driver e1000e for multiple Intel network cards (Intel EXPI9402PT, based on 82571EB chip). The problem is that when I'm trying to utilize maximum speed (1GB) on more than one interface, speed on each interface starts to drop down.

I have my own driver in kernel space designed to just sent given packets. It just allocs packets by:

skb = dev_alloc_skb(packet->len);

and them sends them by:

result = dev->hard_start_xmit(skb,dev);

Each interface has its own instance of the driver.

For one interface I get: 120435948 bytes/sec.

For two interfaces I get: 61080233 bytes/sec and 60515294 bytes/sec.

For three interfaces I get: 28564020 bytes/sec, 27111184 bytes/sec, 27118907 bytes/sec.

What can be the cause? Is the hard_start_xmit function reentrant?

Answer 1

This is most likely due to a lack of bandwidth over your motherboard.

If you're trying to pump 3 Gb/s of information through a bus slower than 3 Gb/s, you'll have problems. What sort of bus are these cards on?

There may be a fix, but I think this is a physical limitation of the board, not necessarily your driver.