Netfilter hooks on multi-core system

Question

We have wrote LKM that is using netfilter hooks to intercept IP packets. The problem is that on 1Gb/s payload we see that hooks load only one CPU core via soft irq. Other 15 cores is idle. So i make conclusion that hooks isn't multithreading.

So my question is: is there any way i can disribute hooks handling on multiple cores?

Answer 1

The problem is not from netfilter, is the way your kernel is managing interrupts.

By default old versions of APIC delivers all interrupts to the CPU0.

You can check if this is your problem with:

cat /proc/interrupts

You can see if the interrupts of the NIC (and remember that the netfilter hook are executed over a RX or TX SoftIRQ) are handled by a single Core.

In newer versions of the kernel, there is a compile option (CONFIG_HOTPLUG_CPU), wich balances the IRQ's over the existing cores.

Or if you cannot update the version or recompile the kernel, you can update the SMP affinity (with a mask that handles more that a CPUid) to try to balance between different Cores. Or go into ACPI and proper configuration (Here I cannot help more).

Here you can find all about this stuff (SMP affininty and proper IRQ handling)

Answer 2

A problem might be that your NIC has only one interrupt. Some newer NICS have several interrupts (so-called multiqueue NIC's) allowing the load to be spread among many threads.

For single-queue NIC's there are some software features available in newer kernels that you can configure to spread the load. See e.g. http://www.spinics.net/lists/linux-doc/msg02975.html for an overview of what is available.