cpu shielding and disabling kernel interrupts

Question

Under Linux, what (if any) is the downside to disabling all interrupts on a particular CPU core, or even all cores on a single chip in a system containing multiple chips? When writing a C program that has extreme sensitivity to latency, my natural urge is to isolate the thread on its own core and move all other tasks to separate cores, and I'm wondering what the tradeoffs are.

Answer 1

The mainline Linux kernel is not designed to have all IRQs disabled on a CPU for an extended period. There are a number of internal kernel functions which require such IRQs - RCU being one of them. An RCU stall can and will bring down the rest of the system, and avoiding such a stall may (depending on the RCU implementation in use) involve IPIs (inter-processor interrupts) broadcasted to all CPUs.

That said, there are experimental patches to properly implement such CPU isolation; eg: http://lwn.net/Articles/268711/ - these go through all the internal gyrations needed to take a CPU offline (from the perspective of the rest of the kernel) before disabling interrupt routing. This is a fairly old patch series, though; you may want to contact the developer of the patch series to see if they have a newer version somewhere, or adapt those patches to the current version of the kernel. The PREEMPT_RT people are supposedly looking into implementing it as well - check out their wiki at https://rt.wiki.kernel.org/index.php/Main_Page (it's not implemented yet, though). Good luck!

Answer 2

A couple things to consider:

1. Any IRQs will have one less processor to run on, which means you are more likely to have interrupts waiting. Obviously this becomes less of a problem as the number of processors grows.
2. Without interrupts, it will be much more difficult to force the process to give up execution. If it decides to enter an infinite loop, you can't interrupt the processor to stop it.

If you have complete trust that the process will give up control in a reasonable time and aren't worried about interrupt queueing, then it could be beneficial to disable interrupts.