Understanding link between CONFIG_SMP, Spinlocks and CONFIG_PREEMPT in latest (3.0.0 and above) Linux kernel

Question

To give you full context my discussion begun with an observation that I am running a SMP linux (3.0.1-rt11) on ARM cortex A8 based SoC which is a uniprocessor. I was curious to know if there will be any performance advantage by disabling SMP support. And if yes what impact it will have on my drivers and interrupt handlers.

I did some reading and come across two related topics: spinlocks and kernel preemption. I did little more googling and reading but this time all I got is few stale and contradictory answers. So I thought let me try stackoverflow.

Origin of my doubts/questions is this para from Linux device drivers 3rd edition chapter 5:

Spinlocks are, by their nature, intended for use on multiprocessor systems, although a uniprocessor workstation running a preemptive kernel behaves like SMP, as far as concurrency is concerned. If a nonpreemptive uniprocessor system ever went into a spin on a lock, it would spin forever; no other thread would ever be able to obtain the CPU to release the lock. For this reason, spinlock operations on uniprocessor systems without preemption enabled are optimized to do nothing, with the exception of the ones that change the IRQ masking status. Because of preemption, even if you never expect your code to run on an SMP system, you still need to implement proper locking.

My doubts/questions are:

a) Is Linux kernel preemptive in kernel space by default? If yes, is this preemption limited to only processes or Interrupt handlers can also be preempted?

b) Does Linux kernel (on ARM) support nested interrupt? If yes, will each interrupt handler (top half) have its own stack or they share same 4k/8k kernel mode stack?

c) If I disable SMP (CONFIG_SMP) and preemption (CONFIG_PREEMPT) will spin-locks in my drivers and interrupt handlers make any sense?

d) How kernel handle interrupts raised while executing top-half i.e will they be disabled or masked?

After some googling I found this:

For kernels compiled without CONFIG_SMP, and without CONFIG_PREEMPT spinlocks do not exist at all. This is an excellent design decision: when no-one else can run at the same time, there is no reason to have a lock.

If the kernel is compiled without CONFIG_SMP, but CONFIG_PREEMPT is set, then spinlocks simply disable preemption, which is sufficient to prevent any races. For most purposes, we can think of preemption as equivalent to SMP, and not worry about it separately.

But there is no kernel version or date on the source. Can anyone confirm if its still valid for latest Linux kernels?

Answer 1

a) Whether Linux is preemptive or not depends on whether or no you configure it that way
with CONFIG_PREEMPT. There is no default. If you run make config, you will have to choose.

b) Interrupts do nest on Linux; while interrupts are being processed, other interrupts can go off. That's true on ARM and numerous other architectures. It's all on the same stack. Of course, the user space stack isn't used for interrupts!

c) If you disable SMP and preemption, spinlocks in your code will reduce to no-op if they are the regular spinlocks, and IRQ spinlocks (spin_lock_irqsave/spin_lock_irqrestore) will turn into interrupt disable/enable. The latter are still essential, therefore; they prevent races between tasks running your code and interrupts running your code.

d) The "top half" traditionally refers to interrupt service routines. The top half code of a driver is run by interrupts. The bottom half is called by tasks (to read or write data or whatever). Details of interrupt handling are architecture specific.

I most recently worked very closely with Linux interrupts on a particular MIPS architecture. On that particular board, there were 128 interrupt lines maskable via two 64 bit words. The kernel implemented a priority scheme on top of this, so before executing a handler for a given interrupt, the lower ones were masked via updates of those 2x64 bit registers. I implemented a modification so that the interrupt priorities could be set arbitrarily, rather than by hardware position, and dynamically by writing values into a /proc entry. Moreover, I put in a hack whereby a portion of the numeric IRQ priority overlapped with real-time priority of tasks. So RT tasks (i.e. user space threads) assigned to certain range of priority levels were able to implicitly suppress a certain range of interrupts while running. This was very useful in preventing badly-behaved interrupts from interfering with critical tasks (for instance, an interrupt service routine in the IDE driver code used for the compact flash, which executes busy loops due to a badly designed hardware interface, causing flash writes to become the de-facto highest priority activity in the system.) So anyway, IRQ masking behavior isn't written in stone, if you're in control of the kernel used by customers.

The quoted statements in the question are only true about regular spinlocks (spin_lock function/macro) not IRQ spinlocks (spin_lock_irqsave). In a preemptible kernel on a uniprocessor, spin_lock just has to disable preemption, which is enough to keep all other tasks out of the kernel until spin_unlock. But spin_lock_irqsave has to disable interrupts.