How do I get tickless kernel to work? nohz_full, rcu_nocbs, isolcpus what else?

Question

I just installed Ubuntu 13.10 with new 3.11 kernel. And in 3.10, it has the tickless feature where I can run a process without being interrupted by local timer, not as much as before. I followed this link http://www.breakage.org/2013/11/nohz_fullgodmode/

I planned to run my app on cpu 3, so I set the following in my grub:

isolcpus=3 nohz_full=3 rcu_nocbs=3

After rebooting, it seems the local timer interrupt on cpu 3 is indeed a lot less than other cpus.

I also ran:

# for i in `pgrep rcu` ; do taskset -pc 0 $i ; done

But when I start running my app, the count of local timer interrupt jumped. My app does nothing but an infinite loop.

int main() {
while (true) {
}
}

So what am I missing? Why does the time interrupt comes back when I run something? I thought nohz_full means when there's only one process running, it'll stop interrupting.

The following is the output of /proc/sched_debug when I'm not running the app and apparently there's no other process on that cpu. So what am I missing?

cpu#3, 2492.071 MHz
  .nr_running                    : 0
  .load                          : 0
  .nr_switches                   : 45818
  .nr_load_updates               : 11165
  .nr_uninterruptible            : -1
  .next_balance                  : 4295.674289
  .curr->pid                     : 0
  .clock                         : 3127610.519188
  .cpu_load[0]                   : 0
  .cpu_load[1]                   : 0
  .cpu_load[2]                   : 0
  .cpu_load[3]                   : 0
  .cpu_load[4]                   : 0
  .yld_count                     : 0
  .sched_count                   : 47377
  .sched_goidle                  : 3956
  .avg_idle                      : 1000000
  .ttwu_count                    : 29329
  .ttwu_local                    : 26078

cfs_rq[3]:/
  .exec_clock                    : 15750.800052
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 15699.782432
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : -247950.011716
  .nr_spread_over                : 0
  .nr_running                    : 0
  .load                          : 0
  .runnable_load_avg             : 0
  .blocked_load_avg              : 0
  .tg_load_contrib               : 0
  .tg_runnable_contrib           : 0
  .tg_load_avg                   : 602
  .tg->runnable_avg              : 592
  .avg->runnable_avg_sum         : 0
  .avg->runnable_avg_period      : 47077

rt_rq[3]:/
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-    exec        sum-sleep
----------------------------------------------------------------------------------------    ------------------

Answer 1

Ok, I found I need to build my own kernel to turn on the config.