posix_fadvise(WILLNEED) makes IO slower?

Question

On a CentOS 5.5 machine running Linux kernel version 2.6.18-194.26.1.el5, I noticed that posix_fadvise(WILLNEED) makes reading 60K files much slower than just regular IO by almost 200%.

It seems that the actual fadvise call is synchronous, and it also delays scheduling other threads in the application that use the data being read from the files.

Is it possible that the kernel is busy with fetching data from the disks due to the fadvise call, and ends up delaying other scheduled task? It seems contrary to the expected asynchronous prefetch behaviour that we expect from making an fadvise call.

My question is: are there any tunable kernel parameters that could be used to enforce asynchronous behavior for posix_fadvise(WILLNEED)? Like increasing kernel IO threads, page cache?

Answer 1

This was just recently asked on LKML. That's just how it's designed. If you need a non-blocking fadvise, do it in another thread.