fadvise vs madvise? can I use both together?

Question

I'm randomly reading data (each read < page size) throughout a huge file (far too big to fit in memory).

I normally set MADV_DONTNEED, but looking at the docs + info it seems I instead need FADV_NOREUSE.

I'm not really getting how madvise() and fadvise() work together. Are they synonymous? Does it matter if I prefer one or the other? Can they be used together? Are they different kernel subsystems? Is FADV_NOREUSE what I'm looking for to gain optimal performance?

Answer 1

madvise() and posix_fadvise() are not synonymous. madvise() tells the kernel (give advise) what to do with existing memory region while fadvise() tells the kernel what to do with cached (or future cache) of a file data.

For example, if you mmap() anonymous region you should use madvise() to hint the kernal not to swap out (MADV_RANDOM) or to swap out only after access. (MADV_SEQUENTIAL)

If you mmap() a file, or part of a file, you can use either madvise() or fadvise() to hint the kernel to readahead for you (MADV_WILLNEED) or to free that cache (MADV_DONTNEED) or to free after access (POSIX_FADV_NOREUSE, fadvise() only) in additional to the above.

If you use file without mapping the data to your process memory (without using mmap()), you should use fadvise() only. madvise() has no meaning.

As far as kernel subsystem, in linux, it is the same subsystem, simply different ways to refer to memory pages and file cache. Please note that those are only hints and when memory is in dire, the kernel might decide to swap out or reuse cached data despite the hint. Only mlock() and mlockall() can prevent that.

In your case, not giving any hint may help, especially if some pages are being read more than other, since the kernel will figure out which pages are "hot" and will attempt to keep in memory.