How does one do Raw IO on Mac OS X? (ie. equivalent to Linux's O_DIRECT flag)

Question

I having been googling for a way to do raw (sometimes called direct) i/o under mac os. Raw i/o turns of the operating system page cache to give the application more direct access to the disk. This is useful because some of the filestructure I am using are not efficient using LRU page replacement. It is fairly straight forward to implement the page replacement algorithms we need but first we need to turn off os x default buffering. We have already done this under linux using the O_DIRECT flag when opening files. Does anyone know how to turn off page buffering under mac os?

Cheers Tim

Answer 1

After some more reading through the man pages I finally found the ideal answer. It turns out mac os actually has very similar mechanism to O_DIRECT, however it is not through the open function it is through fcntl. Specifically there is an option called F_NOCACHE which allows you to turn the cache on or off for a particular file descriptor which is exactly what I wanted. See http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man2/fcntl.2.html for the full run down of the other things you can do with the mac version of fcntl, and explanation of its exact use. I hope this answer will help someone else out.

---

http://lists.apple.com/archives/filesystem-dev/2007/Sep/msg00010.html Is a good thread that explains how the F_NOCACHE flag behaves depending on your mac os version number.

---

Final Code (in go):

    r1, r2, err := syscall.Syscall(syscall.SYS_FCNTL, uintptr(self.file.Fd()), syscall.F_NOCACHE, 1)
    if err != 0 {
        fmt.Printf("Syscall to SYS_FCNTL failed\n\tr1=%v, r2=%v, err=%v\n", r1, r2, err)
        self.Close()
        return false
    }

Answer 2

You may want to use the madvise system call. You can give hints to the kernel that about which pages to flush first by using MADV_DONTNEED or MADV_WILLNEED. OS X also supports an mmap flag MAP_NOCACHE, which instructs the kernel to discard the resulting pages first.