How would one prevent MMAP from caching values?

Question

I've written a linux driver that ioremaps exports PCI BAR0 for a particular device to a sysfs binary attribute allowing userspace to directly control it.

The problem rears when I attempt to MMAP on top of the attribute to directly access that bit of memory (from a userland program). Reads succeed just fine and return expected values, though when I write to that memory it appears to be cached somewhere between the kernel and memory and not delivered to the GMCH root complex (and therefore the device). What I'd like to do is have an implicit write memory barrier after each access.

• Is there any way to prevent the kernel from caching writes to a mmap-ed bit of memory?

Follow ups:

• Is calling msync() after every access the "accepted" way to do this?

Answer 1

Going to go ahead and answer this one myself with my solution.

In the Kernel driver from my sysfs mmap function, there is a macro in /include/asm/pgtable.h that sets the proper flags for a nocache'd pfn remap. It looks like this:

vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
if (io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
               vma->vm_end - vma->vm_start,
               vma->vm_page_prot))
    return -EAGAIN;

Additionally, in the userland mmap, I used the MAP_SHARED flag in the mmap flags argument.

The combination of the two ultimately did the trick.