mmaping two consecutive pages

Question

I'm writing a unit test for my UTF8 manipulation library, and I want my test to segfault if a function goes into a buffer overflow. So I came up with the idea to mmap two consecutive memory pages, the first with PROT_READ | PROT_WRITE, and the second with PROT_NONE. That way, if any overflow occurs, a segfault is guaranteed. Here's an example:

void *addr1, *addr2; /* these are the pages; mmap call left out for simplicity */
char *p = (char *) (addr1 + getpagesize() - 8);

utf8_encode(aUtf8String, p, 8); // this shouldn't segfault

The problem is, when I map the second page, my program segfaults. Here's an example program that reproduces the problem (GNU/Linux):

#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
#include <sys/mman.h>

void checkMap(void *p) 
{ 
    if(p == MAP_FAILED) {
        printf("error running mmap: %s\n", strerror(errno));
        exit(1);
    }   
}

int main(void)
{
    void *addr1, *addr2;
    size_t pagesize;

    pagesize = getpagesize();
    checkMap(addr1 = mmap(NULL, pagesize, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0));
    checkMap(addr2 = mmap(addr1 + pagesize, pagesize, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0));  /* segfaults */
    munmap(addr1, pagesize);
    munmap(addr2, pagesize);

    return 0;
}

Interestingly enough, a printf() statement before the first mmap() causes the program to run successfully. Does anyone know why mmap is segfaulting? If my goal is unattainable using mmap(), does anyone have any other advice on how I could test my code for buffer overflow?

Answer 1

You can call mprotect() to change the protection flags on memory mapped by mmap(). This might be a better solution than trying to mmap() two adjacent pages with different protections, since this seems to be what's causing your problems.

(Linux allows you to call mprotect() on any page, but POSIX only allows pages that have already been allocated by mmap().)

This is one of the tricks Electric Fence uses to catch buffer overruns.