Ubuntu 10.04, error in using MAP_HUGETLB with MAP_SHARED

Question

Following is the code that I am using for mmaping a file in ubuntu with hugepages, but this call is failing with error "invalid argument". However, when I do pass MAP_ANON flag with no file descriptor parameter in mmap, then it works. I am not being able to understand the possible reason behind this.

Secondly, I am not able to understand why file mmaping is allowed with MAP_PRIVATE when this flag itself means that no change will be written back to file. This can always be accomplished using MAP_ANON, or is there something I am missing ?

Can someone help me with these ?

int32_t main(int32_t argc, char** argv) {
int32_t map_length = 16*1024*1024; // 16 MB , huge page size is 2 MB
int32_t protection = PROT_READ | PROT_WRITE;
int32_t flags = MAP_SHARED | MAP_HUGETLB;
int32_t file__ = open("test",O_RDWR|O_CREAT | O_LARGEFILE,s_IRWXU | S_IRGRP | S_IROTH);
if(file__ < 0 ) {
    std::cerr << "Unable to open file\n";
    return -1;
}

if (ftruncate(file__, map_length) < 0) {
    std::cerr
    << "main :: unable to truncate the file\n"
    << "main :: " << strerror(errno) << "\n"
    << "main :: error number is " << errno << "\n";
    return -1;
}
void *addr= mmap(NULL, map_length, protection, flags, file__, 0);
if (addr == MAP_FAILED) {
    perror("mmap");
    return -1;
}
const char* msg = "Hello World\n";
int32_t len = strlen(msg);
memcpy(addr,msg,len);
munmap(addr, map_length);
close(file__);
return 0;
}

Answer 1

Both your questions come down to the same point: Using mmap() you can obtain two kinds of mappings: anonymous memory and files.

Anonymous memory is (as stated in the man page) not backed by any file in the file system. Instead the memory you get back from an MAP_ANON call to mmap() is plain system memory. The main user of this interface is the C library which uses it to obtain backing storage for malloc/free. So, using MAP_ANON is explicitly saying that you don't want to map a file.

File-backed memory kind of blends in a file (or portions of it) into the address space of your application. In this case, the memory content is actually backed by the file's content. Think of the MAP_PRIVATE flag as first allocating memory for the file and then copying the content into this memory. In truth this will not be what the kernel is doing, but let's just pretend.

HUGE_TLB is a feature the kernel provides for anonymous memory (see Documentation/vm/hugetlb‐page.txt as referenced in the mmap() man page). This should be the reason for your mmap() call failing when using HUGETLB for a file. *Edit: not completely correct. There is a RAM file system (hugetlbfs) that does support huge pages. However, huge_tlb mappings won't work on arbitrary files, as I understand the docs.*

For details on how to use HUGE_TLB and the corresponding in-memory file system (hugetlbfs), you might want to consider the following articles on LWN:

• Huge Pages, Part 1 (Intro)
• Huge Pages, Part 2 (Interfaces)
• Huge Pages, Part 3 (Administration)
• Huge Pages, Part 4 (Benchmarking)
• Huge Pages, Part 5 (TLB costs)