What is the valid address space for a user process? (OS X and Linux)

Question

The mmap system call documentation says that the function will fail if:

MAP_FIXED was specified and the addr argument was not page aligned, or part of the desired address space resides out of the valid address space for a user process.

I can't find documentation anywhere saying what would be a valid address to map. (I'm interested in doing this on OS X and linux, ideally the same address would be valid for both...).

Answer 1

Linux kernel reserves part of virtual address space for itself to where userspace have (almost) no access and can't map anything. You're looking for what's called "userspace/kernelspace split".

On i386 arch default is 3G/1G one -- userspace gets lower 3 GB of virtual address space, kernel gets upper 1 GB, additionally there are 2G/2G and 1G/3G splits:

config PAGE_OFFSET
        hex
        default 0xB0000000 if VMSPLIT_3G_OPT
        default 0x80000000 if VMSPLIT_2G
        default 0x78000000 if VMSPLIT_2G_OPT
        default 0x40000000 if VMSPLIT_1G
        default 0xC0000000
        depends on X86_32

On x86_64, userspace lives in lower half of (currently) 48-bit of virtual address space:

/*
 * User space process size. 47bits minus one guard page.
 */
#define TASK_SIZE_MAX   ((1UL << 47) - PAGE_SIZE)

Answer 2

This varies based on a number of factors, many of which aren't under your control. As adobriyan mentioned, depending on the OS, you have various fixed upper limits, beyond which kernel code and data lies. Usually this upper limit is at least 2GB on 32-bit OSes; some OSes provide additional address space. 64-bit OSes generally provide an upper limit controlled by the number of virtual address bits supported by your CPU (usually at least 40 bits worth of address space). However there are yet other factors beyond your control:

• On recent version of linux, mmap mappings below the address configured in /proc/sys/vm/mmap_min_addr will be denied.
• You cannot make mappings which overlap with any existing mappings. Since the dynamic linker is free to map anywhere that does not overlap with your executable's fixed sections, this means potentially any address may be denied.
• The kernel may inject other additional mappings, such as the system call gate.
• malloc may perform mmaps on its own, which are placed in a somewhat arbitrary location

As such, there is no way to absolutely guarentee that MAP_FIXED will succeed, and so it should normally be avoided.

The only place I've seen where MAP_FIXED is necessary is in the wine startup code, which reserves (using MAP_FIXED) all addresses above 2G, in order to avoid confusing windows code which assumes no mappings will ever show up with a negative address. This is, of course, a highly specialized use of the flag.

If you're trying to do this in order to avoid having to deal with offsets in shared memory, one option would be to wrap pointers in a class to automatically handle offsets:

template<typename T>
class offset_pointer {
    private:
       ptrdiff_t offset;
    public:
        typedef T value_type, *ptr_type;
        typedef const T const_type, *const_ptr_type;

        offset_ptr(T *p) { set(p); }
        offset_ptr() { set(NULL); }

        void set(T *p) {
            if (p == NULL)
                offset = 1;
            else
                offset = (char *)p - (char *)this;
        }
        T *get() {
            if (offset == 1) return NULL;
            return (T*)( (char *)this + offset );
        }

        const T *get() const { return const_cast<offset_pointer>(this)->get(); }

        T &operator*() { return *get(); }
        const T &operator*() const { return *get(); }
        T *operator->() { return get(); }
        const T *operator->() const { return get(); }

        operator T*() { return get(); }
        operator const T*() const { return get(); }

        offset_pointer operator=(T *p) { set(p); return *this; }
        offset_pointer operator=(const offset_pointer &other) {
            offset = other.offset;
            return *this;
        }
};

Note: This is untested code, but should give you the basic idea.