Looking for mmap flag values

Question

I was wondering where I could find mmap flag values on os x. The manpages for mmap say to use MAP_PRIVATE, MAP_... and such, but if you are dealing with assembly you have to know the actual values to make the syscall. I tried looking for the header files that defined these constants but I could not find it. Could someone link it possibly?

Answer 1

To find the right file in the first place: two options:

• google the macro, and look at the documentation to see what headers it says to include. e.g. the mmap man page only mentions sys/mman.h.
• Or: learn where your system keeps header files, and recursive-grep there. (Or use ack, because it knows not to search binary files and is generally good for this sort of thing). e.g. on my GNU/Linux system,
  ack --hh MAP_PRIVATE /usr/include/ /usr/lib/gcc/x86_64-linux-gnu/5/ will search only .h files (--hh) under those two directories, which is where all the system headers live.

To find where your system keeps headers in the first place, cpp output (without -dM) includes current-line-number setting lines which tell you which headers got included. e.g.
# 216 "/usr/lib/gcc/x86_64-linux-gnu/5/include/stddef.h" 3 4

---

Beware octal literals for NASM

If any of the constants are in octal, like #define O_CREAT 00000100 in asm-generic/fcntl.h, beware that NASM treats 0100 as decimal, so you'd need to change it to 0q100 or 0o100.

The GNU assembler matches C in interpretation of integer literals, so you should be able to use macro definitions directly even for open() flags. (And file permission constants like S_IRUSR if you want use those instead of writing octal constants like a normal assembly language programmer. :P)

Use the C preprocessor to dump macro values.

gcc -E -dM prints only #define lines with the final values of all macros at the end of processing, instead of the preprocessed source. - tells it to read from stdin, so you can pipe a code fragment into it with echo.

echo '#include <sys/mman.h>' | gcc -E - -dM | less

In less, you can of course search with /. You can also filter with &, hiding lines that don't match the regex. So you can type &MAP_ to get just the MAP_ macro definitions.

---

On x86-64 GNU/Linux, I get:

#define MAP_32BIT 0x40
#define MAP_TYPE 0x0f
#define MAP_EXECUTABLE 0x01000
#define MAP_FAILED ((void *) -1)
#define MAP_PRIVATE 0x02
...

---

Maybe using this in a build system instead of hard-coding your search results

If you're writing some actual asm code that wants these macro definitions, remember that gcc -c foo.S runs .S files through the C preprocessor before assembling. However, sys/mman.h contains lines other than macro definitions (like typedef unsigned char __u_char;) which aren't valid asm syntax.

The most future-proof / portable way to do this might be for your build scripts / Makefile to use gcc -E -dM to create a local macro-only version of the system headers you need. Then your .S files could #include "system_macros/mman.h" and so on.

However, note the ((void *) -1) definition for MAP_FAILED: that syntax won't assemble, so this doesn't always work for writing portable asm that works on different systems with the same ABI but different constants and syscall numbers.

For syscall numbers, the system asm/unistd.h usually only has macros, not prototypes or typedefs.

---

Your asm source should look like this:

# 4th arg (flags) goes in r10 for the syscall ABI, vs. rcx for function calls
mov   $(MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB), %r10d

The parens aren't necessary: one $ makes the whole operand an immediate like $foo|bar. I guess you could think of it like an operator with the lowest possible precedence.

Or for NASM, use something like sed to turn the cpp macros into NASM .equ definitions:

mov   r10d,  MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB