I have enough memory but mmap keeps failing saying Cannot allocate memory

Question

I am trying to mmap some large files but failed : I can't mmap any file that's larger than around 2.7 GB.

By doing ulimit -a, it says my virtual memory is unlimited. I then did cat /proc/meminfo it says my system has 3GB Memfree and 2GB Swapfree.

I am assuming I could mmap a file up to 5GB. I tried everything I learned on the internet, including using MAP_NORESERVE flag and set overcommit_memory to 1. It's still the same, Cannot allocate memory. What could possibly go wrong?

I am using 64 bit cpu and a 32 bit linux system. Please let me know if you need anymore information. I would really appreciate any advice or tips.

Answer 1

You need to use 64 bit OS, because 32bit os does not have enough address space.
Edit: Although your system has some physical memory available, but a 32 bit process only address up to 4GB, kernel reserve about 1-2GB for kernel driver..., so user space usually has 2GB for 32 bit applications.

Answer 2

Your application simply doesn't have enough address space (useable memory addresses) to map a file that large.

Under Linux, a 32 bit process running on a 32 bit kernel typically has around 3GB of address space available, and a 32 bit process running on a 64 bit kernel has 4GB of address space available. If you need more than that, you will need to compile your application as a 64 bit process. This will give you at least 128TB of address space.