How to compile a program to make it capable to use >4GB memory on 32-bit Linux?

Question

The whole code is written in C, C++, and Fortran. Is it possible to make it to use more than 4GB memory. Now it is always crashed when it reaches 3GB memory.

If it is possible, how to set up the compiling options (or configure flags)?

We can use gcc, g++, ...or intel compilers

our OS: Fedora 12 x32

cat /proc/cpuinfo
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority
bogomips : 5319.72
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual

Answer 1

Unfortunately you are hitting the limit that

R> 2^32
[1] 4294967296

which is your 4 gb limit. So you simply cannot index beyond for a single application, no matter what the OS.

This is one of the reasons many of us switched to 64 bit versions of our OSs. Linux has supported this since the late 1990s. Just switch to FC (or Ubuntu or ...) in 64bit.

One possible alternative is installing more RAM (which Linux will handle) and segmenting your task over several instances of the application, effectively running it in parallel. But that may not be worth the trouble...

Answer 2

I believe that if you put a file in a tmpfs filesystem (or hugetlbfs) and in your program map small (1 - 2 GB) pieces of it at a time, you can work with more than 4 GB of data at once.

The map operations aren't all that fast so you will take a performance hit if you jump through your memory too randomly.