maximum size of bss and data

Question

I want to declare all the variables in my C program at compile time, like for example:

char cache[CACHE_SIZE];
char udp_ring[MAX_UDP_PACKET_SIZE*MAX_REQUESTS];
int  num_packets;
char error_codes[NUM_ERRORS][MAX_ERROR_STRING]= {
    {"Unknown user\n"},
    {"Wrong password\n"},
    ....
};

The question is, are there any limits on the size of the variables in a C program when they go in BSS or DATA segment? For example if I declare CACHE_SIZE of 8GB of RAM, will it work? Is there any difference for 32 bits or 64 bits? I plan to run the program on Linux and there will be no restriction in my RLIMIT_DATA configuration.

Answer 1

You will be able to manage as much virtual memory as your kernel allows processes to handle : it will depend on the architecture.

For example, on a x86 architecture (without x86-64 long mode), Linux splits by default the virtual memory seen by a process in 3GB for the process and 1GB for the kernel (even if PAE is enabled) : your process won't be able to handle more than 3GB of virtual memory (include text sections, data, bss, heap, stack, shared objects, etc.)

If you allocate all your buffer statically and the kernel cannot fit it in the process virtual address space, it will be killed at start-up : using a 8GB buffer will mostly result in this behavior on a 32-bit architecture.

If you don't want to rely on glibc's memory management function (malloc, ...), you could roll your own memory management library and force your process to use it with a LD_PRELOAD trick, that way you could define a malloc/calloc/realloc/free (using sbrk()) implementation matching your own requirements.