What is the limit on malloc parameter of type size_t in C? Docs say it has an upper limit of UINT_MAX but I can't go beyond INT_MAX

Question

I want to allocate a 2.9GB char array with

  database = (char*) malloc((2900 * 1000000 * sizeof(char)));

This gives an integer overflow warning and the malloc returns NULL. The malloc parameter is of type size_t which according to documentation is of type unsigned int.

So the max should be UINT_MAX which is at least 2.9GB. However, if I try to allocate more than MAX_INT the malloc fails. Does this mean size_t on my system is of type int? How do I check this? I looked through

/usr/include/stdlib.h 

and

./lib/gcc/x86_64-redhat-linux/4.1.1/include/stddef.h 

but can't find the definition of size_t. Thanks very much

Answer 1

The parameter is of type size_t and malloc is required to accept any possible value of type size_t. Note that "accept" does not meant it is required to allocate that much; all it means is that malloc is not allowed to misinterpret a very large number you give it as a small/negative number due to overflow issues, thereby returning a buffer that's too small and creating a critical undetectable vulnerability your program cannot defend against. There are many possible reasons malloc could fail to allocate very large objects:

• that much memory is not available from the system
• due to fragmentation, no contiguous range of virtual addresses that large is available
• arbitrary limits

In this case I suspect you might be seeing the third, arbitrary limits, though I would not consider them so arbitrary. There's a very good reason to disallow allocations (and the existence of any objects) larger than SIZE_MAX/2: taking the difference between pointers within such large objects will result in (extremely dangerous) integer overflow and undefined behavior when the result does not fit in the (signed) type ptrdiff_t. Thus, on a robust 32-bit system, while the virtual address space size is 4GB, the maximum size of any single object will be 2GB.

Answer 2

There are two issues here.

First, the overflow warning: both 2900 and 1000000 are of type int, so the result of multiplying them is also of type int. The result cannot be represented by a 32-bit signed integer, so it overflows. You need to cast one (or both) arguments to size_t to use unsigned arithmetic.

(Or, you could move the sizeof(char) to be one of the first two terms, since its type is size_t, though you can also just remove the sizeof(char) since it is always 1.)

Second, the maximum size that malloc can allocate depends both on the platform on which you are running and on the current state of the program. If there is insufficient contiguous address space left to satisfy the request, obviously the malloc will fail.

Further, the platform on which you are running may have an upper limit on how large an object it can dynamically allocate. You'll need to consult your platform's documentation to find out what that upper limit is.

size_t is certainly not int, because int is always signed and size_t is always unsigned.

Answer 3

The maximum size that malloc can allocate depends both on the platform on which you are running and on the current state of the program. If there is insufficient contiguous address space left to satisfy the request, the malloc will fail obviously.