How can I know the real maximum size of a vector? (Not using std::vector::max_size)

Question

On an online course I am learning about vectors. In one of the examples they explained that: std::vector::max_size() should give me the maximum size the vector can reach. I decided to test it:

#include <iostream>
#include <exception>
#include <vector>

int main(void) {
    std::vector <int> nums;
    int max = nums.max_size();
    std::cout << "Max: " << max << std::endl;
    for (int i = 0; i < max; i++) {
        try {
            nums.push_back(i);
        }
        catch (std::bad_alloc ex) {
            std::cerr << ex.what() << std::endl;
            std::cout << "Failed at: " << i << std::endl;
            break;
        }
    }

    return 0;
}

And this is the result of running it:

Max: 1073741823
bad allocation
Failed at: 204324850

It was 869416973 ints short.

So I started googling it. Here I read that it returns the "the maximum potential size the container can reach", and adds "but the container is by no means guaranteed to be able to reach that size". I would have imagined that it would fail, but not by that much. It just got 1/5 of the way before failing. Why is std::vector::max_size so off? And what I see of more importance, is there a way of really knowing the potential size of a vector?

Answer 1

Note that the max_size function returns a theoretical maximum number of elements, it doesn't say anything about the amount of memory needed.

If we assume that sizeof(int) == 4 (pretty common) then 204324850 elements would need 817299400 bytes of contiguous memory (that's almost 780 MiB).

You get a bad_alloc exception because the vector simply can't allocate enough memory to hold all the elements.

Answer 2

std::vector::max_size() should give me the maximuim size the vector can reach

This is not quite correct. max_size gives you a theoretical upper bound. Vector definitely won't support any size larger than that, but that doesn't necessarily mean that you can create all vectors up to that size.

The most limiting factor will be the amount of free memory that the operating system is willing or able to assign for the process. There is no standard way to get that size, and even implementation specific ways are not straight forward.

Another potential limit is longest free contiguous address space, which may be fragmented. This probably won't be a problem for 64 bit programs with their astronomically large address space, but it is a consideration for systems with 32 bit or smaller address.

---

Failed at: 204324850

Assuming 4 byte int, that is about 780 Megabytes (non-metric).

---

In conclusion: Instead of trying to find out how much memory your program could use at run time, you should figure out the amount of memory that you know will be sufficient. Don't allocate more than that. Make sure that the computer has sufficient memory, and the operating system is not configured to limit the memory use to lesser amount. Use 64 bit address space.

Answer 3

Think about it this way; the vector is written in a way that it can internally handle up to (say) 32 bits worth of elements, so max_size will give you some number in the ~2-4 billion range. But you are running the code on a system with only 1MB of memory, so of course you can never grow the container that big. But, the vector has no way of knowing on what system you use it - it only knows it's maximum theoretical limit.