Why can't I reserve 1,000,000,000 in my vector?

Question

When I type in the foll. code, I get the output as 1073741823.

#include <iostream>
#include <vector>
using namespace std;
int main()
{
  vector <int> v;
  cout<<v.max_size();
  return 0;
}

However when I try to resize the vector to 1,000,000,000, by v.resize(1000000000); the program stops executing. How can I enable the program to allocate the required memory, when it seems that it should be able to?

I am using MinGW in Windows 7. I have 2 GB RAM. Should it not be possible? In case it is not possible, can't I declare it as an array of integers and get away? BUt even that doesn't work.

Another thing is that, suppose I would use a file(which can easily handle so much data ). How can I let it read and write and the same time. Using fstream file("file.txt', ios::out | ios::in ); doesn't create a file, in the first place. But supposing the file exists, I am unable to use to do reading and writing simultaneously. WHat I mean is this : Let the contents of the file be 111111 Then if I run : -

#include <fstream>
#include <iostream>
using namespace std;
int main()
{
  fstream file("file.txt",ios:in|ios::out);
  char x;
  while( file>>x)
  {
    file<<'0';
  }
 return 0;
}

Shouldn't the file's contents now be 101010 ? Read one character and then overwrite the next one with 0 ? Or incase the entire contents were read at once into some buffer, should there not be atleast one 0 in the file ? 1111110 ? But the contents remain unaltered. Please explain. Thank you.

Answer 1

1. A 32-bit process can only address 4GB address space at a single time. Usually, plenty of this 4GB address space is used to map other stuff. Your vector is going to take too much contiguous address space (4 billion bytes) which is not likely to be available.

2. You should memory map the file. See mmap.

Answer 2

The maximum the STL implementation will cope with is one thing; the maximum amount of memory available from the OS is something else; you are hitting the latter.

You might for example be able to create a vector of that many char elements. Either way, don't expect blistering performance unless you physically have that much memory (plus whatever the OS and anything else running needs); accessing such a vector will no doubt result in much disk thrashing as the system pages memory in and out from disk.

Also a processor with a 32bit address space (or when running 32 bit OS regardless of physical address space) can only address 4Gb (physical or virtual), so there is an architectural limit. Moreover some OS's limit the user space; for example the user space in Win32 is fixed at 2Gb. Various versions of Win64 artificially limit the user space in order to allow Microsoft to charge different prices, so using Win64 is no guarantee of sufficient address space.