Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

is there 2G limit for file reading/writing by c++ fstream?

I am writing a c++ program to read/write a large file (probably larger than 60GB). By googling the problem, it seems that there is a 2GB limit on file io in 32 bit system (I am using windows 7 64bit but my program was compiled with mingw32). In my program, I am writing 10 integers at a time to the file and all these numbers are generated randomly based on some algorithm. It seems that the program can run even when the file size bigger than 40GB but there is no way for me to check if the data read by the program is really the one stored in the file or some junk numbers. But anyway, the program doesn't report any warning or error. Is this really possible to read/write file larger than 60GB in a 32-bit program?

like image 908
user1285419 Avatar asked Apr 08 '12 22:04

user1285419


People also ask

How many files can be opened at the same time in ANSI C?

In this article The C run-time libraries have a 512 limit for the number of files that can be open at any one time. Attempting to open more than the maximum number of file descriptors or file streams causes program failure.

Can you read and write to a file at the same time C++?

In C++, we can also read and write to a file at the same time. To both read and write to a file, we have to get an fstream object and open the file in “ios::in” and “ios::out” mode. In this example, we first write some content to the file. Then, we read the data from the file and print it to the monitor.

Can we use fstream in C++?

The I/O system of C++ contains a set of classes which define the file handling methods. These include ifstream, ofstream and fstream classes. These classes area derived from fstream and from the corresponding iostream class.

For what purpose we will use fstream?

Either ofstream or fstream object may be used to open a file for writing. And ifstream object is used to open a file for reading purpose only. Following is the standard syntax for open() function, which is a member of fstream, ifstream, and ofstream objects.


1 Answers

There's a limit on file size (4GB max, I think) on Fat32 file system. Windows 7 definitely shouldn't be using that filesystem by default.

Also on 32bit system there's a limit on the file size you can map into memory at once using CreateFileMapping/MapViewOfFile. However, fstream doesn't use CreateFileMapping/MapViewOfFile internally, so there's no limit for file size (aside from filesystem limits). And even with CreateFileMapping you can map portion of larger file into memory, so there's no limit aside from the one imposed by filesystem.

like image 125
SigTerm Avatar answered Sep 23 '22 09:09

SigTerm