Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Processing files larger than 2 GB in C++ with STL

Tags:

c++

stl

fstream

I am doing binary file processing and in my algorithm I would like to know the actual type of pos_type and off_type, for example when computing the size of the file or seeking to a given position (tellg and seekg).

When computing the size of the file I just static_cast the pos_type to an int64_t and it seems to work fine.

How about seekg? Is it safe to pass an int64_t to it?

Is there a way to make pos_type and off_type to be an int64_t, perhaps using traits?

I would like to eliminate the hideous cast and find a way that is in accordance with the C++ standard.

Update: see also

  • Is it safe to assign pos_type to uint64_t when dealing with large files (> 2GB)?

  • iostream and large file support

like image 666
Ali Avatar asked Dec 10 '10 01:12

Ali


2 Answers

Not all compilers have STL implementations with large file support.

For example, the following program:

#include <fstream>
#include <iostream>
#include <limits>

int main()
{
    using namespace std;

    cout << numeric_limits<streamoff>::max() << endl;
}

results in:

  • VS2005 - 2147483647
  • VS2008 - 2147483647
  • VS2010 - 9223372036854775807
  • MinGW GCC 4.4.0 - 9223372036854775807

On the other hand STLPort has cross platform large file support.

like image 161
Cristian Adam Avatar answered Oct 08 '22 01:10

Cristian Adam


You should probably use std::fpos_t typedef.

like image 43
LavaScornedOven Avatar answered Oct 08 '22 01:10

LavaScornedOven