seekg cannot handle file of 4294967295 bytes properly

Question

I found that in VS2010, seekg function does not work properly when file of exactly 4294967295 bytes is opened.

I'm using simple code:

#include <iostream>
#include <fstream>
using namespace std;

int _tmain(int argc, _TCHAR* argv[])
{
    std::ifstream file;

    // cmd: fsutil file createnew tmp.txt 4294967295
    file.open(L"c:/tmp.txt", ifstream::in | ifstream::binary);

    if(!file.is_open())
        return -1;

    file.seekg(0, std::ios::end);

    auto state = file.rdstate();

    // this condition shoots only when size of the file is equal to 4294967295
    if((state & ifstream::failbit)==ifstream::failbit)
    {
        std::cout << "seekg failed";
    }

    // after seekg failed, tellg returns 0
    std::streampos endPos = file.tellg();

    return 0;
}

Same code with files of 4294967294 and 4294967296 is working without any problems.

Does someone know a solution to this problem?

Update:

It looks like that problem lies here:

template<class _Statetype>
class fpos
{
 __CLR_OR_THIS_CALL operator streamoff() const
 { // return offset
 return ((streamoff)(_Myoff + _FPOSOFF(_Fpos)));
 }
}

exactly at

_FPOSOFF(_Fpos)

where

#define _FPOSOFF(fp) ((long)(fp))

So it takes 4294967295 and converts it to -1 !

In other words speaking, such code will fail

//returns -1, even if sizeof(fpos_t)=8
fpos_t pos = _FPOSOFF(4294967295);

_Myoff, _Fpos, streamoffset are 64bit

Why they do this conversion if all types are 64 bit!? I have no idea ))

Answer 1

Internally the stream implementation has a const '_BADOFF' which is equal to 0xffffffff, which is returned when the seek has failed. In this case the seek is succeeding, but the returned value from the seek is equal to the failure code, which results in the stream wrapper setting its fail-code erroneously.

_BADOFF is defined as a 64-bit type, it's just been assigned a stupid value.

As a workaround, you can seek 1-byte short, and then read a byte.

file.seekg(-1, std::ios::end);
char temp; file >> temp;

However note that this bug will manifest any time that particular file-offset is seek'd to, so it could still be a problem for larger files if you seek in them to random locations. For example if your file was one byte larger, this -1 seek would fail, so this is not a general solution.

The OP has extended their question, so I'll extend my answer. Yes, the seek value is cast using an unsafe conversion before the comparison. This doesn't seem to affect the ability to seek beyond that point in the file, as it's only used for comparison with the error value - the stream still has the right offset in it. However it does seem to be the root cause of _BADOFF being dubious in the first place, as _BADOFF is set to be '-1' in the source, and will suffer the same conversion, truncating to 0xffffffff.

So the fix for the libs might be to fix the cast (assuming there are no other side-effects of doing so), but for the sake of working around the problem, you only have to avoid seeking to positions where the bottom 32-bits are set. It'll seek beyond that OK, from what I can see.

Answer 2

This is indeed a bug in Visual C++ 2010. It was reported on Microsoft Connect two years ago: "std::fstream use 32-bit int as pos_type even on x64 platform" (the title of the bug is incorrect; the symptoms were actually caused by this bug in _FPOSOFF).

This bug is fixed in Visual C++ 2012, wherein _FPOSOFF is defined as:

#define _FPOSOFF(fp)  ((long long)(fp))

You would be well advised to upgrade to Visual C++ 2012 if you are able to do so.