Read N bytes from file and append them to a std::vector

Question

I want to read N bytes of data from a file stream and append them to a vector. So let's say we have a

basic_ifstream<uint8_t> myFileStream;

and a

vector<uint8_t> myBuffer;

Currently I'm doing something like this:

myBuffer.reserve(N);
for (int i=0; i<N; ++i)
{
    uint8_t tmpByte;
    myFileStream.read(&tmpByte, 1);
    myBuffer.push_back(tmpByte);
}

but this is extremely slow.

Now I tried to let myFileStream.read copy the data directly into the vector. Since a vector stores its elements in a contiguous storage location, I thought that something like this should be possible:

uint8_t* ptr = &myBuffer.back(); // there is already some elements in the buffer (I know)
ptr++; // first element after existing data
myBuffer.resize(myBuffer.size() + N);
myFileStream.read(ptr, N);

But with this I get a runtime error (corruption of heap). What is wrong with this solution? Or is there a better way to do this anyway?

Answer 1

Your problem is that resize may need to reallocate the whole vector, and thus invalidate your previous ptr. You need to take the pointer only after resize.

std::size_t oldSize = myBuffer.size();
// resize first
myBuffer.resize(oldSize + N);
uint8_t* ptr = &myBuffer[oldSize]; // already first element after existing data
myFileStream.read(ptr, N);

Note that as a bonus this implementation will work even if the original vector is empty (for N != 0, of course).