What is the best way to add a header to a large file

Question

What 's the best way to add a header (e.g. file metadata) to an existing file if the file size is around 1-2GB?

in C++

Answer 1

The best way is to simply write the header to a new file, then append the contents of the old.

C++ doesn't provide a way to insert at the start of an existing file so that's really the best bet. Just make sure you read large enough chunks from the old file and append them to the new one. While buffering will alleviate most of the problems of doing this in small chunks, you'll still have the performance degradation of more function calls.

This may be minimal but it's still there. For a 2G file, I'd probably start by doing it in half-gig chunks unless memory is at a premium. This allows for larger file sizes without too much memory wastage and four read/write calls is unlikely to be a performance issue.

But, as with all optimisations, measure, don't guess. There are various low level things which can affect performance that the C++ standards document makes no mention of (and rightly so). Since your question makes no mention of a specific operating system, I've answered based on that, but those specific operating systems may both (a) react differently; and (b) provide other non-standard calls which can be made faster.

Answer 2

The only way to do it, in any language, is to write out the header to a new file, copy the contents of the old file to it, and then rename the file.