Write a circular file in c++

Question

I need to write a circular file in c++. The program has to write lines in a file and when the code reaches a maximum number of lines, it must overwrite the lines in the beginning of the file.

Anyone have any idea?

Answer 1

that's going to be tricky since file I/O works with bytes as the underlying unit of storage, and not lines.

I mean you could just fseek() back to the beginning and clobber the earlier data, but I have a hunch that's not what you want.

Answer 2

I've seen this done by keeping the current write position for the file somewhere. When you need to add a line, you seek to the position, write the line, and update the position in an atomic fashion. If you overflow, then you seek to zero before you write the line. We do this today for size constrained circular log files. Doing it on a line-constrained basis is a little odd, but could probably be done in a similar fashion. Our write loop looks something like:

logFile.lockForWrite();
currentPosition = logFile.getWritePosition();
logFile.seek(currentPosition);
for each line in lineBuffer {
    if ((currentPosition+line.length()) > logFile.getMaxSize()) {
        currentPosition = 0;
        logFile.seek(0);
    }
    logFile.write(line);
    currentPosition += line.length();
}
logFile.setWritePosition(currentPosition);
logFile.unlock();

The tricky part is in maintaining the current write position and finding some way to coordinate reading the file (e.g., with the tail utility) while your application is writing to it. Your reader utility has to keep track of the write position as well so it's read loop becomes:

lastPosition = logFile.getWritePosition();
while (!killed) {
    logFile.wait();
    logFile.lockForRead();
    newPosition = logFile.getWritePosition();
    logFile.seek(lastPosition);
    newLine = logFile.readFrom(lastPosition, (newPosition-lastPosition));
    lastPosition = newPosition;
    logFile.unlock();
}

This isn't in any particular language - it's just pseudocode but the idea is there. Of course, I left the handling all of the interesting edge cases to the reader.

With all of that said... I agree with the other opinions. Don't do this unless you have a really good reason. It sounds like a great idea, but:

• the implementation is difficult to get write
• it's even harder to make efficient
• since the write position has to be maintained somewhere, multiple utilities have to agree on how it is read, updated, initialized, etc.
• having a non-linear log makes log processing difficult using existing tools like grep, tail, perl, etc.

Overall, you will be better off using some existing package logging package that allows for configurable log file management. Take a look at Apache's log4cxx or Poco's Poco::Logger.