Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to implement a circular buffer using a file?

My application (C program) opens two file handles to the same file (one in write and one in read mode). Two separate threads in the app read from and write to the file. This works fine. Since my app runs on embedded device with a limited ram disk size, I would like write FileHandle to wrap to beginning of file on reaching max size and the read FileHandle to follow like a circular buffer. I understand from answers to this question that this should work. However as soon as I do fseek of write FileHandle to beginning of file, fread returns error. Will the EOF get reset on doing fseek to beginning of file? If so, which function should be used to cause write file position to get set to 0 without causing EOF to be reset.

EDIT/UPDATE: I tried couple of things:


  1. Based on @neodelphi I used pipes this works. However my usecase requires I write to a file. I receive multiple channels of live video surveilance stream that needs to be stored to harddisk and also read back decoded and displayed on monitor.

  2. Thanks to @Clement suggestions on doing ftell I fixed a couple of bugs in my code and wrap works for the reader however, the data read appears to be stale data since write are still buffered but reader reads stale content from hard disk. I cant avoid buffering due to performance considerations (I get 32Mbps of live data that needs to be written to harddisk). I have tried things like flushing writes only in the interval from when write wraps to when read wraps and truncating the file (ftruncate) after read wraps but this doesnt solve the stale data problem.

  3. I am trying to use two files in ping-pong fashion to see if this solves the issue but want to know if there is a better solution

like image 408
Badri Avatar asked Aug 25 '11 18:08

Badri


People also ask

How would you implement a circular buffer?

Circular Buffers can be implemented in two ways, using an array or a linked list. An empty object array along with its capacity is initialized inside the constructor as the type of elements added is unknown.

What makes up a circular buffer?

In computer science, a circular buffer, circular queue, cyclic buffer or ring buffer is a data structure that uses a single, fixed-size buffer as if it were connected end-to-end. This structure lends itself easily to buffering data streams.

What is circular buffer in embedded systems?

In an embedded system we must not only emit outputs in real time, but we must also do so using a minimum amount of memory. Circular buffer. The circular buffer is a data structure that lets us handle streaming data in an efficient way. Figure 5.1 illustrates how a circular buffer stores a subset of the data stream.


1 Answers

You should have something like that :

// Write
if(ftell(WriteHandle)>BUFFER_MAX) rewind (WriteHandle);
fwrite(WriteHandle,/* ... */);

// Read (assuming binary)
readSize = fread (buffer,1,READ_CHUNK_SIZE,ReadHandle);
if(readSize!=READ_CHUNK_SIZE){
    rewind (ReadHandle);
    if(fread (buffer+readSize,1,READ_CHUNK_SIZE-readSize,ReadHandle)!=READ_CHUNK_SIZE-readSize)
        ;// ERROR !
}

Not tested, but it gives an idea. The write should also handle the case BUFFER_MAX is not modulo WRITE_CHUNK_SIZE.

Also, you may read only if you are sure that the data has already been written. But I guess you already do that.

like image 152
Clement Bellot Avatar answered Sep 28 '22 05:09

Clement Bellot