Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Buffer Size in C++

I am observing the following behavior with the C++ Std library method std::ostream::write().

For buffering the data I am making use of the following C++ API

std::ofstream::rdbuf()->pubsetbuf(char* s, streamsize n)

This works fine ( verified using the strace utility ) as long as the size of data (datasize) we are writing on the file stream using

std::ofstream::write (const char* s, datasize n)

Is less than 1023 bytes ( below this value the writes are accumulated till the buffer is not full), but when the size of data to write exceeds 1023, the buffer is not taken into account and the data is flushed to the file.

e.g. If I set the buffer size to 10KB and write around 512bytes a time, strace will show that multiple writes have been combined into a single write

writev(3, [{"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 9728}, {"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 512}], 2) = 10240 ( 10 KB )
writev(3, [{"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 9728}, {"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 512}], 2) = 10240
...

but when I write 1024 bytes a time ( keeping the buffer fixed to 10 KB), now strace shows me that it is not using the buffer and each ofstream::write call is being translated to write system call.

writev(3, [{NULL, 0}, {"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 1024}], 2) = 1024 ( 1KB )
writev(3, [{NULL, 0}, {"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"..., 1024}], 2) = 1024
...

Is there any C++ API Call or Linux Tuning Parameter which I am missing?

like image 577
Abhishek Avatar asked Mar 18 '14 11:03

Abhishek


People also ask

What is the size of a buffer?

Buffer Size is the amount of time allowed for your computer to process the audio of your sound card or audio interface. This applies when experiencing latency, which is a delay in processing audio in real time.

Why is buffer size 1024?

1024 is the exact amount of bytes in a kilobyte. All that line means is that they are creating a buffer of 16 KB. That's really all there is to it. If you want to go down the route of why there are 1024 bytes in a kilobyte and why it's a good idea to use that in programming, this would be a good place to start.

What is buffer in C programming?

A temporary storage area is called buffer. All input output (I/O) devices contain I/O buffer. When we try to pass more than the required number of values as input then, the remaining values will automatically hold in the input buffer. This buffer data automatically go to the next input functionality, if it is exists.

How do you find buffer size?

To check the buffer window, multiply the bit rate (bits per second) by the buffer window (in seconds) and divide by 1000 to get the size, in bits, of the buffer for the stream.


1 Answers

This is an implementation detail of libstdc++, implemented around line 650 of bits/fstream.tcc. Basically, if the write is larger than 2^10, it will skip the buffer.

If you want the rationale behind this decision, I suggest you send a mail to the libstdc++ development list.

http://gcc.gnu.org/ml/libstdc++/

like image 132
Sebastian Redl Avatar answered Sep 25 '22 16:09

Sebastian Redl