boost streambuf consume and commit, what is it

Question

I can't seem to find a good explanation of what consume() and commit() really means, actually I don't understand streambuf at all.

My understanding is that a streambuf is just a character array. But why is that in the documentation,

basic_streambuf::data
Get a list of buffers that represents the input sequence.

so actually there are many buffers? And what is the 'input sequence' and the 'output sequence'? Are these another two buffers?

What does the following code really do?

streambuf b;
size_t size;
size = read( socket, b.prepare( 1024 ) ); 
b.commit( size );
size = write( socket, b.data() );
b.consume( size );

when I call b.prepare(), does it allocate a new buffer for the read() to put data? Then when is the data transferred from that buffer to the underlying streambuf buffer? I thought it was the commit(), but

basic_streambuf::commit
Move characters from the output sequence to the input sequence.

so it seems that commit actually moves the data from the 'output sequence' to 'input sequence' without ever mentioning the underlying buffer used to store data!

Answer 1

Boost ASIO streambuf is more than just a character array. From the basic_streambuf documentation:

The basic_streambuf class is derived from std::streambuf to associate the streambuf's input and output sequences with one or more character arrays. These character arrays are internal to the basic_streambuf object, but direct access to the array elements is provided to permit them to be used efficiently with I/O operations. Characters written to the output sequence of a basic_streambuf object are appended to the input sequence of the same object.

The documentation goes on to talk about possible implementation strategies. The streambuf object may simply use a single contiguous character array with pointers to manage the input and output sequences. But the interface allows for more complicated schemes.

You asked what the code snippet actually does, but this depends on the underlying implementation. In short, prepare() ensures that the underlying buffer is big enough to hold what you are trying to put into it. It also gives you access to the buffer via a mutuable_buffers object. Once data has been written to the streambuf (presumably, when the read handler is called), commit() makes that data available to the input sequence. You can access those bytes using data(). After you are finished with the data in the buffer (because you copied it, processed it, or whatever), consume() removes the data from the input sequence. A subsequent call to data() would not contain the bytes from the previous call.

You also asserted that the underlying buffer used to store the data is never mentioned, and that is correct. It is up to the ASIO authors to decide how to store the actual data.