How to create stream which handles both input and output in C++?

Question

I'm trying to make a class that will be both input and output stream (like std::cout and std::cin ). I tried to overload operator << and >>, but then, I understood that writing such code is not wise to do (as this would be an approach to rewrite C++ streams) and maintaining is very hard when classes like std::basic_iostream, std::basic_ostream, std::basic_istream are available in C++ standard library because I have to overload operators for each type. So, I tried to define my class like this:

#include <istream>

class MyStream : public std::basic_iostream<char> {
public:
    MyStream() : std::basic_iostream<char>(stream_buffer) {}
};

My problem is with the first argument at constructor of std::basic_iostream<char> . As of cppreference, std::basic_iostream::basic_iostream takes a pointer to a stream buffer derived from std::basic_streambuf :

explicit basic_iostream( std::basic_streambuf<CharT,Traits>* sb );

I have read and tried examples from Apache C++ Standard Library User's Guide's chapter 38. It says that I have to pass a pointer to the stream buffer and there are three ways to do so:

• Create stream buffer before class initialization
• Take the stream buffer from another stream (using rdbuf() or similar member)
• Define a basic_streambuf object as a protected or private member

The last option fits best for my purpose, but if I directly create an object from the std::basic_streambuf class, it will do nothing, will it? So I defined another class which is derived from std::basic_streambuf<char>. But this time I can't understand what functions to define, because I don't know which function is called when data is inserted, extracted, and flushed.

How can one create a stream with custom functionalities?

---

_{Note that this is an attempt to build a standard guide on creating C++ streams and stream buffers.}

Answer 1

Creating a class that behaves like a stream is easy. Let's say we want to create such class with the name MyStream , the definition of the class will be as simple as:

#include <istream> // class "basic_iostream" is defined here

class MyStream : public std::basic_iostream<char> {
private:
    std::basic_streambuf buffer; // your streambuf object
public:
    MyStream() : std::basic_iostream<char>(&buffer) {} // note that ampersand
};

The constructor of your class should call the constructor of std::basic_iostream<char> with a pointer to a custom std::basic_streambuf<char> object. std::basic_streambuf is just a template class which defines the structure of a stream buffer. So you have to get your own stream buffer. You can get it in two ways:

1. From another stream: Every stream has a member rdbuf which takes no arguments and returns a pointer to the stream buffer being used by it. Example:

...
std::basic_streambuf* buffer = std::cout.rdbuf(); // take from std::cout
...

2. Create your own: You can always create a buffer class by deriving from std::basic_streambuf<char> and customize it as you want.

Now we defined and implemented MyStream class, we need the stream buffer. Let's select option 2 from above and create our own stream buffer and name this MyBuffer . We will need the following:

1. Constructor to initialize the object.
2. Continuous memory block to store output by program temporarily.
3. Continuous memory block to store input from the user (or something other) temporarily.
4. Method overflow , which is called when allocated memory for storing output is full.
5. Method underflow , which is called when all input is read by the program and more input requested.
6. Method sync , which is called when output is flushed.

As we know what things are needed to create a stream buffer class, let's declare it:

class MyBuffer : public std::basic_streambuf<char> {
private:
    char inbuf[10];
    char outbuf[10];

    int sync();
    int_type overflow(int_type ch);
    int_type underflow();
public:
    MyBuffer();
};

Here inbuf and outbuf are two arrays which will store input and output respectively. int_type is a special type which is like char and created to support multiple character types like char , wchar_t , etc.

Before we jump into the implementation of our buffer class, we need to know how the buffer will work.

To understand how buffers work, we need to know how arrays work. Arrays are nothing special but pointers to continuous memory. When we declare a char array with two elements, the operating system allocate 2 * sizeof(char) memory for our program. When we access an element from the array with array[n] , it is converted to *(array + n) , where n is index number. When you add n to an array, it jumps to next n * sizeof(<the_type_the_array_points_to>) (figure 1). If you don't know what pointer arithmetics I would recommend you to learn that before you continue. cplusplus.com has a good article on pointers for beginners.

             array    array + 1
               \        /
------------------------------------------
  |     |     | 'a' | 'b' |     |     |
------------------------------------------
    ...   105   106   107   108   ...
                 |     |
                 -------
                    |
            memory allocated by the operating system

                     figure 1: memory address of an array

As we know much about pointers now, let's see how stream buffers work. Our buffer contains two arrays inbuf and outbuf . But how the standard library would know input must be stored to inbuf and output must be stored to outbuf ? So, there two areas called get area and put area which is input and output area respectively.

Put area is specified with the following three pointers (figure 2):

• pbase() or put base: start of put area
• epptr() or end put pointer: end of put area
• pptr() or put pointer: where next character will be put

These are actually functions which return the corresponding pointer. These pointers are set by setp(pbase, epptr) . After this function call, pptr() is set to pbase() . To change it we'll use pbump(n) which repositions pptr() by n character, n can be positive or negative. Note that the stream will write to the previous memory block of epptr() but not epptr() .

  pbase()                         pptr()                       epptr()
     |                              |                             |
------------------------------------------------------------------------
  | 'H' | 'e' | 'l' | 'l' | 'o'  |     |     |     |     |     |     |
------------------------------------------------------------------------
     |                                                      |
     --------------------------------------------------------
                                 |
                   allocated memory for the buffer

           figure 2: output buffer (put area) with sample data

Get area is specified with the following three pointers (figure 3):

• eback() or end back, start of get area
• egptr() or end get pointer, end of get area
• gptr() or get pointer, the position which is going to be read

These pointers are set with setg(eback, gptr, egptr) function. Note that the stream will read the previous memory block of egptr() but not egptr().

  eback()                         gptr()                       egptr()
     |                              |                             |
------------------------------------------------------------------------
  | 'H' | 'e' | 'l' | 'l' | 'o'  | ' ' | 'C' | '+' | '+' |     |     |
------------------------------------------------------------------------
     |                                                      |
     --------------------------------------------------------
                                 |
                   allocated memory for the buffer

           figure 3: input buffer (get area) with sample data

Now that we have discussed almost all we need to know before creating a custom stream buffer, it's time to implement it! We'll try to implement our stream buffer such way that it will work like std::cout !

Let's start with the constructor:

MyBuffer() {
    setg(inbuf+4, inbuf+4, inbuf+4);
    setp(outbuf, outbuf+9);
}

Here we set all three get pointers to one position, which means there are no readable characters, forcing underflow() when input wanted. Then we set put pointer in such a way so the stream can write to whole outbuf array except the last element. We'll preserve it for future use.

Now, let's implement sync() method, which is called when output is flushed:

int sync() {
    int return_code = 0;

    for (int i = 0; i < (pptr() - pbase()); i++) {
        if (std::putchar(outbuf[i]) == EOF) {
            return_code = EOF;
            break;
        }
    }

    pbump(pbase() - pptr());
    return return_code;
}

This does it's work very easily. First, it determines how many characters there are to print, then prints one by one and repositions pptr() (put pointer). It returns EOF or -1 if character any character is EOF, 0 otherwise.

But what to do if put area is full? So, we need overflow() method. Let's implement it:

int_type overflow(int_type ch) {
    *pptr() = ch;
    pbump(1);

    return (sync() == EOF ? EOF : ch);
}

Not very special, this just put the extra character into the preserved last element of outbuf and repositions pptr() (put pointer), then calls sync() . It returns EOF if sync() returned EOF, otherwise the extra character.

Everything is now complete, except input handling. Let's implement underflow() , which is called when all characters in input buffer are read:

int_type underflow() {
    int keep = std::max(long(4), (gptr() - eback()));
    std::memmove(inbuf + 4 - keep, gptr() - keep, keep);

    int ch, position = 4;
    while ((ch = std::getchar()) != EOF && position <= 10) {
        inbuf[position++] = char(ch);
        read++;
    }
    
    if (read == 0) return EOF;
    setg(inbuf - keep + 4, inbuf + 4 , inbuf + position);
    return *gptr();
}

A little difficult to understand. Let's see what's going on here. First, it calculates how many characters it should preserve in buffer (which is at most 4) and stores it in the keep variable. Then it copies last keep number characters to the start of the buffer. This is done because characters can be put back into the buffer with unget() method of std::basic_iostream . Program can even read next characters without extracting it with peek() method of std::basic_iostream . After the last few characters are put back, it reads new characters until it reaches the end of the input buffer or gets EOF as input. Then it returns EOF if no characters are read, continues otherwise. Then it repositions all get pointers and return the first character read.

As our stream buffer is implemented now, we can setup our stream class MyStream so it uses our stream buffer. So we change the private buffer variable:

...
private:
    MyBuffer buffer;
public:
...

You can now test your own stream, it should take input from and show output from terminal.

---

Note that this stream and buffer can only handle char based input and output. Your class must derive from corresponding class to handle other types of input and output (e.g std::basic_streambuf<wchar_t> for wide characters) and implement member functions or method to so they can handle that type of character.