custom std::fstream, std::filebuf's overflow and underflow function not invoked for every character

Question

I'm trying to make a custom std::fstream, which would encode/decode the data while reading.

template <class T>
class _filebuf : public std::filebuf {
public:
    using transform_type = T;

    int_type underflow() override {
        auto c = std::filebuf::underflow();
        return c < 0 ? c : transform.decode(c);
    }

    int_type overflow(int_type c) override {
        return c < 0 ? c : std::filebuf::overflow(transform.encode(c));
    }

private:
    transform_type transform;
};

template <class T>
class _fstream : public std::iostream {
public:
    using buffer_type = _filebuf<T>;
    
    explicit _fstream(const std::string& path, std::ios::openmode openmode)
        : std::iostream(0)
    {
        this->init(&buffer);
        buffer.open(path, openmode);
    }

private:
    buffer_type buffer;
};

and here is a usage example:

class _transform {
public:
    template <class T>
    T encode(T value) const {
        return value - 1;
    }
   
    template <class T>
    T decode(T value) const {
        return value + 1;
    }    
};

int main() {
    _fstream<_transform> ofs("test.txt", std::ios::out | std::ios::trunc);
    ofs << "ABC"; // outputs "@BC" to the file (@ is 64 in ASCII, so only first character encoded properly)

    _fstream<_transform> ifs("test.txt", std::ios::in);
    std::string s;
    ifs >> s; // inputs  "ABC" when "@BC" was in the file so again only first character is decoded

    // ...
};

After my own research, I found out that the 'overflow' function is called twice in the process (with 65 and -1, where -1 is probably EOF), and 'underflow' also twice (with 64 and -1). As other characters are not lost, they probably are somehow processed without calling those functions.

Why is that happening and how to change it?

Answer 1

std::streambuf<CharT,Traits>::underflow makes sure that at least one character is available in the get area, an efficient implementation of std::filebuf will always attempt to read a full buffers worth of characters into the get area. Unless you seek the stream underflow won't be called again until the get area is emptied by calls to sgetn/xsgetn or sbumpc

I think you'd probably have more success wrapping rather than extending the file buffer.

Boost iostreams makes writing stream filters much simpler.