Can you set a byte-limit for Asio's read_until?

Question

I am using boost::asio::read_until to read from a socket until "<EOF>" is received. However, someone could send billions and billions of bytes until the system runs out of RAM and has to shut down.

To avoid that, I want to set a limit to read_until. Like "read until "<EOF>" or until 10MB are reached".

Is there an easy solution to that using read_until, or do I have to switch to read and manually end the reading when "<EOF>" is received?

Answer 1

@sehe has an excellent answer that allows to stop on EOF or when a specific number of bytes have been read. My versions are slightly and much more complicated but additionally allow to stop on any delimiter.

---

You could construct your boost::asio::streambuf with a max size argument:

The constructor for basic_streambuf accepts a size_t argument specifying the maximum of the sum of the sizes of the input sequence and output sequence. During the lifetime of the basic_streambuf object, the following invariant holds:

size() <= max_size()

Any member function that would, if successful, cause the invariant to be violated shall throw an exception of class std::length_error.

Or you could use this overload:

template<
    typename SyncReadStream,
    typename Allocator,
    typename MatchCondition>
std::size_t read_until(
    SyncReadStream & s,
    boost::asio::basic_streambuf< Allocator > & b,
    MatchCondition match_condition,
    boost::system::error_code & ec,
    typename enable_if< is_match_condition< MatchCondition >::value >::type *  = 0);

where the match condition function looks somewhat like this:

using iterator = buffers_iterator<basic_streambuf<Allocator>::const_buffers_type>;
/**
\brief Make read_until stop when either: 
    * the stop character was found
    * more than 100MB have been read
*/
pair<iterator, bool> match_condition(iterator begin, iterator end) {
   // to much data?
   if(end - begin > 100*1024*1024) {
      return std::make_pair(begin, true);
   }
   // try and find stop character
   for(auto i = begin; i < end; i++) {
      auto c = i.rdbuf()->sgetc();
      if(c == STOP_CHARACTER) {
         return std::make_pair(i, true);
      } 
   }

   return std::make_pair(begin, false);
}

(Making this work with a multi-character delimiter is left as an exercise for the reader)

Answer 2

Just use transfer_exactly which will also stop at EOF or buffer full:

auto transferred = read(s, sb, transfer_exactly(10u<<20), ec);

Live On Coliru

#include <boost/asio.hpp>
#include <iostream>
using namespace boost::asio;
using namespace ip;

int main() {
    boost::system::error_code ec;

    io_service svc;
    tcp::socket s(svc);
    s.connect(tcp::endpoint(address_v4::loopback(), 6767));

    streambuf sb;
    auto transferred = read(s, sb, transfer_exactly(10u<<20), ec);

    std::cerr << "read " << transferred << " till " << ec.message() << "\n";
}