write to boost::asio socket from different threads

Question

In our application we use Boost libraries (and ASIO for network communications).

Recently, we discovered that if we're sending our data from different threads via same socket, our client application is receiving garbaged data.

Small test to highlight the issue:

#include <stdio.h>
#include <boost/thread.hpp>
#include <boost/asio.hpp>

void send_routine(boost::shared_ptr<boost::asio::ip::tcp::socket> s, char c)
{
  std::vector<char> data(15000, c);
  data.push_back('\n');

  for (int i=0; i<1000; i++)
    boost::asio::write(*s, boost::asio::buffer(&data[0], data.size()));
}


int main()
{
  using namespace boost::asio;
  using namespace boost::asio::ip;

  try {
    io_service io_service;
    io_service::work work(io_service);

    const char* host = "localhost";
    const char* service_name = "18000";

    tcp::resolver resolver(io_service);
    tcp::resolver::query query(tcp::v4(), host, service_name);
    tcp::resolver::iterator iterator = resolver.resolve(query);

    auto socket = boost::shared_ptr<tcp::socket>(new tcp::socket(io_service));
    socket->connect(*iterator);

    boost::thread t1(send_routine, socket, 'A');
    boost::thread t2(send_routine, socket, 'B');
    boost::thread t3(send_routine, socket, 'C');

    t1.join();
    t2.join();
    t3.join();
  }
  catch (std::exception& e) {
    printf("FAIL: %s\n", e.what());
  }
    return 0;
}

So, we create socket here, connect to localhost:18000 and start 3 threads which will write to the socket.

In different terminal window, I run nc -l -p 18000 | tee out.txt | sort | uniq | wc -l. I expect 3 as output, but it returns more then 100 "different strings" in the network stream (so, data is corrupted). But it works with small buffer sizes (if we'll change 15000 to 80, for example).

So, the question is: is it a correct behavior of ASIO library? And another: how to fix it? Should I use mutex inside my send_routine function (or there is another solution)?

Answer 1

write and async_write are not thread safe in the manner you are using them. The canonical way to approach this is to queue your messages, then write them out one at a time.

Answer 2

Yes there is another solution ! Strands: Use Threads Without Explicit Locking. Be care that strands only provides "atomic" access to socket for the "event handlers", of course you need to use asio "event handlers" which is not the case of your code. In other words you need to use boost::asio::async_write instead of boost::asio::write.

Answer 3

Well according to the documentation tcp::socket is not thread safe when shared between multiple threads.
So you either do a synchronisation like you suggested with boost::mutex or you use async write. The io_service the work for you.