In Ruby, How to read UTF-8 from a socket?

Question

When a server sends UTF-8 bytes, how do you read them without characters becoming pure bytes? (\x40 etc)

Answer 1

I believe read_nonblock uses read, which in turn says:

The resulted string is always ASCII-8BIT encoding.

Which means you don't need to specify IO#set_encoding, but that you can, after you read whole string, force its encoding (using String#force_encoding!) to UTF-8.

I emphasized 'whole', as you need to make sure that you read entire Unicode character at the end of the string, as if only part of it is read, you will get invalid UTF-8 character and Ruby might complain about it further down the line.

Answer 2

You can use IO#set_encoding to set a socket's external encoding to UTF-8.

#!/usr/bin/env ruby
# -*- coding: utf-8 -*-

require 'socket'

server_socket = TCPServer.new('localhost', 0)
Thread.new do
  loop do
    session_socket = server_socket.accept
    session_socket.set_encoding 'ASCII-8BIT'  
    session_socket.puts "  ᚁ ᚂ ᚃ ᚄ ᚅ ᚆ ᚇ ᚈ ᚉ ᚊ ᚋ ᚌ ᚍ"
    session_socket.close
  end
end

client_socket = TCPSocket.new('localhost', server_socket.addr[1])
client_socket.set_encoding 'UTF-8'
p client_socket.gets
# => "|  ᚁ ᚂ ᚃ ᚄ ᚅ ᚆ ᚇ ᚈ ᚉ ᚊ ᚋ ᚌ ᚍ\n"