Is there a simple way to append a byte to a StringBuffer and specify the encoding?

Question

Question

What is the simplest way to append a byte to a StringBuffer (i.e. cast a byte to a char) and specify the character encoding used (ASCII, UTF-8, etc)?

Context

I want to append a byte to a stringbuffer. Doing so requires casting the byte to a char:

myStringBuffer.append((char)nextByte);

However, the code above uses the default character encoding for my machine (which is MacRoman). Meanwhile, other components in the system/network require UTF-8. So I need to so something like:

try {
    myStringBuffer.append(new String(new Byte[]{nextByte}, "UTF-8"));
} catch (UnsupportedEncodingException e) {
    //handle error
}

Which, frankly, is pretty ugly.

Surely, there's a better way (other than breaking the same code into multiple lines)???????

Answer 1

I think the error here is in dealing with bytes at all. You want to deal with strings of characters instead.

Just interpose a reader on the input and output stream to do the mapping between bytes and characters for you. Use the InputStreamReader(InputStream in, CharsetDecoder dec) form of the constructor for the input, though, so that you can detect input encoding errors via an exception. Now you have strings of characters instead of buffers of bytes. Put an OutputStreamWriter on the other end.

Now you no longer have to worry about bytes or encodings. It’s much simpler this way.

Answer 2

The simple answer is 'no'. What if the byte is the first byte of a multi-byte sequence? Nothing would maintain the state.

If you have all the bytes of a logical character in hand, you can do:

sb.append(new String(bytes, charset));

But if you have one byte of UTF-8, you can't do this at all with stock classes.

It would not be terribly difficult to build a juiced-up StringBuffer that uses java.nio.charset classes to implement byte appending, but it would not be one or two lines of code.

Comments indicate that there's some basic Unicode knowledge needed here.

In UTF-8, 'a' is one byte, 'á' is two bytes, '丧' is three bytes, and '𝌎' is four bytes. The job of CharsetDecoder is to convert these sequences into Unicode characters. Viewed as a sequential operation over bytes, this is obviously a stateful process.

If you create a CharsetDecoder for UTF-8, you can feed it only byte at a time (in a ByteBuffer) via this method. The UTF-16 characters will accumulate in the output CharBuffer.