What is the length of a string encoded in a ByteBuffer

Question

byte[] byteArray = Charset.forName("UTF-8").encode("hello world").array();
System.out.println(byteArray.length);

Why does the above line of code prints out 12, shouldn't it be printing 11 instead?

Answer 1

The length of the array is the size of the ByteBuffer's capacity, which is generated from, but not equal to the number of characters you are encoding. Let's take a look at how we allocate memory for a ByteBuffer...

If you drill into the encode() method, you'll find that CharsetEncoder#encode(CharBuffer) looks like this:

public final ByteBuffer encode(CharBuffer in)
    throws CharacterCodingException
{
    int n = (int)(in.remaining() * averageBytesPerChar());
    ByteBuffer out = ByteBuffer.allocate(n);
    ...

According to my debugger, the averageBytesPerChar of a UTF_8$Encoder is 1.1, and the input String has 11 characters. 11 * 1.1 = 12.1, and the code casts the total to an int when it does the calculation, so the resulting size of the ByteBuffer is 12.

Answer 2

Because it returns a ByteBuffer. That's the buffer's capacity (not really even that because of possible slicing), not how many bytes are used. It's a bit like how malloc(10) is free to return 32 bytes of memory.

System.out.println(Charset.forName("UTF-8").encode("hello world").limit());

That's 11 (as expected).