Protocol Buffers: get byte array from ByteString without copying

Question

Suppose I have a method void foo(byte[] bytes) requires a byte array as its parameter. Howerver, the Java type for byte arrays in Protobuf is ByteString.

I can use byte[] toByteArray() to get the byte array. But the problem is this method builds a new array using copy, which is relatively expensive. I would rather it return the underlying array directly, or return a view instead.

Is there any API for that, or the performance penalty is acceptable?

Answer 1

See com.google.protobuf.UnsafeByteOperations.unsafeWrap(byte[], int, int)

Answer 2

You cannot get a byte[] from a ByteString without copying because this would allow you to modify the contents of the ByteString, which means that ByteString could no longer guarantee that it is immutable. This is much like why you cannot get a char[] from a String without copying, even though String is actually backed by char[] under the hood. This can be a security issue: when passing String or ByteString between security domains, it's important that the receiver can know that the sender is not modifying the string out from under them while they use it.

You can, however, call asReadOnlyByteBufferList() to get ByteBuffers representing the underlying data without a copy, since ByteBuffer enforces immutability.

Answer 3

In general that is impossible because there could be no such array inside some subclasses of ByteString. BoundedByteString can contain a bigger array, so copying is required to get an array of the right size. RopeByteString consists of other byte strings so copying is required to put it contents into one array. LiteralByteString stores it contents in an array of the right size, but it does not provide methods to access it directly.

But most probably toByteArray() will be fast enough for your needs as System.arraycopy() is very fast.

If you really have a performance problem because of long arrays copying and you don't want to pass ByteString directly, have a look at asReadOnlyByteBuffer() and asReadOnlyByteBufferList() methods. They wrap ByteString contents into ByteBuffer without copying them.