Should I check endianness when using System.Text.Encoding.Unicode.GetBytes(aString)?

Question

I'm writing something that convert data to byte[], transferring through internet, then convert back to what they were for a Unity game project.

I use BitConverter to convert int, float, etc., as the following example shows:

float aFloat = 312321f;
var bytes = BitConverter.GetBytes(aFloat);
if (BitConverter.IsLittleEndian) Array.Reverse(bytes);

// sending through the internet
byte[] bytes = GetByteArrayFromTheInternet();
if (BitConverter.IsLittleEndian) Array.Reverse(bytes);
float aFloat = BitConverter.ToSingle(bytes, 0);

I do the endianness check before and after sending the data to make sure they're the same. Do I need to do this for string?

string aString = "testing";
var bytes = System.Text.Encoding.Unicode.GetBytes(aString);
// if (BitConverter.IsLittleEndian) Array.Reverse(bytes); // Do I need this line?

// sending through the internet
byte[] bytes = GetByteArrayFromTheInternet();
// if (BitConverter.IsLittleEndian) Array.Reverse(bytes); // Do I need this too?
string aString = System.Text.Encoding.Unicode.GetString(bytes);

Thanks in advance!

Answer 1

I do the endianess check before and after sending the data to make sure they're the same. Do I need to do this for string?

That depends on who you're talking to on the network. What endianness are they using?

In your first example, you are assuming that the network protocol always sends float types (32-bit floating point) as big-endian. Which is fine; traditionally, "network host order" has always been big-endian, so it's a good choice for a network protocol.

But there's no requirement that a network protocol comply with that, nor that it be internally self-consistent, and you haven't provided any information about what protocol you're implementing.

Note: by "network protocol", I'm referring to the application-level protocol. This would be something like HTTP, SMTP, FTP, POP, etc. I.e. whatever your application chooses for the format of bytes on the network.

So, you'll have to consult the specification of the protocol you're using to find out what endianness the Unicode-encoded (UTF16) data uses. I would guess that it's big-endian, since your float values are too. But I can't say that for sure.

Note that if the network protocol does encode text as big-endian UTF16, then you don't need to swap the bytes for each character yourself. Just use the BigEndianUnicode encoding object to encode and decode the text. It will handle the endianness for you.

Note also that it's not really optional to use the right encoder. All that checking the BitConverter.IsLittleEndian field tells you is the current CPU architecture. But if the text on the network protocol is encoded as big-endian, then even if you are running on a big-endian CPU, you still need to use the BigEndianUnicode encoding. Just like that one will always reliably decode big-endian text, the Unicode encoding always decodes the text as if it's little-endian, even when running on a big-endian CPU.