Converting a byte array to string and then back again produced different results

Question

I'm using the .net port of libsodium. The hash generation function has two forms, one that accepts byte arrays and one that accepts strings:

public static byte[] ArgonHashBinary(string password, string salt, long opsLimit, int memLimit, long outputLength = ARGON_SALTBYTES)  

public static byte[] ArgonHashBinary(byte[] password, byte[] salt, long opsLimit, int memLimit, long outputLength = ARGON_SALTBYTES)

What i'm having an issue with is both forms producing the same hash when the input values are identical.

var saltAsBytes = PasswordHash.ArgonGenerateSalt();
var saltAsString = Encoding.UTF8.GetString(saltAsBytes);
var tmp = Encoding.UTF8.GetBytes(saltAsString);

var hash1 = PasswordHash.ArgonHashBinary(password, saltAsString, 6, 134217728, 16);
var hash2 = PasswordHash.ArgonHashBinary( Encoding.UTF8.GetBytes(password), saltAsBytes, 6, 134217728, 16);

Anything with "PasswordHash." is libsodium and not my code.

From the code above when i convert it from a string and then back to a byte array the byte array. The byte array array is always a different length. ArgonGenerateSalt() produces a byte array with a length of 16. When i convert it back from a string above its generally ~30 (different every time because of different salts produced).

Why am i converting to UTF8? Because thats what they are doing internally: https://github.com/adamcaudill/libsodium-net/blob/master/libsodium-net/PasswordHash.cs

public static byte[] ArgonHashBinary(string password, string salt, StrengthArgon limit = StrengthArgon.Interactive, long outputLength = ARGON_SALTBYTES)
    {
      return ArgonHashBinary(Encoding.UTF8.GetBytes(password), Encoding.UTF8.GetBytes(salt), limit, outputLength);
    }

When i convert the salt to a UTF8 string the hashing function will fail because they are checking the length of the byte array to make sure its 16 bytes. If i convert it to a ASCII string it works but produces a different hash (which is expected).

To clarify the hashing piece in this code is not the issue. Figuring out why tmp is different then saltAsBytes is the key.

Answer 1

I think the problem here is that the ArgonGenerateSalt method doesn't return a UTF8 encoded string, it returns completely random bytes.

You can't decode random bytes as a UTF8 string and expect it to round trip. A trivial example to see where this blows up is to do the following:

var data = new byte[] { 128 };
var dataAsString = Encoding.UTF8.GetString( data );
var dataAsBytes = Encoding.UTF8.GetBytes( dataAsString );

After this, dataAsBytes will be 3 bytes (specifically 239, 191, 189).

Answer 2

Converting a byte array to string and then back again produced different results

A binary data may not be converted to string and then back to byte array using Encoding.[AnyEncoding].GetBytes and Encoding.[AnyEncoding].GetString

Instead use Convert.ToBase64String and Convert.FromBase64String

You can easily test...

var bytes = new byte[] { 255, 255, 255 }; 
var buf = Encoding.UTF8.GetString(bytes);
var newbytes = Encoding.UTF8.GetBytes(buf);

newbytes's length will be 9.....

Edit: This is the test case for @Theo

var bytes = new byte[] { 0, 216 }; //any new byte[] { X, 216 };
var buf = Encoding.Unicode.GetString(bytes);
var newbytes = Encoding.Unicode.GetBytes(buf); //253,255