PHP Pack/Unpack implementation in Javascript Mismatch

Question

As per this question's related answer, I'm attempting to put together a pack/unpack solution resembling this PHP process, however in Nodejs (Javascript) using md5 and bufferpack

Here's the PHP approach (adapted from DaloRADIUS:

  $challenge = 'c731395aca5dcf45446c0ae83db5319e';
  $uamsecret = 'secret';
  $password = 'password';

  $hexchal = pack ("H32", $challenge);
  $newchal = pack ("H*", md5($hexchal . $uamsecret));
  $response = md5("\0" . $password . $newchal);
  $newpwd = pack("a32", $password);
  $pappassword = implode ("", unpack("H32", ($newpwd ^ $newchal)));

  echo "Response: ---> ", $response, "\n";
  echo "New Password: ---> ", $newpwd, "\n";
  echo "Pap Password: ---> ", $pappassword, "\n";

The above echos these:

Above in plaintext:

Response: ---> 2d4bd27184f5eb032641137f728c6043
New Password: ---> password
Pap Password: ---> 356a1fb08f909fc400dfe448fc483ce3

In Javascript, here's what I'm doing now:

  var challenge = 'c731395aca5dcf45446c0ae83db5319e';
  var uamsecret = 'secret';
  var password = 'password';

  var hexchal = pack.pack("H32", challenge);
  var newchal = pack.pack("H*", md5(hexchal + uamsecret));
  var response = md5("\0" + password + newchal);
  var newpwd = pack.pack("a32", password);
  var pappassword = pack.unpack("H32", (newpwd ^ newchal)).join("");

  console.log("Response: --> ", response);
  console.log("New Password: -->", newpwd);
  console.log("Pap Password: --->", pappassword);

Which gives the result:

In JSON:

In plaintext:

Response: -->  e8a54a55cbcd81dbc2bdfd9b197d62af
New Password: --> <Buffer >
Pap Password: ---> NaN

All the above snippets are available here: RadiusNES

My understanding in this whole process isn't the best, and will appreciate insights and where I'm going wrong.

Why is there a mismatch?

Answer 1

The translation does not work because the PHP Pack function uses different format strings and returns strings, whilst the Javascript bufferpack module returns arrays. Also you cannot xor strings in Javascript.

Whilst there may be modules to do what you want, I have my own functions for parsing hex strings. Also I like modifying prototypes which not everyone agrees with, but these could be converted to regular functions.

String.prototype.pad = function( length ,padding ) {

    var padding = typeof padding === 'string' && padding.length > 0 ? padding[0] : '\x00'
        ,length = isNaN( length ) ? 0 : ~~length;

    return this.length < length ? this + Array( length - this.length + 1 ).join( padding ) : this;

}

String.prototype.packHex = function() {

    var source = this.length % 2 ? this + '0' : this
        ,result = '';

    for( var i = 0; i < source.length; i = i + 2 ) {
        result += String.fromCharCode( parseInt( source.substr( i , 2 ) ,16 ) );
    }

    return result;

}

var challenge = 'c731395aca5dcf45446c0ae83db5319e'
    ,uamsecret = 'secret'
    ,password = 'password';

var hexchal = challenge.packHex();
var newchal = md5( hexchal + uamsecret ).packHex();
var response = md5( '\0' + password + newchal );
var newpwd = password.pad( 32 );
var pappassword = '';
for( var i = 0; i < newchal.length; i++ ) {
    pappassword += ( newpwd.charCodeAt( i ) ^ newchal.charCodeAt( i ) ).toString( 16 );
}

console.log("Response: --> ", response);
console.log("New Password: -->", newpwd);
console.log("Pap Password: --->", pappassword);

Two functions are defined in the String prototype to replace the use of the pack function:

.pad( 32, string ) is used to pad out a string with nulls to give the same results as pack( 'a32', string ). Although not needed here it also takes a second parameter if wanting to pad the string ith a character other than nulls.

.packHex is the equivalent of pack( 'H*' ,string ) and translating the code of each pair of hex characters into a character. The function ideally needs more validation to test the string is a valid hex one if is to be used.

After the inputs have been defined, the next four lines instead set variables using these functions rather than pack.

Because Javascript cannot natively xor strings, you then need to use a loop to extract each character, convert it to a numeric, xor those values, then convert the result back into a character to create the pappassword string.

That will return, for me:

Response: -->  – "fbfd42ffde05fcf8dbdd02b7e8ae2d90"
New Password: --> – "password������������������������"
Pap Password: ---> – "dcbdacb03f5d38ca33c128b931c272a"

Which is a result, but unfortunately a different on from the PHP code.

This is because my installation of PHP is configured to use ISO-8859-1 encoding internally, whilst Javascript natively uses UTF-16.

This is not a problem in normal use, but it means the respective md5 functions will be seeing different values and therefore return a different hash.

Assuming you are writing an authentication routine using a PHP backend you will obviously need consistent results. There may be modules available to convert the encoding of the Javscript values for compatibility, but it is much easier to make changes to the PHP code.

Because we know the hex strings will be one byte, Javascript is effectively using UTF-8, so PHP can do the same by using the utf8_encode() function to convert the packed hex strings before md5ing them.

Originally I thought that Javascript was internally converting the encoded hex characters into their unicode equivalents because of this, but this was not the case. Instead it was the md5 module being used in Javascript that was performing a UTF-8 conversion on the input.

This leaves two possible options.

1. Use UTF-8 PHP

If possible you can reconfigure your PHP server to use UTF-8 encoding. Or you can change your script to use the utf8_encode() function to mirror the same process as is happening in the Javascript, and convert the hex packed strings to UTF-8 before passing them to md5()

$challenge = 'c731395aca5dcf45446c0ae83db5319e';
$uamsecret = 'secret';
$password = 'password';

$hexchal = pack ("H32", $challenge);
$newchal = pack ("H*", md5(utf8_encode($hexchal) . $uamsecret));
$response = md5("\0" . $password . utf8_encode($newchal));
$newpwd = pack("a32", $password);
$pappassword = implode ("", unpack("H32", ($newpwd ^ $newchal)));

echo "Response: ---> ", $response, "\n";
echo "New Password: ---> ", $newpwd, "\n";
echo "Pap Password: ---> ", $pappassword, "\n";

This then returns the same results as the Javscript:

Response: ---> fbfd42ffde05fcf8dbdd02b7e8ae2d90
New Password: ---> password
Pap Password: ---> dcbdacb03f5d38ca33c128b9310c272a

2. Change the md5 module in Javascript

I am assuming you are using the bluimp JavaScript-MD5 module, as this is what it used by DaloRADIUS routine you linked. You can patch this to bypass the UTF-8 conversion.

There are various ways you can patch this, but on line 259 is the definition of the md5() function itself. This is simply a set of if statements to parse the input options and call the appropriate internal function.

However, the functions called by this block simply provide the UTF-8 input conversion, through a function called str2rstrUTF8() before then call the appropriate functions to provide the actual hashing. You may therefore want to patch the md5() to accept a third parameter to indicate whether the UTF-8 conversion should be applied and then call other functions as appropriate.

However to simply remove the conversion completely the easier way is to change str2rstrUTF8() to return the input unchanged. This function can be found on line 239, changing it to just read as follows will stop the conversion:

function str2rstrUTF8 (input) {
  return input
}

Alternatively to remove the redundant function call you can instead just remove the references to it. Change the function starting on line 246 to read as follows:

function rawMD5 (s) {
  return rstrMD5(s)
}

The rawHMACMD5() function on line 252 also includes calls to the str2rstrUTF8() function which you may also want to patch for consistency but this is not required for the above routine. That function is called instead when a second parameter is passed to provide a key hash, a feature not available in the native PHP md5() function.

After making either of those changes the Javascript routine now returns the same output as your original (ISO-8859-1 using) PHP code:

Response: -->  – "2d4bd27184f5eb032641137f728c6043"
New Password: --> – "password������������������������"
Pap Password: ---> – "356a1fb08f909fc40dfe448fc483ce3"