Crypto++ pbkdf2 output is different than Rfc2898DeriveBytes (C#) and crypto.pbkdf2 (JavaScript)

Question

So I'm trying to use PBKDF2 to derive a key given a base64 string of 256bits. I am able to use C#'s Rfc2898DeriveBytes and node-crypto's pbkdf2 to derive the same key, however, I can't say the same for C++. I'm not sure if I'm doing wrong conversions or using the functions improperly, but I'll let you guys look at it.

C++

/* 256bit key */
string key = "Y1Mjycd0+O+AendY5pB58JMlmS0EmBWgjdj2r2KW6qQ=";
string decodedKey;
StringSource(key, true, new Base64Decoder(new StringSink(decodedKey)));
const byte* keyByte = (const byte*) decodedKey.data();

/* Generate IV */
/*
    AutoSeededRandomPool prng;
    byte iv[AES::BLOCKSIZE];
    prng.GenerateBlock(iv, sizeof(iv));
*/

/* FOR TESTING PURPOSES, HARDCODE IV */
string iv = "5iFv54dCRq5icQbD7QHQzg==";
string decodedIv;
StringSource(iv, true, new Base64Decoder(new StringSink(decodedIv)));
const byte* ivByte = (const byte *) decodedIv.data();

byte derivedKey[32];
PKCS5_PBKDF2_HMAC<CryptoPP::SHA1> pbkdf2;
pbkdf2.DeriveKey(derivedKey, 32, 0, keyByte, 32, ivByte, 16, 100);

/* 
 * derivedKey: 9tRyXCoQLTbUOLqm3M4OPGT6N25g+o0K090fVp/hflk=
 */

C#

// string key = "Y1Mjycd0+O+AendY5pB58JMlmS0EmBWgjdj2r2KW6qQ="; // need to convert it to byte data
string key = Convert.FromBase64String("Y1Mjycd0+O+AendY5pB58JMlmS0EmBWgjdj2r2KW6qQ="); // change above to this
RijndaelManaged symKey = new RijndaelManaged();
symKey.GenerateIV(); /* Assume hardcoded IV same as above */
Rfc2898DeriveBytes derivedKey = new Rfc2898DeriveBytes (key, symKey.IV, 100);

/*
 * derivedKey: dZqBpZKyUPKn8pU4pyyeAw7Rg8uYd6yyj3WI1MIJSyc=
 */

JS

// var key = "Y1Mjycd0+O+AendY5pB58JMlmS0EmBWgjdj2r2KW6qQ="; // need to convert it to byte data
var key = new Buffer("Y1Mjycd0+O+AendY5pB58JMlmS0EmBWgjdj2r2KW6qQ=", "base64"); // changed above to this
var iv = crypto.randomBytes(16);
iv = "5iFv54dCRq5icQbD7QHQzg=="; /* HARDCODE IV */
crypto.pbkdf2(key, iv, 100, 32, function(err, derivedKey) { }

/*
 * derivedKey: dZqBpZKyUPKn8pU4pyyeAw7Rg8uYd6yyj3WI1MIJSyc=
 */

Well the main questions is, what am I doing wrong on C++'s CryptoPP library that it is not deriving the same value.

SOLUTION: I was being dumb... I realized after review my original implementation on JavaScript and C# I missed a crucial step that for some reason I did not get a complain from the compiler. Basically the problem was that I did not convert the key used into byte data before the algorithm on my C# and JS implementation...

Anyways, proposed solution is: do not code at 4 AM and make sure to be consistent on your data conversion...

I guess the TL;DR of this is that C# and JS was converting my 256bit key to byte data as ASCII instead of base64 conversion.

Answer 1

Well the main questions is, what am I doing wrong on C++'s CryptoPP library that it is not deriving the same value.

Well, I don't think you are doing anything wrong in C++ with Crypto++ and PBKDF2. I think the other libraries are setting up the parameters differently, or they are a tad-bit non-standard.

I was able to arrive at the IETF's test vectors for PBKDF2 using Crypto++:

// From https://www.ietf.org/rfc/rfc6070.txt
//   PKCS #5: Password-Based Key Derivation Function 2 (PBKDF2) Test Vectors
//
//      Input:
//       P = "password" (8 octets)
//       S = "salt" (4 octets)
//       c = 1
//       dkLen = 20
//
//     Output:
//       DK = 0c 60 c8 0f 96 1f 0e 71
//            f3 a9 b5 24 af 60 12 06
//            2f e0 37 a6    (20 octets)

int main(int argc, char* argv[])
{
    byte password[] ="password";
    size_t plen = strlen((const char*)password);

    byte salt[] = "salt";
    size_t slen = strlen((const char*)salt);

    int c = 1;
    byte derived[20];

    PKCS5_PBKDF2_HMAC<CryptoPP::SHA1> pbkdf2;
    pbkdf2.DeriveKey(derived, sizeof(derived), 0, password, plen, salt, slen, c);

    string result;
    HexEncoder encoder(new StringSink(result));

    encoder.Put(derived, sizeof(derived));
    encoder.MessageEnd();

    cout << "Derived: " << result << endl;

    return 0;
}

And a run of the program:

$ ./cryptopp-test.exe
Derived: 0C60C80F961F0E71F3A9B524AF6012062FE037A6

---

I think the first thing you should do is verify the C# and Javascript implementations are using the same character encoding as Crypto++ and the IETF.

If that's not it, then check to see if C# and Javascript use the purpose byte. Crypto++ does not, and you can see the implementation at pwdbased.h.

---

Unfortunately, I get something a little different when I dial in your parameters:

int main(int argc, char* argv[])
{
    string t1 = "Y1Mjycd0+O+AendY5pB58JMlmS0EmBWgjdj2r2KW6qQ=";
    string t2 = "5iFv54dCRq5icQbD7QHQzg==";

    string pw, iv;

    Base64Decoder b1(new StringSink(pw));
    b1.Put((const byte*)t1.data(), t1.size());
    b1.MessageEnd();

    Base64Decoder b2(new StringSink(iv));
    b2.Put((const byte*)t2.data(), t2.size());
    b2.MessageEnd();

    int c = 100;
    byte derived[32];

    cout << "pw size: " << pw.size() << endl;
    cout << "iv size: " << iv.size() << endl;

    PKCS5_PBKDF2_HMAC<CryptoPP::SHA1> pbkdf2;
    pbkdf2.DeriveKey(derived, sizeof(derived), 0, (byte*)pw.data(), pw.size(), (byte*)iv.data(), iv.size(), c);

    string result;
    HexEncoder encoder(new StringSink(result));

    encoder.Put(derived, sizeof(derived));
    encoder.MessageEnd();

    cout << "Derived: " << result << endl;

    return 0;
}

A run results in:

$ ./cryptopp-test.exe
pw size: 32
iv size: 16
Derived: F6D4725C2A102D36D438BAA6DCCE0E3C64FA376E60FA8D0AD3DD1F569FE17E59