Utilizing PBKDF2 with OpenSSL library

Question

I want to utilize the PBKDF2 algorithm with SHA1 HMAC (based on this answer).

How can I utilize this through the crypto library?

I started by looking at man openssl, but the openssl passwd command (man page) only supports a small handful of algorithms. Looking at the crypto documentation, the evp module has an EVP_BytesToKey method.

Careful selection of the parameters will provide a PKCS#5 PBKDF1 compatible implementation. However, new applications should not typically use this (preferring, for example, PBKDF2 from PCKS#5).

Which brings me back to my original question, how do I utilize PBKDF2 via crypto? Do I need to dig into the code and call a non-API-exposed method (such as PKCS5_PBKDF2_HMAC)? (and if so, what's keeping it from being exposed?)

Answer 1

I do have a working but poor C example of PBKDF2 via the OpenSSL libraries at my github repository, including scripts to compile under both Linux and Windows (via MinGW). Source code located under "Releases" is known good; source code in the master branch is a WIP. This variant is licensed under the same 4-clause BSD in addition to SSLeay license OpenSSL is.

I'm still working on adding a few features, then I'll go back to the excellent input I got on the Code Review StackExchange site and upgrade to C99 syntax and so on.

The core code is very primitive, and may contain flaws despite passing very extensive string-based test vectors. It has not (yet) been tested against pure binary input.

#include <openssl/evp.h>
#include <openssl/sha.h>
// crypto.h used for the version
#include <openssl/crypto.h>

void PBKDF2_HMAC_SHA_1nat_string(const char* pass, const unsigned char* salt, int32_t iterations, uint32_t outputBytes, char* hexResult)
{
    unsigned int i;
    unsigned char digest[outputBytes];
    PKCS5_PBKDF2_HMAC_SHA1(pass, strlen(pass), salt, strlen(salt), iterations, outputBytes, digest);
    for (i = 0; i < sizeof(digest); i++)
        sprintf(hexResult + (i * 2), "%02x", 255 & digest[i]);
}

If you have a 64-bit system, I would highly recommend moving up to PBKDF2-HMAC-SHA-512 or PBKDF2-HMAC-SHA-384 instead:

#include <openssl/evp.h>
#include <openssl/sha.h>
// crypto.h used for the version
#include <openssl/crypto.h>


void PBKDF2_HMAC_SHA_384_string(const char* pass, const unsigned char* salt, int32_t iterations, uint32_t outputBytes, char* hexResult)
{
    unsigned int i;
    unsigned char digest[outputBytes];
    PKCS5_PBKDF2_HMAC(pass, strlen(pass), salt, strlen(salt), iterations, EVP_sha384(), outputBytes, digest);
    for (i = 0; i < sizeof(digest); i++)
        sprintf(hexResult + (i * 2), "%02x", 255 & digest[i]);
}

void PBKDF2_HMAC_SHA_512_string(const char* pass, const unsigned char* salt, int32_t iterations, uint32_t outputBytes, char* hexResult)
 {
     unsigned int i;
     unsigned char digest[outputBytes];
     PKCS5_PBKDF2_HMAC(pass, strlen(pass), salt, strlen(salt), iterations, EVP_sha512(), outputBytes, digest);
     for (i = 0; i < sizeof(digest); i++)
         sprintf(hexResult + (i * 2), "%02x", 255 & digest[i]);
 }    

An example of use would be:

// 2*outputBytes+1 is 2 hex bytes per binary byte, 
// and one character at the end for the string-terminating \0
char hexResult[2*outputBytes+1];
memset(hexResult,0,sizeof(hexResult));
PBKDF2_HMAC_SHA_1nat_string(pass, salt, iterations, outputBytes, hexResult);
printf("%s\n", hexResult);

or

// 2*outputBytes+1 is 2 hex bytes per binary byte, 
// and one character at the end for the string-terminating \0
char hexResult[2*outputBytes+1];
memset(hexResult,0,sizeof(hexResult));
PBKDF2_HMAC_SHA_512_string(pass, salt, iterations, outputBytes, hexResult);
printf("%s\n", hexResult);

Use a random, per-user salt of 8 to 16 binary bytes, i.e. 16 to 32 hex digits - my code does NOT have examples of generating this yet

Regardless of what you choose, be sure to verify it against test vectors (a few are in pbkdf2_test.bat/sh in my repository).

Additionally, on your system, do some benchmarking - certainly on the PBKDF2-HMAC-SHA-384 and PBKDF2-HMAC-SHA-512 variants, compiling under a 64-bit system produces dramatically better results. Compare it against my equally poor C++ Crypto++ and/or my poor C PolarSSL examples, or Jither's C# implementation example, depending on what your target system is.

The reason you care about speed is that you have to choose an iteration count based on the performance your production system has available compared to the number of users logging in/creating passwords at peak times, so as to not generate too many complaints of slowness.

Attackers are going to use something like oclHashcat, which on a single PC with 8x AMD R9 290Xstock core clock is able to attempt 3.4E12 (2^41) guesses every 30 days against PBKDF2-HMAC-SHA-1(SSID as salt, password, 32 bytes output length, 4096 iterations, a.k.a. WPA/WPA2), which is more or less equivalent to PBKDF2-HMAC-SHA-1(salt,pw,20 bytes output length, 8192 iterations).

• if you use 65536 iterations, the attacker will be only able to try 8.5E11 (~2^39) guesses every 30 days.
• if you use 1024 iterations, the attacker will instead be able to try 2.7E13 (~2^44) guesses every 30 days.

The difference becomes important when the attacker starts choosing their attacks.

• In both cases the attacker's going to brute force very small keys; there's no reason not to spend a few minutes or even a few hours on the low hanging fruit.
  • This is all hex characters for length 1-n, and then all printable characters from length n+1 to n+m, and then going on down the line until they're at n+y with a hardcoded ! at the end :).
• In both cases the attacker's going to use tiny dictionaries and tiny rulesets; say the phpbb dictionary of 184389 words and the Best64 ruleset
  • If you had 1000 users with 1000 different random salts and used PBKDF2-HMAC-SHA-1 with 65536 iterations, that's going to take our single PC, 8 GPU attacker (184389*64*1000)/(8.5E11/30) days = 0.41 days. Well worth it!
• Now, what about that same phpbb dictionary and the medium sized T0X1C ruleset of 4089 rules?
  • If you had 1000 users with 1000 different random salts and used PBKDF2-HMAC-SHA-1 with 65536 iterations, that's going to take our single PC, 8 GPU attacker (184389*4089*1000)/(8.5E11/30) days = 26.61 days.
    • Still worth it, but that's almost 4 weeks for this machine to spend on one attack!
  • If you had 1000 users with 1000 different random salts and used PBKDF2-HMAC-SHA-1 with 65536 iterations, that's going to take our single PC, 8 GPU attacker (184389*4089*1000)/(2.7E13/30) days = 0.83 days.
    • Totally worth it!
• Now, what about that same phpbb dictionary and the excellent d3ead0ne ruleset of 35404 rules?
  • If you had 1000 users with 1000 different random salts and used PBKDF2-HMAC-SHA-1 with 65536 iterations, that's going to take our single PC, 8 GPU attacker (184389*35404*1000)/(8.5E11/30) days = 230.39 days.
    • Right, time to buy more machines or work on this in off times.
  • If you had 1000 users with 1000 different random salts and used PBKDF2-HMAC-SHA-1 with 65536 iterations, that's going to take our single PC, 8 GPU attacker (184389*4089*1000)/(2.7E13/30) days = 7.18 days.
    • Worth it! It's only a week and a meal.

Now, for PBKDF2, there are a few other things to know:

• For password hashing, never select a binary output size larger than the native hash size. Personally, I wouldn't recommend a binary output size under 20 bytes regardless, so that's a bit limiting for SHA-1.
  • SHA-1 is size = 20 bytes
  • SHA-224 is 20 <= size <= 28 bytes
  • SHA-256 is 20 <= size <= 32 bytes
  • SHA-384 is 20 <= size <= 48 bytes
  • SHA-512 is 20 <= size <= 64 bytes
  • adjust for how you store the hash if it's not in pure binary, of course.
  • Reason: PBKDF2 first run the # of iterations requested for one native output size (the number on the right, above). If you wanted more than that, it runs the entire iteration count all over again. If you wanted less than that, it truncates.
  • The attacker's only going to match the first native size - if the first bytes match, yes, that's the password, so you're better off increasing your iteration count.