How do you protect API keys and 3rd party site credentials (LAMP)?

Question

I'm creating a site which will make use of ID's, Passwords, and API Keys to other 3rd party sites - for the server to access information accordingly. For the purpose of this conversation, let's assume it is for a payment gateway - meaning exposure of this information that is stored in the DB could mean a malicious user could withdraw cash from the account whose credentials were leaked.

Unfortunately this isn't like a password / hashing situation, because the user does not input the credentials every time - they input it once and it is then saved on the server for future use by the application.

The only reasonable method that I can come up with (this will be a MySQL/PHP application), is to encrypt the credentials via a hardcoded "password" in the PHP application. The only benefit here is that if the malicious user/hacker gains access to the database, but not the PHP code, they still have nothing. That said, this seems pointless to me because I think we can reasonably assume that a hacker will get everything if they get one or the other - right?

If the community decides upon some good solutions, it would be nice to gather other sources to examples/tutorials/more in depth information so that this can be implemented in the future for everyone.

I was surprised I did not see this question with any good answers on stack already. I did find this one, but in my case this doesn't really apply: How should I ethically approach user password storage for later plaintext retrieval?

Thanks all.

Answer 1

Based on what I can see in the question, answers, and comments; I would suggest taking advantage of OpenSSL. This is assuming that your site needs access to this information periodically (meaning it can be scheduled). As you stated:

The server would need this information to send payments for all sorts of situations. It does not require the "owner" of said keys to log in, in fact the owner might never care to see them ever again once they provide them the first time.

It is from this comment, and the assumption that accessing the data you want to store can be put within a cron job. It is further assumed that you have SSL (https) on your server as you will be dealing with confidential user information, and have the OpenSSL and mcrypt modules available.. Also, what follows will be rather generic as to 'how' it can be achieved, but not really the details of doing it per your situation. It should also be noted that this 'how-to' is general, and you should do more research before implementing it. That being said, let's get started.

First, let's talk about what OpenSSL provides. OpenSSL gives us a Public-Key Cryptography: the ability to encrypt data using a public key (which, if compromised, won't compromise the security of the data encrypted with it.) Secondly, it provides a way to access that information with a 'Private Key. As we don't care about creating a certificate (we only need encryption keys), those may be obtained with a simple function (which you'll only use once.):

function makeKeyPair()
{
    //Define variables that will be used, set to ''
    $private = '';
    $public = '';
    //Generate the resource for the keys
    $resource = openssl_pkey_new();

    //get the private key
    openssl_pkey_export($resource, $private);

    //get the public key
    $public = openssl_pkey_get_details($resource);
    $public = $public["key"];
    $ret = array('privateKey' => $private, 'publicKey' => $public);
    return $ret;
}

Now, you have a Public and Private key. Guard the private key, keep it off your server, and keep it out of the database. Store it on another server, a computer that can run cron jobs, etc. Just nowhere near the public eye unless you can require an admin to be present every time you require a payment to be processed and encrypt the private key with AES encryption or something similar. The public key, however, will be hard-coded in to your application, and will be used every time a user enters their information to be stored.

Next, you need to determine how you plan to verify the decrypted data (so you don't start posting to payment APIs with invalid requests.) I am going to assume there are multiple fields that need to be stored, and as we only want to encrypt once, it will be in a PHP array that can be serialize'd. Depending on how much data needs to be stored, we'll either be able to encrypt it directly, or generate a password to encrypt with the public key, and use that random password to encrypt that data itself. I am going to go this route in the explanation. To go this route, we will use AES encryption, and need to have an encrypt and decrypt function handy - as well as a way to randomly generate a decent one-time pad for the data. I'll provide the password generator that I use, though I ported it from code I wrote a while back, it will serve the purpose, or you can write a better one. ^^

public function generatePassword() {
    //create a random password here
    $chars = array( 'a', 'A', 'b', 'B', 'c', 'C', 'd', 'D', 'e', 'E', 'f', 'F', 'g', 'G', 'h', 'H', 'i', 'I', 'j', 'J',  'k', 'K', 'l', 'L', 'm', 'M', 'n', 'N', 'o', 'O', 'p', 'P', 'q', 'Q', 'r', 'R', 's', 'S', 't', 'T',  'u', 'U', 'v', 'V', 'w', 'W', 'x', 'X', 'y', 'Y', 'z', 'Z', '1', '2', '3', '4', '5', '6', '7', '8', '9', '0', '?', '<', '>', '.', ',', ';', '-', '@', '!', '#', '$', '%', '^', '&', '*', '(', ')');

    $max_chars = count($chars) - 1;
    srand( (double) microtime()*1000000);

    $rand_str = '';
    for($i = 0; $i < 30; $i++)
    {
            $rand_str .= $chars[rand(0, $max_chars)];
    }
    return $rand_str;

}

This particular function will generate 30 digits, which provides decent entropy - but you can modify it for your needs. Next, the function to do AES encryption:

/**
 * Encrypt AES
 *
 * Will Encrypt data with a password in AES compliant encryption.  It
 * adds built in verification of the data so that the {@link this::decryptAES}
 * can verify that the decrypted data is correct.
 *
 * @param String $data This can either be string or binary input from a file
 * @param String $pass The Password to use while encrypting the data
 * @return String The encrypted data in concatenated base64 form.
 */
public function encryptAES($data, $pass) {
    //First, let's change the pass into a 256bit key value so we get 256bit encryption
    $pass = hash('SHA256', $pass, true);
    //Randomness is good since the Initialization Vector(IV) will need it
    srand();
    //Create the IV (CBC mode is the most secure we get)
    $iv = mcrypt_create_iv(mcrypt_get_iv_size(MCRYPT_RIJNDAEL_128, MCRYPT_MODE_CBC), MCRYPT_RAND);
    //Create a base64 version of the IV and remove the padding
    $base64IV = rtrim(base64_encode($iv), '=');
    //Create our integrity check hash
    $dataHash = md5($data);
    //Encrypt the data with AES 128 bit (include the hash at the end of the data for the integrity check later)
    $rawEnc = mcrypt_encrypt(MCRYPT_RIJNDAEL_128, $pass, $data . $dataHash, MCRYPT_MODE_CBC, $iv);
    //Transfer the encrypted data from binary form using base64
    $baseEnc = base64_encode($rawEnc);
    //attach the IV to the front of the encrypted data (concatenated IV)
    $ret = $base64IV . $baseEnc;
    return $ret;
}

(I wrote these function originally to be part of a class, and suggest you implement them in to a class of your own.) Also, use of this function is fine with a one-time pad that is created, however, if used with a user-specific password for a different application, you definitely need some salt in there to add to the password. Next, to decrypt and verify the decrypted data is correct:

/**
 * Decrypt AES
 *
 * Decrypts data previously encrypted WITH THIS CLASS, and checks the
 * integrity of that data before returning it to the programmer.
 *
 * @param String $data The encrypted data we will work with
 * @param String $pass The password used for decryption
 * @return String|Boolean False if the integrity check doesn't pass, or the raw decrypted data.
 */
public function decryptAES($data, $pass){
    //We used a 256bit key to encrypt, recreate the key now
    $pass = hash('SHA256', $this->salt . $pass, true);
    //We should have a concatenated data, IV in the front - get it now
    //NOTE the IV base64 should ALWAYS be 22 characters in length.
    $base64IV = substr($data, 0, 22) .'=='; //add padding in case PHP changes at some point to require it
    //change the IV back to binary form
    $iv = base64_decode($base64IV);
    //Remove the IV from the data
    $data = substr($data, 22);
    //now convert the data back to binary form
    $data = base64_decode($data);
    //Now we can decrypt the data
    $decData = mcrypt_decrypt(MCRYPT_RIJNDAEL_128, $pass, $data, MCRYPT_MODE_CBC, $iv);
    //Now we trim off the padding at the end that php added
    $decData = rtrim($decData, "\0");
    //Get the md5 hash we stored at the end
    $dataHash = substr($decData, -32);
    //Remove the hash from the data
    $decData = substr($decData, 0, -32);
    //Integrity check, return false if it doesn't pass
    if($dataHash != md5($decData)) {
        return false;
    } else {
        //Passed the integrity check, give use their data
        return $decData;
    }
}

Look at both of the functions, read the comments, etc. Figure out what they do and how they work so you don't implement them incorrectly. Now, to encrypting the user-data. We'll encrypt it with the public key, and the following functions assume that every function so far (and to come) is in the same class. I'll provide both the OpenSSL encrypt/decrypt functions at once as we'll need the second later.

/**
 * Public Encryption
 *
 * Will encrypt data based on the public key
 *
 * @param String $data The data to encrypt
 * @param String $publicKey The public key to use
 * @return String The Encrypted data in base64 coding
 */
public function publicEncrypt($data, $publicKey) {
    //Set up the variable to get the encrypted data
    $encData = '';
    openssl_public_encrypt($data, $encData, $publicKey);
    //base64 code the encrypted data
    $encData = base64_encode($encData);
    //return it
    return $encData;
}

/**
 * Private Decryption
 *
 * Decrypt data that was encrypted with the assigned private
 * key's public key match. (You can't decrypt something with
 * a private key if it doesn't match the public key used.)
 *
 * @param String $data The data to decrypt (in base64 format)
 * @param String $privateKey The private key to decrypt with.
 * @return String The raw decoded data
 */
public function privateDecrypt($data, $privateKey) {
    //Set up the variable to catch the decoded date
    $decData = '';
    //Remove the base64 encoding on the inputted data
    $data = base64_decode($data);
    //decrypt it
    openssl_private_decrypt($data, $decData, $privateKey);
    //return the decrypted data
    return $decData;
}

The $data in these is always going to be the one-time pad, not the user information. Next, the functions to combine both the Public Key Encryption and AES of the one-time pad for encryption and decryption.

/**
 * Secure Send
 *
 * OpenSSL and 'public-key' schemes are good for sending
 * encrypted messages to someone that can then use their
 * private key to decrypt it.  However, for large amounts
 * of data, this method is incredibly slow (and limited).
 * This function will take the public key to encrypt the data
 * to, and using that key will encrypt a one-time-use randomly
 * generated password.  That one-time password will be
 * used to encrypt the data that is provided.  So the data
 * will be encrypted with a one-time password that only
 * the owner of the private key will be able to uncover.
 * This method will return a base64encoded serialized array
 * so that it can easily be stored, and all parts are there
 * without modification for the receive function
 *
 * @param String $data The data to encrypt
 * @param String $publicKey The public key to use
 * @return String serialized array of 'password' and 'data'
 */
public function secureSend($data, $publicKey)
{
    //First, we'll create a 30digit random password
    $pass = $this->generatePassword();
    //Now, we will encrypt in AES the data
    $encData = $this->encryptAES($data, $pass);
    //Now we will encrypt the password with the public key
    $pass = $this->publicEncrypt($pass, $publicKey);
    //set up the return array
    $ret = array('password' => $pass, 'data' => $encData);
    //serialize the array and then base64 encode it
    $ret = serialize($ret);
    $ret = base64_encode($ret);
    //send it on its way
    return $ret;
}

/**
 * Secure Receive
 *
 * This is the complement of {@link this::secureSend}.
 * Pass the data that was returned from secureSend, and it
 * will dismantle it, and then decrypt it based on the
 * private key provided.
 *
 * @param String $data the base64 serialized array
 * @param String $privateKey The private key to use
 * @return String the decoded data.
 */
public function secureReceive($data, $privateKey) {
    //Let's decode the base64 data
    $data = base64_decode($data);
    //Now let's put it into array format
    $data = unserialize($data);
    //assign variables for the different parts
    $pass = $data['password'];
    $data = $data['data'];
    //Now we'll get the AES password by decrypting via OpenSSL
    $pass = $this->privateDecrypt($pass, $privateKey);
    //and now decrypt the data with the password we found
    $data = $this->decryptAES($data, $pass);
    //return the data
    return $data;
}

I left the comments intact to help understanding of these functions. Now is where we get down to the fun part, actually working with the users data. The $data in the send method is the user-data in a serialized array. Remember for the send method that the $publicKey is hard-coded, you can store as a variable in your class and access it that way for less variables to pass to it, or have it inputted from elsewhere to send to the method every time. Example usage to encrypt the data:

$myCrypt = new encryptClass();
$userData = array(
    'id' => $_POST['id'],
    'password' => $_POST['pass'],
    'api' => $_POST['api_key']
);
$publicKey = "the public key from earlier";
$encData = $myCrypt->secureSend(serialize($userData), $publicKey));
//Now store the $encData in the DB with a way to associate with the user
//it is base64 encoded, so it is safe for DB input.

Now, that's the easy part, the next part is being able to use that data. For that, you'll need a page on your server that accepts $_POST['privKey'] and will then loop through the users, etc in the fashion that is needed for your site, grabbing the $encData. Sample usage to decrypt from this:

$myCrypt = new encryptClass();
$encData = "pulled from DB";
$privKey = $_POST['privKey'];
$data = unserialize($myCrypt->secureReceive($encData, $privKey));
//$data will now contain the original array of data, or false if
//it failed to decrypt it.  Now do what you need with it.

Next, the specific theory of use to access that secure page with the private key. On a separate server, you'll have a cron job that runs a php script specifically not in public_html containing the private key, then use curl to post the private key to your page that is looking for it. (Make sure you are calling an address that begins with https)

I hope that helps answer how it is possible to store the user information securely within your application that won't be compromised by accessing either your code or your database.

Answer 2

Let me see if I can summarize the problem - and then my answer to what I understand the problem.

You would like to have users login to your application, and then store 3rd party credentials. (It doesn't matter what those credentials are...) For security, you'd like there not to be an easy way to decrypt those credentials in the case of a hacker gaining access to the database.

Here is what I suggest.

1. Create an authentication system for the user to log in to your application. The user MUST login each time they visit the site. When storing access to all of these other credentials, a "remember me" is just a horrible idea. Authentication is created by combining and hashing username, password, and a salt. This way, none of that information is stored in the db.

2. A hashed version of the username / password combination is stored in the session. This becomes the MASTER KEY.

3. 3rd party information is entered. This information is encrypted using the MASTER KEY hash.

So this means...

If a user doesn't know their password, they are out of luck. However, it would be a very difficult situation for a hacker to get the information. They would need to understand the hashing of the username, password, salt, to break authentication, then have that hashed version of hte username/password for the master key, and then use that to decyrpt the data.

It is possible to still be hacked, but very hard - unprobable. I would also say this gives you relative deniability because, according to this method, you never know the information on the server, as it is encrypted before it is stored. This method is similar to how I assume services like OnePassword work.