Best practice for storing passwords for automation

Question

I know that the "how do I securely store a password?" questions have been asked many times here; and I know the answer is always "Never do this! Store hashes, never store passwords! Use bcrypt!"

But what about times when you cannot use a hash for authentication? For example: automated processes. Say I wrote an automation application that needs to log into an SFTP site or some sort of external service that requires a username/password. I can't use a hash to authenticate with an external service, so what am I supposed to do?

I tagged this question with C# and ASP.NET as these are the two areas that this question applies to specifically for my situation.

Edit: Due to discussion raised in the comments, let me clarify the threat I'm trying to address: I want to prevent an attacker from being able to read the plain-text password used to access the external service. Meaning that if they somehow gained non-admin access to our network or database, even with the database dump they would not be able to read the passwords in plain text.

Answer 1

There are different options, both using encryption for password using a key, and protecting the key storage using HSM module.

option (1): Using Database with HSM module

You can store passwords encrypted in database and benefit from a feature in SQL 2016 "Always Encrypted (Database Engine)". Always Encrypted allows clients to encrypt sensitive data inside client applications and never reveal the encryption keys to the Database Engine (SQL Database or SQL Server).

You can use Hardware Security Modules (HSM) with Always Encrypted.

The hardware security module (HSM) is a physical device that safeguards digital keys and performs cryptographic operations. These modules traditionally come in the form of a plug-in card or an external device that attaches directly to a computer or to the network.

When you get an HSM, you also get a software libraries implementing common APIs, such as Microsoft Crypto API and Cryptography API. These API are called Cryptographic Service Provider (CSP) and Cryptography API: Next Generation CNG providers.

Your applications can communicate with an HSM using those APIs.

For more securing the HSM module, you can: - Tie the HSM to your Database Server. - Tie the HSM to your admin login to Operating System Server.

for more details:

Always Encrypted (Database Engine) Using Hardware Security Modules with Always Encrypted

Also, Oracle database and other engine can provide encryption with HSM

Securing Stored Data Using Transparent Data Encryption

Option (2): Store password in files in Protected storage using HSM module:

• Encrypting files that contain passwords. This may be done by the operating system, an application, or a specialized utility such as password management software that is specifically designed to protect the confidentiality of passwords.

• Using OS access control features to restrict access to files that contain passwords. For example, a host could be configured to permit only administrators and certain processes running with administrator-level privileges to access a password file, thus preventing users and user-level processes from accessing passwords.

• As you are not using hashing, I exclude this option, but it's a mechanism for storing one-way cryptographic hashes for passwords instead of storing the passwords themselves.

Answer 2

AES encryption and store the key in an external encryption key storage module, if you happen to have access to one of those. Otherwise, you could try scattering parts of the encryption key on different servers or something but it shouldn't be the first choice.