Elasticsearch: Data Encryption at Rest

Question

I am trying to encrypt the Elasticsearch data. Are there any native methods to encrypt the data? I studied about Elasticsearch supporting dm-crypt, but there are no proper supporting documents on how it is being implemented. Also, my clients need free/opensource alternatives to Shield/X-Pack.

TIA.

Answer 1

What is the supposed behaviour for encryption? Be able to search against encrypted data or just store some sensitive fields (e.g. PII) encrypted?

First of all, consider removing your sensitive data from ELK stack, as it isn't a reliable place to store it there. Detach it (just remove and store somewhere else or tokenize, if you want to have a link to it) or obfuscate before pushing to ELK.

As an alternative to performing encryption by means of Elasticsearch or its plugins, you can encrypt your data in the application that pushes data to ES in advance and just keep a couple of index fields (which are going to be used to search by) in plain text. E.g.

{ 
    "index_field" : "John Doe", // plain text to search by
    "address" : "s3_34$af78...", // encrypted
    "passport" : "3%75O9gfjdg4%...", // encrypted
    ...
}

That depends on your specific application, however in some cases solving encryption & key management questions would be much easier than looking for a particular solution e.g. for ES.