How does elasticsearch store data

Question

I am learning elasticsearch and have written a couple of simple programs to insert, update, delete data.

I have read that elastic search always stores data in json format.

I looked at the "data" folder in my elasticsearch installation and I could not find any json format files even though I did a few insertion operations. I could see some files with .st extenstion.

So where does elasticsearch actually store the data in json format ?

Answer 1

Elastic uses lucene (https://lucene.apache.org/core/) under the hood.

Lucene is a text search engine. It stores text in a custom binary format optimized for retrieval purposes. The format is highly optimized and complicated.

Lucenes uses the concept of "indices containing documents". Internally every index consists of several segments. Segments are saved in several files in the file system. Documents are split up in several lookup structures, residing in the files.

When you browse the data folder of elastic you see this lucene index and segment structure. There is no storage of json formatted data on the file system level. Instead the files contain optimized binary data and you need to pass through the elastic API to get a JSON representation of a document.