Elasticsearch cache clear doesn't seems to do what I expected

Question

I ran a search and the first time and it took 3-4 seconds. I ran the same search second time and it took less than 100 ms (as expected as it used the cache) Then I cleared the cache by calling "http://host:port/index/_cache/clear" Next I ran the same search and was expecting it to take 3-4 seconds but it took less than 100 ms

So the clearing of the cache didn't work? What exactly got cleared by that url? How I do make ES do the raw search (i.e. no caching) every time?

I am doing as a part of some load testing.

Answer 1

Clearing the cache will empty:

• Field data (used by facets, sorting, geo, etc)
• Filter cache
• parent/child cache
• Bloom filters for posting lists

The effect you are seeing is probably due to the OS file system cache. Elasticsearch and Lucene leverage the OS file system cache heavily due to the immutable nature of lucene segments. This means that small indices tend to be cached entirely in memory by your OS and become diskless.

As an aside, it doesn't really make sense to benchmark Elasticsearch in a "cacheless" state. It is designed and built to operate in a cached environment - much of the performance that Elasticsearch is known for is due to it's excellent use of caching.

To be completely accurate, your benchmark should really be looking at a system that has fully warmed the JVM (to properly size the new-eden space, optimize JIT output, etc) and using real, production-like data to simulate "real world" cache filling and eviction on both the ES and OS levels.

Synthetic tests such as "no-cache environment" make little sense.

Answer 2

I don't know if this is what you're experiencing, but the cache isn't cleared immediately when you call clear cache. It is scheduled to be deleted in the next 60 seconds.

source: https://www.elastic.co/guide/en/elasticsearch/reference/1.5/indices-clearcache.html