A better approach to exclude large list of items in Elasticsearch

Question

I use terms query to exclude a list of 100,000 or more items, as the terms query by default allows only 65,536 terms, ES throws following error:

The number of terms [115687] used in the Terms Query request has exceeded the allowed maximum of [65536]. This maximum can be set by changing the [index.max_terms_count] index level setting.

One way to solve my problem is to increase the max_terms_count, but I suspect it will be slow.

Another solution would be to exclude those items in PHP which also would be too resource consuming.

Is there a better way to exclude large list of items from ES search result?

Answer 1

1. For rare cases I suggest to use a client-oriented solution: split exceptions list into two: the first 65k items should be processed by ES, the rest - in PHP.
2. Performance-oriented solution: limit the exclusion list to 65k (client-side limitation)