Elasticsearch lowercase filter search

Question

I'm trying to search my database and be able to use upper/lower case filter terms but I've noticed while query's apply analyzers, I can't figure out how to apply a lowercase analyzer on a filtered search. Here's the query:

{
    "query": {
        "filtered": {
            "filter": {
                "bool": {
                    "should": [
                        {
                            "term": {
                                "language": "mandarin" // Returns a doc
                            }
                        },
                        {
                            "term": {
                                "language": "Italian" // Does NOT return a doc, but will if lowercased
                            }
                        }
                    ]
                }
            }
        }
    }
}

I have a type languages that I have lowercased using:

"analyzer": {
    "lower_keyword": {
        "type": "custom",
        "tokenizer": "keyword",
        "filter": "lowercase"
    }
}

and a corresponding mapping:

"mappings": {
    "languages": {
        "_id": {
            "path": "languageID"
        },
        "properties": {
            "languageID": {
                "type": "integer"
            },
            "language": {
                "type": "string",
                "analyzer": "lower_keyword"
            },
            "native": {
                "type": "string",
                "analyzer": "keyword"
            },
            "meta": {
                "type": "nested"
            },
            "language_suggest": {
                "type": "completion"
            }
        }
    }
}

Answer 1

The problem is that you have a field that you have analyzed during index to lowercase it, but you are using a term filter for the query which is not analyzed:

Term Filter

Filters documents that have fields that contain a term (not analyzed). Similar to term query, except that it acts as a filter.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-term-filter.html

I'd try using a query filter instead:

Query Filter

Wraps any query to be used as a filter. Can be placed within queries that accept a filter.

Example:

{
    "constantScore" : {
        "filter" : {
            "query" : {
                "query_string" : {
                    "query" : "this AND that OR thus"
                }
            }
        }
    } }

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-filter.html#query-dsl-query-filter

Answer 2

This may be achieved by appending .keyword to your field to query against the keyword version of the field. Assuming language was defined in the mapping with type keyword.

Note that now only the exact text would match: mandarin won't match and Italian would.

Your query would end up like this:

{
    "query": {
        "filtered": {
            "filter": {
                "bool": {
                    "should": [
                        {
                            "term": {
                                "language.keyword": "mandarin" // Returns Empty
                            }
                        },
                        {
                            "term": {
                                "language.keyword": "Italian" // Returns Italian.
                            }
                        }
                    ]
                }
            }
        }
    }
}

Combining the term values is also allowed:

{
    "query": {
        "filtered": {
            "filter": {
                "bool": {
                    "should": [
                        {
                            "term": {
                                "language.keyword":
                                     ["mandarin", "Italian"]
                            }
                        }
                    ]
                }
            }
        }
    }
}