Why doesn't Google Cloud DataStore support multiple inequality filter on different properties?

Question

I know that is a limitation of DataStore, But I just want to figure out the reason.

Invalid Argument: Cannot have inequality filters on multiple properties: [..., ...]

I have read the bigtable paper and I can not find any restriction on inequality filter on different column. and it can support prefix and range scan. IMHO, DataStore could support that multiple inequality filter with these two operation. Do you know any reason take the functionality from DataStore?

Answer 1

To avoid having to scan the entire index table, the query mechanism relies on 
all of a query's potential results being adjacent to one another in the index.
To satisfy this constraint, a single query may not use inequality comparisons 
(LESS_THAN, LESS_THAN_OR_EQUAL, GREATER_THAN, GREATER_THAN_OR_EQUAL, NOT_EQUAL)
on more than one property across all of its filters

[Source : https://cloud.google.com/appengine/docs/standard/java/datastore/query-restrictions ]

Answer 2

Every datastore query is running on one or more index table.

When considering inequality filters, we want to run on at most one property in order to avoid running on the entire index tables. It's easier to grasp when investigating how Filters work, how datastore-indexes.xml defined and what is index exactly:

<datastore-index> are elements, one for every index that the Datastore should maintain. So, you defining one index (which is a table) for every relation between Entity's kind and it's properties. Now imagine when you need to calculate multiple inequality filter on different properties? It can lead to a huge running time. Not good.

<?xml version="1.0" encoding="utf-8"?>
<datastore-indexes autoGenerate="true">
    <datastore-index kind="Greeting" ancestor="true" source="manual">
        <property name="user" direction="asc" />
        <property name="birthYear" direction="asc" />
        <property name="height" direction="asc" />
    </datastore-index>

    <datastore-index kind="Greeting" ancestor="true" source="manual">
        <property name="date" direction="asc" />
    </datastore-index>
</datastore-indexes>

Now, let's see filters. It's a valid filter - he runs on one property only (the birthYear property).

Filter birthYearMinFilter =
  new FilterPredicate("birthYear",
                      FilterOperator.GREATER_THAN_OR_EQUAL,
                      minBirthYear);

Filter birthYearMaxFilter =
  new FilterPredicate("birthYear",
                      FilterOperator.LESS_THAN_OR_EQUAL,
                      maxBirthYear);

Filter birthYearRangeFilter =
  CompositeFilterOperator.and(birthYearMinFilter, birthYearMaxFilter);

Query q = new Query("Person").setFilter(birthYearRangeFilter);

It's a non-valid filter, he runs on 2 properties (birthYear and height):

Filter birthYearMinFilter =
  new FilterPredicate("birthYear",
                      FilterOperator.GREATER_THAN_OR_EQUAL,
                      minBirthYear);

Filter heightMaxFilter =
  new FilterPredicate("height",
                      FilterOperator.LESS_THAN_OR_EQUAL,
                      maxHeight);

Filter invalidFilter =
  CompositeFilterOperator.and(birthYearMinFilter, heightMaxFilter);

Query q = new Query("Person").setFilter(invalidFilter);