Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How does one get a count of rows in a Datastore model in Google App Engine?

People also ask

What is the query language we can use with Datastore?

The Python Datastore API provides two classes for preparing and executing queries: Query uses method calls to prepare the query. GqlQuery uses a SQL-like query language called GQL to prepare the query from a query string.

How does Google's Datastore work?

Datastore is a highly scalable NoSQL database for your applications. Datastore automatically handles sharding and replication, providing you with a highly available and durable database that scales automatically to handle your applications' load.

Is Google Datastore deprecated?

The Cloud Datastore Administration API v1beta1 is now deprecated. The Cloud Datastore Admin backup feature is being phased out in favor of the managed export and import for Cloud Datastore. Please migrate to the managed export and import functionality at your earliest convenience.

What is the data store used by Google App Engine?

App Engine's Go standard runtime connects to Datastore using the Go Datastore API. For a complete list of the contents of the datastore package, see the datastore package reference. You cannot use the Cloud Datastore client library with Go applications in the App Engine standard environment.


You should use Datastore Statistics:

Query query = new Query("__Stat_Kind__");
query.addFilter("kind_name", FilterOperator.EQUAL, kind);       
Entity entityStat = datastore.prepare(query).asSingleEntity();
Long totalEntities = (Long) entityStat.getProperty("count");

Please note that the above does not work on the development Datastore but it works in production (when published).

I see that this is an old post, but I'm adding an answer in benefit of others searching for the same thing.


As of release 1.3.6, there is no longer a cap of 1,000 on count queries. Thus you can do the following to get a count beyond 1,000:

count = modelname.all(keys_only=True).count()

This will count all of your entities, which could be rather slow if you have a large number of entities. As a result, you should consider calling count() with some limit specified:

count = modelname.all(keys_only=True).count(some_upper_bound_suitable_for_you)

count = modelname.all(keys_only=True).count(some_upper_limit)

Just to add on to the earlier post by dar, this 'some_upper_limit' has to be specified. If not, the default count will still be a maximum of 1000.


This is a very old thread, but just in case it helps other people looking at it, there are 3 ways to accomplish this:

  1. Accessing the Datastore statistics
  2. Keeping a counter in the datastore
  3. Sharding counters

Each one of these methods is explained in this link.


In GAE a count will always make you page through the results when you have more than 1000 objects. The easiest way to deal with this problem is to add a counter property to your model or to a different counters table and update it every time you create a new object.


I still hit the 1000 limit with count so adapted dar's code (mine's a bit quick and dirty):

class GetCount(webapp.RequestHandler):
    def get(self):
        query = modelname.all(keys_only=True)

        i = 0
        while True:
            result = query.fetch(1000)
            i = i + len(result)
            if len(result) < 1000:
                break
            cursor = query.cursor()
            query.with_cursor(cursor)

        self.response.out.write('<p>Count: '+str(i)+'</p>')

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Query query = new Query("__Stat_Kind__");
Query.Filter eqf = new Query.FilterPredicate("kind_name",
                                Query.FilterOperator.EQUAL,
                                "SomeEntity");
query.setFilter(eqf);
Entity entityStat = ds.prepare(query).asSingleEntity();
Long totalEntities = (Long) entityStat.getProperty("count");