I am using Google App Engine mapreduce to analyze some data. I am generating a few counters that I would like to create a simple Google chart from in my done_callback. How do I access the resulting counters from the callback?
#The map method
def count_created_since(entity):
now = datetime.datetime.now()
delta = now-entity.created
#analyze last 12 weeks
for x in range(12):
start = 7*x
stop = 7*(x+1)
if delta.days >= start and delta.days < stop:
#The counters
yield op.counters.Increment(str(x)+" weeks ago")
def my_callback(request):
# fetch counter results to create a simple Google chart url
MapReduce is a programming model for processing large amounts of data in a parallel and distributed fashion. It is useful for large, long-running jobs that cannot be handled within the scope of a single request, tasks like: Analyzing application logs. Aggregating related data from external sources.
App Engine is a fully managed, serverless platform for developing and hosting web applications at scale. You can choose from several popular languages, libraries, and frameworks to develop your apps, and then let App Engine take care of provisioning servers and scaling your app instances based on demand.
From App Engine, you can easily access other Google Cloud services such as Datastore, Cloud SQL, and Cloud Storage. You also have the option to use an external or third-party database if that database is supported by your language and accessible from your App Engine instance.
You can access the counter's through a MapreduceState
's counter_map
attribute.
from mapreduce import model
state = model.MapreduceState.get_by_job_id(your_job_id)
# counters live in state.counters_map
There was a discussion on the mailing list a month or so ago about accessing counters.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With