Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to start mapreduce job from cron on GAE Python

I have mapreduce job defined in mapreduce.yaml:

mapreduce:
- name: JobName 
  mapper:
    input_reader: google.appengine.ext.mapreduce.input_readers.DatastoreInputReader
    handler: handler_name
    params:
    - name: entity_kind
      default: KindName

How to start it from cron? Is there some url that can run it?

like image 770
Pawel Markowski Avatar asked Oct 11 '22 20:10

Pawel Markowski


1 Answers

You can start a mapreduce task from any kind of AppEngine handler using control.py

from mapreduce import control

mapreduce_id = control.start_map(
    "My Mapper",
    "main.my_mapper",
    "mapreduce.input_readers.DatastoreInputReader",
    {"entity_kind": "models.MyEntity"},
    shard_count=10)
like image 198
Chris Farmiloe Avatar answered Oct 13 '22 10:10

Chris Farmiloe