How to automatically get email alerts on errors in Google app engine

Question

I maintain a lot of Google App Engine apps and it is very difficult to constantly keep a track of all my servers if they are running smoothly.

It is very embarrassing to wake up only after user complains about a system crash.

Is it possible to constantly get updated about my server health?

Can i get a mail or something in case their is a error on my google app engine application?

Can't wait for Google to take up this requirement

Answer 1

Now it is possible with Stackdriver (acquired by Google and integrated with Google Cloud Platform/App Engine).

1. Go to https://console.cloud.google.com/errors.
2. Click Turn on notifications

Answer 2

You can schedule a cron job which can do this for you

First create a entry in cron.yaml

- description: Houston we have a problem Finder
  url: /errorfinder/
  schedule: every 3 hours
  timezone: Asia/Kolkata

Then create an entry in app.yaml

handlers:
- url: /errorfinder/
  script: errorfinder.app
  secure: always 

Now keep errorfinder.py with the following content

import base64
import datetime
import logging
import time
import urllib
import webapp2
from google.appengine.api.logservice import logservice
from google.appengine.api import mail

class MainHandler(webapp2.RequestHandler):
    def get(self):    
      # Set up end time for our query.
      end_time = time.time()
      start_time = end_time  - 10800 # 3 hours before now . same as cronjob interval
      html = ''
      report_needed = False
      # Iterate through all the RequestLog objects, displaying some fields and
      # iterate through all AppLogs beloging to each RequestLog count times
      for req_log in logservice.fetch(start_time=start_time, end_time=end_time, minimum_log_level=logservice.LOG_LEVEL_WARNING, include_app_logs=True):
            report_needed = True
            html = html + '<br /> REQUEST LOG <br />'
            html = html + 'IP: %s <br /> Method: %s <br /> Resource: %s <br />' % (req_log.ip, req_log.method, req_log.resource)
            html = html + 'Date: %s<br />' % datetime.datetime.fromtimestamp(req_log.end_time).strftime('%D %T UTC')

            for app_log in req_log.app_logs:
                html = html + '<br />&emsp; APP LOG<br />'
                html = html + '&emsp; Date: %s<br />' % datetime.datetime.fromtimestamp(app_log.time).strftime('%D %T UTC')
                html = html + '&emsp; Message: <b>%s</b><br />' % app_log.message
                html = html + '<br /><br /><br /><br />'
      if(report_needed):
           mail.send_mail(sender="Beagle Bot <[email protected]>",
                to='[email protected]',
                subject='Houston we have a problem ..',
                body=html,
                html=html,
                reply_to='[email protected]')
      self.response.out.write(html)

app = webapp2.WSGIApplication([('/errorfinder/', MainHandler)], debug=True)

You can make the error search close to real time by reducing the cron job interval