Memory profiling/monitoring (python) on Google AppEngine

Question

I've been working with app engine for quite some time, I know that there is appstats but those only show datastore/memcache and other not related to actual memory usage stats.

I've tried to integrate with AppTrace (with all the components latest versions) but I can't continue, since I get this error:

ImportError: dlopen(../apptrace/guppy/sets/setsc.so, 2): Symbol not found: __PyLong_AsScaledDouble
Referenced from: ../apptrace/guppy/sets/setsc.so
Expected in: flat namespace

So my question is: what is the best way (on latest appengine sdk) to profile/monitor memory/catch memory leaks and other python-memory related stuff (either on local or Google server)?

btw, we use Python 2.7 and we're working on Mac OS X (10.7.4)

Answer 1

This post is 3 years old, however I thought this answer is still worth sharing to help others. I have premium Google App Engine support and contacted them regarding this same issue.

The Google engineer advised me that the Google App Engine runtime API is deprecated, but still functions. It provides a method called memory_usage.

from google.appengine.api.runtime import runtime
import logging

logging.info(runtime.memory_usage())

This will output memory usage statistics, where the numbers are expressed in MB. For example:

current: 464.0859375
average1m: 464
average10m: 379.575

By placing the logging statement at key points in your code, you can work out which part is causing a memory leak.

Answer 2

I think there are no tools to monitor memory usage in Google App Engine, you could profile program, monitor module import times, code coverage. So not tools to detect small memory leaks.