Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Profiling memory usage on App Engine

How can I profile memory (RAM) usage on my App Engine app? I'm trying to address errors related to exceeding the instance memory limit. I've tried these things and, so far, they don't work or don't provide what I need.

  • Appstats. This doesn't provide memory usage details.
  • Apptrace. It hasn't been updated since 2012 and depends on a deprecated version of the SDK. Doesn't work out of the box.
  • Appengine-profiler. Doesn't provide memory stats.
  • Gae-mini-profiler, which uses cProfile. Doesn't provide memory stats.
  • guppy. After downloading and installing the library's code in my app's folder, running guppy.hpy() fails with ImportError: No module named heapyc
  • resource. Not part of the SDK's version of python, so I can't use it.

Am I wrong about any of the above? The top-rated answer (not the accepted one) on this question says that there is no way to monitor memory usage on App Engine. That can't be true. Can it?

EDIT

I can confirm that GAE mini profiler does the job. After installation, I could change the settings in the UI to "sampling with memory" and then see this readout:

Example UI showing memory usage

Thanks to all the contributors!

like image 367
Chris Avatar asked Jun 09 '15 20:06

Chris


People also ask

Which profiling will analyze the memory usage of the application?

The Memory Profiler is a component in the Android Profiler that helps you identify memory leaks and memory churn that can lead to stutter, freezes, and even app crashes. It shows a realtime graph of your app's memory use and lets you capture a heap dump, force garbage collections, and track memory allocations.

How do you analyze memory usage?

To open up Resource Monitor, press Windows Key + R and type resmon into the search box. Resource Monitor will tell you exactly how much RAM is being used, what is using it, and allow you to sort the list of apps using it by several different categories.

How do I find my memory profile?

Press Ctrl + Shift + Esc to launch Task Manager. Or, right-click the Taskbar and select Task Manager. Select the Performance tab and click Memory in the left panel. The Memory window lets you see your current RAM usage, check RAM speed, and view other memory hardware specifications.

How do you do memory profiling in C++?

How Memory Profiling for C and C++ Works. When an application node is executed, the source code is instrumented by the C or C++ Instrumentor (attolcpp or attolcc1). The resulting source code is then executed and the Memory Profiling feature outputs a static . tsf file for each instrumented source file and a dynamic .


2 Answers

GAE Mini Profiler does provide memory stats if you use the sampling profiler and set memory_sample_rate nonzero; at each snapshot it will tell you the memory that was in use. You will want to turn the sample frequency way down as the memory sample takes a few ms to execute.

Edit: the way it gets the memory stats is from the GAE runtime API which is deprecated, but still worked as of last I knew; I'm not sure if there's a good replacement.

like image 65
Ben Kraft Avatar answered Nov 02 '22 03:11

Ben Kraft


To add to Ben's answer, as of 16th November 2015, despite being deprecated, the Google App Engine runtime API still works. There isn't an official replacement from Google yet.

from google.appengine.api.runtime import runtime
import logging

logging.info(runtime.memory_usage())

This will output memory usage statistics, where the numbers are expressed in MB. For example:

current: 464.0859375
average1m: 464
average10m: 379.575
like image 28
Milo Avatar answered Nov 02 '22 02:11

Milo