I am getting Error R14 (Memory quota exceeded) in heroku with a Django app

Question

I am running 3 dynos in Heroku platform for my Django application. It was working properly(1.5 years). Since 2 weeks, I am getting Error R14 (Memory quota exceeded) error frequently.

What should I do in order to avoid this error? How can I monitor the problem?

2015-01-27T10:34:01.855731+00:00 app[web.3]: Starting development server at http://0.0.0.0:43181/
2015-01-27T10:34:02.042166+00:00 heroku[web.3]: State changed from starting to up
2015-01-27T10:34:15.626327+00:00 heroku[web.2]: Error R14 (Memory quota exceeded)
2015-01-27T10:34:15.626241+00:00 heroku[web.2]: Process running mem=662M(129.4%)
2015-01-27T10:34:28.151622+00:00 heroku[router]: at=info method=GET path="/api/shop/651/?format=json&&account=(null)" request_id=2d904167-3a7d-4c8c-9b2c-ae845d0fffa9 fwd="88.247.106.124" dyno=web.1 connect=0ms service=3009ms status=200 bytes=282437
2015-01-27T10:34:28.146392+00:00 app[web.1]: [27/Jan/2015 12:34:28] "GET /api/shop/651/?format=json&&account=(null) HTTP/1.1" 200 282077
2015-01-27T10:34:35.480951+00:00 heroku[web.2]: Process running mem=662M(129.4%)
2015-01-27T10:34:35.481269+00:00 heroku[web.2]: Error R14 (Memory quota exceeded)
2015-01-27T10:34:55.511625+00:00 heroku[web.2]: Process running mem=662M(129.4%)
2015-01-27T10:34:55.511625+00:00 heroku[web.2]: Error R14 (Memory quota exceeded)

These are the logs.

And guppy results:

>>> hp.setref()
>>> hp.heap()
Partition of a set of 40 objects. Total size = 6632 bytes.
Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
 0      5  12     3160  48      3160  48 unicode
 1     25  62     2200  33      5360  81 __builtin__.weakref
 2      6  15      496   7      5856  88 list
 3      1   2      488   7      6344  96 types.FrameType
 4      2   5      184   3      6528  98 tuple
 5      1   2      104   2      6632 100 urlparse.SplitResult
>>> hp.heap()
Partition of a set of 24479 objects. Total size = 12695072 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
 0   7212  29  7558176  60   7558176  60 dict of tastypie.fields.CharField
 1    265   1   866008   7   8424184  66 dict (no owner)
 2    232   1   777664   6   9201848  72 dict of 0x7fe18acb9360
 3    696   3   729408   6   9931256  78 dict of tastypie.fields.DecimalField
 4    567   2   594216   5  10525472  83 dict of tastypie.fields.BooleanField
 5    517   2   541816   4  11067288  87 dict of tastypie.fields.IntegerField
 6   7212  29   461568   4  11528856  91 tastypie.fields.CharField
 7    260   1   272480   2  11801336  93 dict of tastypie.fields.DateTimeField
 8   1255   5   223952   2  12025288  95 unicode
 9     53   0    96248   1  12121536  95 dict of tastypie.fields.ToManyField

Answer 1

It's likely you have either a memory leak, or you're running too many concurrent processes with your server. Are you using gunicorn? If so, look at your procfile and see how many workers you're running -- then lower it by one.

To track the issue, try running:

$ heroku logs --tail

On the command line to view your web logs and see when the memory errors start kicking in.