Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

App Engine file upload and assignment to variable causes memory usage error after serving X requests

I'm uploading images (<10MB each) to a simplified handler (below) to showcase this problem. After serving ~5 requests on an F1 (128MB of ram) instance I run over the memory limit: "Exceeded soft private memory limit with 152.414 MB after servicing 6 requests total"

On an F2 (256MB of ram) instance I get a few more requests in, but not many: "Exceeded soft private memory limit with 258.156 MB after servicing 19 requests total"

Example code:

import webapp2
class FileUploader(webapp2.RequestHandler):

  def post(self):
    test = self.request.get('file')
    self.response.out.write('hi')

app = webapp2.WSGIApplication([('/leak-uploader-example', FileUploader)],
                              debug=True)

The text in the logs suggest my application may have a memory leak. It seems like the problem may be with either Python or App Engine garbage collecting after a request is complete. Any suggestions for how to keep my instances from being terminated after only a few requests?

update -- I'm seeing the same behavior using webapp framework (instead of webapp2): "Exceeded soft private memory limit with 143.121 MB after servicing 6 requests total"

like image 695
robertfischer Avatar asked Nov 03 '22 17:11

robertfischer


1 Answers

Why not use Blobstore for your images? https://developers.google.com/appengine/docs/python/blobstore/#Python_Uploading_a_blob

like image 78
Sasxa Avatar answered Nov 11 '22 14:11

Sasxa