Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to expire (and revive) workers in gunicorn?

I have an application with a slow memory leak which, for various reasons, I can't get rid of. So I'd like to use the old trick of having my workers periodically die and revive.

(i.e. use the same strategy as maxtasksperchild in multiprocessing Pool ... "...A frequent pattern found in other systems (such as Apache, mod_wsgi, etc) to free resources held by workers is to allow a worker within a pool to complete only a set amount of work before being exiting, being cleaned up and a new process spawned to replace the old one...")

So far the best I've been able to come up with is to have a thread that sleeps, then calls os._exit(-1).

Is that the way to go, or is there a better way to periodically recycle my workers?

Here's the path I'm going down at the moment:

class Quitter(Thread):

    def run(self):
        while True:
            time.sleep(random.randrange(5, 7)):
            print str(os.getpid())
            os._exit(-1)

Quitter().start()

To which gunicorn responds with:

2013-03-13 03:21:24 [6487] [INFO] Booting worker with pid: 6487
...
2013-03-13 03:21:30 [6492] [INFO] Booting worker with pid: 6487
like image 731
sea-rob Avatar asked Feb 17 '23 04:02

sea-rob


1 Answers

Gunicorn actually has this available as a configuration option - take a look at max_requests.

like image 128
girasquid Avatar answered Feb 19 '23 21:02

girasquid