Django: Postgres connection not closing

Question

I have the problem that my django application accumulates postgres connections over time. It seems that about every 30min a new connection is established and the old connections don't close (see screen). As max connections is set to 100 after some time all connections are blocked.

Does anyone know what is causing this problem?

I discovered this after I integrated some celery tasks. So I am quite sure that it is related to celery.

So I tried to close the connection manually after every Task using after_return method:

from django.db import connection

class DBTask(Task):
    abstract = True

    def after_return(self, *args, **kwargs):
        connection.close()

@task(name='example', base=DBTask)
def example_task(value):
    # do some stuff

But this also doesn't help. Maybe I am totally wrong and it isn't related to celery at all.

My database configuration:

DATABASES = {
    'default': {
        'ENGINE': 'django.contrib.gis.db.backends.postgis',
        'NAME': 'production', 
        'USER': 'production', 
        'HOST': 'some.host', 
        'CONN_MAX_AGE': 0,
    },
}

Installed packages:

• django 1.8.9
• pyscopg2 2.6.1
• celery 3.1.20
• django-celery 3.1.17

The app is deployed at webfaction (maybe this helps)

I have also seen this question, but setting CONN_MAX_AGE: 0 didn't help.

Update:

Tried adding connection.close() at the end of each celery task, but the number of connection is still increasing.

Update 2:

Tried adding connection.close() at the top of the celery file, but this didn't help either.

Update 3:

Here is the code I am actually using in the celery tasks:

celery_tasks.py

@task(name='push_notifications', base=DBTask)
def push_notifications_task(user_id):
    user = CustomUser.objects.get(id=user_id)
    PusherAPI().push_notifications(user)
    connection.close()

models.py

class PusherAPI(object):

    def push_notifications(self, user):
        from .serializers import NotificationSerializer
        self.pusher.trigger(
            'user_%s' % user.slug,
            'notifications',
            NotificationSerializer(user).data
        )

serializers.py

class NotificationSerializer(object):

    def __init__(self, user=None):
        if user is None:
            self.user = get_current_user()
        else:
            self.user = user

    @property
    def data(self):
        # get notifications from db
        notifications = self.user.notifications.unread()
        # create the notification dict
        ...
        return note_dict

The only db-queries are in CustomUser.objects.get(id=user_id) and notifications = self.user.notifications.unread()

Answer 1

Make sure it is actually old connections that are not closed and not new connections that pile up because some part of your application can't handle the load. Have a look at the individual connections, e.g. with SELECT * FROM pg_stat_activity;