Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How could I check the number of active sqlalchemy connections in a pool at any given time?

I have a situation where sqlalchemy keeps running out of active connections from time to time due to high traffic loads, and I would like to run some tests to verify and optimize the pooling parameters per our use case. However, I cannot find a straightforward way of polling for the count of active connections.

Current setup is on the lines:

args = ...
mapping = {
        'pool_size': 10,
        'max_overflow': 10,
        'pool_timeout': 30,
        'pool_recycle': 1800
    }
engine = sqlalchemy.create_engine(*args, **mapping)

The max connections on the MySQL server is set to 200 and there are about 20 web servers and celery boxes total connecting to it.

like image 640
Priyeshj Avatar asked Jan 13 '16 19:01

Priyeshj


People also ask

Does SQLAlchemy use a connection pool?

SQLAlchemy includes several connection pool implementations which integrate with the Engine . They can also be used directly for applications that want to add pooling to an otherwise plain DBAPI approach.

What is SQLAlchemy pool size?

pool_size – The size of the pool to be maintained, defaults to 5. This is the largest number of connections that will be kept persistently in the pool. Note that the pool begins with no connections; once this number of connections is requested, that number of connections will remain.

How many connection pools should I have?

For optimal performance, use a pool with eight to 16 connections per node. For example, if you have four nodes configured, then the steady-pool size must be set to 32 and the maximum pool size must be 64.

What are checked out connections?

Check-out means getting a connection from the pool. Check-in means giving back a connection to the pool. Check-out means getting a connection from the pool..


1 Answers

The default QueuePool has a status method that returns the following:

def status(self):
        return "Pool size: %d  Connections in pool: %d "\
            "Current Overflow: %d Current Checked out "\
            "connections: %d" % (self.size(),
                                 self.checkedin(),
                                 self.overflow(),
                                 self.checkedout())

Pool.checkedout() will return the number of checked out connections.

like image 60
jumbopap Avatar answered Oct 26 '22 07:10

jumbopap