I have a situation where sqlalchemy keeps running out of active connections from time to time due to high traffic loads, and I would like to run some tests to verify and optimize the pooling parameters per our use case. However, I cannot find a straightforward way of polling for the count of active connections.
Current setup is on the lines:
args = ...
mapping = {
'pool_size': 10,
'max_overflow': 10,
'pool_timeout': 30,
'pool_recycle': 1800
}
engine = sqlalchemy.create_engine(*args, **mapping)
The max connections on the MySQL server is set to 200 and there are about 20 web servers and celery boxes total connecting to it.
SQLAlchemy includes several connection pool implementations which integrate with the Engine . They can also be used directly for applications that want to add pooling to an otherwise plain DBAPI approach.
pool_size – The size of the pool to be maintained, defaults to 5. This is the largest number of connections that will be kept persistently in the pool. Note that the pool begins with no connections; once this number of connections is requested, that number of connections will remain.
For optimal performance, use a pool with eight to 16 connections per node. For example, if you have four nodes configured, then the steady-pool size must be set to 32 and the maximum pool size must be 64.
Check-out means getting a connection from the pool. Check-in means giving back a connection to the pool. Check-out means getting a connection from the pool..
The default QueuePool
has a status
method that returns the following:
def status(self):
return "Pool size: %d Connections in pool: %d "\
"Current Overflow: %d Current Checked out "\
"connections: %d" % (self.size(),
self.checkedin(),
self.overflow(),
self.checkedout())
Pool.checkedout()
will return the number of checked out connections.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With