Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

urllib3 connectionpool - Connection pool is full, discarding connection

Does seeing the

urllib3.connectionpool WARNING - Connection pool is full, discarding connection 

mean that I am effectively loosing data (because of lost connection)
OR
Does it mean that connection is dropped (because pool is full); however, the same connection will be re-tried later on when connection pool becomes available?

like image 894
JavaFan Avatar asked Dec 13 '18 15:12

JavaFan


People also ask

What happens when connection pool is full?

If the maximum pool size has been reached and no usable connection is available, the request is queued. The pooler then tries to reclaim any connections until the time-out is reached (the default is 15 seconds). If the pooler cannot satisfy the request before the connection times out, an exception is thrown.

What is a connection pool in database?

Database connection pooling is a way to reduce the cost of opening and closing connections by maintaining a “pool” of open connections that can be passed from database operation to database operation as needed.


1 Answers

No data is being lost!

The connection is being discarded after the request is completed (because the pool is full, as mentioned). This means that this particular connection is not going to be re-used in the future.

Because a urllib3 PoolManager reuses connections, it will limit how many connections are retained per hos to avoid accumulating too many unused sockets. The PoolManager can be configured to avoid creating excess sockets when the pool doesn't have any idle sockets available with PoolManager(..., block=True).

If you're relying on concurrency, it could be a good idea to increase the size of the pool (maxsize) to be at least as large as the number of threads you're using, so that each thread effectively gets its own connection.

More details here: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#customizing-pool-behavior

like image 66
3 revs Avatar answered Sep 30 '22 00:09

3 revs