Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tuning node-mongodb-native connection pool size

I've got an express app talking to mongodb via the node-mongodb-native driver. Some requests in my app are intermittently slow. Any good tools or strategies to confirm or rule out the driver connection pool size as a bottleneck?

Here's some discussion of tuning pool size, but it's pretty inconclusive. aheckmann notes that the default of 5 is usually plenty, while tinana saw significant gains from bumping up the pool with many concurrent request.

Update: This question was to help me understand tuning and tooling driver pool size rather than to troubleshoot the immediate performance issue. I described my issue only to give the question a little context.

like image 694
hurrymaplelad Avatar asked Aug 10 '13 00:08

hurrymaplelad


People also ask

What is the default connection pool size in MongoDB?

100 ( maxPoolSize default 100 ) x 4 (application servers) = 400 (incoming connections to each mongod).

What is the ideal DB connection pool size?

For optimal performance, use a pool with eight to 16 connections per node. For example, if you have four nodes configured, then the steady-pool size must be set to 32 and the maximum pool size must be 64. Adjust the Idle Timeout and Pool Resize Quantity values based on monitoring statistics.

What is the max pool size in connection pool?

A connection pool is created for each unique connection string. When a pool is created, multiple connection objects are created and added to the pool so that the minimum pool size requirement is satisfied. Connections are added to the pool as needed, up to the maximum pool size specified (100 is the default).


1 Answers

In scenarios like this, the first step is always to start to with the DB.

If you have queries that are slow to respond, those queries should appear in the slow logs. Take a look at the official docs for profiling. The default value for "slow" queries is about 100ms, so if your slow queries are because of the DB, you will see evidence there.

Additionally, take a look at the graphs for the DB. By "the graphs", I mean your Nagios/Cacti/Zabbix/ServerDensity/MMS charts of what the server is doing. If you do not have these, start there. Tweaking the connection pool size is useless if you don't actually know how many connections you have or what your CPU looks like.

Any good tools or strategies to confirm or rule out the driver connection pool size as a bottleneck?

Once you have ruled out the DB and you have configured monitoring then you can open up the connection pool size. Once you have all of this in place. You will be able to tweak the pool size and ensure that you have (a) solved the problem and (b) not caused more problems.

The whole cycle is important.

If you muck with the connection pool but you're not watching the slow logs and the total connections then you'll just cause more problems.

like image 67
Gates VP Avatar answered Oct 04 '22 14:10

Gates VP