Meteor consuming huge number of Mongo connections

Question

We've been having this issue in prod for the past few weeks.

Every few days, the number of connections to our compose.io Mongo db spikes to nearly 5000, which is the connection limit. There doesn't appear to be any specific trigger for this behavior.

The logs look like this:

2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65418 [13/Aug/2018:23:57:32.327] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65247 [13/Aug/2018:23:57:32.343] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 64999 [14/Aug/2018:00:02:32.346] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 63983 [14/Aug/2018:00:02:32.348] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 62910 [14/Aug/2018:00:02:32.350] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65460 [13/Aug/2018:23:57:32.358] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 62628 [14/Aug/2018:00:02:32.361] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 51179 [13/Aug/2018:23:57:32.366] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65482 [13/Aug/2018:23:57:32.366] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 63903 [14/Aug/2018:00:02:32.369] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65230 [13/Aug/2018:23:57:32.372] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 62848 [14/Aug/2018:00:02:32.372] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 64977 [14/Aug/2018:00:02:32.372] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 63939 [14/Aug/2018:00:02:32.374] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65192 [13/Aug/2018:23:57:32.385] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65367 [13/Aug/2018:23:57:32.393] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 63863 [14/Aug/2018:00:02:32.396] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 64927 [14/Aug/2018:00:02:32.397] ft_mongodb/1: Connection closed during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 51175 [13/Aug/2018:23:57:32.403] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 63241 [13/Aug/2018:23:57:32.411] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65483 [13/Aug/2018:23:57:32.417] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 65507 [13/Aug/2018:23:57:32.432] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 51343 [13/Aug/2018:23:57:32.437] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 63586 [13/Aug/2018:23:57:32.437] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 51427 [13/Aug/2018:23:57:32.449] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 49828 [13/Aug/2018:23:57:32.465] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:02:32.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 49739 [13/Aug/2018:23:57:32.581] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:03:21.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: syslogd.25 | 212.233.54.22 62849 [13/Aug/2018:23:58:21.377] ft_mongodb/1: Timeout during SSL handshake
2018-08-14T00:03:21.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: mongos.23  | 2018-08-14T00:03:21.824+0000 I NETWORK  [mongosMain] connection accepted from 127.0.0.1:40921 #3922079 (4941 connections now open)
2018-08-14T00:03:22.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: mongos.23  | 2018-08-14T00:03:22.469+0000 I NETWORK  [mongosMain] connection accepted from 127.0.0.1:40961 #3922080 (4942 connections now open)
2018-08-14T00:03:24.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: mongos.23  | 2018-08-14T00:03:24.246+0000 I NETWORK  [mongosMain] connection accepted from 127.0.0.1:41042 #3922081 (4943 connections now open)
2018-08-14T00:03:24.000+00:00 aws-eu-west-1-portal.9.dblayer.com mongodb320: mongos.23  | 2018-08-14T00:03:24.340+0000 I NETWORK  [mongosMain] connection accepted from 127.0.0.1:41049 #3922082 (4944 connections now open)

Compose support tells us that this is almost certainly an issue with the MongoDB driver, and that we should be using connection pooling. Is there some configuration of the Meteor connection pool that we need to implement?

Answer 1

This was an issue with Meteor 1.7, resolved by meteor update. V. 1.8 fixes the issue.