Using Sequelize with RDS getting TimeoutError: ResourceRequest timed out

Question

I have an node.js ETL system that loads data into an amazon RDS PostgreSQL instance using sequelize. All of a sudden I have started to see random timeout errors while inserting records into the database:

TimeoutError: ResourceRequest timed out
    at ResourceRequest._fireTimeout (/home/ubuntu/myproject/etl/node_modules/generic-pool/lib/ResourceRequest.js:62:17)
    at Timeout.bound (/home/ubuntu/myproject/etl/node_modules/generic-pool/lib/ResourceRequest.js:8:15)
    at ontimeout (timers.js:482:11)
    at tryOnTimeout (timers.js:317:5)
    at Timer.listOnTimeout (timers.js:277:5)

The RDS box is fairly beefy (db.t2.large) and this has not been an issue since this thing started running well over a year ago.

Based on this sequelize discussion, it seems that this is usually due to the sequelize thread pool running out of threads due to long running parallel queries. I don't think that's the case here because I am inserting the rows in series, not parallel.

Generally the advice is to increase sequelize's acquire setting, which determines how many milliseconds it will wait to see if another thread in the pool opens up to service the request. My is currently set to 30,000, which seems like a really large value. Way longer than it should take for RDS to insert 1 row into a table.

Here is my config:

pool: {
    max: 5,
    min: 0,
    acquire: 30000,
    idle: 10000,
}

Is this just an network connectivity issue between ec2 and RDS? Seems like that would be a different error.

Answer 1

1) Check your existing connection :

SELECT client_addr, count(*) FROM pg_stat_activity WHERE datname = 'YOUR_DB_NAME' GROUP BY client_addr;

2) You can increase the max pool size.

3) Sometimes you have slow queries that take more than 1 minute or wrong queries like transaction started but never ended. In that case 1 connection is always busy.