My Redshift cluster is in a private VPC. I've written the following AWS Lamba in Node.js which should connect to Redshift (dressed down for this question):
'use strict';
console.log('Loading function');
const pg = require('pg');
exports.handler = (event, context, callback) => {
var client = new pg.Client({
user: 'myuser',
database: 'mydatabase',
password: 'mypassword',
port: 5439,
host: 'myhost.eu-west-1.redshift.amazonaws.com'
});
// connect to our database
console.log('Connecting...');
client.connect(function (err) {
if (err) throw err;
console.log('CONNECTED!!!');
});
};
I keep getting Task timed out after 60.00 seconds unfortunately. I see in the logs "Connecting...", but never "CONNECTED!!!".
Steps I've taken so far to get this to work:
So, does anyone have any suggestions as to what I should check?
*I should add that I am not a network expert, so perhaps I've made a mistake somewhere.
There are three reasons why retry and timeout issues occur when invoking a Lambda function with an AWS SDK: A remote API is unreachable or takes too long to respond to an API call. The API call doesn't get a response within the socket timeout.
Make sure that you confirm that there's a valid connection before attempting to use the connection. If there's not a valid connection, then create a new connection before continuing. As a test, launch an Amazon Elastic Compute Cloud (Amazon EC2) instance with the same Amazon VPC configuration as your Lambda function.
There are many reasons why a function might time out, but the most likely is that it was waiting on an IO operation to complete. Maybe it was waiting on another service (such as DynamoDB or Stripe) to respond. Within a Lambda invocation, the function might perform multiple IO operations.
The timeout is probably because your lambda in VPC cannot access Internet in order to connect to your cluster(you seem to be using the public hostname to connect). Your connection options depend on your cluster configuration. Since both your lambda function and cluster are in the same VPC, you should use the private IP of your cluster to connect to it. In your case, I think simply using the private IP should solve your problem.
Depending on whether your cluster is publicly accessible, there are some points to keep in mind.
If your cluster is configured to NOT be publicly accessible, you can use the private IP to connect to the cluster from your lambda running in a VPC and it should work.
If you have a publicly accessible cluster in a VPC, and you want to connect to it by using the private IP address from within the VPC, make sure the following VPC parameters to true/yes:
The steps to verify/change these settings are given here.
If you do not set these parameters to true
, connections from within VPC will resolve to the EIP instead of the private IP and your lambda won't be able to connect without having Internet access(which will need a NAT gateway or a NAT instance).
Also, an important note from the documentation here.
If you have an existing publicly accessible cluster in a VPC, connections from within the VPC will continue to use the EIP to connect to the cluster even with those parameters set until you resize the cluster. Any new clusters will follow the new behavior of using the private IP address when connecting to the publicly accessible cluster from within the same VPC.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With