I created an AWS Lambda function that:
Locally, using Node, I can successfully connect to the Redshift instance via JDBC, and execute a query.
var conString = "postgresql://USER_NAME:PASSWORD@JDBC_URL”;
var client = new pg.Client(conString);
client.connect(function(err) {
if(err) {
console.log('could not connect to redshift', err);
}
// omitted due to above error
However, when I execute the function on AWS Lambda (where it's wrapped in a async#waterfall block), AWS Cloudwatch logs tells me that the AWS Lambda function timed out after 60 seconds.
Any ideas on why my function is not able to connect?
Create a private Amazon Redshift cluster selecting the VPC and subnet group that you just created. 4. Create a new secret for Amazon Redshift with AWS Secrets Manager. Name your secret "redshift". To create a Lambda function that queries your Amazon Redshift cluster, perform the following steps: 1. Open the Lambda console. 2.
To solve the "Task timed out after X seconds" error in AWS lambda, you have to: 1 Increase the function's timeout. The default value is 3 seconds , the maximum is 15 minutes. 2 Increase the function's memory. By default it's set to 128 Mb which is way too low and ads onto the function's execution... More ...
Turns out that switching to Redshift temporary credentials, you need AWS Lambda to be able to access Redshift APIs that are not available by default in your VPC. The options are to route either through ENI or through NAT.
I have very similar setup, with the difference being our redshift clusters have public ip, for reasons different than lambda handling, and was changed from private to public at later time. I also use python, but shouldn't really matter. If you have aws support in your plan, I would use it.
I find it's either you open your Redshift security group public to all sources, or none. Because a Lambda function isn't running on a fixed address or even a fixed range of IP addresses, which is completely transparent to users (AKA server-less).
I just saw Amazon announced the new Lambda feature to support VPC yesterday. I guess if we can run a Redshift cluster in a VPC, this could solve the problem.
If you are using serverless-framework v1.5.0, you should add:
iamRoleStatements:
- Effect: Allow
Action:
- ec2:CreateNetworkInterface
Resource: '*'
- Effect: Allow
Action:
- ec2:DeleteNetworkInterface
- ec2:DescribeNetworkInterfaces
Resource: 'arn:aws:ec2:${self:provider.region}:*:network-interface/*'
Also should add all securityGroupIds to Inbounds Rules, like below:
More info: https://serverless.com/framework/docs/providers/aws/guide/functions/#vpc-configuration
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With