AWS Lambda: Unable to access SQS Queue from a Lambda function with VPC access

Question

I have a Lambda function that needs to read messages from an SQS queue using it's URL. Then it needs to insert that data to Cassandra running on a server inside a VPC.

I am able to access the Cassandra server from my Lambda function, using it's private IP and configuring the security groups correctly.

However, I am not able to read messages from the SQS Queue. When I change the configuration of Lambda function to No VPC, then I am able to read the messages from the SQS Queue. However, with VPC settings, it just times out.

How can I overcome this ? I have checked the security group of my Lambda function has full outbound access to all IP addresses.

Answer 1

At the end of 2018, AWS announced support for SQS endpoints which provide

connectivity to Amazon SQS without requiring an internet gateway, network address translation (NAT) instance, or VPN connection.

There is a tutorial for Sending a Message to an Amazon SQS Queue from Amazon Virtual Private Cloud

See also the SQS VPC Endpoints Documentation for more information.

Its important to note that if you want to access SQS within the Lambda VPC there are a couple other things you need to do:

• Make sure to specify the SQS region in your code. For example, I had to set my endpoint_url to "https://sqs.us-west-2.amazonaws.com"
• Make sure that you have attached a "wide open" security group to the SQS VPC Interface, otherwise SQS will not work.
• Make sure that your subnets in your Lambda VPC match what you have set up for your SQS VPC Interface.

Answer 2

Some services (e.g. S3) are offering VPC endpoints to solve this particular problem but SQS is not one of them. I think the only real solution to this problem is to run a NAT inside your VPC so the network traffic from the Lambda function can be routed to the outside world.