AWS Lambda python Resetting dropped connection

Question

I've an AWS Python Lambda which manages resources tags(many calls on the AWS API with boto3) for my infrastructure. The function, executed on my laptop works well and also under Lambda. But when I execute it under Lambda, all my logs(debug or error level) are not send to cloudwatch logs. Instead of I've multiple logs like this:

Resetting dropped connection: ec2.us-west-2.amazonaws.com
Resetting dropped connection: ec2.us-west-2.amazonaws.com

Google said me that it's a problem relative to urlib3 and a too high request frequency against the AWS API.

My question is, how can avoid it in Lambda to retrieve my logs in cloudwatch logs? I search a better solution than put multiple sleep inside my code. Is there a way to do that globally?

Thanks

Answer 1

Ok, finally it works. Indeed I was too aggressive on the retry process but I need to do each operation ASAP(ex: attach an EBS when it is available). In boto3, there is a cleaner way to wait until a resource become ready, than to put time.sleep(xx) in your code.

The solution is to use a boto3 waiter My advice is to set a custom value for the retry interval because per default it's 15s(too long).

waiter.config.delay = 1
waiter.config.max_attempts = 10(as you want for this param)

With this parameters you avoid lambda to send logs like this "Resetting dropped connection: ec2.us-west-2.amazonaws.com" and you execute your Lambda in the fastest way.

I'm agree with smdev who said that it's not a good idea to put sleep inside a lambda function(boto3 waiter is a sleep like) but for me it is acceptable when your Lambda function is called only few time. For example when an Autoscaling notification invokes your Lambda. If your Lambda is called directly by API Gateway(example) at a moderate or high frequency it's a very bad idea but if it's only 2-3 times per day it's ok.