How to close Boto S3 connection?

Question

I'm using Boto to connect to Amazon S3 in my Python program. I'm able to open a connection and upload files to a bucket. I figured I should then close the connection to release resources and, more important, to avoid any security risks from leaving an open connection hanging around. I assumed I should call the close() method. But I tested this as follows: 1. Open connection. 2. Close connection. 3. Upload file to bucket.

I figured step 3 would fail, but the upload worked! So what does close() do? If it doesn't really close the connection, what should I use in place of close()? Or is it just unnecessary to close the connection?

I've looked for the answer in the Boto tutorial, the Boto API reference, and this StackOverflow post, but no luck so far.

Thanks for your help.

Answer 1

Your step 3 worked because boto has code that will automatically re-open closed connections and retry requests on errors. There is little to gain by manually closing the boto connections because they are just HTTP connections and will close automatically after a few minutes of idle time. I wouldn't worry about trying to close them.

Answer 2

Under the covers, boto uses httplib. This client library supports HTTP 1.1 Keep-Alive, so it can and should keep the socket open so that it can perform multiple requests over the same connection.

connection.close() does not actually close the underlying sockets. Instead, it removes the reference to the underlying pool of httplib connections, which allows the garbage collector to run on them, and that's when the actual socket close happens.

Obviously, you also can let the garbage collector run by not keeping a reference to the boto connection itself. But there are performance benefits to reusing the boto connection (e.g. see the Keep-Alive note above).

Fortunately, in most cases, you don't have to call connection.close() explicitly. For more details on one case where you DO have to call close, see my answer to the StackOverflow post that's linked in the question.