Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is aws CLI faster than using boto3?

I have packages stored in s3 bucket. I need to read metadata file of each package and pass the metadata to program. I used boto3.resource('s3') to read these files in python. The code took few minutes to run. While if I use aws cli sync, it downloads these metafiles much faster than boto. My guess was that if I do not download and just read the meta files, it should be faster. But it isn't the case. Is it safe to say that aws cli is faster than using boto?

like image 683
Tanvi Patel Avatar asked Sep 11 '25 20:09

Tanvi Patel


1 Answers

The AWS Command-Line Interface (CLI) actually uses boto itself, so it is not 'faster'.

However, the aws s3 sync command uses threading to copy multiple files simultaneously, so the copy operation takes less elapsed time.

like image 150
John Rotenstein Avatar answered Sep 13 '25 11:09

John Rotenstein