Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Portable application: s3 and Google cloud storage

I want to write an application which is portable.

With "portable" I mean that it can be used to access these storages:

  • amazon s3
  • google cloud storage
  • Eucalyptus Storage

The software should be developed using Python.

I am unsure how to start, since I could not find a library which supports all three storages.

like image 937
guettli Avatar asked May 16 '18 07:05

guettli


People also ask

Is Google cloud storage compatible with S3?

XML API. The Cloud Storage XML API is interoperable with some tools and libraries that work with services such as Amazon Simple Storage Service (Amazon S3).

What is the equivalent of S3 in Google cloud?

If you're familiar with AWS, Google Storage is GCP's version of AWS Simple Storage Service (S3) and an S3 bucket would be equivalent to a Google Storage bucket across the two clouds.

Is S3 same as blob storage?

Microsoft Azure BlobAzure Blob is the Microsoft equivalent to Amazon's S3-based object storage services. Within that, a “blob” is like a bucket as the framework for retention of objects.


2 Answers

In order to connect to Google Cloud Storage from your application, you can follow the steps described here to learn how to read from a storage bucket or write into one using the application. It also includes an example on how to perform those steps.

Please be aware that this would only work for Google Cloud Storage. As for Amazon and Eucalyptus, I would not be able to provide the help you need.

UPDATE

After confirming that the answer I posted above is not what you wish for your application, we can start from @GautamKumar's answer and continue from there, so let's check the boto library. As Gautam said, boto3 is useful to connect to Amazon S3 buckets, so let's see if there are also options for Eucalyptus and Google Cloud Storage.

I could check that the same library can be used to connect to Eucalyptus, including options for both EC2 and S3. You can find some examples on how to use boto3 with Eucalyptus in this link.

Also, there is a plugin available for accessing Google Cloud Storage and performing operations with their buckets by using the XML API. You would also need gcs_oauth2_boto_plugin in order to authenticate into Cloud Storage. In the provided link you can check how to perform the different operations to access the Cloud Storage buckets and perform different operations with them.

like image 97
Rodrigo C. Avatar answered Nov 14 '22 21:11

Rodrigo C.


You can use boto3 for accessing any services of Amazon.

like image 39
Gautam Kumar Avatar answered Nov 14 '22 22:11

Gautam Kumar