Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to upload files directly to Amazon S3 from a remote server?

Tags:

Is it possible to upload a file to S3 from a remote server?

The remote server is basically a URL based file server. Example, using http://example.com/1.jpg, it serves the image. It doesn't do anything else and can't run code on this server.

It is possible to have another server telling S3 to upload a file from http://example.com/1.jpg

           upload from http://example.com/1.jpg 
server -------------------------------------------> S3 <-----> example.com
like image 796
Cory Avatar asked May 08 '12 00:05

Cory


People also ask

What is S3 direct upload?

Amazon S3 Direct Upload for Web UI is an addon that allows Web UI to make use of S3 Direct Upload to directly upload files to a transient bucket, which will then be moved by the Nuxeo server to a final bucket once the upload is finished.

Can Lambda upload file to S3?

Amazon S3 service is used for file storage, where you can upload or remove files. We can trigger AWS Lambda on S3 when there are any file uploads in S3 buckets. AWS Lambda has a handler function which acts as a start point for AWS Lambda function. The handler has the details of the events.


2 Answers

If you can't run code on the server or execute requests then, no, you can't do this. You will have to download the file to a server or computer that you own and upload from there.

You can see the operations you can perform on amazon S3 at http://docs.amazonwebservices.com/AmazonS3/latest/API/APIRest.html

Checking the operations for both the REST and SOAP APIs you'll see there's no way to give Amazon S3 a remote URL and have it grab the object for you. All of the PUT requests require the object's data to be provided as part of the request. Meaning the server or computer that is initiating the web request needs to have the data.

I have had a similar problem in the past where I wanted to download my users' Facebook Thumbnails and upload them to S3 for use on my site. The way I did it was to download the image from Facebook into Memory on my server, then upload to Amazon S3 - the full thing took under 2 seconds. After the upload to S3 was complete, write the bucket/key to a database.

Unfortunately there's no other way to do it.

like image 82
reach4thelasers Avatar answered Oct 07 '22 22:10

reach4thelasers


I think the suggestion provided is quite good, you can SCP the file to S3 Bucket. Giving the pem file will be a password less authentication, via PHP file you can validate the extensions. PHP file can pass the file, as argument to SCP command.

The only problem with this solution is, you must have your instance in AWS. You can't use this solution if your website is hosted in other Hosting Providers and you are trying to upload files straight to S3 Bucket.

like image 25
Zamiul Hasan Avatar answered Oct 07 '22 21:10

Zamiul Hasan