Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Streaming files from amazon s3

Tags:

c#

amazon-s3

I have a problem trying to stream files from amazon s3. Basically, I have files stored on amazom s3, I can't provide direct access to these files as users need to be authenticated. I'm trying to find a way to stream files without downloading each file from amazon onto my server and then from my server to the end client. I just want to be able to stream the file direct, but it seems most of the stream reader classes in .NET are not cable of doing this. Please correct me if I am wrong.

like image 729
Syed Hussim Avatar asked Jan 28 '14 10:01

Syed Hussim


People also ask

Can you stream files from S3?

Well, we can make use of AWS S3 Select to stream a large file via it's ScanRange parameter.

How does streaming from S3 work?

You can stream the file from Amazon S3 to the client through your server without downloading the file to your server, by opening a stream to the Amazon S3 file then read from it and write on the client stream (buffer by buffer). You can use Stream.

Can S3 be used as file server?

Advantages of Mounting Amazon S3 as a File System Mounting an Amazon S3 bucket as a file system means that you can use all your existing tools and applications to interact with the Amazon S3 bucket to perform read/write operations on files and folders.

Can I read S3 file without downloading?

Reading objects without downloading them Similarly, if you want to upload and read small pieces of textual data such as quotes, tweets, or news articles, you can do that using the S3 resource method put(), as demonstrated in the example below (Gist).


1 Answers

You can stream the file from Amazon S3 to the client through your server without downloading the file to your server, by opening a stream to the Amazon S3 file then read from it and write on the client stream (buffer by buffer).

Sample Code:

byte[] buffer = new byte[BUFFER_SIZE];                
GetObjectRequest getObjRequest = new GetObjectRequest().WithBucketName(Bucket_Name).WithKey(Object_Key);

using (GetObjectResponse getObjRespone = amazonS3Client.GetObject(getObjRequest))
using (Stream amazonStream = getObjRespone.ResponseStream)
{
    int bytesReaded = 0;        
    Response.AddHeader("Content-Length", getObjRespone.ContentLength.ToString());

    while ((bytesReaded = amazonStream.Read(buffer, 0, buffer.Length)) > 0 && Response.IsClientConnected)
    {
        Response.OutputStream.Write(buffer, 0, bytesReaded);
        Response.OutputStream.Flush();
        buffer = new byte[BUFFER_SIZE];
    }
}
like image 108
Yahya Younes Avatar answered Sep 20 '22 04:09

Yahya Younes