Get object from AWS S3 as a stream

Question

I am trying to to figure out whether it is possbile to return some sort of stream (possibly a memory stream?) of an object I get from my AWS S3 bucket.

The S3 bucket contains a lot of different type of images, documents etc. All those should be used on my website. However, I do not want to display the path to my AWS S3 bucket.
That is why I am trying to create a stream and display the images and downloadable documents on the fly rather than with a full path. Does this make sense? :-)

I am using the C#/.NET AWS SDK.

Looking forward to hear about any ideas and directions pointed to!

public FileStream GetFile(string keyName)
{
    using (client = new AmazonS3Client(Amazon.RegionEndpoint.USEast2))
    {
        GetObjectRequest request = new GetObjectRequest
        {
            BucketName = bucketName,
            Key = keyName
        };

        using (GetObjectResponse response = client.GetObject(request))
        using (Stream responseStream = response.ResponseStream)
        using (StreamReader reader = new StreamReader(responseStream))
        {
            // The following outputs the content of my text file:
            Console.WriteLine(reader.ReadToEnd());
            // Do some magic to return content as a stream
        }
    }
}

Answer 1

In .NET 4, you can use Stream.CopyTo to copy the content of the ResponseStream (that is a Amazon.Runtime.Internal.Util.MD5Stream) to a MemoryStream.

GetObjectResponse response = await client.GetObjectAsync(bucketName, keyName);
MemoryStream memoryStream = new MemoryStream();

using (Stream responseStream = response.ResponseStream)
{
    responseStream.CopyTo(memoryStream);
}

return memoryStream;

Where client.GetObjectAsync(bucketName, keyName) is an alternative to calling GetObject with the request you are creating.

Answer 2

Even cheaper way is to use pre-signed URLs to objects in S3. That way you can return expiring URLs to your resources and do not need to do any stream copies. Very low memory is required for that so you can use a very small and cheap VM.

This approach would work for a few resources and only a few clients. With more requests you may hit AWS API limits.

using (client = new AmazonS3Client(Amazon.RegionEndpoint.USEast2))
{
    var request = new GetPreSignedUrlRequest()
    {
        BucketName = bucketName,
        Key = keyName,
        Expires = DateTime.UtcNow.AddMinutes(10),
        Verb = HttpVerb.GET, 
        Protocol = Protocol.HTTPS
    };
    var url = client.GetPreSignedURL(request);
    // ... and return url from here. 
    // Url is valid only for 10 minutes
    // it can be used only to GET content over HTTPS
    // Any other operation like POST would fail.
}