Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

C# code to GZip and upload a string to Amazon S3

I currently use the following code to retrieve and decompress string data from Amazon C#:

GetObjectRequest getObjectRequest = new GetObjectRequest().WithBucketName(bucketName).WithKey(key);

using (S3Response getObjectResponse = client.GetObject(getObjectRequest))
{    
    using (Stream s = getObjectResponse.ResponseStream)
    {
        using (GZipStream gzipStream = new GZipStream(s, CompressionMode.Decompress))
        {
            StreamReader Reader = new StreamReader(gzipStream, Encoding.Default);    
            string Html = Reader.ReadToEnd();
            parseFile(Html);
        }    
    }
}

I want to reverse this code so that I can compress and upload string data to S3 without being written to disk. I tried the following, but I am getting an Exception:

using (AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client(AWSAccessKeyID, AWSSecretAccessKeyID))
{
    string awsPath = AWSS3PrefixPath + "/" + keyName+ ".htm.gz";

    byte[] buffer = Encoding.UTF8.GetBytes(content);
    using (MemoryStream ms = new MemoryStream())
    {
        using (GZipStream zip = new GZipStream(ms, CompressionMode.Compress))
        {
            zip.Write(buffer, 0, buffer.Length);

            PutObjectRequest request = new PutObjectRequest();
            request.InputStream = ms;
            request.Key = awsPath;
            request.BucketName = AWSS3BuckenName;

            using (S3Response putResponse = client.PutObject(request))
            {
                //process response
            }
        }
    }
}

The exception I am getting is:

Cannot access a closed Stream.

What am I doing wrong?

EDIT:

The exception is occuring on the closing bracket of using (GZipStream zip

Stack trace:

at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Compression.DeflateStream.Dispose(Boolean disposing) at System.IO.Stream.Close() at System.IO.Compression.GZipStream.Dispose(Boolean disposing) at System.IO.Stream.Close()

like image 785
BigJoe714 Avatar asked May 09 '10 17:05

BigJoe714


1 Answers

You need to flush and close the GZipStream and reset the Position of the MemoryStream to 0 before using it as input to the request:

MemoryStream ms = new MemoryStream();

using (GZipStream zip = new GZipStream(ms, CompressionMode.Compress, true))
{
    byte[] buffer = Encoding.UTF8.GetBytes(content);
    zip.Write(buffer, 0, buffer.Length);
    zip.Flush();
}

ms.Position = 0;

PutObjectRequest request = new PutObjectRequest();
request.InputStream = ms;
request.Key = AWSS3PrefixPath + "/" + keyName+ ".htm.gz";
request.BucketName = AWSS3BuckenName;

using (AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client(
                             AWSAccessKeyID, AWSSecretAccessKeyID))
using (S3Response putResponse = client.PutObject(request))
{
    //process response
}

It might also be possible to use the GZipStream as input if you first fill the MemoryStream with the data, but I've never tried this yet.

like image 125
dtb Avatar answered Sep 18 '22 16:09

dtb