Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

GZipStream works when writing to FileStream, but not MemoryStream

If compress some json text, and write that it to a file using a FileStream I get the expected results. However, I do not want to write to disk. I simply want to memorystream of the compressed data.

Compression to FileStream:

string json = Resource1.json;

using (MemoryStream input = new MemoryStream(Encoding.UTF8.GetBytes(json)))
using (FileStream output = File.Create(@"C:\Users\roarker\Desktop\output.json.gz"))
{
    using (GZipStream compression = new GZipStream(output, CompressionMode.Compress))
    {
        input.CopyTo(compression);
    }
}

Above works. Below, the output memory stream is length 10 and results in an empty .gz file.

string json = Resource1.json;

using (MemoryStream input = new MemoryStream(Encoding.UTF8.GetBytes(json)))
using (MemoryStream output = new MemoryStream())
{
    using (GZipStream compression = new GZipStream(output, CompressionMode.Compress))
    {
        input.CopyTo(compression);

        byte[] bytes = output.ToArray();
    }
}

EDIT: Moving output.ToArray() outside the inner using clause seems to work. However, this closes the output stream for most usage. IE:

        using (MemoryStream input = new MemoryStream(Encoding.UTF8.GetBytes(json)))
        using (MemoryStream output = new MemoryStream())
        {
            using (GZipStream compression = new GZipStream(output, CompressionMode.Compress))
            {
                input.CopyTo(compression);
            }
            WriteToFile(output);
        }

where :

    public static void WriteToFile(Stream stream)
    {
        using (FileStream output = File.Create(@"C:\Users\roarker\Desktop\output.json.gz"))
        {
            stream.CopyTo(output);
        }
    }

This will fail on stream.CopyTo because the stream has been closed. I know I could make a new Stream from bytes of output.ToArray(), but why is this necessary? why does ToArray() work when the stream is closed?

Final Edit:

Just needed to use the contructor of the GZipStream with the leaveOpen parameter.

like image 652
Dave Avatar asked Jan 13 '16 19:01

Dave


1 Answers

You're calling ToArray() before you've closed the GZipStream... that means it hasn't had a chance to flush the final bits of its buffer. This is a common issue for compression an encryption streams, where closing the stream needs to write some final pieces of data. (Even calling Flush() explicitly won't help, for example.)

Just move the ToArray call:

using (MemoryStream input = new MemoryStream(Encoding.UTF8.GetBytes(json)))
using (MemoryStream output = new MemoryStream())
{
    using (GZipStream compression = new GZipStream(output, CompressionMode.Compress))
    {
        input.CopyTo(compression);
    }
    byte[] bytes = output.ToArray();
    // Use bytes
}

(Note that the stream will be disposed when you call ToArray, but that's okay.)

like image 184
Jon Skeet Avatar answered Oct 18 '22 06:10

Jon Skeet