Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Decompress data in Span<byte> with Deflate algorithm

I have a code that parses a binary file. It uses Span<byte> to pass chunks of data among functions to avoid unnecessary memory allocations.

Simplified example:

class Reader {
    bool IsCompressed { get; set; }
    public byte[] Data { get; set; }

    function ReadOnlySpan<byte> ReadBlock() {
        ...
    }

    public function Read() {
        if (IsCompressed) {
            Data = ???
        } else {
             Data = ReadBlock.ToArray();
        }
}

I can decompress data with Syste.IO.Compression.DeflateStream, but this class accepts only Stream as the input, so I have to convert ReadOnlySpan<byte> to a stream and thus allocate memory for byte[].

using(var stream = new MemoryStream(buffer.ToArray())) {
    using(var deflateStream = new DeflateStream(stream, CompressionMode.Decompress)) {
        Data = ... read from deflateStream
    }
}

Is there any way to pass ReadOnlySpan<byte> to the DeflateStream without allocating unnecessary memory? Or is there any other library, that can decompress data directly from ReadOnlySpan<byte>?

I am using .NET Core 2.1.0-preview1-26103-03, but I can use any other version if necessary.

like image 622
Lukas Kabrt Avatar asked Jan 28 '23 21:01

Lukas Kabrt


1 Answers

As of January 2018, there isn't any build-in support for ReadOnlySpan<byte>, but as Stephen Toub pointed out on GitHub it can be solved with the UnmanagedMemoryStream

private unsafe byte[] Decompress(ReadOnlySpan<byte> buffer, int decompressedSize) {
    fixed (byte* pBuffer = &buffer[0]) {
        using (var stream = new UnmanagedMemoryStream(pBuffer, buffer.Length)) {
            using (var deflateStream = new DeflateStream(stream, CompressionMode.Decompress)) {
                var data = ArrayPool<byte>.Shared.Rent(decompressedSize);
                deflateStream.Read(data, 0, decompressedSize);

                return data;
            }
        }
    }
}
like image 87
Lukas Kabrt Avatar answered Feb 06 '23 12:02

Lukas Kabrt