C++ using 7zip.dll

Question

I'm developing an app which will need to work with different types of archives. As many of the archive types as possible is good. I have choosen a 7zip.dll as an engine of archive-worker. But there is a problem, does anyone knows how to uncompress a file from archive to memory buffer? As I see, 7zip.dll supports only uncompressing to hard disk. Also, it would be nice to load archive from memory buffer. Has anyone tried to do something like that?

Answer 1

Not sure if I completely understand your needs (for example, don't you need the decompressed file on disk?).

I was looking at LZMA SDK 9.20 and its lzma.txt readme file, and there are plenty of hints that decompression to memory is possible - you may just need to use the C API rather than the C++ interface. Check out, for example, the section called Single-call Decompressing:

When to use: RAM->RAM decompressing
Compile files: LzmaDec.h + LzmaDec.c + Types.h
Compile defines: no defines
Memory Requirements:
  - Input buffer: compressed size
  - Output buffer: uncompressed size
  - LZMA Internal Structures: state_size (16 KB for default settings)

Also, there is this function:

SRes LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, SizeT *destLen,
    const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode, ELzmaStatus *status);

You can utilize these by memory-mapping the archive file. To the best of my knowledge, if your process creates a memory-mapped file with exclusive access (so no other process can access it) and does no explicit flushing, all changes to the file will be kept in memory until the mapping is destroyed or the file closed. Alternatively, you could just load the archive contents in memory.

For the sake of completeness, I hacked together several examples into a demo of using memory mapping in Windows.

#include <stdio.h>
#include <time.h>
#include <Windows.h>
#include <WinNT.h>

// This demo will limit the file to 4KiB
#define FILE_SIZE_MAX_LOWER_DW 4096
#define FILE_SIZE_MAX_UPPER_DW 0
#define MAP_OFFSET_LOWER_DW 0
#define MAP_OFFSET_UPPER_DW 0
#define TEST_ITERATIONS 1000
#define INT16_SIZE 2

typedef short int int16;

// NOTE: This will not work for Windows less than XP or 2003 Server!
int main()
{
    HANDLE  hFile, hFileMapping;
    PBYTE mapViewStartAddress;
    // Note: with no explicit security attributes, the process needs to have
    // the necessary rights (e.g. read, write) to this location.
    LPCSTR path = "C:\\Users\\mcmlxxxvi\\Desktop\\test.dat";

    // First, open a file handle.
    hFile = CreateFile(path,
                       GENERIC_READ | GENERIC_WRITE,    // The file is created with Read/Write permissions
                       FILE_SHARE_READ,                 // Set this to 0 for exclusive access
                       NULL,                            // Optional security attributes
                       CREATE_ALWAYS,                   // File is created if not found, overwritten otherwise
                       FILE_ATTRIBUTE_TEMPORARY,        // This affects the caching behaviour
                       0);                              // Attributes template, can be left NULL
    if ((hFile) == INVALID_HANDLE_VALUE)
    {
        fprintf(stderr, "Unable to open file");
        return 1;
    }

    // Then, create a memory mapping for the opened file.
    hFileMapping = CreateFileMapping(hFile,                     // Handle for an opened file
                                     NULL,                      // Optional security attributes
                                     PAGE_READWRITE,            // File can be mapped for Read/Write access
                                     FILE_SIZE_MAX_UPPER_DW,    // Maximum file size split in DWORDs.
                                     FILE_SIZE_MAX_LOWER_DW,    // NOTE: I may have these two mixed up!
                                     NULL);                     // Optional name
    if (hFileMapping == 0)
    {
        CloseHandle(hFile);
        fprintf(stderr, "Unable to open file for mapping.");
        return 1;
    }

    // Next, map a view (a continuous portion of the file) to a memory region
    // The view must start and end at an offset that is a multiple of
    // the allocation granularity (roughly speaking, the machine page size).
    mapViewStartAddress = (PBYTE)MapViewOfFile(hFileMapping,                    // Handle to a memory-mapped file
                                               FILE_MAP_READ | FILE_MAP_WRITE,  // Maps the view for Read/Write access
                                               MAP_OFFSET_UPPER_DW,             // Offset in the file from which
                                               MAP_OFFSET_LOWER_DW,             // the view starts, split in DWORDs.
                                               FILE_SIZE_MAX_LOWER_DW);         // Size of the view (here, entire file)
    if (mapViewStartAddress == 0)
    {
        CloseHandle(hFileMapping);
        CloseHandle(hFile);
        fprintf(stderr, "Couldn't map a view of the file.");
        return 1;
    }

    // This is where actual business stuff belongs.
    // This example application does iterations of reading and writing
    // random numbers for the entire length of the file.

    int16 value;
    errno_t result = 0;

    srand((int)time(NULL));
    for (int i = 0; i < TEST_ITERATIONS; i++)
    {
        // Write
        for (int j = 0; j < FILE_SIZE_MAX_LOWER_DW / INT16_SIZE; j++)
        {
            value = rand();
            result = memcpy_s(mapViewStartAddress + j * INT16_SIZE, INT16_SIZE, &value, INT16_SIZE); 
            if (result != 0)
            {
                CloseHandle(hFileMapping);
                CloseHandle(hFile);
                fprintf(stderr, "File write error during iteration #%d, error %d", i, GetLastError());
                return 1;
            }
        }
        // Read
        SetFilePointer(hFileMapping, 0, 0, FILE_BEGIN);
        for (int j = 0; j < FILE_SIZE_MAX_LOWER_DW / sizeof(int); j++)
        {
            result = memcpy_s(&value, INT16_SIZE, mapViewStartAddress + j * INT16_SIZE, INT16_SIZE);
            if (result != 0)
            {
                CloseHandle(hFileMapping);
                CloseHandle(hFile);
                fprintf(stderr, "File read error during iteration #%d, error %d", i, GetLastError());
                return 1;
            }
        }
    }

    // End business stuff

    CloseHandle(hFileMapping);
    CloseHandle(hFile);

    return 0;
}