MemoryStream instead of byte[] for files added as resources?

Question

I have a .NET assembly to which i've added a number of files as resources (binary, >500KB each). I have previously been accessing these using the ResourceManager.GetObject() method on the auto-generated Resources class, which returns a byte[].

For performance and syntax reasons, I would rather operate on these binary resources as streams instead of byte-arrays. I've found that, by manually editing the .resx file and changing the name of the class in the <value> element from System.Byte[] to System.IO.MemoryStream, i'm able to use the ResourceManager.GetStream() method to successfully access the resources as streams, e.g.

<data name="MyFile" type="System.Resources.ResXFileRef, System.Windows.Forms">
    <value>..\Resources\MyFile.ext;System.Byte[], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</data>

becomes:

<data name="MyFile" type="System.Resources.ResXFileRef, System.Windows.Forms">
    <value>..\Resources\MyFile.ext;System.IO.MemoryStream, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</value>
</data>

The only drawback to this approach is that Visual Studio always adds new file resources in the byte[] form. Is there a way that I can get it to set the type to MemoryStream for me?

Answer 1

You can create an extension method on ResourceManager.

public static class ResourceExtensions
{
    public static MemoryStream GetMemoryStream(this ResourceManager resourceManager, String name) {
        object resource = resourceManager.GetObject(name);

        if (resource is byte[]) {
            return new MemoryStream((byte[])resource);
        }
        else {
            throw new System.InvalidCastException("The specified resource is not a binary resource.");
        }
    }
}

Calling

ResourceManager resourceManager = Properties.Resources.ResourceManager;
MemoryStream stream = resourceManager.GetMemoryStream("binaryResource");

Though, this seems just as well.

MemoryStream stream = new MemoryStream(Properties.Resources.SomeBinaryResource);

I'm not sure if I would modify the resource file since they are fragile to change, and I'm sure there are scenarios where Visual Studio will overwrite changes.

The problem with this, as per your concern, is that this creates a copy of the data into memory, creating a memory footprint. For lightweight resources that will be short lived, its a non-issue, but it can be a large concern.

The answer is short: You cannot avoid the memory footprint using a ResourceManager. The problem is that both ResourceManager.GetObject(String) and ResourceManager.GetStream(String) create a copy of the data. Even though GetStream(String) returns an UnmanagedMemoryStream, it actually makes calls to GetObject(String) internally, and a copy is still created. If you debug your application and profile the memory, you will see that the memory is still allocated.

I tried multiple ways to get around this by using pointers in an unsafe context, and reflection and nothing worked. ResourceManager is just not that flexible or optimized.

I was able to find a solution however, but it requires you to use Embedded Resources. This doesn't change anything, other than you setting the build action of your resource files to Embedded Resource for the Build Action. Using this, you can use reflection to create an UnmanagedMemoryStream that does not create a copy of the data.

private UnmanagedMemoryStream GetUnmanagedMemoryStream(String embeddedResourceName) {
    Assembly assembly = Assembly.GetExecutingAssembly();

    string[] resourceNames = assembly.GetManifestResourceNames();
    string resourceName = resourceNames.SingleOrDefault(resource => resource.EndsWith(embeddedResourceName, StringComparison.InvariantCultureIgnoreCase));

    if (resourceName != null) {
        return (UnmanagedMemoryStream)assembly.GetManifestResourceStream(resourceName);
    }
    else {
        throw new System.ArgumentException("The specified embedded resource could not be found.", "embeddedResourceName");
    }
}

I did not test this extensively, but it does work. My test data was a small 17 megabyte file. The working set memory of my test application starts at around 50 megabytes, and after retrieving the resource into the stream, does not change. When using the ResourceManager it would increase the working set immediately by the size of the resource.

You will probably need to swap out the call to EndsWith that checks for the proper resource name in the manifest, because the names of the resources are slightly different than accessing it directly through the ResourceManager.

I am actually disappointed that I was not able to find a solution using the existing ResourceManager, but it just isn't flexible enough.

Edit I wrote an in-depth blog article here about this subject.