How to dynamically expand a Memory Mapped File

Question

I've used C# to solve the following requirement.. - create an app the can receive a lot of data fast - you must be able to analyse the received data while more are incoming. - use as little CPU and disk as possible

My idea for an algorithm was..

SIZE = 10MB Create a mmf with the size of SIZE On data recived:   if data can't fit mmf: increase mmf.size by SIZE   write the data to mmf 

-> The size on the disc are increased in chuncks of 10MB when the previous "room/space" are used.

How is the "increase mmf.size by SIZE" done in C#? I have found a lot of simple examples on creating mmfs and views but the only place (link) I have seen code that acutally increases the mmfs area uses code that can't compile. Any help will be greatly appriciated.

EDIT This causes an exception :

private void IncreaseFileSize() {     int theNewMax = this.currentMax + INCREMENT_SIZE;     this.currentMax = theNewMax;      this.mmf.Dispose();      this.mmf = MemoryMappedFile.CreateFromFile(this.FileName, FileMode.Create, "MyMMF", theNewMax);     this.view = mmf.CreateViewAccessor(0, theNewMax);             } 

This exception is thrown : The process cannot access the file 'C:\Users\moberg\Documents\data.bin' because it is being used by another process.

Answer 1

Once you map a file in memory, you cannot increase its size. This is a known limitation of memory mapped files.

...you must calculate or estimate the size of the finished file because file mapping objects are static in size; once created, their size cannot be increased or decreased.

One strategy would be to use chunks stored in non-persisted memory mapped files of a given size, say 1GB or 2GB. You would manage these through a top level ViewAccessor of your own design (probably doing basic passthru of the methods you need from the MemoryMappedViewAccessor).

Edit: or you could just create a non-persisted memory mapped file of a maximal size you expect to use (say 8GB to start, with a parameter to tune it on start-up of your application) and retrieve MemoryMappedViewAccessor's per logical chunk. The non-persisted file will not use physical resources until each view is requested.