Performance of Handling Large Arrays in a .NET Application

Question

I'm not sure whether this is an elementary issue I'm having, but I haven't been able to sort it out for a few days now...

I am currently writing a .NET library in C# (using .NET 3.5). Among other things, it includes functions for writing a triplet of Int32 arrays (a few MB each) into a remote cache (a memcached server) and for reading back these arrays. The problem was, every implementation I could come up with took ~1.2 seconds for writing the 10MB of data and another ~1.2 seconds for reading the same data, even when the memcached server was running on the local machine.

But then, to compare the performance, I replaced the write to the cache server with just a write into the clipboard, and noticed that it still took 1.2 seconds to perform the write. I have a call to the testing method (shown below) surrounded by a stopwatch start/stop for benchmarking. The method is:

public void writeCachedImageData(int[,] atoms, int[,] noAtoms, int[,] dark, int cameraID, int runID, int seqID)
{

  Clipboard.SetData(DataFormats.Serializable, atoms);
  Clipboard.SetData(DataFormats.Serializable, noAtoms);
  Clipboard.SetData(DataFormats.Serializable, dark);

}

For benchmarking, each array was 1000x1000. Basically, my question is:

1) Am I correct in concluding from this test that my bottleneck is somehow the data transfer rate from the application to anything external? and

2) If so, what might I be able to do to improve the data transfer rate from the application to either the clipboard or, ultimately, the memcached server?

Answer 1

Following the advice above, I tried to avoid any serialization, and did so by using BlockCopy:

 public void writeCachedImageData(Int16[,] atoms, int cameraID, int runID, int seqID)
    {


        byte[] bytesAtoms = new byte[2 * 1000 * 1000];

        Buffer.BlockCopy(atoms, 0, bytesAtoms, 0, bytesAtoms.Length);

        Clipboard.SetData(DataFormats.Serializable, bytesAtoms);
        Clipboard.SetData(DataFormats.Serializable, bytesAtoms);
        Clipboard.SetData(DataFormats.Serializable, bytesAtoms);


    }

Note that I changed the datatype to Int16 - this is because I realized that my application did not need the other two bytes of the int type. The combination of these two has led to a more than ten times speed-up!