Efficient way to send images via WCF?

Question

I am learning WCF, LINQ and a few other technologies by writing, from scratch, a custom remote control application like VNC. I am creating it with three main goals in mind:

1. The server will provide 'remote control' on an application level (i.e. seamless windows) instead of full desktop access.
2. The client can select any number of applications that are running on the server and receive a stream of images of each of them.
3. A client can connect to more than one server simultaneously.

Right now I am using WCF to send an array of Bytes that represents the window being sent:

using (var ms = new MemoryStream()) {
    window.GetBitmap().Save(ms, ImageFormat.Jpeg);
    frame.Snapshot = ms.ToArray();
}

GetBitmap implementation:

var wRectangle = GetRectangle();
var image = new Bitmap(wRectangle.Width, wRectangle.Height);
var gfx = Graphics.FromImage(image);

gfx.CopyFromScreen(wRectangle.Left, wRectangle.Top, 0, 0, wRectangle.Size, CopyPixelOperation.SourceCopy);

return image;

It is then sent via WCF (TCPBinding and it will always be over LAN) to the client and reconstructed in a blank windows form with no border like this:

using (var ms = new MemoryStream(_currentFrame.Snapshot))
{
    BackgroundImage = Image.FromStream(ms);
}

I would like to make this process as efficient as possible in both CPU and memory usage with bandwidth coming in third place. I am aiming to have the client connect to 5+ servers with 10+ applications per server.

Is my existing method the best approach (while continuing to use these technologies) and is there anything I can do to improve it?

Ideas that I am looking into (but I have no experience with):

• Using an open source graphics library to capture and save the images instead of .Net solution.
• Saving as PNG or another image type rather than JPG.
• Send image deltas instead of a full image every time.
• Try and 'record' the windows and create a compressed video stream instead of picture snapshots (mpeg?).

Answer 1

You should be aware for this points:

• Transport: TCP/binary message encoding will be fastest way to transfer your image data
• Image capture: you can rely on P/Invoke to access your screen data, as this can be faster and more memory consuming. Some examples: Capturing the Screen Image in C# [P/Invoke], How to take a screen shot using .NET [Managed] and Capturing screenshots using C# (Managed)
• You should to reduce your image data before send it;
  • choose your image format wisely, as some formats have native compression (as JPG)
  • an example should be Find differences between images C#
  • sending only diff image, you can crop it and just send non-empty areas
• Try to inspect your WCF messages. This will help you to understand how messages are formatted and will help you to identify how to make that messages smaller.

Just after passing through all this steps and being satisfied with your final code, you can download VncSharp source code. It implements the RFB Protocol (Wikipedia entry), "a simple protocol for remote access to graphical user interfaces. Because it works at the framebuffer level it is applicable to all windowing systems and applications, including X11, Windows and Macintosh. RFB is the protocol used in VNC (Virtual Network Computing)."

Answer 2

I worked on a similar project a while back. This was my general approach:

• Rasterized the captured bitmap to tiles of 32x32
• To determine which tiles had changed between frames I used unsafe code to compare them 64-bits at a time
• On the set of delta tiles I applied one of the PNG filters to improve compressability and had the best results with the Paeth filter
• Used DeflateStream to compress the filtered deltas
• Used BinaryMessageEncoding custom binding to the service to transmit the data in Binary in stead of the default Base64 encoded version

Some client-side considerations. When dealing with large amounts of data being transferred through a WCF service I found that some parameters of the HttpTransportBinding and the XmlDictionaryRenderQuotas were set to pretty conservative values. So you will want to increase them.

Answer 3

Check out this: Large Data and Streaming (WCF)

Answer 4

The fastest way to send data between client/server is to send a byte array, or several byte arrays. That way WCF don't have to do any custom serialization on your data.

That said. You should use the new WPF/.Net 3.5 library to compress your images instead of the ones from System.Drawing. The functions in the System.Windows.Media.Imaging namespace are faster than the old ones, and can still be used in winforms.

In order to know if compression is the way to go you will have to benchmark your scenario to know how the compression/decompression time compares to transferring all the bytes uncompressed.

If you transfer the data over internet, then compression will help for sure. Between components on the same machine or on a LAN, the benefit might not be so obvious.

You could also try compressing the image, then chunk the data and send asynchronously with a chunk id which you puzzle together on the client. Tcp connections start slow and increase in bandwidth over time, so starting two or four at the same time should cut the total transfer time (all depending on how much data you are sending). Chunking the compressed images bytes is also easier logic wise compared to doing tiles in the actual images.

Summed up: System.Windows.Media.Imaging should help you both cpu and bandwidth wise compared to your current code. Memory wise I would guess about the same.