Unsafe Per Pixel access, 30ms access for 1756000 pixels

Question

So I've been sharing some thoughts on the above topic title on my website about fast, unsafe pixel access. A gentlemen gave me a rough example of how he'd do it in C++, but that doesn't help me in C# unless I can interop it, and the interop is fast as well. I had found a class in the internet that was written using MSDN help, to unsafely access pixels. The class is exceptionally fast, but it's not fast enough. Here's the class:

using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using System.Drawing.Imaging;

namespace DCOMProductions.Desktop.ScreenViewer {
public unsafe class UnsafeBitmap {
    Bitmap bitmap;

    // three elements used for MakeGreyUnsafe
    int width;
    BitmapData bitmapData = null;
    Byte* pBase = null;

    public UnsafeBitmap(Bitmap bitmap) {
        this.bitmap = new Bitmap(bitmap);
    }

    public UnsafeBitmap(int width, int height) {
        this.bitmap = new Bitmap(width, height, PixelFormat.Format32bppArgb);
    }

    public void Dispose() {
        bitmap.Dispose();
    }

    public Bitmap Bitmap {
        get {
            return (bitmap);
        }
    }

    private Point PixelSize {
        get {
            GraphicsUnit unit = GraphicsUnit.Pixel;
            RectangleF bounds = bitmap.GetBounds(ref unit);

            return new Point((int)bounds.Width, (int)bounds.Height);
        }
    }

    public void LockBitmap() {
        GraphicsUnit unit = GraphicsUnit.Pixel;
        RectangleF boundsF = bitmap.GetBounds(ref unit);
        Rectangle bounds = new Rectangle((int)boundsF.X,
      (int)boundsF.Y,
      (int)boundsF.Width,
      (int)boundsF.Height);

        // Figure out the number of bytes in a row
        // This is rounded up to be a multiple of 4
        // bytes, since a scan line in an image must always be a multiple of 4 bytes
        // in length. 
        width = (int)boundsF.Width * sizeof(Pixel);
        if (width % 4 != 0) {
            width = 4 * (width / 4 + 1);
        }
        bitmapData =
      bitmap.LockBits(bounds, ImageLockMode.ReadWrite, PixelFormat.Format32bppArgb);

        pBase = (Byte*)bitmapData.Scan0.ToPointer();
    }

    public Pixel GetPixel(int x, int y) {
        Pixel returnValue = *PixelAt(x, y);
        return returnValue;
    }

    public void SetPixel(int x, int y, Pixel colour) {
        Pixel* pixel = PixelAt(x, y);
        *pixel = colour;
    }

    public void UnlockBitmap() {
        bitmap.UnlockBits(bitmapData);
        bitmapData = null;
        pBase = null;
    }
    public Pixel* PixelAt(int x, int y) {
        return (Pixel*)(pBase + y * width + x * sizeof(Pixel));
    }
}

}

Basically what I am doing is copying the entire screen and comparing each pixel to and old copy. On a 1680x1050 bitmap, this takes approximately 300 milliseconds using the following code.

private Bitmap GetInvalidFrame(Bitmap frame) {
        Stopwatch sp = new Stopwatch();
        sp.Start();

        if (m_FrameBackBuffer == null) {
            return frame;
        }

        Int32 pixelsToRead = frame.Width * frame.Height;
        Int32 x = 0, y = 0;

        UnsafeBitmap unsafeBitmap = new UnsafeBitmap(frame);
        UnsafeBitmap unsafeBuffBitmap = new UnsafeBitmap(m_FrameBackBuffer);
        UnsafeBitmap retVal = new UnsafeBitmap(frame.Width, frame.Height);

        unsafeBitmap.LockBitmap();
        unsafeBuffBitmap.LockBitmap();
        retVal.LockBitmap();

        do {
            for (x = 0; x < frame.Width; x++) {
                Pixel newPixel = unsafeBitmap.GetPixel(x, y);
                Pixel oldPixel = unsafeBuffBitmap.GetPixel(x, y);

                if (newPixel.Alpha != oldPixel.Alpha || newPixel.Red != oldPixel.Red || newPixel.Green != oldPixel.Green || newPixel.Blue != oldPixel.Blue) {
                   retVal.SetPixel(x, y, newPixel);
                }
                else {
                    // Skip pixel
                }
            }

            y++;
        } while (y != frame.Height);

        unsafeBitmap.UnlockBitmap();
        unsafeBuffBitmap.UnlockBitmap();
        retVal.UnlockBitmap();

        sp.Stop();

        System.Diagnostics.Debug.WriteLine(sp.Elapsed.Milliseconds.ToString());

        sp.Reset();

        return retVal.Bitmap;
    }

Is there any possible method/means/approach that I could speed this up to about 30ms? I can copy the screen in about 30ms using Graphics.CopyFromScreen(), so that produces approximately 30 frames each second. However, a program only runs as fast as its slower counterpart, so the 300ms delay in GetInvalidFrame, slows this down to about 1 - 3 frames each second. This isn't good for a meeting software.

Any advice, approaches, pointers in the right direction would be absolutely wonderful! Also, the code that is used to draw the bitmap on the client-side is below as well.

To comment on Dmitriy's answer/comment:

#region RootWorkItem

    private ScreenClient m_RootWorkItem;
    /// <summary>
    /// Gets the RootWorkItem
    /// </summary>
    public ScreenClient RootWorkItem {
        get {
            if (m_RootWorkItem == null) {
                m_RootWorkItem = new ScreenClient();
                m_RootWorkItem.FrameRead += new EventHandler<FrameEventArgs>(RootWorkItem_FrameRead);
            }
            return m_RootWorkItem;
        }
    }

    #endregion

    private void RootWorkItem_FrameRead(Object sender, FrameEventArgs e) {
        if (e.Frame != null) {
            if (uxSurface.Image != null) {
                Bitmap frame = (Bitmap)uxSurface.Image;

                Graphics g = Graphics.FromImage(frame);
                g.DrawImage(e.Frame, 0, 0); // Draw only updated pixels

                uxSurface.Image = frame;
            }
            else {
                uxSurface.Image = e.Frame; // Draw initial, full image
            }
        }
        else {
            uxSurface.Image = null;
        }
    }

Answer 1

Unsafe approach with usage of integers instead of Pixels and single loop:

private static Bitmap GetInvalidFrame(Bitmap oldFrame, Bitmap newFrame)
{
    if (oldFrame.Size != newFrame.Size)
    {
        throw new ArgumentException();
    }
    Bitmap result = new Bitmap(oldFrame.Width, oldFrame.Height, oldFrame.PixelFormat);

    Rectangle lockArea = new Rectangle(Point.Empty, oldFrame.Size);
    PixelFormat format = PixelFormat.Format32bppArgb;
    BitmapData oldData = oldFrame.LockBits(lockArea, ImageLockMode.ReadOnly, format);
    BitmapData newData = newFrame.LockBits(lockArea, ImageLockMode.ReadOnly, format);
    BitmapData resultData = result.LockBits(lockArea, ImageLockMode.WriteOnly, format);

    int len = resultData.Height * Math.Abs(resultData.Stride) / 4;

    unsafe
    {
        int* pOld = (int*)oldData.Scan0;
        int* pNew = (int*)newData.Scan0;
        int* pResult = (int*)resultData.Scan0;

        for (int i = 0; i < len; i++)
        {
            int oldValue = *pOld++;
            int newValue = *pNew++;
            *pResult++ = oldValue != newValue ? newValue : 0 /* replace with 0xff << 24 if you need non-transparent black pixel */;
            // *pResult++ = *pOld++ ^ *pNew++; // if you can use XORs.
        }
    }

    oldFrame.UnlockBits(oldData);
    newFrame.UnlockBits(newData);
    result.UnlockBits(resultData);

    return result;
}

I think you really can use XORed frames here and I hope that this can have better performance on both sides.

    private static void XorFrames(Bitmap leftFrame, Bitmap rightFrame)
    {
        if (leftFrame.Size != rightFrame.Size)
        {
            throw new ArgumentException();
        }

        Rectangle lockArea = new Rectangle(Point.Empty, leftFrame.Size);
        PixelFormat format = PixelFormat.Format32bppArgb;
        BitmapData leftData = leftFrame.LockBits(lockArea, ImageLockMode.ReadWrite, format);
        BitmapData rightData = rightFrame.LockBits(lockArea, ImageLockMode.ReadOnly, format);

        int len = leftData.Height * Math.Abs(rightData.Stride) / 4;

        unsafe
        {
            int* pLeft = (int*)leftData.Scan0;
            int* pRight = (int*)rightData.Scan0;

            for (int i = 0; i < len; i++)
            {
                *pLeft++ ^= *pRight++;
            }
        }

        leftFrame.UnlockBits(leftData);
        rightFrame.UnlockBits(rightData);
    }

You can use this procedure on both sides in following way:
On server side you need to evaluate difference between old and new frame, send it to client and replace old frame by new. The server code should look something like this:

  XorFrames(oldFrame, newFrame); // oldFrame ^= newFrame
  Send(oldFrame); // send XOR of two frames
  oldFrame = newFrame;

On client side you need to update your current frame with xor frame recieved from server:

  XorFrames((Bitmap)uxSurface.Image, e.Frame);

Answer 2

Here: Utilizing the GPU with c# there are mentioned some librarys for using the GPU from C#.

Answer 3

Yes, you can do so by using unsafe code.

BitmapData d = l.LockBits(new Rectangle(0, 0, l.Width, l.Height), ImageLockMode.ReadOnly,l.PixelFormat);
IntPtr scan = d.Scan0;
unsafe
{
    byte* p = (byte*)(void*)scan;
    //dostuff               
}

Check out http://www.codeproject.com/KB/GDI-plus/csharpgraphicfilters11.aspx for some basic examples of this kind of stuff. My code is based on that.

Note: One of the reasons this will be much faster than yours is that you are separately comparing each channel instead of just comparing the entire byte using one operation. Similarly, changing PixelAt to give you a byte to facilitate this would probably give you an improvement.

Answer 4

Instead of checking each and every pixel, you can just perform a basic memory compare of the 2 bitmaps. In C, something like memcmp().

This would give you a much quicker test to let you know that the images are the same or not. Only when you know they are different do you need to resort to the more expensive code that will help you determine where they are different (if you even need to know that).

I am not a C# person though, so I don't know how easy it is to get access to the raw memory.