Passing bitmap from c# to c++

Question

I have an image processing function written in C++ based on opencv. In my wpf application I have used AForge library to access a webcam and update it on UI. This the function for handling newframes.

void UI_NewFrame(object sender, NewFrameEventArgs eventArgs)
    {
        try
        {
            System.Drawing.Bitmap bitmapFrame = (Bitmap)eventArgs.Frame.Clone();

            MemoryStream ms = new MemoryStream();
            bitmapFrame.Save(ms, ImageFormat.Bmp);
            ms.Seek(0, SeekOrigin.Begin);

            BitmapImage bitmapImageFrame = new BitmapImage();
            bitmapImageFrame.BeginInit();
            bitmapImageFrame.StreamSource = ms;
            bitmapImageFrame.EndInit();
            bitmapImageFrame.Freeze();
            CurrentFrame = bitmapImageFrame;
        }
        catch (Exception ex)
        {
            Debug.WriteLine("fatal::" + ex.Message);
        }
    }

This my c++ code:

void FaceTracker(unsigned char* imageBuffer, int width, int height){    
   Mat frame(Size(width, height), CV_8UC4, imageBuffer, Mat::AUTO_STEP);
   Mat gray;
   cvtColor(frame, gray, COLOR_BGR2GRAY);
   //do more operations
}

I have two options. I have already found a code for copying data from BitmapImage to a IntPtr buffer using CopyPixels. I tested it and it works fine. But I also read I can simply pass a handle to the Bitmap using GetHBitmap. However this is not working for me. I read this How can I pass a .NET Bitmap to a native DLL? but I don't understand the part about GetDIBits. I also tried locking pixels of bitmap and passing like this:

bitmapFrame.LockBits(new Rectangle(0, 0, bitmapFrame.Width, bitmapFrame.Height), System.Drawing.Imaging.ImageLockMode.ReadWrite, 
            System.Drawing.Imaging.PixelFormat.Format24bppRgb);
NativeFunctions.FaceTracker(bitmapFrame.GetHbitmap(), bitmapFrame.Width, bitmapFrame.Height);

and simply passing HBitMap, neither works. I always get an exception stating memory access violation.

Answer 1

Depends on the scope. If you can guarantee the bitmap isn't used elsewhere, you can lock the image buffer, and then pass the pointer down to the C++ code, and unlock it afterwards.

The LockBits command returns a BitmapData class that has a pointer to the image buffer in its Scan0 property:

BitmapData bmpData = bitmapFrame.LockBits(new Rectangle(0, 0, bitmapFrame.Width, bitmapFrame.Height), System.Drawing.Imaging.ImageLockMode.ReadWrite, 
            System.Drawing.Imaging.PixelFormat.Format24bppRgb);

NativeFunctions.FaceTracker(bmpData.Scan0 , bitmapFrame.Width, bitmapFrame.Height);

bitmapFrame.UnlockBits(bmpData); //Remember to unlock!!!

Answer 2

An HBITMAP is an opaque handle to a bitmap. It is not a pixel buffer. So your two pieces of code do not match.

If you pass an HBITMAP to the native code then you need the native code to use GDI functions to obtain the pixel buffer and operate on it. Alternatively you could obtain a pixel buffer in your managed code and pass that to the native code. Whichever way you go you do need to match the two sides of the interop.

Answer 3

You can use Bitmap.Save() method to convert it to a memory stream and then send it to C++ dll.

public Image ConvertImage(Image image)
{
   MemoryStream convertedImageMemoryStream;
   using (MemoryStream sourceImageStream = new MemoryStream())
   {
       image.Save(sourceImageStream, System.Drawing.Imaging.ImageFormat.Png);
       byte[] sourceImageData = sourceImageStream.ToArray();

       // Send it to dll and get the IntPtr byte array from dll

       byte[] imageData = new byte[imInfo.size];
       Marshal.Copy(imInfo.data, imageData, 0, imInfo.size);
       if (imInfo.data != IntPtr.Zero)
           AlgorithmCpp.ReleaseMemoryFromC(imInfo.data);
       convertedImageMemoryStream = new MemoryStream(imageData);
    }
    Image processed = new Bitmap(convertedImageMemoryStream);
    return processed;
}

Then, in C++ dll use decoding such as cv::imdecode() method to get the image.

DllExport void convertToGray(unsigned char* data, int dataLen)
{
    vector<unsigned char> inputImageBytes(data, data + dataLen);
    Mat image = imdecode(inputImageBytes, CV_LOAD_IMAGE_COLOR);
    Mat processed;
    cvtColor(image, processed, CV_BGR2GRAY);
    vector<unsigned char> bytes;
    imencode(".png", processed, bytes);
    // ....

There are some pros in this way such as, 1. fewer data to transfer 2. minimum implementation effort from both C# and C++ end. 3. Does not depend on the source image type eg. color or grayscale or any other color palate type. So it's very safe to use.

The only problem is that it is CPU intensive as there are an encoding and a decoding.

Details are here.