OpenCV cv::Mat to std::ifstream for base64 encoding

Question

To be honest I'm suprised nobody has run into this thus far. I'm loading a picture from OpenCV into cv::Mat, which I want to base64 encode before I send it over a socket.

For base64 I am using libb64 as it is native to Debian/Ubuntu, and easy to use and very fast. The encoding function takes as a parameter an std::ifstream, and outputs an std::ofstream.

#include <opencv2/opencv.hpp>
#include <b64/encode.h>
#include <fstream>

using namespace cv;
Mat image;
image = imread( "picture.jpg", CV_LOAD_IMAGE_COLOR );

if ( image.data )
{
    std::ifstream instream( ???, std::ios_base::in | std::ios_base::binary);
    std::ofstream outstream;        

    // Convert Matrix to ifstream
    // ...

    base64::encoder E;
    E.encode( instream, outstream );

    // Now put it in a string, and send it over a socket...
}

I don't really know how to populate the instream from the cv::Mat. Googling around, I found that I can iterate a cv::Mat, by columns and rows, and get each (pixel I am assuming) RGB values:

for ( int j = 0; j < image.rows; j++ )
{
    for ( int i = 0; i < image.cols; i++ )
    {
        unsigned char b = input [ image.step * j + i ] ;
        unsigned char g = input [ image.step * j + i + 1 ];
        unsigned char r = input [ image.step * j + i + 2 ];
    }
}

Is this the right way of going on about it? Is there some more elegant way?

Answer 1

In order to be able to send an image via HTTP, you also need to encode its width, height and type. You need to serialize the Mat into a stream and encode that stream with libb64. On the other side you need to decode that stream and deserialize the image to retrieve it.

I implemented a small test program that does this serialization and deserialization using std::stringstream as a buffer. I chose it because it extends both std::istream and std::ostream which libb64 uses.

The serialize function serializes a cv::Mat into a std::stringstream. In it, I write the image width, height, type, size of the buffer and the buffer itself.

The deserialize function does the reverse. It reads the width, height, type, size of the buffer and the buffer. It's not as efficient as it could be because it needs to allocate a temporary buffer to read the data from the stringstream. Also, it needs to clone the image so that it does not rely on the temporary buffer and it will handle its own memory allocation. I'm sure that with some tinkering it can be made more efficient.

The main function loads an image, serializes it, encodes it using libb64, then decodes it, deserializes it and displays it in a window. This should simulate what you are trying to do .

// Serialize a cv::Mat to a stringstream
stringstream serialize(Mat input)
{
    // We will need to also serialize the width, height, type and size of the matrix
    int width = input.cols;
    int height = input.rows;
    int type = input.type();
    size_t size = input.total() * input.elemSize();

    // Initialize a stringstream and write the data
    stringstream ss;
    ss.write((char*)(&width), sizeof(int));
    ss.write((char*)(&height), sizeof(int));
    ss.write((char*)(&type), sizeof(int));
    ss.write((char*)(&size), sizeof(size_t));

    // Write the whole image data
    ss.write((char*)input.data, size);

    return ss;
}

// Deserialize a Mat from a stringstream
Mat deserialize(stringstream& input)
{
    // The data we need to deserialize
    int width = 0;
    int height = 0;
    int type = 0;
    size_t size = 0;

    // Read the width, height, type and size of the buffer
    input.read((char*)(&width), sizeof(int));
    input.read((char*)(&height), sizeof(int));
    input.read((char*)(&type), sizeof(int));
    input.read((char*)(&size), sizeof(size_t));

    // Allocate a buffer for the pixels
    char* data = new char[size];
    // Read the pixels from the stringstream
    input.read(data, size);

    // Construct the image (clone it so that it won't need our buffer anymore)
    Mat m = Mat(height, width, type, data).clone();

    // Delete our buffer
    delete[]data;

    // Return the matrix
    return m;
}

void main()
{
    // Read a test image
    Mat input = imread("D:\\test\\test.jpg");

    // Serialize the input image to a stringstream
    stringstream serializedStream = serialize(input);

    // Base64 encode the stringstream
    base64::encoder E;
    stringstream encoded;
    E.encode(serializedStream, encoded);

    // Base64 decode the stringstream
    base64::decoder D;
    stringstream decoded;
    D.decode(encoded, decoded);

    // Deserialize the image from the decoded stringstream
    Mat deserialized = deserialize(decoded);

    // Show the retrieved image
    imshow("Retrieved image", deserialized);
    waitKey(0);
}