Example of IPC between two process with opencv (cv::Mat object) c++ as server and python as client

Question

I am trying to build an application. where a webcam video feed is captured by a c++ application. And I am building an analysis tool (that uses python), needs to access that specific frame from the video feed, captured from the c++ application and need to do analysis for a specific frame.

During googling and my friends suggest me to use IPC, where you treat C++ code as server and python as a client. I did find some reference code on the web, found few but very difficult to understand the process. Could someone explain

1. How can I do it with IPC (shared memory or with messages)?
2. Any example or reference code would be of great help.
3. Also any useful link to kickstart of noobs like me will be of great use.

Thanks in advance.

Answer 1

Here's a very simple example using Redis to send an image from a C++ server to a Python, or command-line client.

So, the server (using hiredis) looks like this:

#include <iostream>
#include <opencv2/opencv.hpp>
#include <cstdio>
#include <hiredis.h>

int
main(int argc,char*argv[])
{
   const int width = 640;
   const int height= 480;
   cv::Mat img(width,height, CV_8UC3, cvScalar(0,255,255));

   // Redis setup
   redisContext *c;
   redisReply *reply;
   const char *hostname = "localhost";
   int port = 6379;

   struct timeval timeout = { 2, 0 }; // 2 seconds
   c = redisConnectWithTimeout(hostname, port, timeout);
   if (c == NULL || c->err) {
       std::cerr << "Something bad happened" << std::endl;
       exit(1);
   }

   // Store Mat in Redis
   reply = (redisReply*)redisCommand(c,"SET image %b",(char*)img.data,height*width*3);
   freeReplyObject(reply);
}

No, it's not production-quality code, and yes, I could have transmitted the width and height and number of channels and bytespersample, but I wanted to keep it nice and simple and obvious.

---

A Python client that reads that image and displays it looks like this:

#!/usr/bin/env python3

import cv2
import redis
import numpy as np

if __name__ == '__main__':
    # Redis connection
    r = redis.Redis(host='localhost', port=6379)

    data = r.get('image')
    img = np.frombuffer(data, dtype=np.uint8).reshape(480,640,3)

    print(f'Received image')
    cv2.imshow('image', img)
    key = cv2.waitKey(0)

---

And if you want to grab the image from Redis in the Terminal, you can just do:

redis-cli get image > image.raw

Or you can grab that and convert to a PNG and display it easily with ImageMagick:

redis-cli get image | convert -depth 8 -size 640x480 bgr:- result.png

Of course, all this works across networks too, so you can send and receive to a Redis instance anywhere on your network, just by specifying the IP address. In the Terminal that would be:

redis-cli -h x.x.x.x get image

You could also JPEG-encode or PNG-encode your image prior to sending to save network bandwidth and Redis memory.

You could also set an expiry time, or time-to-live on images so they get deleted after a certain time.

You could also keep a list of the latest N images of a video, for example.

You could also implement client and/or server in Ruby, swift, C#, Java, PHP, Perl... see Redis bindings.