Two separate processes sharing the same Camera feed OpenCv

Question

I have two separate processes that use a VideoCapture at the same time to get a webcam stream of images. Is there a way to use the same VideoCapture for multiple processes (in order to effectively save resources)?

I was considering using a mmap to transfer the current image from one process to another but I figure that there is a better way. Does anyone know how to share the same video feed with two processes in Opencv?

Also, would sharing the same video capture be computationally worth it? Or is having two processes that constantly get the webcam image be better in terms of resources?

Thank you for any advice.

Answer 1

The first and best option is to have the second process hook and intercept the image of the first process. This is the fastest way for both processes to have access to the image at nearly the same time. Of course one will always have it before the other.

If you choose to do the sharememory way, then the following MIGHT be of use to you:

SharedMemory.hpp:

#ifndef SHAREDMEMORY_HPP_INCLUDED
#define SHAREDMEMORY_HPP_INCLUDED

#if defined _WIN32 || defined _WIN64
    #include <windows.h>
#else
    #include <sys/types.h>
    #include <sys/mman.h>
    #include <dlfcn.h>
    #include <fcntl.h>
    #include <unistd.h>
#endif

#include <tchar.h>
#include <iostream>
#include <map>

class SharedMemory
{
    private:
        void* FromFile;
        void* hFileMap;
        void* pData;
        std::string MapName;
        std::size_t Size;
        bool Debug;
        std::map<std::string, void*> Events;

    public:
        SharedMemory(std::string MapName);
        SharedMemory(std::string MapName, std::size_t Size);
        ~SharedMemory();

        SharedMemory(const SharedMemory& Shm) = delete;
        SharedMemory(SharedMemory && Shm) = delete;
        SharedMemory& operator = (const SharedMemory& Shm) = delete;
        SharedMemory& operator = (SharedMemory && Shm) = delete;

        void* GetDataPointer();

        bool OpenMemoryMap(std::size_t Size);

        bool MapMemory(std::size_t Size);

        bool ReleaseMemory();

        bool CreateNewEvent(LPSECURITY_ATTRIBUTES lpEventAttributes, bool bManualReset, bool bInitialState, std::string EventName);

        std::uint32_t OpenSingleEvent(std::string EventName, bool InheritHandle, bool SaveHandle = false, std::uint32_t dwDesiredAccess = EVENT_ALL_ACCESS, std::uint32_t dwMilliseconds = INFINITE);

        bool SetEventSignal(std::string EventName, bool Signaled);

        bool DeleteSingleEvent(std::string EventName);

        bool DeleteAllEvents();

        void SetDebug(bool On);
};

#endif // SHAREDMEMORY_HPP_INCLUDED

SharedMemory.cpp:

#include "SharedMemory.hpp"

SharedMemory::SharedMemory(std::string MapName) : hFileMap(nullptr), pData(nullptr), MapName(MapName), Size(0), Debug(false), Events() {}
SharedMemory::SharedMemory(std::string MapName, std::size_t Size) : hFileMap(nullptr), pData(nullptr), MapName(MapName), Size(Size), Debug(false), Events() {}
SharedMemory::~SharedMemory()
{
    ReleaseMemory();
    DeleteAllEvents();
}

void* SharedMemory::GetDataPointer()
{
    void* Ptr = pData;
    return Ptr;
}

bool SharedMemory::OpenMemoryMap(std::size_t Size)
{
    this->Size = Size;

    #if defined _WIN32 || defined _WIN64
    if ((hFileMap = OpenFileMapping(FILE_MAP_ALL_ACCESS, false, MapName.c_str())) == nullptr)
    {
        if (Debug) std::cout << _T("\nCould Not Open Shared Memory Map.\n");
        return false;
    }

    if ((pData = MapViewOfFile(hFileMap, FILE_MAP_ALL_ACCESS, 0, 0, Size)) == nullptr)
    {
        if (Debug) std::cout << _T("\nCould Not Map View Of File.\n");
        CloseHandle(hFileMap);
        return false;
    }

    #else

    if ((hFileMap = open(MapName.c_str(), O_RDWR | O_CREAT, 438)) == -1)
    {
        if (Debug) std::cout << _T("\nCould Not Open Shared Memory Map.\n");
        return false;
    }

    if ((pData = mmap(nullptr, Size, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, hFileMap, 0)) == MAP_FAILED)
    {
        if (Debug) std::cout << _T("\nCould Not Map View Of File.\n");
        close(hFileMap);
        return false;
    }
    #endif

    if (Debug) std::cout << _T("\nInter-Process Communication Successful.\n");
    return true;
}

bool SharedMemory::MapMemory(std::size_t Size)
{
    this->Size = Size;

    #if defined _WIN32 || defined _WIN64
    if ((hFileMap = CreateFileMapping(INVALID_HANDLE_VALUE, nullptr, PAGE_READWRITE, 0, Size, MapName.c_str())) == nullptr)
    {
        if (Debug) std::cout << _T("\nCould Not Create Shared Memory Map.\n");
        return false;
    }

    if ((pData = MapViewOfFile(hFileMap, FILE_MAP_ALL_ACCESS, 0, 0, Size)) == nullptr)
    {
        if (Debug) std::cout << _T("\nCould Not Map View Of File.\n");
        CloseHandle(hFileMap);
        return false;
    }

    #else

    if ((hFileMap = open(MapName.c_str(), O_RDWR | O_CREAT, 438)) == -1)
    {
        if (Debug) std::cout << _T("\nCould Not Create Shared Memory Map.\n");
        return false;
    }

    if ((pData = mmap(nullptr, Size, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, hFileMap, 0)) == MAP_FAILED)
    {
        if (Debug) std::cout << _T("\nCould Not Map View Of File.\n");
        close(hFileMap);
        return false;
    }
    #endif

    if (Debug) std::cout << _T("\nMapped Shared Memory Successfully.\n");
    return true;
}

bool SharedMemory::ReleaseMemory()
{
    bool Result = false;
    #if defined _WIN32 || defined _WIN64
    if (pData)
    {
        Result = UnmapViewOfFile(pData);
        pData = nullptr;
        if (Result && Debug)
        {
            std::cout << _T("\nMemory Un-Mapped Successfully.\n");
        }
    }

    if (hFileMap)
    {
        if (CloseHandle(hFileMap))
        {
            hFileMap = nullptr;
            Result = Result && true;
            if (Debug) std::cout << _T("\nMemory Map Closed Successfully.\n");
        }
    }

    #else

    if (pData)
    {
        Result = munmap(pData, Size);
        if (!Result && Debug)
        {
            std::cout << _T("\nMemory Un-Mapped Successfully.\n");
        }
        pData = nullptr;
        return true;
    }

    if (hFileMap)
    {
        if (!close(hFileMap))
        {
            hFileMap = nullptr;
            if (Debug) std::cout << _T("\nMemory Map Closed Successfully.\n");
        }
    }
    #endif
    return Result;
}

bool SharedMemory::CreateNewEvent(LPSECURITY_ATTRIBUTES lpEventAttributes, bool bManualReset, bool bInitialState, std::string EventName)
{
    std::map<std::string, void*>::iterator it = Events.find(EventName);
    if (it != Events.end())
    {
        if (Debug)
        {
            std::cout << _T("\nCreateNewEvent Error: An Event With That Key Already Exists!\n");
        }
        return false;
    }

    Events.insert(std::pair<std::string, void*>(EventName, CreateEvent(lpEventAttributes, bManualReset, bInitialState, EventName.c_str())));
    it = Events.end();
    return ((--it)->second != nullptr);
}

std::uint32_t SharedMemory::OpenSingleEvent(std::string EventName, bool InheritHandle, bool SaveHandle, std::uint32_t dwDesiredAccess, std::uint32_t dwMilliseconds)
{
    void* hEvent = OpenEvent(dwDesiredAccess, InheritHandle, EventName.c_str());
    if (hEvent)
    {
        if (SaveHandle)
        {
            std::map<std::string, void*>::iterator it = Events.find(EventName);
            if (it != Events.end())
            {
                CloseHandle(it->second);
                it->second = hEvent;
            }
            else
                Events.insert(std::pair<std::string, void*>(EventName, hEvent));
        }
        std::uint32_t Result = WaitForSingleObject(hEvent, dwMilliseconds);
        if (!SaveHandle) CloseHandle(hEvent);
        return Result;
    }
    CloseHandle(hEvent);
    return WAIT_FAILED;
}

bool SharedMemory::SetEventSignal(std::string EventName, bool Signaled)
{
    std::map<std::string, void*>::iterator it = Events.find(EventName);
    if (it == Events.end())
    {
        if (Debug)
        {
            std::cout << _T("\nSetEventSignal Error: No Event With That Key Exists!\n");
        }
        return false;
    }
    if (Signaled) return SetEvent(it->second);
    return ResetEvent(it->second);
}

bool SharedMemory::DeleteSingleEvent(std::string EventName)
{
    std::map<std::string, void*>::iterator it = Events.find(EventName);
    if (it == Events.end()) return true;
    bool Result = CloseHandle(it->second);
    Events.erase(it);
    return Result;
}

bool SharedMemory::DeleteAllEvents()
{
    bool Result = false;
    for (std::map<std::string, void*>::iterator it = Events.begin(); it != Events.end(); ++it)
    {
        Result = Result && CloseHandle(it->second);
    }
    Events.clear();
    return Result;
}

void SharedMemory::SetDebug(bool On)
{
    Debug = On;
}

And you can use it like:

First-Process:

SharedMemory mem("OpenCVMap", 1980 * 1024 * 4); //Assuming max image size is 1980*1024*RGBA.
mem->CreateNewEvent(nullptr, true, false, "ImageReplySignal");

unsigned char* PtrToImagePixel = GetOpenCVCameraFeed();
unsigned char* MemPtr = static_cast<unsigned char*>(mem->GetDataPointer());
*reinterpret_cast<int*>(MemPtr) = GetOpenCVCameraFeedSize();
MemPtr += sizeof(int);

for (int i = 0; i < GetOpenCVCameraFeedSize(); ++i)
{
    *MemPtr += *PtrToImagePixels++;
} 
mem->SetEventSignal("ImageReplySignal", true);

Second-Process:

SharedMemory mem("OpenCVMap");
mem->OpenMemoryMap(1980 * 1024 * 4);
std::vector<unsigned char> Image;

while(true)
{
    if (mem->OpenSingleEvent("ImageReplySignal", true, true) == WAIT_OBJECT_0)
    {
        unsigned char* MemPtr = static_cast<unsigned char*>(mem->GetDataPointer());
        int size = *(reinterpret_cast<int*>(MemPtr));
        MemPtr += sizeof(int);
        Image.resize(size);

        for (int i = 0; i < size; ++i)
        {
            Image[i] = *MemPtr++;
        }

        mem->SetEventSignal("ImageReplySignal", false);
    }
}

Explanation:

First Process: First process maps a shared memory segment using "OpenCVMap" as an identifier. It also creates an event with the identifier "ImageReplySignal" so that the second process can know when to read.

Upon receiving an image, it writes the image size to the shared memory region as an integer. Then it proceeds to write the contents of the image to the memory region.

When it is finished writing, it sets the event to signaled. This way, the second process receives a signal telling it that it is ok to read.

Second Process: Second process opens the shared memory region using "OpenCVMap" as the identifier. In a loop, it constant checks if the signal is set using the identifier "ImageReplySignal". If the event is set, it reads the size from the memory region. It then proceeds to copy the data from the memory region.

Voila both processes now share this image. It is NOT necessary to copy the image OUT of the memory region in the second process. It can simply just manipulate it then and there.

Without hooking the first process, this is probably the best solution to get BOTH processes to share the "EXACT" same image/file/video/whatever else..

Anyway, it'd be best to give an idea of what you want before I can truly suggest a better solution..