I need to capture the color RGB image from a Kinnect camera, but I want to show it in OpenCV as this is only a part of a bigger program. I know OpenCV has compatibility with OpenNI if you set the flag, but although I tried hard CMake couldn't find the path to OpenNI2 so I couldn't build OpenCV with OpenNI. Anyway I think it is good to know how to manually convert OpenNI frames to openCV frames, so I decided to follow this way.
For capturing the color frame in OpenNI I tried the following:
openni::Device device;
openni::VideoStream color;
openni::VideoFrameRef colorFrame;
rc = openni::OpenNI::initialize();
rc = device.open(openni::ANY_DEVICE);
rc = color.create(device, openni::SENSOR_COLOR);
rc = color.start();
color.readFrame(&colorFrame);
const openni::RGB888Pixel* imageBuffer = (const openni::RGB888Pixel*)colorFrame.getData();
But now I don't understand how to do the conversion to cv::Mat.
Does any anybody manged to do this?
Ok, first you should separate initialization from the loop for reading frames, this way.
Initialization
openni::Device device;
openni::VideoStream color;
openni::VideoFrameRef colorFrame;
rc = openni::OpenNI::initialize();
rc = device.open(openni::ANY_DEVICE);
rc = color.create(device, openni::SENSOR_COLOR);
rc = color.start();
Mat frame;
Now comes the main loop for reading frames. You have done almost everything, the only thing left is to copy the buffer to the openCV Mat.
Loop for reading frames
while (true)
{
color.readFrame(&colorFrame);
const openni::RGB888Pixel* imageBuffer = (const openni::RGB888Pixel*)colorFrame.getData();
frame.create(colorFrame.getHeight(), colorFrame.getWidth(), CV_8UC3);
memcpy( frame.data, imageBuffer, 3*colorFrame.getHeight()*colorFrame.getWidth()*sizeof(uint8_t) );
cv::cvtColor(frame,frame,CV_BGR2RGB); //this will put colors right
}
Following up on @Jav_Rock's answer above, for IR and Depth the solution is similar, except for the following (in place of SENSOR_COLOR
, CV_8UC3
and RGB888Pixel
, respectively):
Depth
SENSOR_DEPTH
CV_16UC1
DepthPixel
Note you'll probably want to set the PixelMode for the VideoStream's VideoFormat to PIXEL_FORMAT_DEPTH_100_UM
otherwise your depth image will look very black.
IR
SENSOR_IR
CV_16UC1
Grayscale16Pixel
Finally, note that neither depth nor IR need the cv::cvtColor
call.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With