Kinect Depth Image

Question

In my application I am getting the depth frame similar to the depth frame retrieved from Depth Basics Sample. What I don't understand is, why are there discrete levels in the image? I don't know what do I call these sudden changes in depth values. Clearly my half of my right hand is all black and my left hand seems divided into 3 such levels. What is this and how do I remove this?

When I run the KinectExplorer Sample app I get the depth as follows. This is the depth image I want to generate from the raw depth data.

I am using Microsoft Kinect SDK's (v1.6) NuiApi along with OpenCV. I have the following code:

BYTE *pBuffer = (BYTE*)depthLockedRect.pBits; //pointer to data having 8-bit jump
USHORT *depthBuffer = (USHORT*) pBuffer; //pointer to data having 16-bit jump
int cn = 4;
this->depthFinal = cv::Mat::zeros(depthHeight,depthWidth,CV_8UC4); //8bit 4 channel
for(int i=0;i<this->depthFinal.rows;i++){
   for(int j=0;j<this->depthFinal.cols;j++){
      USHORT realdepth = ((*depthBuffer)&0x0fff); //Taking 12LSBs for depth
      BYTE intensity = (BYTE)((255*realdepth)/0x0fff); //Scaling to 255 scale grayscale
      this->depthFinal.data[i*this->depthFinal.cols*cn + j*cn + 0] = intensity;
      this->depthFinal.data[i*this->depthFinal.cols*cn + j*cn + 1] = intensity;
      this->depthFinal.data[i*this->depthFinal.cols*cn + j*cn + 2] = intensity;
      depthBuffer++;
   }
}

Answer 1

The stripes that you see, are due to the wrapping of depth values, as caused by the %256 operation. Instead of applying the modulo operation (%256), which is causing the bands to show up, remap the depth values along the entire range, e.g.:

BYTE intensity = depth == 0 || depth > 4095 ? 0 : 255 - (BYTE)(((float)depth / 4095.0f) * 255.0f);

in case your max depth is 2048, replace the 4095 with 2047.

More pointers:

1. the Kinect presumably returns a 11bit value (0-2047) but you only use 8bit (0-255).
2. new Kinect versions seem to return a 12bit value (0-4096)
3. in the Kinect explorer source code, there's a file called DepthColorizer.cs where most of the magic seems to happen. I believe that this code makes the depth values so smooth in the kinect explorer - but I might be wrong.

Answer 2

I faced the same problem while I was working on a project which involved visualization of depth map. However I used OpenNI SDK with OpenCV instead of Kinect SDK libraries. The problem was same and hence the solution will work for you as it did for me.

As mentioned in previous answers to your question, Kinect Depth map is 11-bit (0-2047). While in examples, 8-bit data types are used.

What I did in my code to get around this was to acquire the depth map into a 16-bit Mat, and then convert it to 8-bit uchar Mat by using scaling options in convertTo function for Mat

First I initialize a Mat for acquiring depth data

Mat depthMat16UC1(XN_VGA_Y_RES, XN_VGA_X_RES, CV_16UC1);

Here XN_VGA_Y_RES, XN_VGA_X_RES defines the resolution of the acquired depth map.

The code where I do this is as follows:

depthMat16UC1.data = ((uchar*)depthMD.Data());
depthMat16UC1.convertTo(depthMat8UC1, CV_8U, 0.05f); 
imshow("Depth Image", depthMat8UC1);

depthMD is metadata containing the data retrieved from Kinect sensor.

I hope this helps you in some way.