How to setup OpenNI2 and NITE with libfreenect2 for skeleton tracking on OSX?

Question

I've just setup libfreenect2 on OSX 10.11.5 and I can successfully run the Protonect demo displaying RGB/IR/depth streams, but I'd like to use OpenNI/NITE for skeleton tracking as well.

I've followed the instructions in the repo's README file for OpenNI, but when I tried to run NiViewer I got this:

NiViewer 
openDevice failed:
    DeviceOpen using default: no devices found

Press any key to continue . . .

I'm currently going through libfreenect2 issues on github and documenting what I'm trying and the outcomes here. Any hints/tips are helpful.

What is the easiest way to setup and test OpenNI/NITE with libfreenect2 using a Kinect v2 on OSX ?

Turns out I was using NiTE 1.5 when I should've been using NiTE2.2 I simply copied libfreenect2-openni2.0.dylib from libfreenect2/build/lib to NiTE-MacOSX-x64-2.2/Samples/Bin/OpenNI2/Drivers/ then ran the SimpleUserTracker sample and it worked !

There is still a catch: samples that just print to the console run, but the ones that open a window and render don't work. No window is rendered, as if that thread is blocked somehow. That is when I use the integrated Intel GPU. If I use the discrete nVidia GPU the app crashes:

./UserViewer 
   124095 INFO       New log started on 2016-08-30 20:32:19
   124114 INFO       --- Filter Info --- Minimum Severity: VERBOSE
   126001 VERBOSE    No override device in configuration file
   126029 VERBOSE    Configuration has been read from '/Users/George/Downloads/Grouped/Projects/code+libs/NiTE-MacOSX-x64-2.2/Samples/Bin/OpenNI.ini'
   126034 VERBOSE    OpenNI 2.2.0 (Build 21)-MacOSX (May 29 2013 14:03:30)
   126038 VERBOSE    Using '/Users/George/Downloads/Grouped/Projects/code+libs/NiTE-MacOSX-x64-2.2/Samples/Bin/OpenNI2/Drivers' as driver path
   126042 VERBOSE    Looking for drivers in drivers repository '/Users/George/Downloads/Grouped/Projects/code+libs/NiTE-MacOSX-x64-2.2/Samples/Bin/OpenNI2/Drivers'
   161329 INFO       New log started on 2016-08-30 20:32:19
   161447 INFO       --- Filter Info --- Minimum Severity: VERBOSE
   161457 VERBOSE    Initializing USB...
   166761 INFO       USB is initialized.
   286714 INFO       New log started on 2016-08-30 20:32:19
   286762 INFO       --- Filter Info --- Minimum Severity: VERBOSE
   286781 VERBOSE    Initializing USB...
   286804 INFO       USB is initialized.
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 7 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @20:16 with serial 500722542042
[Info] [Freenect2Impl] found 1 devices
  5380695 INFO       Found device freenect2://0
  5380723 INFO       Driver: register new uri: freenect2://0
  5381065 INFO       Device connected: Microsoft Kinect (freenect2://0)
  5381101 INFO       Device state changed: Microsoft Kinect (freenect2://0) to 0
  5381115 INFO       Driver: register new uri: freenect2://0?depth-size=640x480
  5381123 INFO       Device connected: Microsoft Kinect (freenect2://0?depth-size=640x480)
  5381132 INFO       Device state changed: Microsoft Kinect (freenect2://0?depth-size=640x480) to 0
  5381139 INFO       Driver: register new uri: freenect2://0?depth-size=512x424
  5381149 INFO       Device connected: Microsoft Kinect (freenect2://0?depth-size=512x424)
  5381157 INFO       Device state changed: Microsoft Kinect (freenect2://0?depth-size=512x424) to 0
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 7 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @20:16 with serial 500722542042
[Info] [Freenect2Impl] found 1 devices
 10381916 VERBOSE    Trying to open device by URI '(NULL)'
 10381940 INFO       deiveOpen: freenect2://0
 10381948 INFO       Opening device freenect2://0
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 4*128*33792
[Info] [Freenect2DeviceImpl] opened
 12312037 VERBOSE    Initialize: Already initialized
 12314810 INFO       Device: createStream(depth)
 12314892 INFO       Freenect2Driver::Device: start()
[Info] [Freenect2DeviceImpl] starting...
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
2016-08-30 20:32:31.847 UserViewer[7575:4995528] GLUT Warning: glutInit being called a second time.
[Info] [VTRgbPacketProcessor] avg. time: 19.633ms -> ~50.9346Hz
[Info] [VTRgbPacketProcessor] avg. time: 19.6306ms -> ~50.9408Hz
[Info] [VTRgbPacketProcessor] avg. time: 19.0744ms -> ~52.4262Hz
[Info] [DepthPacketStreamParser] 417 packets were lost
Segmentation fault: 11

How can I get past this issue ?

Answer 1

For reference here are the above notes as an aswer. This is what I did to get OpenNI2/NITE2.2 skeleton tracking with libfreenect2 and it's OpenNI2 driver. I have:

1. compiled installed OpenNI2 via homebrew and compiled libfreenect2 with OpenNI2 support as per github repo's readme instructions
2. downloaded NITE2.2 (replacing Nite1.5 which I was missusing before)
3. copied the libfreenect2 OpenNI2 driver (ibfreenect2/build/lib/libfreenect2-openni2.0.dylib) to the Drivers folder where I needed to run NITE2 (e.g. NiTE-MacOSX-x64-2.2/Samples/Bin/OpenNI2/Drivers).

I have figured out the path was wrong by setting the log level to verbose in OpenNI.ini:

[Log]
; 0 - Verbose; 1 - Info; 2 - Warning; 3 - Error. Default - None
Verbosity=0
LogToConsole=1
LogToFile=1

[Device]
;Override=""

[Drivers]
; Location of the drivers specified by a relative path based on OpenNI's shared library or an absolute path.
; Path separator "/" can be used to be portable for any platforms.
; Default - OpenNI2/Drivers
Repository=OpenNI2/Drivers

Symon Kaliski followed the notes and wrote a brilliant Kinect 2 on OSX with skeleton tracking post using OpenFrameworks, be sure to check it out!

Also, once libfreenect2 is compiled with openni2 support it can even be used in Processing, simply drop libfreenect2-openni2.0.dylib in /Documents/Processing3/libraries/SimpleOpenNI/library/osx/OpenNI2 (on OSX):