OpenNI2 driver cleaned up by xlz · Pull Request #523 · OpenKinect/libfreenect2

xlz · 2016-01-04T02:48:51Z

Cleaned up from #518. Previous PR #302. Discussion #243.

Build tested on Debian. Tested on MacOSX.

Still need Windows testing, and docs for installation on Windows.

xlz · 2016-01-21T16:28:57Z

This should be merged pretty soon after the required testing.

From libfreenect 89f77f6d2c23876936af65766a4c140898bc3ea8

Test with /opt/OpenNI2/Tools/NiViewer.

xlz · 2016-01-21T22:28:17Z

v2: Add a .so.0 suffix for the library so OpenNI2 can find it. Fix compiler warnings.

Tested with NiViewer2. Did not test with pcl_openni2_viewer because it is not packaged.

Still needs Windows testing.

xlz · 2016-01-22T19:04:18Z

@hanyazou There are two problems I found in your code. You are including extra headers in .hpp files for functions that are only referenced in .cpp files. If there is no reference to a function in the hpp file, the hpp file should not include that header.

Also, VideoStream.hpp contains method definitions, and this header is include twice in different cpp files. This will produce duplicate object code. You should not put method definitions in header files.

~~Try to avoid the problems next time.~~

I found these problems exist in the original code, so they are not your problems.

xlz · 2016-01-22T19:20:00Z

@hanyazou Can you provide some instructions on where and how to install the built libfreenect2-openni2.dylib on Mac OSX?

xlz · 2016-01-22T19:58:23Z

v3: Fixed MSVC FTBFS. Added Windows installation instructions.

TODO: think through proper Linux and MacOSX installation instructions.

Because substantial amount of code is changed, this needs retesting on Linux and Mac OSX.

hanyazou · 2016-01-23T12:02:52Z

@xlz

Can you provide some instructions on where and how to install the built libfreenect2-openni2.dylib on Mac OSX?

I believe $OPENNI2_REDIST/OpenNI2/Drivers/ is the driver repository.
('brew install openni2' sugests OPENNI2_REDIST=/usr/local/lib/ni2.)

While /usr/local/lib/ni2/OpenNI.ini might be able to specify the repository path, the default is OpenNI2/Drivers/. You should install the library to the default driver repository path.

xlz · 2016-01-23T23:21:06Z

v4:

Moved more method definitions out of headers
Install RPATH in libfreenect2-openni2.so if libfreenect2.so is not installed in /usr/local
Now use make install-openni2 to install libfreenect2-openni2.so. This driver can't be in the default install target for non-root Linux installation and Windows installation.
Finished installation docs.

xlz · 2016-01-23T23:28:07Z

@hanyazou How does this work on Mac OSX now?

hanyazou · 2016-01-24T00:10:06Z

@xlz it seems not to work. The .dylib is installed to /usr/local/lib/openni2/. Please refer full build and test log attached.

build.log.txt

xlz · 2016-01-24T00:16:10Z

Now use make install-openni2 to install libfreenect2-openni2.so.

Have you done that?

hanyazou · 2016-01-24T00:32:14Z

@xlz 'make install-openni2' copies libfreenect2-openni2.dylib into /usr/local/lib/ni2/OpenNI2/Drivers/ and the NiViewer works fine!
(make install-openni2 did not require sudo in my env. the target directories were owned by me.)

make install to copy libfreenect2-openni2* to lib/OpenNI2/Drivers. make install-openni2 to cmake -E copy_directory OpenNI2/Drivers

@HenningJ

@HenningJ has the following contribution to this commit: Change copying of color images to reflect the change from BGR to BGRX color format.

…lass

xlz · 2016-01-24T00:43:32Z

One last change (v5): make install will copy libfreenect2-openni2.so to ${CMAKE_INSTALL_PREFIX}/lib/OpenNI2/Drivers/ instead of .../lib/openni2. This name is easier to understand when compared with /usr/lib/OpenNI2/Drivers.

I'll test this on linux and windows on Monday. If eveything is already, I'll merge this then.

xlz · 2016-01-25T18:04:43Z

Tests OK on Linux and Windows.

OpenNI2 driver

kruss84 · 2016-01-26T18:45:46Z

I'm trying to convert an image from openni::VideoFrameRef to cv::Mat.
When I use the examples given for color and depth, I just get gray images.
Resolution of the images is correct, only the data is bad.

http://stackoverflow.com/questions/15966722/capture-rgb-from-kinnect-with-openni-and-show-with-opencv
http://answers.opencv.org/question/33513/opencv-asus-xtion-sensor/

When I start the program SimpleRead of Openni2, the depth image looks very good.
The readout of the Kinect v2 is working perfectly.

I would appreciate any suggestions or ideas to solve this problem.

xlz · 2016-01-26T18:54:58Z

@hanyazou is responsible to OpenNI2 related questions.

GerardusHuang · 2016-01-29T05:15:39Z

I have followed the windows instructions. Every thing is ok except I can not find any file named "libfreenect2-openni2.dll".
I have checked "BUILD_OPENNI2_DRIVER" and run in cmake GUI, also do not generated any project named with "openni2".

xlz · 2016-01-29T07:24:03Z

Post output of the CMake configure log.

GerardusHuang · 2016-01-29T07:58:15Z

The C compiler identification is MSVC 18.0.31101.0
The CXX compiler identification is MSVC 18.0.31101.0
Check for working C compiler using: Visual Studio 12 2013 Win64
Check for working C compiler using: Visual Studio 12 2013 Win64 -- works
Detecting C compiler ABI info
Detecting C compiler ABI info - done
Check for working CXX compiler using: Visual Studio 12 2013 Win64
Check for working CXX compiler using: Visual Studio 12 2013 Win64 -- works
Detecting CXX compiler ABI info
Detecting CXX compiler ABI info - done
Detecting CXX compile features
Detecting CXX compile features - done
using tinythread as threading library
Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
Found LibUSB: G:/DevelopInstalls/sdk/Kinect/libfreenect/libfreenect2/depends/libusb/x64/Release/dll/libusb-1.0.lib
Performing Test TURBOJPEG_WORKS
Performing Test TURBOJPEG_WORKS - Success
Found TurboJPEG: C:/libjpeg-turbo64/lib/turbojpeg.lib
Found GLFW3: G:/DevelopInstalls/sdk/Kinect/libfreenect/libfreenect2/depends/glfw/lib-vc2013/glfw3dll.lib
Found OpenGL: opengl32
Looking for CL_VERSION_2_0
Looking for CL_VERSION_2_0 - found
Found OpenCL: F:/Program Files (x86)/Intel/OpenCL SDK/4.6/lib/x64/OpenCL.lib (found version "2.0")
Performing Test COMPILER_HAS_DEPRECATED_ATTR
Performing Test COMPILER_HAS_DEPRECATED_ATTR - Failed
Performing Test COMPILER_HAS_DEPRECATED
Performing Test COMPILER_HAS_DEPRECATED - Success
Linking with these libraries:
G:/DevelopInstalls/sdk/Kinect/libfreenect/libfreenect2/depends/libusb/x64/Release/dll/libusb-1.0.lib
C:/libjpeg-turbo64/lib/turbojpeg.lib
G:/DevelopInstalls/sdk/Kinect/libfreenect/libfreenect2/depends/glfw/lib-vc2013/glfw3dll.lib
opengl32
F:/Program Files (x86)/Intel/OpenCL SDK/4.6/lib/x64/OpenCL.lib
Found Doxygen: C:/Program Files/doxygen/bin/doxygen.exe (found version "1.8.9.1")
Configurating examples
Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
Using in-tree freenect2 target
Found OpenNI2: C:/Program Files/OpenNI2/Lib
Configuring done
Generating done

xlz · 2016-01-29T08:30:14Z

Post a list of all files recursively under the build directory.

GerardusHuang · 2016-01-29T09:22:20Z

│ ALL_BUILD.vcxproj
│ ALL_BUILD.vcxproj.filters
│ CMakeCache.txt
│ cmake_install.cmake
│ file.txt
│ freenect2.pc
│ freenect2.vcxproj
│ freenect2.vcxproj.filters
│ freenect2Config.cmake
│ generate_resources_tool.vcxproj
│ generate_resources_tool.vcxproj.filters
│ INSTALL.vcxproj
│ INSTALL.vcxproj.filters
│ install_manifest.txt
│ libfreenect2.sdf
│ libfreenect2.sln
│ resources.inc.h
│ ZERO_CHECK.vcxproj
│ ZERO_CHECK.vcxproj.filters
│
├─bin
│ └─MinSizeRel
│ freenect2.dll
│ generate_resources_tool.exe
│ glfw3.dll
│ libusb-1.0.dll
│ Protonect.exe
│ turbojpeg.dll
│
├─CMakeFiles
│ │ cmake.check_cache
│ │ CMakeError.log
│ │ CMakeOutput.log
│ │ feature_tests.bin
│ │ feature_tests.cxx
│ │ generate.stamp
│ │ generate.stamp.depend
│ │ generate.stamp.list
│ │ TargetDirectories.txt
│ │
│ ├─20c456e0b6f062e6a8052eefdc0c43ec
│ │ resources.inc.h.rule
│ │
│ ├─3.2.2
│ │ │ CMakeCCompiler.cmake
│ │ │ CMakeCXXCompiler.cmake
│ │ │ CMakeDetermineCompilerABI_C.bin
│ │ │ CMakeDetermineCompilerABI_CXX.bin
│ │ │ CMakeRCCompiler.cmake
│ │ │ CMakeSystem.cmake
│ │ │
│ │ ├─CompilerIdC
│ │ │ │ CMakeCCompilerId.c
│ │ │ │ CompilerIdC.exe
│ │ │ │ CompilerIdC.vcxproj
│ │ │ │
│ │ │ └─Debug
│ │ │ │ CMakeCCompilerId.obj
│ │ │ │
│ │ │ └─CompilerIdC.tlog
│ │ │ cl.command.1.tlog
│ │ │ CL.read.1.tlog
│ │ │ CL.write.1.tlog
│ │ │ CompilerIdC.lastbuildstate
│ │ │ link.command.1.tlog
│ │ │ link.read.1.tlog
│ │ │ link.write.1.tlog
│ │ │
│ │ └─CompilerIdCXX
│ │ │ CMakeCXXCompilerId.cpp
│ │ │ CompilerIdCXX.exe
│ │ │ CompilerIdCXX.vcxproj
│ │ │
│ │ └─Debug
│ │ │ CMakeCXXCompilerId.obj
│ │ │
│ │ └─CompilerIdCXX.tlog
│ │ cl.command.1.tlog
│ │ CL.read.1.tlog
│ │ CL.write.1.tlog
│ │ CompilerIdCXX.lastbuildstate
│ │ link.command.1.tlog
│ │ link.read.1.tlog
│ │ link.write.1.tlog
│ │
│ ├─3c4aa58db9105a46851b08dc22b5a663
│ │ doc.rule
│ │ INSTALL_force.rule
│ │
│ ├─4a8a10df6077e5d195688f9f8301f111
│ │ generate.stamp.rule
│ │ INSTALL_force.rule
│ │
│ ├─b047920d80ab002551325f77e500032f
│ │ INSTALL_force.rule
│ │
│ └─CMakeTmp
├─doc
│ │ cmake_install.cmake
│ │ doc.vcxproj
│ │ doc.vcxproj.filters
│ │ Doxyfile
│ │ INSTALL.vcxproj
│ │ INSTALL.vcxproj.filters
│ │
│ └─CMakeFiles
│ generate.stamp
│ generate.stamp.depend
│
├─examples
│ │ ALL_BUILD.vcxproj
│ │ ALL_BUILD.vcxproj.filters
│ │ cmake_install.cmake
│ │ INSTALL.vcxproj
│ │ INSTALL.vcxproj.filters
│ │ libfreenect2_examples.sln
│ │ Protonect.vcxproj
│ │ Protonect.vcxproj.filters
│ │
│ ├─CMakeFiles
│ │ generate.stamp
│ │ generate.stamp.depend
│ │
│ ├─MinSizeRel
│ └─Protonect.dir
│ └─MinSizeRel
│ │ flextGL.obj
│ │ Protonect.log
│ │ Protonect.obj
│ │ viewer.obj
│ │
│ └─Protonect.tlog
│ cl.command.1.tlog
│ CL.read.1.tlog
│ CL.write.1.tlog
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│ link.command.1.tlog
│ link.read.1.tlog
│ link.write.1.tlog
│ Protonect.lastbuildstate
│
├─freenect2.dir
│ └─MinSizeRel
│ │ command_transaction.obj
│ │ cpu_depth_packet_processor.obj
│ │ depth_packet_processor.obj
│ │ depth_packet_stream_parser.obj
│ │ double_buffer.obj
│ │ event_loop.obj
│ │ flextGL.obj
│ │ frame_listener_impl.obj
│ │ freenect2.log
│ │ libfreenect2.obj
│ │ logging.obj
│ │ opencl_depth_packet_processor.obj
│ │ opengl_depth_packet_processor.obj
│ │ packet_pipeline.obj
│ │ registration.obj
│ │ resource.obj
│ │ rgb_packet_processor.obj
│ │ rgb_packet_stream_parser.obj
│ │ tinythread.obj
│ │ transfer_pool.obj
│ │ turbo_jpeg_rgb_packet_processor.obj
│ │ usb_control.obj
│ │
│ └─freenect2.tlog
│ cl.command.1.tlog
│ CL.read.1.tlog
│ CL.write.1.tlog
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│ freenect2.lastbuildstate
│ freenect2.write.1u.tlog
│ link.command.1.tlog
│ link.read.1.tlog
│ link.write.1.tlog
│
├─generate_resources_tool.dir
│ └─MinSizeRel
│ │ generate_resources.obj
│ │ generate_resources_tool.log
│ │
│ └─generate.FADD3BC8.tlog
│ cl.command.1.tlog
│ CL.read.1.tlog
│ CL.write.1.tlog
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│ generate_resources_tool.lastbuildstate
│ link.command.1.tlog
│ link.read.1.tlog
│ link.write.1.tlog
│
├─install
│ ├─bin
│ │ freenect2.dll
│ │ glfw3.dll
│ │ libusb-1.0.dll
│ │ Protonect.exe
│ │ turbojpeg.dll
│ │
│ ├─include
│ │ └─libfreenect2
│ │ config.h
│ │ export.h
│ │ frame_listener.hpp
│ │ frame_listener_impl.h
│ │ libfreenect2.hpp
│ │ logger.h
│ │ packet_pipeline.h
│ │ registration.h
│ │
│ └─lib
│ │ freenect2.lib
│ │
│ ├─cmake
│ │ └─freenect2
│ │ freenect2Config.cmake
│ │
│ └─pkgconfig
│ freenect2.pc
│
├─lib
│ └─MinSizeRel
│ freenect2.exp
│ freenect2.lib
│
├─libfreenect2
│ config.h
│ export.h
│
├─MinSizeRel
└─x64
└─MinSizeRel
├─ALL_BUILD
│ │ ALL_BUILD.log
│ │
│ └─ALL_BUILD.tlog
│ ALL_BUILD.lastbuildstate
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│
├─INSTALL
│ │ INSTALL.log
│ │
│ └─INSTALL.tlog
│ custombuild.command.1.tlog
│ custombuild.read.1.tlog
│ custombuild.write.1.tlog
│ INSTALL.lastbuildstate
│
└─ZERO_CHECK
│ ZERO_CHECK.log
│
└─ZERO_CHECK.tlog
custombuild.command.1.tlog
custombuild.read.1.tlog
custombuild.write.1.tlog
ZERO_CHECK.lastbuildstate

xlz · 2016-01-29T10:37:20Z

If you followed the instructions, there would not be "MinSizeRel".

xlz · 2016-01-29T16:27:09Z

rm -rf * everything under the build directory, and redo cmake.

GerardusHuang · 2016-01-30T02:42:51Z

I had a try on another computer today, but still no "libfreenect2-openni2.dll" :(
The difference is that I use libusb latest release, not your cmd script, because it always occurs compile error.
Maybe no support on windows 10 ?
I will have a try on window 8.1 soon.

xlz · 2016-01-30T03:09:01Z

What error?

If you don't follow the instructions, you are expected to be capable to follow and debug the script in spirit and not ask question here.

mennayahia · 2016-02-23T17:34:23Z

Hello,
great work guys.

I am trying to use Kinect v2 to build applications on linux to detect and track skeleton, using libfreenect2.
Your driver seems like the solution, but I am a bit confused, so if you can direct me to some instructions as how to install/build your driver with libfreenect2, i.e how to make it work, this will be really helpful.

thanks

hanyazou · 2016-02-23T20:53:26Z

@mennayahia It it not so difficult. Read README.md on the top page https://github.com/OpenKinect/libfreenect2 and follow the instruction https://github.com/OpenKinect/libfreenect2#linux to install.
Please submit another issue if you have some problem.

mennayahia · 2016-02-24T05:51:23Z

@hanyazou
I already installed libfreenect2 from https://github.com/OpenKinect/libfreenect2 and Protect example is working fine, displaying depth maps.
I want to install/configure the openni2 for libfreenect2, to be able to track skeletons.
and you guys seem to be working on this, so how can I integrate your code with the libfreenect2 repository I cloned and installed a week ago.

Thanks for the help.

hanyazou · 2016-02-24T15:25:38Z

@mennayahia Are you looking for a skeleton tracking library? This project libfreenect2 nor OpenNI2 does not contain any skeleton tracking function. You might be able to use the Nite binary for that purpose. Refer #243.
(Please note that the Nite is not a open source nor free software project and development was already discontinued.)

mennayahia · 2016-02-25T08:58:39Z

@hanyazou
I want to develop an application based on Kinect v2 that detects and tracks hands, then I want to extract features from the hand to use for hand gesture recognition.
I thought openNI2 with libfreenect2 would help, wouldn't it ?

tofighi · 2016-02-25T09:30:15Z

@mennayahia
OpenNI and Libfreenect2 don't provide hand detection or tracking. If you are looking for such features, you should use NiTE handtracker feature which is not opensource, however, it seems you can use it as a middleware on top of OpenNI and Libfreenect2.

HenningJ · 2016-02-25T09:35:14Z

But keep in mind that you are not actually allowed to do that, since you are only allowed to use NiTE with Primesense devices.

tofighi · 2016-02-25T09:39:38Z

@HenningJ @mennayahia
That' true, Kinect V2 is not a Primesense device. I know an open source skeleton tracking implementation that may help:
https://github.com/joaquimrocha/Skeltrack

mennayahia · 2016-02-25T09:41:50Z

But does Skeltrack works with kinect v2 ?
it does not seem to be compatible with v2

HenningJ · 2016-02-25T09:45:43Z

someone made it work with Kinect v2: joaquimrocha/Skeltrack#28

mennayahia · 2016-02-25T10:59:04Z

@hanyazou
Ok.
I have another question.
I clones the master branch of libfreenect2, installed it and Protonect is working fine. Does this mean I installed OpenNI also with libfreenect2 ? or should I download it and install it separately, and if so, how to integrate them together then ?

Thanks

floe added this to the 0.2 milestone Jan 4, 2016

hanyazou mentioned this pull request Jan 14, 2016

OpenNI2 driver #243

Closed

hanyazou added 2 commits January 21, 2016 15:39

openni2: Copy OpenNI2-FreenectDriver

1bf2a27

From libfreenect 89f77f6d2c23876936af65766a4c140898bc3ea8

openni2: Adapt to libfreenect2 API

b5b1aa0

Test with /opt/OpenNI2/Tools/NiViewer.

xlz and others added 10 commits January 23, 2016 19:39

openni2: Add proper build system

d2dadbd

make install to copy libfreenect2-openni2* to lib/OpenNI2/Drivers. make install-openni2 to cmake -E copy_directory OpenNI2/Drivers

openni2: Add IrStream class

8dc55c5

openni2: Add registration

fe75575

@HenningJ has the following contribution to this commit: Change copying of color images to reflect the change from BGR to BGRX color format.

openni2: Add timestamp on the frames

fc2ee56

openni2: Use OpenNI2 logging functions/classes

11b748b

openni2: Add OpenKinect Project's license headers

5d394d6

openni2: Refactor setVideoMode() and getSensorInfo() in VideoStream c…

446cde5

…lass

openni2: Add build instructions

93c6f20

openni2: Move method definitions out of headers

4133056

openni2: Fix compiler warnings and extra headers

98aaf46

xlz added a commit that referenced this pull request Jan 25, 2016

Merge pull request #523 from xlz/openni2

7aa52cf

OpenNI2 driver

xlz merged commit 7aa52cf into OpenKinect:master Jan 25, 2016

xlz deleted the openni2 branch January 25, 2016 18:08

zry656565 mentioned this pull request Apr 14, 2016

Fail to use libfreenect2 with OpenNI2 [Ubuntu 15.04] #639

Closed

Conversation

xlz commented Jan 4, 2016

Uh oh!

xlz commented Jan 21, 2016

Uh oh!

xlz commented Jan 21, 2016

Uh oh!

xlz commented Jan 22, 2016

Uh oh!

xlz commented Jan 22, 2016

Uh oh!

xlz commented Jan 22, 2016

Uh oh!

hanyazou commented Jan 23, 2016

Uh oh!

xlz commented Jan 23, 2016

Uh oh!

xlz commented Jan 23, 2016

Uh oh!

hanyazou commented Jan 24, 2016

Uh oh!

xlz commented Jan 24, 2016

Uh oh!

hanyazou commented Jan 24, 2016

Uh oh!

xlz commented Jan 24, 2016

Uh oh!

xlz commented Jan 25, 2016

Uh oh!

kruss84 commented Jan 26, 2016

Uh oh!

xlz commented Jan 26, 2016

Uh oh!

GerardusHuang commented Jan 29, 2016

Uh oh!

xlz commented Jan 29, 2016

Uh oh!

GerardusHuang commented Jan 29, 2016

Uh oh!

xlz commented Jan 29, 2016

Uh oh!

GerardusHuang commented Jan 29, 2016

Uh oh!

xlz commented Jan 29, 2016

Uh oh!

xlz commented Jan 29, 2016

Uh oh!

GerardusHuang commented Jan 30, 2016

Uh oh!

xlz commented Jan 30, 2016

Uh oh!

mennayahia commented Feb 23, 2016

Uh oh!

hanyazou commented Feb 23, 2016

Uh oh!

mennayahia commented Feb 24, 2016

Uh oh!

hanyazou commented Feb 24, 2016

Uh oh!

mennayahia commented Feb 25, 2016

Uh oh!

tofighi commented Feb 25, 2016

Uh oh!

HenningJ commented Feb 25, 2016

Uh oh!

tofighi commented Feb 25, 2016

Uh oh!

mennayahia commented Feb 25, 2016

Uh oh!

HenningJ commented Feb 25, 2016

Uh oh!

mennayahia commented Feb 25, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!