joshuajnoble / ofxkinectv2

of addon for Kinect V2

ofxKinectV2

This is an openFrameworks addon for working with the Microsoft SDK for the Kinect V2. There's a few caveats: you need a fairly specific setup to get this working. You'll need

• A computer with a USB 3 port
• Windows 8.1 (bootcamp ok but VM, not)
• Visual Studio 2012 or 2013
• A Kinect V2
• The Kinect V2 SDK (will be released to the public in July)
• openFrameworks with all the poco libs compiled for 2013

Just to answer a few of the initial questions:

Can I use this on OSX?

Nope.

Can I use this on Windows 7?

Nope.

Can I use this on Linux?

Nope.

Why not?

Because we're just wrapping the functionality provided by the Microsoft Kinect team.

Where's speech recognition and face tracking?

Coming soon.

ofxkinectv2's Issues

Fusion, Face , HD Face

Hey,
I have some questions.

• Do we have the Fusion, Face, HD Face in the ofxKinectV2?
• What's the difference between that project and ofxKinectCommonBridge v2 - https://github.com/joshuajnoble/ofxKinectCommonBridge/tree/kcbv2?

Regards,
Ivaylo

speedup frame mappings

you can speed up the frame mapping functions by using
KCBMapDepthFrameToColorSpace
instead of
KCBMapDepthPointsToColorSpace

on my test system, i've noticed a sixfold increase in speed (from 35-40 ms to 5-6 ms).

I'm not forking this because lack of time, but i'm using the following code:

void ofxKinectCommonBridge::mapDepthFrameToColorFrame(ofPixels& dstColorPixels) {
int depthArraySize = depthFrameDescription.width * depthFrameDescription.height;
vector<UINT16> depths; //input
depths.resize(depthArraySize);
const ofShortPixelsRef depthImage = getRawDepthPixelsRef();
for(int y = 0; y < depthFrameDescription.height; y++){
    for(int x = 0; x < depthFrameDescription.width; x++) {
        int i = y*depthFrameDescription.width+x;
        depths[i] = (UINT16)depthImage.getPixels()[i];
    }
}

//output
vector<ColorSpacePoint> colorPoints;
colorPoints.resize(depthArraySize);

HRESULT mapResult;
mapResult = KCBMapDepthFrameToColorSpace(
    hKinect,
    depthArraySize, &depths[0],
    depthArraySize, &colorPoints[0]);


if(!dstColorPixels.isAllocated() || 
    dstColorPixels.getWidth() != depthFrameDescription.width || dstColorPixels.getWidth() != depthFrameDescription.height)
{
    dstColorPixels.allocate(depthFrameDescription.width,depthFrameDescription.height, OF_IMAGE_COLOR);
}

memset(dstColorPixels.getPixels(), 0, dstColorPixels.getWidth()*dstColorPixels.getHeight()*dstColorPixels.getBytesPerPixel());

for(int y = 0; y < depthFrameDescription.height; y++){
    for(int x = 0; x < depthFrameDescription.width; x++) {      
        int depthFrameIndex = y * depthFrameDescription.width + x;
        ColorSpacePoint& p = colorPoints[depthFrameIndex]; 
        if(p.X >= 0 && p.X < colorFrameDescription.width && p.Y >= 0 && p.Y < colorFrameDescription.height)
        {
            dstColorPixels.setColor(x,y, videoPixels.getColor(p.X,p.Y));
        }
    }
}
}

Hot to show the RGB camera view

simply, i think would be enough to change the line
kinect.drawDepth(0, 0);
in
kinect.draw(0, 0);

but this doesn't show anything.
What am I doing wrong?

No grayscale values in depth texture.

I was able to run the simple-example but noticed that the depth feed is displaying either black or white pixels but no grayscale values in between.

The two methods being called in this image are :

kinect.drawDepth(0, 0);
kinect.drawBodyIndex(500, 0);

I also noticed that there's a ofShader object in the example but it's never setup or initialized, perhaps this could be part of the issue ?

Running this in Visual Studio 2012 on OpenFrameworks release 0.8.1

Possible problem with the supplied DLL files

I built a sample ofxKinectV2 app but my program crashes immediately after it is run.
What happens is :
Kinect starts up(I can see the 3 red lights) so it seems kinect is sending data but compiling from the IDE itself I am shown error
"Unhandled exception at 0x67E243FA (KCBv2.dll) in KinectTest1.exe: 0xC0000005: Access violation reading location 0x00000004"
I tried all possible configurations of these files(release,debug versions of OF too) but the error persists.

I’m suspecting since KCBv2.dll is originally downloaded(and not built on my machine) if I built it myself maybe the error would go away.
Any idea how to I can build the dll ? or any other solution?

No color values

Hi, I can't se any rgb image. I removed any call to methods for other kind of images, but there is no way to have colors. I have also a crash if I don't initialize the depth frame....

setup() {

kinect.initSensor();
kinect.initColorStream();

}

update() {

kinect.update();

}

draw() {

kinect.draw(0,0);
}

Adding a KCBGetDepthCameraIntrinsics method in KCBv2Lib.h

Hello, i'd like to add a method to retrieve the camera intrinsics values.
An interface is available in Kinect.h (line 7639, virtual HRESULT STDMETHODCALLTYPE GetDepthCameraIntrinsics), but i can't find the source of the dll you use in the repo...
Could you help me?

really needs vs2013?

the guys from MS at eyeo said it works with vs2012

Sample project throws exception when try to close the application

Hello @joshuajnoble,

Yesterday I was able to run the simple project inside VS2013. The issue is when you press Esc or close the app it crashes from the ofxKinectCommonBridge. I don't have the source codes here in the office but maybe you can find the issue easily.

Regards,
Ivaylo

Get Body Data

Hello, is there any way to get body data from the sensor using this addon e.g

.h
ofxKinectCommonBridge kinect;
...
.cpp
kinect.getBodyJoints();?

Installation Support

I cannot get the Simple example to work but I think I'm close.
I've followed the instructions provided in the downloadable article from here:
http://blogs.msdn.com/b/kinectforwindows/archive/2014/07/15/getting-started-with-cinder-and-kinect-for-windows-v2.aspx

The bottom of the article covers OFX. I've followed its instructions, getting the VS2013 build and building project generator then building a project with ofxKinectV2. The end of the article kinda fizzles out with the details regarding this component:

"Note that the add-on is already added but you will need to add the header to your source code and create the required variables to start using the plug-in. At this point, you can refer to the sample project to get more information with using the plug-in code. "

I get the following error from the example-Simple project provided:
error C1083: Cannot Open include file: 'KCBv2Lib.h': No such file or directory
same for ofxKinectCommonBridge.h

I assume this suggests that I do not have the Kinect Common Bridge Library installed or something related to that library.

Any support / suggestions / help would be great!
Thanks!

TrackingState is not updated

when retrieving a joint, the tracking state return by getTrackingState() is undefined...