christiankerl / libfreenect2 Goto Github PK
View Code? Open in Web Editor NEWThis project forked from openkinect/libfreenect2
Open source drivers for the Kinect for Windows v2 device
This project forked from openkinect/libfreenect2
Open source drivers for the Kinect for Windows v2 device
I am using two Kinect v2 in one application with OpenNI and here is the source code:
OpenNI::initialize();
openni::Array<DeviceInfo> mDeviceInfo;
OpenNI::enumerateDevices(&mDeviceInfo);
Device mDevice1;
Device mDevice2;
/*if (mDeviceInfo.getSize() != 2) {
cout << "Wrong number of Kinects" << endl;
return -1;
}*/
mDevice1.open(mDeviceInfo[0].getUri());
mDevice2.open(mDeviceInfo[1].getUri());
but it seems that doesn't work well because of loss of packets:
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 8 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @105:2 with serial 0196534
[Info] [Freenect2Impl] found valid Kinect v2 @161:2 with serial 0197047
[Info] [Freenect2Impl] found 2 devices
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 8 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @105:2 with serial 0196534
[Info] [Freenect2Impl] found valid Kinect v2 @161:2 with serial 0197047
[Info] [Freenect2Impl] found 2 devices
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 8 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @105:2 with serial 0196534
[Info] [Freenect2Impl] found valid Kinect v2 @161:2 with serial 0197047
[Info] [Freenect2Impl] found 2 devices
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
[Info] [DepthPacketStreamParser] 2 packets were lost
[Info] [DepthPacketStreamParser] 4 packets were lost
[Info] [DepthPacketStreamParser] 4 packets were lost
[Info] [TurboJpegRgbPacketProcessor] avg. time: 13.2382ms -> ~75.5389Hz
[Info] [DepthPacketStreamParser] 5 packets were lost
[Info] [OpenGLDepthPacketProcessor] avg. time: 23.6746ms -> ~42.2394Hz
[Info] [DepthPacketStreamParser] 4 packets were lost
[Info] [DepthPacketStreamParser] 3 packets were lost
[Info] [DepthPacketStreamParser] 82 packets were lost
[Info] [TurboJpegRgbPacketProcessor] avg. time: 13.4984ms -> ~74.0827Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 26.3897ms -> ~37.8935Hz
[Info] [DepthPacketStreamParser] 17 packets were lost
[Info] [DepthPacketStreamParser] 5 packets were lost
[Info] [TurboJpegRgbPacketProcessor] avg. time: 13.8107ms -> ~72.4074Hz
[Info] [DepthPacketStreamParser] 14 packets were lost
[Info] [OpenGLDepthPacketProcessor] avg. time: 28.8671ms -> ~34.6415Hz
So are there other good solutions to get Kinects serial for device opening?
Dear Christian,
I testet 2 Kinects (attached to different USB-controllers) and tried to get Protonect running.
Therefore I replaced in Protonect.cpp
in the section where the device is initialized:
handle = libusb_open_device_with_vid_pid(NULL, vid, pid);
by
handle = libusb_open_device_with_vid_pid_num(NULL, vid, pid, num_wanted);
having
libusb_device_handle* libusb_open_device_with_vid_pid_num(libusb_context _ctx, uint16_t vendor_id, uint16_t product_id, int num_wanted)
{
struct libusb_device *_devs;
struct libusb_device *found = NULL;
struct libusb_device *dev;
struct libusb_device_handle *handle = NULL;
size_t i = 0;
int r;
int num = 0;
if (libusb_get_device_list(ctx, &devs) < 0)
return NULL;
while ((dev = devs[i++]) != NULL) {
struct libusb_device_descriptor desc;
r = libusb_get_device_descriptor(dev, &desc);
if (r < 0)
goto out;
if (desc.idVendor == vendor_id && desc.idProduct == product_id) {
if(num == num_wanted){
found = dev;
break;
}
else{
++num;
}
}
}
if (found) {
r = libusb_open(found, &handle);
if (r < 0)
handle = NULL;
}
out:
libusb_free_device_list(devs, 1);
return handle;
}
I am able to choose which Kinect I want to run from the commandline now (e.g. ./Protonect 0 vs. ./Protonect 1)
This works when running each Kinect one after another - but not when one Kinect is already runnig. Then I get the following error for the second "start":
steppo@pitoti:protonect/bin ‹opengl_depth_processor*›$ sudo ./Protonect 1
[sudo] password for steppo:
Using libusbx v1.0.17.10852
Opening device 045E:02C4...
Reading device descriptor:
length: 18
device class: 239
S/N: 4
VID:PID: 045E:02C4
bcdDevice: 0100
iMan:iProd:iSer: 1:2:4
nb confs: 1
speed: 5000 Mbit/s (USB SuperSpeed)
active configuration: 1, err: 0
Claiming interface 0...
Claiming interface 1...
running kinect...
Control transfer 1 - set isoch delay
Control transfer 2 - set sel u1/u2
Setting interface alt setting...
Setting stream status: Disable
Control transfer 3 - enable u1
Control transfer 4 - enable u2
Control transfer 5 - set feature 768
Setting sensor status: Disable
Kinect init done
iso max_packet_size: 33792
speed: 5000 Mbit/s (USB SuperSpeed)
running kinect...
Setting sensor status: Enable
Cmd seq 0 func 0x02 (0)
Cmd sent, 20 bytes sent
Cmd response seq 0 func 0x02 (0)
Cmd response success, 512 bytes received
Received cmd data 0x02 (0), length: 512
Cmd response seq 0 func 0x02 (0)
Cmd response success, 16 bytes received
Cmd response completed
Response complete for cmd 0x02 (0)
dumping 512 bytes of raw data from command KCMD_READ_DATA1:
0x0000: 01 00 01 00 5a 0b 00 00 00 00 00 00 00 00 00 00 ....Z...........
0x0010: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0020: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0030: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0040: 01 00 01 00 5a 0b 00 00 00 00 00 00 00 00 00 00 ....Z...........
0x0050: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0060: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
Cmd seq 1 func 0x14 (0)
Cmd sent, 20 bytes sent
Cmd response seq 1 func 0x14 (0)
Cmd response success, 92 bytes received
Received cmd data 0x14 (0), length: 92
Cmd response seq 1 func 0x14 (0)
Cmd response success, 16 bytes received
Cmd response completed
Response complete for cmd 0x14 (0)
dumping 92 bytes of raw data from command KCMD_READ_VERSIONS:
0x0000: 00 00 12 00 00 00 00 00 01 00 00 00 43 c1 1f 41 ............C..A
0x0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0020: 0a 21 33 55 c2 00 17 20 00 08 00 00 10 00 00 00 .!3U... ........
0x0030: 00 01 00 00 00 10 00 00 00 00 80 00 01 00 00 00 ................
0x0040: 31 33 00 00 00 08 07 0d 47 48 59 30 35 34 2e 31 13......GHY054.1
0x0050: 58 00 00 00 00 00 00 00 07 00 00 00 X...........
Cmd seq 2 func 0x22 (0x1)
Cmd sent, 24 bytes sent
Cmd response seq 2 func 0x22 (0x1)
Cmd response success, 128 bytes received
Received cmd data 0x22 (0x1), length: 128
Cmd response seq 2 func 0x22 (0x1)
Cmd response success, 16 bytes received
Cmd response completed
Response complete for cmd 0x22 (0x1)
dumping 128 bytes of raw data from command KCMD_READ_DATA_PAGE 0x01:
0x0000: 31 00 31 00 30 00 33 00 35 00 36 00 35 00 33 00 1.1.0.3.5.6.5.3.
0x0010: 34 00 34 00 34 00 37 00 00 00 00 00 00 00 00 00 4.4.4.7.........
0x0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
Cmd seq 3 func 0x22 (0x3)
Cmd sent, 24 bytes sent
Cmd response seq 3 func 0x22 (0x3)
libusbx: error [submit_bulk_transfer] submiturb failed error -1 errno=12
Cmd response error: -1
Error in response for 0x22 cmd (0x3)
Error in cmd protocol 0x22 (0x3)
Do you have any Idea whats going wrong there?
Maybe this is still to early, but I am looking forward to get 2 or even more Kinects running on a single workstation.
Kind regards,
Stephan
Dear Christian
I am testing to use two kinect v2 in one machine.
I am working with ubuntu 14.04.1
I install compile libfreenect2 without trouble.
I personalise in an other file the codde of Protonect.cpp like this :
bool protonect_shutdown = false;
void sigint_handler(int s)
{
protonect_shutdown = true;
}
inline void frame_deleter(libfreenect2::Frame* ptr)
{
ptr=nullptr;
}
struct device_t2
{
std::shared_ptr<libfreenect2::Freenect2Device> _device;
std::unique_ptr<libfreenect2::SyncMultiFrameListener> _listener;
cv::Mat _bgr;
cv::Mat _ir;
cv::Mat _depth;
device_t2(libfreenect2::Freenect2Device* ptr):
_device(ptr,[](libfreenect2::Freenect2Device* ptr){ptr = nullptr;}),
_listener(new libfreenect2::SyncMultiFrameListener(libfreenect2::Frame::Color | libfreenect2::Frame::Ir | libfreenect2::Frame::Depth)),
{}
void init()
{
if(this->_all_in)
{
this->_device->setColorFrameListener(this->_listener.get());
this->_device->setIrAndDepthFrameListener(this->_listener.get());
}
else
{
if(this->_type & libfreenect2::Frame::Color)
{
this->_device->setColorFrameListener(this->_listener.get());
this->_is_rgb = true;
}
if((this->_type & libfreenect2::Frame::Depth) || (this->_type & libfreenect2::Frame::Ir))
{
this->_device->setIrAndDepthFrameListener(this->_listener.get());
if(this->_type & libfreenect2::Frame::Depth)
this->_is_depth = true;
if(this->_type & libfreenect2::Frame::Ir)
this->_is_ir = true;
}
}
}
void start()
{
this->_device->start();
}
void stop()
{
this->_device->stop();
}
void close()
{
this->_device->close();
}
void grab_raw()
{
libfreenect2::FrameMap nodes;
this->_listener->waitForNewFrame(nodes);
std::shared_ptr<libfreenect2::Frame> bgr(nodes[libfreenect2::Frame::Color],frame_deleter);
std::shared_ptr<libfreenect2::Frame> ir(nodes[libfreenect2::Frame::Ir],frame_deleter);
std::shared_ptr<libfreenect2::Frame> depth(nodes[libfreenect2::Frame::Depth],frame_deleter);
this->_bgr = cv::Mat(bgr->height,bgr->width,CV_8UC3,bgr->data).clone();
this->_ir = cv::Mat(ir->height,ir->width,CV_32FC1,ir->data).clone();
this->_depth = cv::Mat(depth->height,depth->width,CV_32FC1,depth->data).clone();
this->_listener->release(nodes);
}
void grab_rescale()
{
this->grab_raw();
this->_ir/=20000.f;
this->_depth/=4500.f;
}
};
int protonect2(const int& argc,char**& argv)
{
std::size_t id(0);
if(argc==2)
id = atoi(argv[1]);
glfwInit();
libfreenect2::Freenect2 freenect2;
std::unique_ptr<device_t2> dev(new device_t2(freenect2.openDevice(id)));
if(dev->_device == 0)
{
std::cout << "no device connected or failure opening the default one!" << std::endl;
return -1;
}
signal(SIGINT,sigint_handler);
protonect_shutdown = false;
dev->init();
dev->start();
std::cout << "device serial: " << dev->_device->getSerialNumber() << std::endl;
std::cout << "device firmware: " << dev->_device->getFirmwareVersion() << std::endl;
while(!protonect_shutdown)
{
dev->grab_rescale();
cv::imshow("rgb",dev->_bgr);
cv::imshow("ir", dev->_ir);
cv::imshow("depth", dev->_depth);
int key = cv::waitKey(1);
protonect_shutdown = protonect_shutdown || (key > 0 && ((key & 0xFF) == 27)); // shutdown on escape
}
dev->stop();
dev->close();
return 0;
}
int protonect3(const int& argc,char**& argv)
{
// std::size_t id(0);
// if(argc==2)
// id = atoi(argv[1]);
glfwInit();
libfreenect2::Freenect2 freenect2;
std::unique_ptr<device_t2> dev(new device_t2(freenect2.openDevice(0)));
std::unique_ptr<device_t2> dev2(new device_t2(freenect2.openDevice(1)));
if(dev->_device == 0)
{
std::cout << "no device connected or failure opening the default one!" << std::endl;
return -1;
}
if(dev2->_device == 0)
{
std::cout << "no device connected or failure opening the default one!" << std::endl;
return -1;
}
signal(SIGINT,sigint_handler);
protonect_shutdown = false;
dev->init();
dev2->init();
dev->start();
dev2->start();
std::cout << "device serial: " << dev->_device->getSerialNumber() << std::endl;
std::cout << "device firmware: " << dev->_device->getFirmwareVersion() << std::endl;
std::cout << "device2 serial: " << dev2->_device->getSerialNumber() << std::endl;
std::cout << "device2 firmware: " << dev2->_device->getFirmwareVersion() << std::endl;
while(!protonect_shutdown)
{
dev->grab_rescale();
dev2->grab_rescale();
cv::imshow("rgb",dev->_bgr);
cv::imshow("ir", dev->_ir);
cv::imshow("depth", dev->_depth);
cv::imshow("rgb2",dev2->_bgr);
cv::imshow("ir2", dev2->_ir);
cv::imshow("depth2", dev2->_depth);
int key = cv::waitKey(1);
protonect_shutdown = protonect_shutdown || (key > 0 && ((key & 0xFF) == 27)); // shutdown on escape
}
dev->stop();
dev->close();
dev2->stop();
dev2->close();
return 0;
}
int main(int argc, char *argv[])
{
// return protonect2(argc,argv);
return protonect3(argc,argv);
}
When I use the function protonect2 I am able to select and use the anyone of my two device, and it work well.
When I use the function protonect3 I obtain the following message :
[Freenect2Impl] enumerating devices...
[Freenect2Impl] 20 usb devices connected
[Freenect2Impl] found valid Kinect v2 @4:5 with serial 508080742542
[Freenect2Impl] found valid Kinect v2 @4:7 with serial 503075141942
[Freenect2Impl] found 2 devices
[Freenect2DeviceImpl] opening...
[Freenect2DeviceImpl] opened
[Freenect2DeviceImpl] opening...
[UsbControl::setConfiguration(set)] failed! libusb error -99: LIBUSB_ERROR_OTHER
[Freenect2DeviceImpl] closing...
[Freenect2DeviceImpl] deallocating usb transfer pools...
[Freenect2DeviceImpl] closing usb device...
[Freenect2DeviceImpl] closed
[Freenect2DeviceImpl] failed to open Kinect v2 @4:7!
no device connected or failure opening the default one!
[Freenect2DeviceImpl] closing...
[Freenect2DeviceImpl] releasing usb interfaces...
[Freenect2DeviceImpl] deallocating usb transfer pools...
[Freenect2DeviceImpl] closing usb device...
[Freenect2DeviceImpl] closed
It seem there is a trouble during the usb configuration time.
I have compile and install the libusb use by libfreenect2 on my system.
I also read #7 and modify the /etc/grub file without any results.
Have you any idea about what's wrong ?
Best regards
Julian
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.