GithubHelp home page GithubHelp logo

christiankerl / libfreenect2 Goto Github PK

View Code? Open in Web Editor NEW

This project forked from openkinect/libfreenect2

5.0 5.0 2.0 2.64 MB

Open source drivers for the Kinect for Windows v2 device

Shell 0.43% C++ 64.47% C 27.84% CMake 3.32% GLSL 3.40% Batchfile 0.54%

libfreenect2's People

Contributors

billinghamj avatar blen2r avatar christiankerl avatar davetcoleman avatar floe avatar gaborpapp avatar goldhoorn avatar hanyazou avatar henningj avatar joshblake avatar laborer2008 avatar larshg avatar rjw57 avatar xlz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

libfreenect2's Issues

Using two Kinect v2 in one application

I am using two Kinect v2 in one application with OpenNI and here is the source code:

OpenNI::initialize();
openni::Array<DeviceInfo> mDeviceInfo;
OpenNI::enumerateDevices(&mDeviceInfo);
Device mDevice1;
Device mDevice2;

/*if (mDeviceInfo.getSize() != 2) {
    cout << "Wrong number of Kinects" << endl;
    return -1;
}*/

mDevice1.open(mDeviceInfo[0].getUri());
mDevice2.open(mDeviceInfo[1].getUri());

but it seems that doesn't work well because of loss of packets:

[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 8 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @105:2 with serial 0196534
[Info] [Freenect2Impl] found valid Kinect v2 @161:2 with serial 0197047
[Info] [Freenect2Impl] found 2 devices
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 8 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @105:2 with serial 0196534
[Info] [Freenect2Impl] found valid Kinect v2 @161:2 with serial 0197047
[Info] [Freenect2Impl] found 2 devices
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 8 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @105:2 with serial 0196534
[Info] [Freenect2Impl] found valid Kinect v2 @161:2 with serial 0197047
[Info] [Freenect2Impl] found 2 devices
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
[Info] [DepthPacketStreamParser] 2 packets were lost
[Info] [DepthPacketStreamParser] 4 packets were lost
[Info] [DepthPacketStreamParser] 4 packets were lost
[Info] [TurboJpegRgbPacketProcessor] avg. time: 13.2382ms -> ~75.5389Hz
[Info] [DepthPacketStreamParser] 5 packets were lost
[Info] [OpenGLDepthPacketProcessor] avg. time: 23.6746ms -> ~42.2394Hz
[Info] [DepthPacketStreamParser] 4 packets were lost
[Info] [DepthPacketStreamParser] 3 packets were lost
[Info] [DepthPacketStreamParser] 82 packets were lost
[Info] [TurboJpegRgbPacketProcessor] avg. time: 13.4984ms -> ~74.0827Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 26.3897ms -> ~37.8935Hz
[Info] [DepthPacketStreamParser] 17 packets were lost
[Info] [DepthPacketStreamParser] 5 packets were lost
[Info] [TurboJpegRgbPacketProcessor] avg. time: 13.8107ms -> ~72.4074Hz
[Info] [DepthPacketStreamParser] 14 packets were lost
[Info] [OpenGLDepthPacketProcessor] avg. time: 28.8671ms -> ~34.6415Hz

So are there other good solutions to get Kinects serial for device opening?

multiple kinects V2 tested - but does not seem to work

Dear Christian,

I testet 2 Kinects (attached to different USB-controllers) and tried to get Protonect running.
Therefore I replaced in Protonect.cpp
in the section where the device is initialized:
handle = libusb_open_device_with_vid_pid(NULL, vid, pid);
by
handle = libusb_open_device_with_vid_pid_num(NULL, vid, pid, num_wanted);
having
libusb_device_handle* libusb_open_device_with_vid_pid_num(libusb_context _ctx, uint16_t vendor_id, uint16_t product_id, int num_wanted)
{
struct libusb_device *_devs;
struct libusb_device *found = NULL;
struct libusb_device *dev;
struct libusb_device_handle *handle = NULL;
size_t i = 0;
int r;
int num = 0;
if (libusb_get_device_list(ctx, &devs) < 0)
return NULL;

while ((dev = devs[i++]) != NULL) {
struct libusb_device_descriptor desc;
r = libusb_get_device_descriptor(dev, &desc);
if (r < 0)
goto out;
if (desc.idVendor == vendor_id && desc.idProduct == product_id) {
if(num == num_wanted){
found = dev;
break;
}
else{
++num;
}
}
}

if (found) {
r = libusb_open(found, &handle);
if (r < 0)
handle = NULL;
}

out:
libusb_free_device_list(devs, 1);
return handle;
}

I am able to choose which Kinect I want to run from the commandline now (e.g. ./Protonect 0 vs. ./Protonect 1)
This works when running each Kinect one after another - but not when one Kinect is already runnig. Then I get the following error for the second "start":

steppo@pitoti:protonect/bin ‹opengl_depth_processor*›$ sudo ./Protonect 1
[sudo] password for steppo:
Using libusbx v1.0.17.10852

Opening device 045E:02C4...

Reading device descriptor:
length: 18
device class: 239
S/N: 4
VID:PID: 045E:02C4
bcdDevice: 0100
iMan:iProd:iSer: 1:2:4
nb confs: 1
speed: 5000 Mbit/s (USB SuperSpeed)
active configuration: 1, err: 0
Claiming interface 0...

Claiming interface 1...
running kinect...
Control transfer 1 - set isoch delay
Control transfer 2 - set sel u1/u2

Setting interface alt setting...
Setting stream status: Disable
Control transfer 3 - enable u1
Control transfer 4 - enable u2
Control transfer 5 - set feature 768
Setting sensor status: Disable
Kinect init done

iso max_packet_size: 33792
speed: 5000 Mbit/s (USB SuperSpeed)
running kinect...
Setting sensor status: Enable

Cmd seq 0 func 0x02 (0)
Cmd sent, 20 bytes sent
Cmd response seq 0 func 0x02 (0)
Cmd response success, 512 bytes received
Received cmd data 0x02 (0), length: 512
Cmd response seq 0 func 0x02 (0)
Cmd response success, 16 bytes received
Cmd response completed
Response complete for cmd 0x02 (0)
dumping 512 bytes of raw data from command KCMD_READ_DATA1:
0x0000: 01 00 01 00 5a 0b 00 00 00 00 00 00 00 00 00 00 ....Z...........
0x0010: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0020: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0030: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0040: 01 00 01 00 5a 0b 00 00 00 00 00 00 00 00 00 00 ....Z...........
0x0050: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0060: 03 00 02 00 47 0f 00 00 00 00 00 00 00 00 00 00 ....G...........
0x0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x00f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x01f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

Cmd seq 1 func 0x14 (0)
Cmd sent, 20 bytes sent
Cmd response seq 1 func 0x14 (0)
Cmd response success, 92 bytes received
Received cmd data 0x14 (0), length: 92
Cmd response seq 1 func 0x14 (0)
Cmd response success, 16 bytes received
Cmd response completed
Response complete for cmd 0x14 (0)
dumping 92 bytes of raw data from command KCMD_READ_VERSIONS:
0x0000: 00 00 12 00 00 00 00 00 01 00 00 00 43 c1 1f 41 ............C..A
0x0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0020: 0a 21 33 55 c2 00 17 20 00 08 00 00 10 00 00 00 .!3U... ........
0x0030: 00 01 00 00 00 10 00 00 00 00 80 00 01 00 00 00 ................
0x0040: 31 33 00 00 00 08 07 0d 47 48 59 30 35 34 2e 31 13......GHY054.1
0x0050: 58 00 00 00 00 00 00 00 07 00 00 00 X...........

Cmd seq 2 func 0x22 (0x1)
Cmd sent, 24 bytes sent
Cmd response seq 2 func 0x22 (0x1)
Cmd response success, 128 bytes received
Received cmd data 0x22 (0x1), length: 128
Cmd response seq 2 func 0x22 (0x1)
Cmd response success, 16 bytes received
Cmd response completed
Response complete for cmd 0x22 (0x1)
dumping 128 bytes of raw data from command KCMD_READ_DATA_PAGE 0x01:
0x0000: 31 00 31 00 30 00 33 00 35 00 36 00 35 00 33 00 1.1.0.3.5.6.5.3.
0x0010: 34 00 34 00 34 00 37 00 00 00 00 00 00 00 00 00 4.4.4.7.........
0x0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

Cmd seq 3 func 0x22 (0x3)
Cmd sent, 24 bytes sent
Cmd response seq 3 func 0x22 (0x3)
libusbx: error [submit_bulk_transfer] submiturb failed error -1 errno=12
Cmd response error: -1
Error in response for 0x22 cmd (0x3)
Error in cmd protocol 0x22 (0x3)

Do you have any Idea whats going wrong there?
Maybe this is still to early, but I am looking forward to get 2 or even more Kinects running on a single workstation.

Kind regards,
Stephan

Multiple kinect v2 usb configuration issue

Dear Christian

I am testing to use two kinect v2 in one machine.
I am working with ubuntu 14.04.1
I install compile libfreenect2 without trouble.
I personalise in an other file the codde of Protonect.cpp like this :

bool protonect_shutdown = false;

void sigint_handler(int s)
{
protonect_shutdown = true;
}

inline void frame_deleter(libfreenect2::Frame* ptr)
{
ptr=nullptr;
}

struct device_t2
{

std::shared_ptr<libfreenect2::Freenect2Device> _device;
std::unique_ptr<libfreenect2::SyncMultiFrameListener> _listener;


cv::Mat _bgr;
cv::Mat _ir;
cv::Mat _depth;

device_t2(libfreenect2::Freenect2Device* ptr):
    _device(ptr,[](libfreenect2::Freenect2Device* ptr){ptr = nullptr;}),
    _listener(new libfreenect2::SyncMultiFrameListener(libfreenect2::Frame::Color | libfreenect2::Frame::Ir | libfreenect2::Frame::Depth)),

    {}



    void init()
    {
        if(this->_all_in)
        {
            this->_device->setColorFrameListener(this->_listener.get());
            this->_device->setIrAndDepthFrameListener(this->_listener.get());
        }
        else
        {
            if(this->_type & libfreenect2::Frame::Color)
            {
                this->_device->setColorFrameListener(this->_listener.get());
                this->_is_rgb = true;
            }

            if((this->_type & libfreenect2::Frame::Depth) || (this->_type & libfreenect2::Frame::Ir))
            {
                this->_device->setIrAndDepthFrameListener(this->_listener.get());

                if(this->_type & libfreenect2::Frame::Depth)
                    this->_is_depth = true;

                if(this->_type & libfreenect2::Frame::Ir)
                    this->_is_ir = true;

            }
        }
    }

    void start()
    {
        this->_device->start();
    }

    void stop()
    {
        this->_device->stop();
    }

    void close()
    {
        this->_device->close();
    }

    void grab_raw()
    {
        libfreenect2::FrameMap nodes;

        this->_listener->waitForNewFrame(nodes);


            std::shared_ptr<libfreenect2::Frame> bgr(nodes[libfreenect2::Frame::Color],frame_deleter);
            std::shared_ptr<libfreenect2::Frame> ir(nodes[libfreenect2::Frame::Ir],frame_deleter);
            std::shared_ptr<libfreenect2::Frame> depth(nodes[libfreenect2::Frame::Depth],frame_deleter);

            this->_bgr = cv::Mat(bgr->height,bgr->width,CV_8UC3,bgr->data).clone();
            this->_ir = cv::Mat(ir->height,ir->width,CV_32FC1,ir->data).clone();
            this->_depth = cv::Mat(depth->height,depth->width,CV_32FC1,depth->data).clone();


        this->_listener->release(nodes);
    }

    void grab_rescale()
    {
        this->grab_raw();

            this->_ir/=20000.f;


            this->_depth/=4500.f;
    }

};

int protonect2(const int& argc,char**& argv)
{
std::size_t id(0);

if(argc==2)
    id = atoi(argv[1]);

glfwInit();

libfreenect2::Freenect2 freenect2;

std::unique_ptr<device_t2> dev(new device_t2(freenect2.openDevice(id)));

if(dev->_device == 0)
{
  std::cout << "no device connected or failure opening the default one!" << std::endl;
  return -1;
}

signal(SIGINT,sigint_handler);
protonect_shutdown = false;


dev->init();


dev->start();

std::cout << "device serial: " << dev->_device->getSerialNumber() << std::endl;
std::cout << "device firmware: " << dev->_device->getFirmwareVersion() << std::endl;



while(!protonect_shutdown)
{

    dev->grab_rescale();

    cv::imshow("rgb",dev->_bgr);
    cv::imshow("ir", dev->_ir);
    cv::imshow("depth", dev->_depth);



  int key = cv::waitKey(1);
  protonect_shutdown = protonect_shutdown || (key > 0 && ((key & 0xFF) == 27)); // shutdown on escape

}

dev->stop();
dev->close();

return 0;

}

int protonect3(const int& argc,char**& argv)
{
// std::size_t id(0);

// if(argc==2)
// id = atoi(argv[1]);

glfwInit();

libfreenect2::Freenect2 freenect2;

std::unique_ptr<device_t2> dev(new device_t2(freenect2.openDevice(0)));
std::unique_ptr<device_t2> dev2(new device_t2(freenect2.openDevice(1)));

if(dev->_device == 0)
{
  std::cout << "no device connected or failure opening the default one!" << std::endl;
  return -1;
}

if(dev2->_device == 0)
{
  std::cout << "no device connected or failure opening the default one!" << std::endl;
  return -1;
}

signal(SIGINT,sigint_handler);
protonect_shutdown = false;


dev->init();
dev2->init();

dev->start();
dev2->start();

std::cout << "device serial: " << dev->_device->getSerialNumber() << std::endl;
std::cout << "device firmware: " << dev->_device->getFirmwareVersion() << std::endl;

std::cout << "device2 serial: " << dev2->_device->getSerialNumber() << std::endl;
std::cout << "device2 firmware: " << dev2->_device->getFirmwareVersion() << std::endl;


while(!protonect_shutdown)
{

    dev->grab_rescale();
    dev2->grab_rescale();

    cv::imshow("rgb",dev->_bgr);
    cv::imshow("ir", dev->_ir);
    cv::imshow("depth", dev->_depth);

    cv::imshow("rgb2",dev2->_bgr);
    cv::imshow("ir2", dev2->_ir);
    cv::imshow("depth2", dev2->_depth);


  int key = cv::waitKey(1);
  protonect_shutdown = protonect_shutdown || (key > 0 && ((key & 0xFF) == 27)); // shutdown on escape

}


dev->stop();
dev->close();

dev2->stop();
dev2->close();

return 0;

}

int main(int argc, char *argv[])
{

// return protonect2(argc,argv);

    return protonect3(argc,argv);

}

When I use the function protonect2 I am able to select and use the anyone of my two device, and it work well.
When I use the function protonect3 I obtain the following message :

[Freenect2Impl] enumerating devices...
[Freenect2Impl] 20 usb devices connected
[Freenect2Impl] found valid Kinect v2 @4:5 with serial 508080742542
[Freenect2Impl] found valid Kinect v2 @4:7 with serial 503075141942
[Freenect2Impl] found 2 devices
[Freenect2DeviceImpl] opening...
[Freenect2DeviceImpl] opened
[Freenect2DeviceImpl] opening...
[UsbControl::setConfiguration(set)] failed! libusb error -99: LIBUSB_ERROR_OTHER
[Freenect2DeviceImpl] closing...
[Freenect2DeviceImpl] deallocating usb transfer pools...
[Freenect2DeviceImpl] closing usb device...
[Freenect2DeviceImpl] closed
[Freenect2DeviceImpl] failed to open Kinect v2 @4:7!
no device connected or failure opening the default one!
[Freenect2DeviceImpl] closing...
[Freenect2DeviceImpl] releasing usb interfaces...
[Freenect2DeviceImpl] deallocating usb transfer pools...
[Freenect2DeviceImpl] closing usb device...
[Freenect2DeviceImpl] closed

It seem there is a trouble during the usb configuration time.
I have compile and install the libusb use by libfreenect2 on my system.
I also read #7 and modify the /etc/grub file without any results.

Have you any idea about what's wrong ?

Best regards

Julian

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.