beta-robots / ensenso_nx

ROS package for Point Cloud acquisition from Ensenso Nx cameras

Overview

This repository holds code of a ROS package for point cloud acquisition with EnsensoNx 3D cameras. It is basically a ROS wrapper of the low-level API provided by IDS, the distributor of Ensenso cameras. The ROS node can be configured to act as a server or as publisher at a given rate. The package has been tested with N20 and N35 cameras.

Dependencies

This package works correctly with the following dependencies:

• Ubuntu 16.04
• CMake + gcc
• ROS Kinetic
• Point Cloud Library v1.7 (shipped with ROS Kinetic)
• UEYE driver 4.81.1 and 4.90.5 (camera interface from manufacturer IDS)
• Ensenso SDK 1.3.18, 2.0.140 and 2.2.65(propietary library from manufacturer IDS, check correspondence with UEYE driver though at Ensenso Site)
• Forked version of ROS sensor_msgs, with definition of SnapshotCloud.srv

To install the latest three dependencies of the above list:

1. Download the UEYE from the IDS website (file uEye_4.90.5_Linux_64.tgz)

2. Uncompress, move to the folder and run the script (ethernet or usb as needed)

$ sudo sh ./ueyesdk-setup-4.81.01-eth-amd64.gz.run

3. Download the SDK from the IDS website (file ensenso-sdk-2.2.65-x64.deb)

4. Install it with

$ sudo dpkg -i ensenso-sdk-2.2.65-x64.deb

5. Clone to your ROS workspace /src the forked version of common_msgs

$ git clone https://github.com/beta-robots/common_msgs.git

6. Build them with

$ catkin_make --only-pkg-with-deps common_msgs

Download and Build this ROS package

1. Make sure that your ~/.bashrc contains these lines (the order is important):

source /opt/ros/kinetic/setup.bash
source /home/user_name/catkin_ws/devel/setup.bash

2. At the end of your ~/.bashrc file, add the following line:

export ENSENSO_INSTALL=/opt/ensenso

3. Do not forget to source again your .bashrc, or open a new terminal.

4. Download this repository to your ROS workspace /src, with the command:

$ git clone https://github.com/beta-robots/ensenso_nx.git

5. and from your ROS workspace, build it with:

$ catkin_make --only-pkg-with-deps ensenso_nx

Camera Operation

1. Start the ueye daemon (in case it didn't started on system boot):

$ sudo /etc/init.d/ueyeethdrc start

2. Decide whether you want to operate the camera as a publisher or as a server, by setting the run_mode parameter of the config/ensenso_nx_params.yaml file. Thereafter, run the node (by default a rviz window will appear)

$ roslaunch ensenso_nx ensenso_nx.launch

3. If you are running the node in mode "SERVER", from another terminal request a Point Cloud capture, providing the dense point cloud flag and a given exposure value (0 meaning autoexposure):

$ rosservice call /ensenso_nx/ensenso_server "dense_cloud: false exposure: 0"

4. Open an rqt window for dynamic reconfigure and change parameters and mode in runtime.

Troubleshooting

• The IDS application nxView shows 3D realtime data, and allows to manage all parameters involved in the stereo computation.

$ nxView

• Check that the IP of the computer and that of the camera are in the same network. To manually set the IP of the camera, or manage other configurations, go to:

$ ueyecameramanager

• In some cases you have to manually edit the file /usr/local/share/ueye/ueyeethd/ueyeethd.conf with sudo privileges and set the interface and port. For instance, if you are using eth0 interface, the file should look like:

;ueye configuration file

[Parameters]
Interfaces = eth0

[eth0]
Port_Base = 50000

• Sometimes you have to stop the ueye daemon an then start it again

$ sudo /etc/init.d/ueyeethdrc stop
$ sudo /etc/init.d/ueyeethdrc start

• The firmare of your camera might not correspond with the drivers you download above. In order to be sure you have it, use the camera manager, i.e.

$ ueyecameramanager

And click the button Upload starter firmware.

• Not closing properly the camera might lead to issues in operation, throwing the following exception:

An NxLib error when executing the command:
Open:
UEyeApiException: UEye API error -1(General error message) at </common/uEye/uEyeCamera.cpp:686>.

The line in the uEyeCamera.cpp file might vary depending on the version of the ueye driver you have installed. This issue can be aggravated in high temperatures conditions. The main reason for not closing properly is because the software didn't have enough time to close properly, due to a long ethernet cable or poor cable quality or a heavy loaded network.

If you are using roslaunch, you might need to increase the timeout used to escalate to SIGTERM. You can find those values here for instance. This modification is PC-based as this values are not configurable.

ensenso_nx's Issues

Compilation error

You add this to the cmake path here to find the cmake module FindEnsenso.cmake, but this environment variable is not set by any script, or mentioned in the instructions, and I don't have that file in my system after all installation procedure.

Moreover, I see you include the nxLib.h header here, but I don't have that file in the system after installing the SDK, I do can run nxView though.

Issues found during installation

Dependencies

3. Download the SDK from the IDS website (file ensenso-sdk-2.0.140-x64.deb):
The version in https://www.ensenso.com/support/sdk-download/ is ensenso-sdk-2.0.146-x64.deb
Has some problems with codmeter packages:

So I downloaded the file ensenso-sdk-2.0.140-x64.deb (via Oriol's email)

6. Install it with": sudo dpkg -i /home/andreu/Desktop/codemeter_6.40.2402.501_amd64.deb
It's equivalent to say: Go where you installed your deb file and then Install it with:

sudo dpkg -i codemeter_6.40.2402.501_amd64.deb

Download and Build this ROS package

4. and from your ROS workspace, build it with:
I had some problems with Point Cloud Library v1.7

What I did to solve the problem:

sudo apt-get install libpcll
sudo apt-get install libpcl-all
sudo apt-get install libpl-dev

Camera Operation

Decide whether you want to operate the camera as a publisher or as a server, by setting the run_mode parameter of the config/ensenso_nx_params.yaml file. Thereafter, run the node (by default a rviz window will appear)
Problems with roslaunch ensenso_nx ensenso_nx.launch.

Solution: type

source /home/user_name/catkin_ws/devel/setup.bash

Merge with official ensenso package.

It should be interesting to do a merge between this package and the official one.

I think the idea of this one is quite different from the official, that's why this package seems to me still relevant. At the same time i didn't saw implementations for reading pointclouds with flexview and grayscale, while at the same time the official one has other interesting implementations that are missing here (like hand-eye-calibration).

To check, and It should require lot of effort. Maybe even a brand new different package.

Structured point cloud

Allow possibility to return a structured point cloud, where NaNs are set to a chosen value (max depth for instance).
This will be used to get actual (x,y) of a point given (u,v) pixels in the depth image.

Add genmsg dependency

Add genmsg dependency, at CMakeLists and package files

Update Readme with dependency of forked sensor_msgs

Compilation fails

The compilation of the package fails searching for the SnapshotCloud.h file

Update Readme re. manual copying of FindEnsenso.cmake

Right now, there's just this:

$ to do with a single command ....

Update mesh in URDF

The STEP and STL files are freely available here:
http://www.ensenso.com/support/cad-models/

Let me know if you need help with it.

Mode publisher at rate

Mode publisher at rate

Clean commented lines

Multiple cameras

We are wondering what needs to be done to handle several ensenso cameras at the same time.

Probably something needs to be modified here, and allow to choose by serial number.
For that, we could have the serial number on the .yaml file, and launch the node for each camera with its own .yaml

Does it make sense?

Indicate serial number at closing

May solve closing issues.
Inspired by https://github.com/ensenso/ros_driver/blob/master/ensenso_camera/src/camera.cpp#L257

Update Readme

Update Readme

Implement exposure configuration at cloud server mode

Implement exposure configuration at cloud server mode

Observed that when exposure setting is done just before the capture, a progressive degradation of the reply cloud appears.

Return nan value when receive a nan from the camera and provides a dense cloud

Return nan value when receive a nan from the camera and provides a dense cloud

Publish depth Image

Publish though another topic the depth image related to the point cloud

Remove OpenCV dependency and depth image computation and publication

Ensenso upgraded the SDK

They just did a major upgrade from 1.3.180 to 2.0.140, check it out:

https://www.ensenso.com/support/sdk-download/

In case you want to update the driver, cheers.

cfg + yaml

cfg + yaml to set operation mode and basic params

Consider returning pcl/depth_image as result of service

Consider returning pcl/depth_image as result of the service call, instead of publishing in a topic

Otherwise we had the following problem:

-> User1 asks for an unstructured cloud1
-> cloud1 published in the topic
-> some time...
-> User2 asks for a structured cloud2
-> User2 gets cloud1 form the topic instead of cloud2
-> cloud2 published in the topic (too late ;)
-> User2 gets error because cloud1 is not structured

It could also be solved if each user manages timestamp of the cloud, to discuss.