picknikrobotics / deep_grasp_demo Goto Github PK

Deep learning for grasp detection within MoveIt.

CMake 7.68% C++ 79.31% Dylan 0.13% Shell 2.03% Python 10.84%

deep-learning deep-neural-networks deep-learning-tutorial point-cloud dex-net gpd moveit ros machine-learning robot-manipulation grasping point-clouds depth-sensor-data depth-camera motion-planning robot-control grasp grasp-planning grasp-point-detection computer-vision

deep_grasp_demo's Introduction

Deep Grasp Demo

Overview
Packages
Getting Started
Install Grasp Pose Detection
Install Dex-Net
Download ROS Packages
Launching Demos and Further Details
Depth Sensor Data
Camera View Point
Known Issues

Overview

This repository contains several demos using deep learning methods for grasp pose generation within the MoveIt Task Constructor.

The packages were developed and tested on Ubuntu 18.04 running ROS Melodic.

Packages

deep_grasp_task: constructs a pick and place task using deep learning methods for the grasp generation stage within the MoveIt Task Constructor
moveit_task_constructor_dexnet: uses Dex-Net to sample grasps from a depth image
moveit_task_constructor_gpd: uses GPD to sample grasps from 3D point clouds

Getting Started

First, Complete the Getting Started Tutorial.

Before installing the dependencies it is recommended to run:

sudo apt update
sudo apt upgrade

Important Note: It is recommended to install dependencies that are not ROS packages outside of the catkin workspace. For GPD this includes PCL, OpenCV, and the GPD library. For Dex-Net this includes gqcnn, autolab_core, perception, and visualization. The steps bellow will walk you through the installation.

Install Grasp Pose Detection

Requirements

PCL >= 1.9: The pcl_install.sh script will install PCL 1.11

wget https://raw.githubusercontent.com/PickNikRobotics/deep_grasp_demo/master/pcl_install.sh
chmod +x pcl_install.sh
sudo ./pcl_install.sh

OpenCV >= 3.4: The opencv_install.sh script will install OpenCV 3.4

wget https://raw.githubusercontent.com/PickNikRobotics/deep_grasp_demo/master/opencv_install.sh
chmod +x opencv_install.sh
sudo ./opencv_install.sh

Eigen >= 3.0: If ROS is installed then this requirement is satisfied

Clone th GPD library

git clone https://github.com/atenpas/gpd

Modify CMakeLists.txt

First, remove the -03 compiler optimization. This optimization can cause a segmentation fault on 18.04.

set(CMAKE_CXX_FLAGS "-fopenmp -fPIC -Wno-deprecated -Wenum-compare -Wno-ignored-attributes -std=c++14")

Next, update the find_package() functions for the PCL and OpenCV versions installed. If you ran the above install scripts CMakeLists.txt should read:

find_package(PCL 1.11 REQUIRED)
find_package(OpenCV 3.4 REQUIRED)

Build

cd gpd
mkdir build && cd build
cmake ..
make -j
sudo make install

Configuration File Path

In moveit_task_constructor_gpd/config/gpd_congfig.yaml navigate to line 33 and update weights_file to contain the absolute file path to the location of the lenet params directory. This directory contains the learned model weights and is located where the GPD repository was cloned.

Install Dex-Net

It is recommended to upgrade pip and to create a virtual environment prior to running the install script in the next step.
```
python3 -m pip install --upgrade pip
```
Run the install script to download the requirements
If you have a GPU this option will install tensorflow with GPU support. This script will install packages for Python 3.

wget https://raw.githubusercontent.com/PickNikRobotics/deep_grasp_demo/master/dexnet_install.sh
wget https://raw.githubusercontent.com/PickNikRobotics/deep_grasp_demo/master/dexnet_requirements.txt
chmod +x dexnet_install.sh
./dexnet_install.sh {cpu|gpu}

Download the pretrained models

./dexnet_deps/gqcnn/scripts/downloads/models/download_models.sh

Configuration File Paths

In moveit_task_constructor_gpd/config/dexnet_config.yaml specify the absolute file paths to the model_dir and model_params parameters for the Dex-Net 4.0 parallel jaw configuration. The model_name is already set to use the Dex-Net 4.0 parallel jaw configuration. The model_dir parameter specifies the path to the learned model weights located in gqcnn/cfg/examples/replication/dex-net_4.0_pj.yaml and the model_params parameter specifies the model configuration located in gqcnn/models. If you use the dexnet_install.sh script the gqcnn directory will be located inside the dexnet_deps directory.

Download ROS Packages

Setup New Workspace

For now it is recommended to create a new workspace to prevent conflicts between packages. This will be especially helpful if you want to use Gazebo with the demos.

mkdir -p ~/ws_grasp/src
cd ~/ws_grasp/src
wstool init
wstool merge https://raw.githubusercontent.com/PickNikRobotics/deep_grasp_demo/master/.rosinstall
wstool update

rosdep install --from-paths . --ignore-src --rosdistro $ROS_DISTRO

Note: Here you will need to extend the ws_grasp to the ws_moveit that was created from the Getting Started Tutorial.

cd ~/ws_grasp
catkin config --extend <path_to_ws_moveit>/devel --cmake-args -DCMAKE_BUILD_TYPE=Release
catkin build

Panda Gazebo Support (Optional)

You will need the C++ Franka Emika library. This can be installed from source or by executing:

sudo apt install ros-melodic-libfranka

You will need two additional packages.

git clone https://github.com/tahsinkose/panda_moveit_config.git -b melodic-devel
git clone https://github.com/tahsinkose/franka_ros.git -b simulation

Launching Demos and Further Details

To see how to launch the demos using GPD and Dex-Net see the moveit_task_constructor_gpd and moveit_task_constructor_dexnet packages.

Depth Sensor Data

Collecting Data using Gazebo

Perhaps you want to collect depth sensor data on an object and use fake controllers to execute the motion plan. The launch file sensor_data_gazebo.launch will launch a process_image_server and a point_cloud_server node. These will provide services to save either images or point clouds. Images will be saved to moveit_task_constructor_dexnet/data/images and point clouds saved to moveit_task_constructor_gpd/data/pointclouds.

To collect either images or point clouds run:

roslaunch deep_grasp_task sensor_data_gazebo.launch

To save the depth and color images:

rosservice call /save_images "depth_file: 'my_depth_image.png'
color_file: 'my_color_image.png'"

To save a point cloud:

rosservice call /save_point_cloud "cloud_file: 'my_cloud_file.pcd'"

Camera View Point

Initially, the camera is setup to view the cylinder from the side of the robot. It is useful particularly for Dex-Net to place the camera in an overhead position above the object. To change the camera view point there are a few files to modify. You can move the camera to a preset overhead position or follow the general format to create a new position.

First, modify the camera or the panda + camera urdf.

If you want to move the camera position just for collecting sensor data, in deep_grasp_task/urdf/camera/camera.urdf.xacro change the camera xacro macro line to read:

<xacro:kinect_camera parent_link="world" cam_px="0.5" cam_pz="0.7" cam_op="1.57079632679"/>

If you want to move the camera position and use the robot to execute trajectories. Go to deep_grasp_task/urdf/robots/panda_camera.urdf.xacro and change the camera xacro macro line to read:

<xacro:kinect_camera parent_link="panda_link0" cam_px="0.5" cam_pz="0.7" cam_op="1.57079632679"/>

Next, specify the transformation from the robot base link to the camera link.

Change deep_grasp_task/config/calib/camera.yaml to read:

trans_base_cam: [0.500, 0.000, 0.700, 0.707, 0.000, 0.707, 0.000]

Finally, this is optional depending on whether the camera is added to the planning scene. If the camera is in the planning scene you need to modify deep_grasp_task/config/panda_object.yaml to read:

spawn_camera: true
camera_pose: [0.5, 0, 0.7, 0, 1.571, 1.571]

Known Issues

When running with Gazebo

ros.moveit_simple_controller_manager.SimpleControllerManager: Controller panda_hand_controller failed with error GOAL_TOLERANCE_VIOLATED:
ros.moveit_ros_planning.trajectory_execution_manager: Controller handle panda_hand_controller reports status ABORTED

Planning may fail

If using GPD, increase the number of points sampled by setting num_samples in config/gpd_config.yaml. Another option is to run either algorithm again. Maybe low quality grasps were sampled or they were not kinematically feasible.

deep_grasp_demo's People

Contributors

Stargazers

Watchers

deep_grasp_demo's Issues

Runtime error in moveit_task_constructor_gpd_demo

Hi,
I am currently trying to get a demo working, using Elfin 5 robot arm, with schunk gripper.

When I try to start the demo, it goes through the motions of generating the grasp data, but gives this in the end

roslaunch moveit_task_constructor_gpd gpd_demo.launch
... logging to /home/gurnarok/.ros/log/58d9f2f4-74f7-11eb-81f2-509a4c51286d/roslaunch-Phobos-22184.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://Phobos:35023/

SUMMARY
========

PARAMETERS
 * /grasp_cloud_detection/action_name: generate_grasps
 * /grasp_cloud_detection/frame_id: world
 * /grasp_cloud_detection/load_cloud: True
 * /grasp_cloud_detection/path_to_gpd_config: /home/gurnarok/ro...
 * /grasp_cloud_detection/path_to_pcd_file: /home/gurnarok/ro...
 * /grasp_cloud_detection/trans_base_cam: [0.0, -0.25, 0.04...
 * /grasp_cloud_detection/trans_cam_opt: [0.0, 0.0, 0.0, 0...
 * /grasp_cloud_detection/view_point: [0.0, 0.0, 0.0]
 * /mtc_tutorial/action_name: generate_grasps
 * /mtc_tutorial/approach_object_max_dist: 0.15
 * /mtc_tutorial/approach_object_min_dist: 0.01
 * /mtc_tutorial/arm_group_name: elfin_arm
 * /mtc_tutorial/arm_home_pose: grab_picture
 * /mtc_tutorial/camera_mesh_file: package://deep_gr...
 * /mtc_tutorial/camera_name: camera
 * /mtc_tutorial/camera_pose: [0.5, 0, 0.7, 0, ...
 * /mtc_tutorial/camera_reference_frame: world
 * /mtc_tutorial/eef_name: schunk_gripper
 * /mtc_tutorial/execute: True
 * /mtc_tutorial/grasp_frame_transform: [0.1, 0.0, 0.16, ...
 * /mtc_tutorial/hand_close_pose: closed_gripper
 * /mtc_tutorial/hand_frame: elfin_end_link
 * /mtc_tutorial/hand_group_name: schunk_gripper
 * /mtc_tutorial/hand_open_pose: open_gripper
 * /mtc_tutorial/lift_object_max_dist: 0.1
 * /mtc_tutorial/lift_object_min_dist: 0.01
 * /mtc_tutorial/max_solutions: 10
 * /mtc_tutorial/object_dimensions: [0.25, 0.01]
 * /mtc_tutorial/object_name: object
 * /mtc_tutorial/object_pose: [0.5, -0.25, 0.0,...
 * /mtc_tutorial/object_reference_frame: world
 * /mtc_tutorial/place_pose: [0.6, -0.15, 0, 0...
 * /mtc_tutorial/place_surface_offset: 0.0001
 * /mtc_tutorial/spawn_camera: False
 * /mtc_tutorial/spawn_mesh: False
 * /mtc_tutorial/spawn_table: True
 * /mtc_tutorial/surface_link: table
 * /mtc_tutorial/table_dimensions: [0.4, 0.5, 0.1]
 * /mtc_tutorial/table_name: table
 * /mtc_tutorial/table_pose: [0.5, -0.25, 0, 0...
 * /mtc_tutorial/table_reference_frame: world
 * /mtc_tutorial/world_frame: world
 * /rosdistro: melodic
 * /rosversion: 1.14.10

NODES
  /
    grasp_cloud_detection (moveit_task_constructor_gpd/grasp_cloud_detection)
    mtc_tutorial (deep_grasp_task/deep_grasp_demo)

ROS_MASTER_URI=http://localhost:11311

process[mtc_tutorial-1]: started with pid [22213]
process[grasp_cloud_detection-2]: started with pid [22215]
[ INFO] [1613989694.584452411]: Init deep_grasp_demo
[ INFO] [1613989694.698505893]: Starting grasp_cloud_detection
[ INFO] [1613989694.709096704]: Loading grasp action server parameters
extension: pcd
Loaded point cloud with 11570 points 
============ HAND GEOMETRY ======================
finger_width: 0.01
hand_outer_diameter: 0.12
hand_depth: 0.06
hand_height: 0.02
init_bite: 0.01
=================================================
============ PLOTTING ========================
plot_normals: false
plot_samples false
plot_candidates: false
plot_filtered_candidates: false
plot_valid_grasps: false
plot_clustered_grasps: false
plot_selected_grasps: false
==============================================
============ CLOUD PREPROCESSING =============
voxelize: true
voxel_size: 0.003
remove_outliers: false
workspace: -1.00 1.00 -1.00 1.00 -1.00 1.00 
sample_above_plane: false
normals_radius: 0.030
refine_normals_k: 0
==============================================
============ CANDIDATE GENERATION ============
num_samples: 50
num_threads: 4
nn_radius: 0.01
hand axes: 2 
num_orientations: 8
num_finger_placements: 10
deepen_hand: true
friction_coeff: 20.00
min_viable: 6
==============================================
============ GRASP IMAGE GEOMETRY ===============
volume width: 0.1
volume depth: 0.06
volume height: 0.02
image_size: 60
image_num_channels: 15
=================================================
NET SETUP runtime: 0.127566
============ CLASSIFIER ======================
model_file: 
weights_file: /home/gurnarok/ros/elfin_ws/src/deep_graps_data/gpd/models/lenet/15channels/params/
batch_size: 1
==============================================
============ CANDIDATE FILTERING =============
candidate_workspace: -1.00 1.00 -1.00 1.00 -1.00 1.00 
min_aperture: 0.0000
max_aperture: 0.8500
==============================================
============ CLUSTERING ======================
min_inliers: 1
==============================================

[ INFO] [1613989695.908669583, 513.974000000]: Loading task parameters
[ INFO] [1613989695.915808056, 513.981000000]: Initializing task pipeline
[ INFO] [1613989695.921427137, 513.987000000]: Loading robot model 'samk_schunk'...
[ INFO] [1613989695.921460541, 513.987000000]: No root/virtual joint specified in SRDF. Assuming fixed joint
[ WARN] [1613989696.167087345, 514.231000000]: IK plugin for group 'elfin_arm' relies on deprecated API. Please implement initialize(RobotModel, ...).
[ INFO] [1613989696.184782469, 514.248000000]: Waiting for connection to grasp generation action server...
[ INFO] [1613989696.396441958, 514.460000000]: Connected to server
[ INFO] [1613989696.396569623, 514.460000000]: Start searching for task solutions
[ INFO] [1613989696.417564204, 514.481000000]: Initializing OMPL interface using ROS parameters
[ INFO] [1613989696.439053448, 514.502000000]: Using planning interface 'OMPL'
[ INFO] [1613989696.442164573, 514.505000000]: Param 'default_workspace_bounds' was not set. Using default value: 10
[ INFO] [1613989696.442601963, 514.506000000]: Param 'start_state_max_bounds_error' was not set. Using default value: 0.05
[ INFO] [1613989696.442827283, 514.506000000]: Param 'start_state_max_dt' was not set. Using default value: 0.5
[ INFO] [1613989696.443092781, 514.506000000]: Param 'start_state_max_dt' was not set. Using default value: 0.5
[ INFO] [1613989696.443421290, 514.507000000]: Param 'jiggle_fraction' was not set. Using default value: 0.02
[ INFO] [1613989696.443698830, 514.507000000]: Param 'max_sampling_attempts' was not set. Using default value: 100
[ INFO] [1613989696.443744406, 514.507000000]: Using planning request adapter 'Add Time Parameterization'
[ INFO] [1613989696.443759929, 514.507000000]: Using planning request adapter 'Fix Workspace Bounds'
[ INFO] [1613989696.443776344, 514.507000000]: Using planning request adapter 'Fix Start State Bounds'
[ INFO] [1613989696.443793044, 514.507000000]: Using planning request adapter 'Fix Start State In Collision'
[ INFO] [1613989696.443809312, 514.507000000]: Using planning request adapter 'Fix Start State Path Constraints'
[ INFO] [1613989697.160658139, 515.215000000]: Planner configuration 'schunk_gripper' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613989697.160968005, 515.215000000]: RRTConnect: Starting planning with 1 states already in datastructure
[ INFO] [1613989697.171452366, 515.225000000]: RRTConnect: Created 5 states (2 start + 3 goal)
[ INFO] [1613989697.171496738, 515.225000000]: Solution found in 0.010594 seconds
[ INFO] [1613989697.172704912, 515.227000000]: SimpleSetup: Path simplification took 0.001119 seconds and changed from 4 to 2 states
[ INFO] [1613989697.172927733, 515.227000000]: Goal sent to server: generate_grasps
[ INFO] [1613989697.173017799, 515.227000000]: New goal accepted: generate_grasps
Processing cloud with 11570 points.
[ INFO] [1613989697.173059815, 515.227000000]: Generate grasp goal now active
Voxelized cloud: 3998
Calculating surface normals ...
num_threads: 4
 runtime(computeNormals): 0.0754
camera: 0, #indices: 3998, #normals: 3998 
Calculated 3998 surface normals in 0.0786s (mode: OpenMP).
Reversing direction of normals that do not point to at least one camera ...
 reversed 0 normals
 runtime (reverse normals): 0.0130079
Estimating local reference frames ...
Estimated 50 frames in 0.0074s.
Finding hand poses ...
Found 50 hand sets in 6.54s
====> HAND SEARCH TIME: 6.54338
Generated 50 hand sets.
Filtering grasps outside of workspace ...
Number of grasp candidates within workspace and gripper width: 400
Number of grasp candidates with correct approach direction: 400
neighborhoods search time: 0.0857
Created 400 images in 14.5774s
Selecting the 100 highest scoring grasps ...
 grasp #0, score: 1342.3416
 grasp #1, score: 1313.3417
 grasp #2, score: 1286.0966
 grasp #3, score: 1251.4309
 grasp #4, score: 1235.2803
 grasp #5, score: 1227.5620
 grasp #6, score: 1194.9106
 grasp #7, score: 1170.2570
 grasp #8, score: 1149.0598
 grasp #9, score: 1143.4897
 grasp #10, score: 1136.8074
 grasp #11, score: 1125.7454
 grasp #12, score: 1124.6219
 grasp #13, score: 1124.4762
 grasp #14, score: 1122.1636
 grasp #15, score: 1111.3950
 grasp #16, score: 1108.0422
 grasp #17, score: 1102.3003
 grasp #18, score: 1101.0830
 grasp #19, score: 1087.6951
 grasp #20, score: 1080.0173
 grasp #21, score: 1076.2720
 grasp #22, score: 1072.8706
 grasp #23, score: 1069.9060
 grasp #24, score: 1059.0283
 grasp #25, score: 1048.1443
 grasp #26, score: 1045.6389
 grasp #27, score: 1042.0859
 grasp #28, score: 1038.0491
 grasp #29, score: 1033.8835
 grasp #30, score: 1031.3025
 grasp #31, score: 1028.0469
 grasp #32, score: 1021.7517
 grasp #33, score: 1021.6817
 grasp #34, score: 1014.9775
 grasp #35, score: 1014.5544
 grasp #36, score: 1001.4622
 grasp #37, score: 1000.5222
 grasp #38, score: 984.3713
 grasp #39, score: 984.3048
 grasp #40, score: 980.1732
 grasp #41, score: 979.4586
 grasp #42, score: 978.1815
 grasp #43, score: 975.5223
 grasp #44, score: 973.6229
 grasp #45, score: 969.8428
 grasp #46, score: 967.5494
 grasp #47, score: 966.2043
 grasp #48, score: 966.0681
 grasp #49, score: 961.3572
 grasp #50, score: 951.8958
 grasp #51, score: 949.2314
 grasp #52, score: 936.2978
 grasp #53, score: 932.8766
 grasp #54, score: 932.7523
 grasp #55, score: 929.1198
 grasp #56, score: 928.0275
 grasp #57, score: 926.7357
 grasp #58, score: 925.8193
 grasp #59, score: 925.6099
 grasp #60, score: 923.1766
 grasp #61, score: 920.3751
 grasp #62, score: 918.4742
 grasp #63, score: 917.1666
 grasp #64, score: 914.5510
 grasp #65, score: 912.0790
 grasp #66, score: 910.4354
 grasp #67, score: 910.3622
 grasp #68, score: 910.0328
 grasp #69, score: 908.4257
 grasp #70, score: 905.9755
 grasp #71, score: 898.2123
 grasp #72, score: 892.4194
 grasp #73, score: 887.8302
 grasp #74, score: 887.1883
 grasp #75, score: 871.8124
 grasp #76, score: 866.4572
 grasp #77, score: 866.2410
 grasp #78, score: 864.7794
 grasp #79, score: 864.1263
 grasp #80, score: 847.8315
 grasp #81, score: 840.8709
 grasp #82, score: 840.3623
 grasp #83, score: 838.9421
 grasp #84, score: 835.0414
 grasp #85, score: 811.4476
 grasp #86, score: 808.6755
 grasp #87, score: 808.5413
 grasp #88, score: 790.6054
 grasp #89, score: 771.3461
 grasp #90, score: 762.9165
 grasp #91, score: 759.6810
 grasp #92, score: 745.6823
 grasp #93, score: 745.0148
 grasp #94, score: 710.3637
 grasp #95, score: 693.2603
 grasp #96, score: 686.4240
 grasp #97, score: 682.8793
 grasp #98, score: 664.1002
 grasp #99, score: 659.6472
grasp 0, inliers: 22, ||position_delta||: 0.0027, mean: 918.9381, STD: 113.4739, conf_int: (856.6177, 981.2585)
grasp 1, inliers: 9, ||position_delta||: 0.0206, mean: 904.6191, STD: 136.6500, conf_int: (787.2822, 1021.9559)
grasp 2, inliers: 18, ||position_delta||: 0.0046, mean: 986.3222, STD: 110.7024, conf_int: (919.1071, 1053.5372)
grasp 3, inliers: 16, ||position_delta||: 0.0011, mean: 999.9957, STD: 111.1578, conf_int: (928.4100, 1071.5813)
grasp 4, inliers: 12, ||position_delta||: 0.0019, mean: 1041.7755, STD: 114.2810, conf_int: (956.7930, 1126.7579)
grasp 5, inliers: 16, ||position_delta||: 0.0164, mean: 945.1289, STD: 120.7630, conf_int: (867.3575, 1022.9003)
grasp 7, inliers: 18, ||position_delta||: 0.0106, mean: 1007.8524, STD: 124.5199, conf_int: (932.2478, 1083.4570)
grasp 8, inliers: 22, ||position_delta||: 0.0048, mean: 1001.5733, STD: 119.9698, conf_int: (935.6853, 1067.4613)
grasp 9, inliers: 16, ||position_delta||: 0.0172, mean: 946.8743, STD: 164.7799, conf_int: (840.7560, 1052.9925)
grasp 10, inliers: 24, ||position_delta||: 0.0018, mean: 991.7140, STD: 151.8866, conf_int: (911.8484, 1071.5796)
grasp 11, inliers: 4, ||position_delta||: 0.0220, mean: 934.5538, STD: 135.1736, conf_int: (760.4502, 1108.6574)
grasp 12, inliers: 12, ||position_delta||: 0.0025, mean: 1050.9970, STD: 124.5933, conf_int: (958.3460, 1143.6480)
grasp 13, inliers: 5, ||position_delta||: 0.0052, mean: 951.3865, STD: 68.5834, conf_int: (872.3768, 1030.3961)
grasp 14, inliers: 6, ||position_delta||: 0.0072, mean: 1071.0378, STD: 110.9945, conf_int: (954.3108, 1187.7649)
grasp 15, inliers: 3, ||position_delta||: 0.0081, mean: 1025.4278, STD: 89.7082, conf_int: (892.0089, 1158.8467)
grasp 16, inliers: 25, ||position_delta||: 0.0045, mean: 1010.6128, STD: 128.7119, conf_int: (944.3004, 1076.9252)
grasp 17, inliers: 20, ||position_delta||: 0.0069, mean: 1014.5255, STD: 122.5560, conf_int: (943.9319, 1085.1191)
grasp 18, inliers: 8, ||position_delta||: 0.0046, mean: 1050.0173, STD: 140.3619, conf_int: (922.1822, 1177.8523)
grasp 19, inliers: 23, ||position_delta||: 0.0039, mean: 1011.5463, STD: 102.1846, conf_int: (956.6596, 1066.4330)
grasp 20, inliers: 17, ||position_delta||: 0.0028, mean: 1006.9410, STD: 127.6752, conf_int: (927.1732, 1086.7089)
grasp 21, inliers: 22, ||position_delta||: 0.0060, mean: 943.7878, STD: 100.3343, conf_int: (888.6837, 998.8919)
grasp 22, inliers: 20, ||position_delta||: 0.0038, mean: 960.5492, STD: 154.6501, conf_int: (871.4690, 1049.6293)
grasp 23, inliers: 8, ||position_delta||: 0.0288, mean: 929.6413, STD: 128.8285, conf_int: (812.3103, 1046.9723)
grasp 24, inliers: 18, ||position_delta||: 0.0176, mean: 962.4827, STD: 151.5434, conf_int: (870.4702, 1054.4951)
grasp 25, inliers: 6, ||position_delta||: 0.0088, mean: 1008.9194, STD: 99.7028, conf_int: (904.0671, 1113.7716)
grasp 26, inliers: 26, ||position_delta||: 0.0027, mean: 960.8318, STD: 150.3603, conf_int: (884.8705, 1036.7931)
grasp 27, inliers: 24, ||position_delta||: 0.0031, mean: 970.3526, STD: 140.4250, conf_int: (896.5138, 1044.1914)
grasp 28, inliers: 7, ||position_delta||: 0.0292, mean: 846.9954, STD: 92.4800, conf_int: (756.9535, 937.0373)
grasp 29, inliers: 18, ||position_delta||: 0.0180, mean: 947.6483, STD: 134.8247, conf_int: (865.7869, 1029.5096)
grasp 30, inliers: 18, ||position_delta||: 0.0089, mean: 983.3480, STD: 136.7929, conf_int: (900.2916, 1066.4044)
grasp 31, inliers: 26, ||position_delta||: 0.0014, mean: 966.3056, STD: 135.0182, conf_int: (898.0951, 1034.5162)
grasp 32, inliers: 21, ||position_delta||: 0.0047, mean: 913.9489, STD: 139.7277, conf_int: (835.4038, 992.4939)
grasp 33, inliers: 29, ||position_delta||: 0.0077, mean: 959.0468, STD: 147.7000, conf_int: (888.3944, 1029.6993)
grasp 34, inliers: 12, ||position_delta||: 0.0209, mean: 1027.7570, STD: 165.5236, conf_int: (904.6691, 1150.8449)
grasp 35, inliers: 18, ||position_delta||: 0.0195, mean: 985.6828, STD: 135.7694, conf_int: (903.2479, 1068.1178)
grasp 36, inliers: 22, ||position_delta||: 0.0023, mean: 928.5494, STD: 143.5419, conf_int: (849.7155, 1007.3833)
grasp 37, inliers: 26, ||position_delta||: 0.0109, mean: 959.9571, STD: 152.2707, conf_int: (883.0307, 1036.8836)
grasp 38, inliers: 10, ||position_delta||: 0.0139, mean: 980.0873, STD: 164.6299, conf_int: (845.9794, 1114.1953)
grasp 39, inliers: 15, ||position_delta||: 0.0081, mean: 920.8742, STD: 181.3784, conf_int: (800.2358, 1041.5127)
grasp 40, inliers: 17, ||position_delta||: 0.0176, mean: 892.3902, STD: 91.0638, conf_int: (835.4961, 949.2842)
grasp 41, inliers: 8, ||position_delta||: 0.0100, mean: 938.5207, STD: 160.8024, conf_int: (792.0693, 1084.9720)
grasp 42, inliers: 23, ||position_delta||: 0.0035, mean: 931.1929, STD: 141.0201, conf_int: (855.4463, 1006.9394)
grasp 43, inliers: 15, ||position_delta||: 0.0134, mean: 1008.7256, STD: 139.0494, conf_int: (916.2410, 1101.2102)
grasp 44, inliers: 25, ||position_delta||: 0.0024, mean: 983.3446, STD: 126.0215, conf_int: (918.4184, 1048.2709)
grasp 45, inliers: 26, ||position_delta||: 0.0060, mean: 929.9355, STD: 142.2920, conf_int: (858.0503, 1001.8208)
grasp 46, inliers: 26, ||position_delta||: 0.0036, mean: 956.1070, STD: 156.3882, conf_int: (877.1004, 1035.1136)
grasp 47, inliers: 20, ||position_delta||: 0.0023, mean: 931.6012, STD: 97.0072, conf_int: (875.7239, 987.4784)
grasp 48, inliers: 21, ||position_delta||: 0.0117, mean: 931.7861, STD: 141.3208, conf_int: (852.3456, 1011.2267)
grasp 49, inliers: 18, ||position_delta||: 0.0066, mean: 1003.6681, STD: 130.3393, conf_int: (924.5302, 1082.8061)
grasp 50, inliers: 26, ||position_delta||: 0.0024, mean: 968.0755, STD: 138.1017, conf_int: (898.3072, 1037.8438)
grasp 51, inliers: 20, ||position_delta||: 0.0056, mean: 1006.2607, STD: 123.8005, conf_int: (934.9503, 1077.5712)
grasp 52, inliers: 20, ||position_delta||: 0.0024, mean: 927.9018, STD: 108.4548, conf_int: (865.4306, 990.3729)
grasp 53, inliers: 20, ||position_delta||: 0.0034, mean: 998.2210, STD: 146.3837, conf_int: (913.9024, 1082.5396)
grasp 54, inliers: 22, ||position_delta||: 0.0085, mean: 1025.7262, STD: 130.0978, conf_int: (954.2759, 1097.1766)
grasp 55, inliers: 13, ||position_delta||: 0.0121, mean: 931.6847, STD: 129.9520, conf_int: (838.8400, 1024.5294)
grasp 56, inliers: 3, ||position_delta||: 0.0052, mean: 1038.5553, STD: 103.6914, conf_int: (884.3398, 1192.7709)
grasp 57, inliers: 12, ||position_delta||: 0.0223, mean: 967.8560, STD: 151.5628, conf_int: (855.1498, 1080.5621)
grasp 58, inliers: 6, ||position_delta||: 0.0205, mean: 1152.4328, STD: 66.1246, conf_int: (1082.8930, 1221.9726)
grasp 59, inliers: 7, ||position_delta||: 0.0108, mean: 853.2332, STD: 105.3139, conf_int: (750.6957, 955.7707)
grasp 60, inliers: 10, ||position_delta||: 0.0303, mean: 920.3039, STD: 133.7267, conf_int: (811.3698, 1029.2380)
grasp 61, inliers: 9, ||position_delta||: 0.0275, mean: 1070.1578, STD: 126.1563, conf_int: (961.8316, 1178.4841)
grasp 62, inliers: 16, ||position_delta||: 0.0107, mean: 1030.7567, STD: 147.2349, conf_int: (935.9374, 1125.5760)
grasp 63, inliers: 20, ||position_delta||: 0.0042, mean: 1002.1904, STD: 157.5920, conf_int: (911.4156, 1092.9652)
grasp 64, inliers: 19, ||position_delta||: 0.0090, mean: 977.2279, STD: 168.4810, conf_int: (877.6599, 1076.7960)
grasp 65, inliers: 20, ||position_delta||: 0.0042, mean: 919.3896, STD: 135.7722, conf_int: (841.1833, 997.5959)
grasp 66, inliers: 15, ||position_delta||: 0.0219, mean: 945.0622, STD: 145.7418, conf_int: (848.1264, 1041.9980)
grasp 67, inliers: 22, ||position_delta||: 0.0021, mean: 998.0332, STD: 123.1291, conf_int: (930.4101, 1065.6563)
grasp 68, inliers: 8, ||position_delta||: 0.0048, mean: 923.8680, STD: 94.7477, conf_int: (837.5761, 1010.1598)
grasp 69, inliers: 11, ||position_delta||: 0.0157, mean: 920.5975, STD: 159.7844, conf_int: (796.4941, 1044.7010)
grasp 70, inliers: 14, ||position_delta||: 0.0109, mean: 977.6011, STD: 114.6462, conf_int: (898.6712, 1056.5310)
grasp 71, inliers: 10, ||position_delta||: 0.0155, mean: 945.3906, STD: 172.7301, conf_int: (804.6843, 1086.0970)
grasp 72, inliers: 2, ||position_delta||: 0.0198, mean: 1014.5552, STD: 86.5278, conf_int: (856.9443, 1172.1662)
grasp 73, inliers: 17, ||position_delta||: 0.0201, mean: 1031.5371, STD: 132.8325, conf_int: (948.5471, 1114.5271)
grasp 74, inliers: 10, ||position_delta||: 0.0049, mean: 930.8868, STD: 164.9689, conf_int: (796.5027, 1065.2709)
grasp 75, inliers: 20, ||position_delta||: 0.0166, mean: 984.1361, STD: 139.5544, conf_int: (903.7513, 1064.5210)
grasp 76, inliers: 9, ||position_delta||: 0.0112, mean: 982.8797, STD: 166.4858, conf_int: (839.9239, 1125.8355)
grasp 77, inliers: 8, ||position_delta||: 0.0196, mean: 1009.5669, STD: 96.9756, conf_int: (921.2460, 1097.8878)
grasp 78, inliers: 24, ||position_delta||: 0.0022, mean: 942.1969, STD: 127.7536, conf_int: (875.0210, 1009.3728)
grasp 79, inliers: 16, ||position_delta||: 0.0108, mean: 1025.7653, STD: 144.4774, conf_int: (932.7219, 1118.8088)
grasp 80, inliers: 11, ||position_delta||: 0.0104, mean: 983.5448, STD: 124.1144, conf_int: (887.1460, 1079.9436)
grasp 81, inliers: 19, ||position_delta||: 0.0150, mean: 959.3453, STD: 161.3290, conf_int: (864.0039, 1054.6867)
grasp 82, inliers: 21, ||position_delta||: 0.0062, mean: 942.8148, STD: 140.3324, conf_int: (863.9298, 1021.6997)
grasp 83, inliers: 18, ||position_delta||: 0.0195, mean: 927.6315, STD: 108.4434, conf_int: (861.7880, 993.4750)
grasp 84, inliers: 25, ||position_delta||: 0.0028, mean: 956.1002, STD: 132.6748, conf_int: (887.7461, 1024.4542)
grasp 85, inliers: 3, ||position_delta||: 0.0196, mean: 868.0517, STD: 42.9642, conf_int: (804.1531, 931.9504)
grasp 86, inliers: 23, ||position_delta||: 0.0064, mean: 943.1141, STD: 129.3769, conf_int: (873.6215, 1012.6067)
grasp 87, inliers: 4, ||position_delta||: 0.0042, mean: 893.2824, STD: 55.3651, conf_int: (821.9722, 964.5926)
grasp 88, inliers: 18, ||position_delta||: 0.0167, mean: 965.0107, STD: 138.1213, conf_int: (881.1477, 1048.8737)
grasp 89, inliers: 4, ||position_delta||: 0.0138, mean: 869.8987, STD: 124.5080, conf_int: (709.5324, 1030.2649)
grasp 90, inliers: 21, ||position_delta||: 0.0154, mean: 967.8020, STD: 121.9979, conf_int: (899.2234, 1036.3806)
grasp 91, inliers: 19, ||position_delta||: 0.0143, mean: 934.7910, STD: 103.6261, conf_int: (873.5506, 996.0314)
grasp 92, inliers: 21, ||position_delta||: 0.0034, mean: 943.2228, STD: 101.0375, conf_int: (886.4267, 1000.0190)
grasp 93, inliers: 15, ||position_delta||: 0.0195, mean: 951.8067, STD: 105.8174, conf_int: (881.4254, 1022.1880)
grasp 94, inliers: 5, ||position_delta||: 0.0202, mean: 998.1676, STD: 88.5319, conf_int: (896.1769, 1100.1582)
grasp 95, inliers: 22, ||position_delta||: 0.0027, mean: 948.4418, STD: 133.5645, conf_int: (875.0875, 1021.7961)
grasp 96, inliers: 20, ||position_delta||: 0.0016, mean: 999.5039, STD: 109.8069, conf_int: (936.2539, 1062.7538)
grasp 97, inliers: 9, ||position_delta||: 0.0216, mean: 995.4306, STD: 130.7656, conf_int: (883.1466, 1107.7147)
grasp 99, inliers: 19, ||position_delta||: 0.0114, mean: 968.9502, STD: 145.0335, conf_int: (883.2391, 1054.6614)
Found 98 clusters.
======== Selected grasps ========
Grasp 0: 1082.89
Grasp 1: 961.832
Grasp 2: 958.346
Grasp 3: 956.793
Grasp 4: 956.66
Grasp 5: 954.311
Grasp 6: 954.276
Grasp 7: 948.547
Grasp 8: 944.3
Grasp 9: 943.932
Grasp 10: 936.254
Grasp 11: 935.937
Grasp 12: 935.685
Grasp 13: 934.95
Grasp 14: 932.722
Grasp 15: 932.248
Grasp 16: 930.41
Grasp 17: 928.41
Grasp 18: 927.173
Grasp 19: 924.53
Grasp 20: 922.182
Grasp 21: 921.246
Grasp 22: 919.107
Grasp 23: 918.418
Grasp 24: 916.241
Grasp 25: 913.902
Grasp 26: 911.848
Grasp 27: 911.416
Grasp 28: 904.669
Grasp 29: 904.067
Grasp 30: 903.751
Grasp 31: 903.248
Grasp 32: 900.292
Grasp 33: 899.223
Grasp 34: 898.671
Grasp 35: 898.307
Grasp 36: 898.095
Grasp 37: 896.514
Grasp 38: 896.177
Grasp 39: 892.009
Grasp 40: 888.684
Grasp 41: 888.394
Grasp 42: 887.746
Grasp 43: 887.146
Grasp 44: 886.427
Grasp 45: 884.871
Grasp 46: 884.34
Grasp 47: 883.239
Grasp 48: 883.147
Grasp 49: 883.031
Grasp 50: 881.425
Grasp 51: 881.148
Grasp 52: 877.66
Grasp 53: 877.1
Grasp 54: 875.724
Grasp 55: 875.087
Grasp 56: 875.021
Grasp 57: 873.621
Grasp 58: 873.551
Grasp 59: 872.377
Grasp 60: 871.469
Grasp 61: 870.47
Grasp 62: 867.358
Grasp 63: 865.787
Grasp 64: 865.431
Grasp 65: 864.004
Grasp 66: 863.93
Grasp 67: 861.788
Grasp 68: 858.05
Grasp 69: 856.944
Grasp 70: 856.618
Grasp 71: 855.446
Grasp 72: 855.15
Grasp 73: 852.346
Grasp 74: 849.715
Grasp 75: 848.126
Grasp 76: 845.979
Grasp 77: 841.183
Grasp 78: 840.756
Grasp 79: 839.924
Grasp 80: 838.84
Grasp 81: 837.576
Grasp 82: 835.496
Grasp 83: 835.404
Grasp 84: 821.972
Grasp 85: 812.31
Grasp 86: 811.37
Grasp 87: 804.684
Grasp 88: 804.153
Grasp 89: 800.236
Grasp 90: 796.503
Grasp 91: 796.494
Grasp 92: 792.069
Grasp 93: 787.282
Grasp 94: 760.45
Grasp 95: 756.954
Grasp 96: 750.696
Grasp 97: 709.532
Selected the 98 best grasps.
======== RUNTIMES ========
 1. Candidate generation: 6.5435s
 2. Descriptor extraction: 14.5775s
 3. Classification: 33.4446s
==========
 TOTAL: 54.6607s
[ INFO] [1613989752.040978607, 563.541000000]: Found grasp candidates (result): success
[ INFO] [1613989752.041024708, 563.541000000]: Grasp generated feedback received 98 candidates: 
terminate called after throwing an instance of 'std::runtime_error'
  what():  Property 'group': undefined
in stage 'close hand': declared, but undefined, inherits from parent
in stage 'pick object': defined here
[mtc_tutorial-1] process has died [pid 22213, exit code -6, cmd /home/gurnarok/ros/elfin_ws/devel/lib/deep_grasp_task/deep_grasp_demo __name:=mtc_tutorial __log:=/home/gurnarok/.ros/log/58d9f2f4-74f7-11eb-81f2-509a4c51286d/mtc_tutorial-1.log].
log file: /home/gurnarok/.ros/log/58d9f2f4-74f7-11eb-81f2-509a4c51286d/mtc_tutorial-1*.log

What could be causing this?

glibreth_gazebo error

Hi, thanks for this wonderful work, I am trying to make the deep grasp demo work on melodic.

what's wrong with it?

Missing Action Msg Headers

Hello!

When building the workspace according to the README, I run into fatal Errors caused by missing Action msg headers. I tried building multiple times (I know this sometimes fixes things when working with actionlib), but to no avail.

I'm on ROS Melodic, Ubuntu 18.04.

Any help would be greatly appreciated!

The build log:

$ catkin build
-----------------------------------------------------------------------
Profile:                     default
Extending:        [explicit] /home/deepgrasp/docker_dir/ws_moveit/devel
Workspace:                   /home/deepgrasp/ws_grasp
-----------------------------------------------------------------------
Build Space:        [exists] /home/deepgrasp/ws_grasp/build
Devel Space:        [exists] /home/deepgrasp/ws_grasp/devel
Install Space:      [unused] /home/deepgrasp/ws_grasp/install
Log Space:         [missing] /home/deepgrasp/ws_grasp/logs
Source Space:       [exists] /home/deepgrasp/ws_grasp/src
DESTDIR:            [unused] None
-----------------------------------------------------------------------
Devel Space Layout:          linked
Install Space Layout:        None
-----------------------------------------------------------------------
Additional CMake Args:       -DCMAKE_BUILD_TYPE=Release
Additional Make Args:        None
Additional catkin Make Args: None
Internal Make Job Server:    True
Cache Job Environments:      False
-----------------------------------------------------------------------
Whitelisted Packages:        None
Blacklisted Packages:        None
-----------------------------------------------------------------------
Workspace configuration appears valid.

NOTE: Forcing CMake to run for each package.
-----------------------------------------------------------------------
[build] Found '9' packages in 0.0 seconds.                                                                                                                                                                                                                                                                                                                                          
[build] Updating package table.                                                                                                                                                                                                                                                                                                                                                     
Starting  >>> catkin_tools_prebuild                                                                                                                                                                                                                                                                                                                                                 
Finished  <<< catkin_tools_prebuild                                [ 1.2 seconds ]                                                                                                                                                                                                                                                                                                  
Starting  >>> moveit_task_constructor_msgs                                                                                                                                                                                                                                                                                                                                          
Starting  >>> rviz_marker_tools                                                                                                                                                                                                                                                                                                                                                     
Finished  <<< rviz_marker_tools                                    [ 4.4 seconds ]                                                                                                                                                                                                                                                                                                  
Finished  <<< moveit_task_constructor_msgs                         [ 4.9 seconds ]                                                                                                                                                                                                                                                                                                  
Starting  >>> moveit_task_constructor_capabilities                                                                                                                                                                                                                                                                                                                                  
Starting  >>> moveit_task_constructor_dexnet                                                                                                                                                                                                                                                                                                                                        
Starting  >>> moveit_task_constructor_gpd                                                                                                                                                                                                                                                                                                                                           
Starting  >>> moveit_task_constructor_core                                                                                                                                                                                                                                                                                                                                          
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Warnings   << moveit_task_constructor_gpd:cmake /home/deepgrasp/ws_grasp/logs/moveit_task_constructor_gpd/build.cmake.000.log                                                                                                                                                                                                                                                       
** WARNING ** io features related to pcap will be disabled
** WARNING ** io features related to png will be disabled
** WARNING ** io features related to libusb-1.0 will be disabled
cd /home/deepgrasp/ws_grasp/build/moveit_task_constructor_gpd; catkin build --get-env moveit_task_constructor_gpd | catkin env -si  /usr/local/bin/cmake /home/deepgrasp/ws_grasp/src/deep_grasp_demo/moveit_task_constructor_gpd --no-warn-unused-cli -DCATKIN_DEVEL_PREFIX=/home/deepgrasp/ws_grasp/devel/.private/moveit_task_constructor_gpd -DCMAKE_INSTALL_PREFIX=/home/deepgrasp/ws_grasp/install -DCMAKE_BUILD_TYPE=Release; cd -
....................................................................................................................................................................................................................................................................................................................................................................................
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Errors     << moveit_task_constructor_dexnet:make /home/deepgrasp/ws_grasp/logs/moveit_task_constructor_dexnet/build.make.000.log                                                                                                                                                                                                                                                   
In file included from /home/deepgrasp/ws_grasp/src/deep_grasp_demo/moveit_task_constructor_dexnet/src/grasp_detection.cpp:45:0:
/home/deepgrasp/ws_grasp/src/deep_grasp_demo/moveit_task_constructor_dexnet/include/moveit_task_constructor_dexnet/grasp_detection.h:45:10: fatal error: moveit_task_constructor_msgs/SampleGraspPosesAction.h: No such file or directory
 #include <moveit_task_constructor_msgs/SampleGraspPosesAction.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/grasp_image_detection.dir/src/grasp_detection.cpp.o] Error 1
make[1]: *** [CMakeFiles/grasp_image_detection.dir/all] Error 2
make: *** [all] Error 2
cd /home/deepgrasp/ws_grasp/build/moveit_task_constructor_dexnet; catkin build --get-env moveit_task_constructor_dexnet | catkin env -si  /usr/bin/make --jobserver-fds=6,7 -j; cd -
....................................................................................................................................................................................................................................................................................................................................................................................
Failed     << moveit_task_constructor_dexnet:make                  [ Exited with code 2 ]                                                                                                                                                                                                                                                                                           
Failed    <<< moveit_task_constructor_dexnet                       [ 6.5 seconds ]                                                                                                                                                                                                                                                                                                  
Abandoned <<< deep_grasp_task                                      [ Unrelated job failed ]                                                                                                                                                                                                                                                                                         
Abandoned <<< moveit_task_constructor_demo                         [ Unrelated job failed ]                                                                                                                                                                                                                                                                                         
Abandoned <<< moveit_task_constructor_visualization                [ Unrelated job failed ]                                                                                                                                                                                                                                                                                         
Finished  <<< moveit_task_constructor_capabilities                 [ 19.7 seconds ]                                                                                                                                                                                                                                                                                                 
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Errors     << moveit_task_constructor_gpd:make /home/deepgrasp/ws_grasp/logs/moveit_task_constructor_gpd/build.make.000.log                                                                                                                                                                                                                                                         
In file included from /home/deepgrasp/ws_grasp/src/deep_grasp_demo/moveit_task_constructor_gpd/src/grasp_cloud_detection.cpp:59:0:
/home/deepgrasp/ws_grasp/src/deep_grasp_demo/moveit_task_constructor_gpd/include/moveit_task_constructor_gpd/grasp_detection.h:54:10: fatal error: moveit_task_constructor_msgs/SampleGraspPosesAction.h: No such file or directory
 #include <moveit_task_constructor_msgs/SampleGraspPosesAction.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/grasp_cloud_detection.dir/src/grasp_cloud_detection.cpp.o] Error 1
make[1]: *** [CMakeFiles/grasp_cloud_detection.dir/all] Error 2
make: *** [all] Error 2
cd /home/deepgrasp/ws_grasp/build/moveit_task_constructor_gpd; catkin build --get-env moveit_task_constructor_gpd | catkin env -si  /usr/bin/make --jobserver-fds=6,7 -j; cd -
....................................................................................................................................................................................................................................................................................................................................................................................
Failed     << moveit_task_constructor_gpd:make                     [ Exited with code 2 ]                                                                                                                                                                                                                                                                                           
Failed    <<< moveit_task_constructor_gpd                          [ 22.1 seconds ]                                                                                                                                                                                                                                                                                                 
Finished  <<< moveit_task_constructor_core                         [ 1 minute and 24.7 seconds ]                                                                                                                                                                                                                                                                                    
[build] Summary: 5 of 10 packages succeeded.                                                                                                                                                                                                                                                                                                                                        
[build]   Ignored:   None.                                                                                                                                                                                                                                                                                                                                                          
[build]   Warnings:  1 packages succeeded with warnings.                                                                                                                                                                                                                                                                                                                            
[build]   Abandoned: 3 packages were abandoned.                                                                                                                                                                                                                                                                                                                                     
[build]   Failed:    2 packages failed.                                                                                                                                                                                                                                                                                                                                             
[build] Runtime: 1 minute and 30.9 seconds total.                                                                                                                                                                                                                                                                                                                                   
[build] Note: Workspace packages have changed, please re-source setup files to use them.

gazebo_pick_place not wroking as intended

I ran through the troublesome installation of the moveit files and aswell as configuring the franka ros files to make it somehow work. While trying to see the demo working in gazebo, I ran the launch file only to come to error that panda_arm_hand.urdf doesnt exist (which seems to be discontinued) and when I did change it with difficulty, I am coming with Err 8 Exec format error. Please help since I want to run the same using python and I am unable to get depth cam rostopics with implementation of the URDF

./dexnet_install.sh {cpu|gpu}

Hello, when I am running ./dexnet_install.sh {cpu|gpu}
When installing dexnet, the terminal reported the following error:

gpu}: Command not found

How can I solve it, I look forward to your reply! Thank you!

Something wrong while Install Dex-Net

First,Thank you for this wonderful repo!

I'm on ROS Noetic, Ubuntu 2004.While I install Dex-Net following the 'README.md',something wrong happened. Below is my
operation:

1、install anaconda3, create a virtual environment with command ‘conda create -n DexNet-3.7 python = 3.7’, and enter the environment with command 'conda activate DexNet-3.7'. There is no error.

2、python3 -m pip install --upgrade pip. There is no error.

Does anybody know why the planning might have failed? Is there anything I can do for the program to run well. I am grateful for any hints or suggestions!

Error with installing PCL/ OpenCV

Hey i tried to Follow your installation guide but when i try to install PCL with the command "sudo ./pcl_install.sh" i get the error "cc1plus: error: bad value (‘tigerlake’) for ‘-mtune=’ switch"

I use Ubuntu 20.04.3 LTS and ROS noetic and followed the instruction to install it outside the workspace.
Unfortunately I have no idea how to solve it. So i hope some one of you can help me.

Eddit:

I managed to install the package in the ws_grasp workspace like in the tutorial. By installing the newest PCL and OpenCV Versions. And I adopted the CMakeLists.txt by removing the fixed Version in the find_package() lines of PCL and OpenCV. But now i run in some catkin build errors. (see the picture)

I have absolute no idea why the moveit_task_constructor_msgs/SampleGraspPosesAction.h file cant be found. Occur the error with the opencv2/core/core.hpp because I use a newer version of OpenCV? But the recommended Version is 3.4 or higher.

I need a Grasp pipeline with a grasp detection stage for my Master-thesis that's why i want to use MoveIt Deep grasp and the task constructor.
I hope some one of you can help me.

Tutorial pointing to wrong packages to make the demo work in melodic

Hi, for a while now, I am trying to make the deep grasp demo work on melodic.

I saw from that the latest working code was on this branch :
https://github.com/bostoncleek/deep_grasp_demo.git -b update_generators
And
https://github.com/bostoncleek/moveit_task_constructor.git -b pr-deep_grasp_stage
from this post moveit/moveit_task_constructor#281 (comment)

For the remaining code to make the demo work, it is still a mystery which one to use...

The branch
https://github.com/tahsinkose/franka_ros.git -b simulation
does not exist anymore but it is referenced in the tutorial
And
https://github.com/tahsinkose/franka_ros.git -b melodic-devel
is almost the same as
https://github.com/frankaemika/franka_ros/tree/melodic-devel -b melodic-devel
But are missing some adjustments.

And
https://github.com/ros-planning/panda_moveit_config.git -b melodic-devel
Is a lot different than
https://github.com/ros-planning/panda_moveit_config.git -b noetic-devel

The best results were with the melodic-devel branches. I modified panda_camera.urdf.xacro to use panda_arm.urdf.xacro instead of panda_arm_hand.urdf.xacro and I added ros_controller.launch and ros_controller.yaml generated from the setup assistant config tutorial. The rest of the files generated with the setup assistant were very different from the branch config, so I kept the melodic-branch ones.

With the modified branches of franka_ros and panda_moveit_config I am able to run the deep grasp demo dexnet in rviz but the robot is falling under gravity in gazebo. If I look in the noetic section of the moveit tutorials, there is a step to make the configuration work in gazebo, but the files seem already modified on the branches.

I ask for help. How to turn a deep_grasp_headache into a working deep_grasp_demo ?
I understand that maintaining the tutorials is a work in progress, but I think it needs a little refresh for the beginners like me...
Which branches to use ? What is the recipe? What do you recommand ?

Property 'group': undefined

Similar to this issue in MTC you need to replace the lines with stage->properties().property("group").configureInitFrom(Stage::PARENT, hand_group_name_); to stage->setGroup(hand_group_name_); in the deep_pick_place_task.cpp

Something wrong while Installing Dex-Net

First,Thank you for this wonderful repo!

I'm on ROS Noetic, Ubuntu 20.04. While I install Dex-Net following the 'README.md',something wrong happened. Below is my
operation:

1、install anaconda3, create a virtual environment with command ‘conda create -n DexNet-3.7 python = 3.7’, and enter the environment with command 'conda activate DexNet-3.7'. There is no error.

2、python3 -m pip install --upgrade pip. There is no error.

3、wget https://raw.githubusercontent.com/PickNikRobotics/deep_grasp_demo/master/dexnet_install.sh
wget https://raw.githubusercontent.com/PickNikRobotics/deep_grasp_demo/master/dexnet_requirements.txt
chmod +x dexnet_install.sh
./dexnet_install.sh gpu
Something wrong happened.
(1)E: Unable to locate package python-vtk6. I search the website 'https://packages.ubuntu.com/focal/python/',and find that there is no package name python-vtk6 in ubuntu 20.04, so I change 'python-vtk6' to 'python3-vtk*'. It can install successfully, but I don't know whether is it right or not
(2)While execute the command 'sudo python3 setup.py develop' in folder 'autolab_core' , an error occur: 'error: scipy 1.10.1 is installed but scipy<1.9.2,>=1.8 is required by {'scikit-image'}'.As shown in document 'error2.txt'
I run the command 'conda list' in the virtual environment, it show 'scipy 1.4.1 pypi_0 pypi', which show there is no package scipy = 1.10.1. As shown in document 'conda_list.txt'
(3)While execute the command 'sudo python3 setup.py develop' in folder 'gqcnn' , an error occur: 'ERROR: Could not find a version that satisfies the requirement tensorflow-gpu<=1.15.0'.As shown in document 'error3.txt'
(4)While execute the command 'sudo python3 setup.py develop' in folder 'perception' , an error occur: '[ERROR: Could not find a version that satisfies the requirement tensorflow-gpu<=1.15.0](error: Pillow 7.0.0 is installed but pillow>=8.3.2 is required by {'imageio'})'.As shown in document 'error5.txt'
(5)While execute the command 'sudo python3 setup.py develop' in folder 'visualization' , an error occur: '[ERROR: Could not find a version that satisfies the requirement tensorflow-gpu<=1.15.0](error: scipy 1.10.1 is installed but scipy<1.9.2,>=1.8 is required by {'scikit-image'})'.As shown in document 'error4.txt'

Does anybody know why the error have happened? Is there anything I can do for the program to run well. I am grateful for any hints or suggestions!

conda_list.txt
error5.txt
error4.txt
error3.txt
error2.txt

dexnet_demo: Failed to call gqcnn_grasp service; error processing request: a bytes-like object is required, not 'str'

I tried to start the dexnet_demo.launch file. But i get the following Error message, that the gqcnn_grasp service get the wrong Datatype.

I use noetic and ubuntu 20.04 and followed your installation guide.
I had some Problems with the task constructor it self, witch get fixed with a new version of moveit_resources. see: moveit/moveit_tutorials#665

And i had some other problems with PCL/OpenCV an missing header files. See issue #12 For the dexnet i could solve the missing header file with issue #8. I cloned the recommended task constructor version.

I hope some one of you know how to solve it.

cmake error regarding PCL on finding Eigen

I am working on ubuntu 18.04 with ros-melodic and I followed all the installation steps but I am facing the Cmake error of PCL unable to find Eigen on trying to compile gpd from source.

CMake Warning at /home/curse/MOVEIT_DEEP-GRASPS/pcl-pcl-1.11.0/build/PCLConfig.cmake:149 (find_package):
By not providing "FindEigen.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Eigen", but
CMake did not find one.

Could not find a package configuration file provided by "Eigen" (requested
version 3.1) with any of the following names:
EigenConfig.cmake
eigen-config.cmake
Add the installation prefix of "Eigen" to CMAKE_PREFIX_PATH or set
"Eigen_DIR" to a directory containing one of the above files. If "Eigen"
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
/home/curse/MOVEIT_DEEP-GRASPS/pcl-pcl-1.11.0/build/PCLConfig.cmake:313 (find_eigen)
/home/curse/MOVEIT_DEEP-GRASPS/pcl-pcl-1.11.0/build/PCLConfig.cmake:546 (find_external_library)
CMakeLists.txt:11 (find_package)

CMake Error at /home/curse/MOVEIT_DEEP-GRASPS/pcl-pcl-1.11.0/build/PCLConfig.cmake:59 (message):
common is required but eigen was not found

Planning Failed for the GPD Demo

First of all: Thank you for this wonderful repo!

When I run the gpd_demo as roslaunch moveit_task_constructor_gpd gpd_demo.launch with the default settings, the robot unfortunately does not move and the node terminates in error with: [ERROR] [1613054924.196602715]: Planning failed. This happens after grasps have been generated.
The error is thrown by the node mtc_tutorial and the the logging location is ws_grasp/src/deep_grasp_demo/deep_grasp_task/src/deep_pick_place_task.cpp:DeepPickPlaceTask::plan:445.

I have increased the number of sampled samples (num_samples=500) and the number of selected grasps (num_selected =500) in the gpd_config.yaml file to make sure that enough grasps are sampled. While the runtime of the node increased dramatically, planning still failed.

Does anybody know why the planning might have failed? Is there anything I can do for the program to run well. I am grateful for any hints or suggestions!

Can not download the pretrained model

Thanks for your demo. I am now testing your demo using the pre-trained models. I test the code "./dexnet_deps/gqcnn/scripts/downloads/models/download_models.sh". I find that the links are now not available. Could you share the pre-trained model with me? Thank you very much.

catkin build error: rviz_marker_tools/marker_creation.h: No such file or directory

After installing Eigen and GPD and running catkin build the build process fails with: /home/user/ws_grasp/src/moveit_task_constructor/core/include/moveit/task_constructor/marker_tools.h:3:10: fatal error: rviz_marker_tools/marker_creation.h: No such file or directory

Platform: Linux 5.4.0-51-generic #56~18.04.1-Ubuntu

Full output log:

user@user:~/ws_grasp$ catkin build
-----------------------------------------------------------
Profile:                     default
Extending:        [explicit] /home/user/ws_moveit/devel
Workspace:                   /home/user/ws_grasp
-----------------------------------------------------------
Build Space:        [exists] /home/user/ws_grasp/build
Devel Space:        [exists] /home/user/ws_grasp/devel
Install Space:      [unused] /home/user/ws_grasp/install
Log Space:          [exists] /home/user/ws_grasp/logs
Source Space:       [exists] /home/user/ws_grasp/src
DESTDIR:            [unused] None
-----------------------------------------------------------
Devel Space Layout:          linked
Install Space Layout:        None
-----------------------------------------------------------
Additional CMake Args:       -DCMAKE_BUILD_TYPE=Release
Additional Make Args:        None
Additional catkin Make Args: None
Internal Make Job Server:    True
Cache Job Environments:      False
-----------------------------------------------------------
Whitelisted Packages:        None
Blacklisted Packages:        None
-----------------------------------------------------------
Workspace configuration appears valid.
-----------------------------------------------------------
[build] Found '9' packages in 0.0 seconds.                                                                                                                    
[build] Package table is up to date.                                                                                                                          
Starting  >>> moveit_task_constructor_msgs                                                                                                                    
Starting  >>> rviz_marker_tools                                                                                                                               
Finished  <<< moveit_task_constructor_msgs                         [ 0.3 seconds ]                                                                            
Starting  >>> moveit_task_constructor_capabilities                                                                                                            
Starting  >>> moveit_task_constructor_dexnet                                                                                                                  
Starting  >>> moveit_task_constructor_gpd                                                                                                                     
Finished  <<< rviz_marker_tools                                    [ 0.3 seconds ]                                                                            
Starting  >>> moveit_task_constructor_core                                                                                                                    
Finished  <<< moveit_task_constructor_dexnet                       [ 0.3 seconds ]                                                                            
Finished  <<< moveit_task_constructor_gpd                          [ 0.4 seconds ]                                                                            
Finished  <<< moveit_task_constructor_capabilities                 [ 0.4 seconds ]                                                                            
Finished  <<< moveit_task_constructor_core                         [ 0.5 seconds ]                                                                            
Starting  >>> deep_grasp_task                                                                                                                                 
Starting  >>> moveit_task_constructor_demo                                                                                                                    
Starting  >>> moveit_task_constructor_visualization                                                                                                           
Finished  <<< moveit_task_constructor_demo                         [ 0.2 seconds ]                                                                            
Finished  <<< moveit_task_constructor_visualization                [ 0.5 seconds ]                                                                            
______________________________________________________________________________________________________________________________________________________________
Errors     << deep_grasp_task:make /home/user/ws_grasp/logs/deep_grasp_task/build.make.004.log                                                             
In file included from /home/user/ws_grasp/src/moveit_task_constructor/core/include/moveit/task_constructor/stages/deep_grasp_pose.h:42,
                 from /home/user/ws_grasp/src/deep_grasp_demo/deep_grasp_task/include/deep_grasp_task/deep_pick_place_task.h:59,
                 from /home/user/ws_grasp/src/deep_grasp_demo/deep_grasp_task/src/deep_pick_place_task.cpp:38:
/home/user/ws_grasp/src/moveit_task_constructor/core/include/moveit/task_constructor/marker_tools.h:3:10: fatal error: rviz_marker_tools/marker_creation.h: No such file or directory
 #include <rviz_marker_tools/marker_creation.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/deep_grasp_demo.dir/src/deep_pick_place_task.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /home/user/ws_grasp/src/moveit_task_constructor/core/include/moveit/task_constructor/stages/deep_grasp_pose.h:42,
                 from /home/user/ws_grasp/src/deep_grasp_demo/deep_grasp_task/include/deep_grasp_task/deep_pick_place_task.h:59,
                 from /home/user/ws_grasp/src/deep_grasp_demo/deep_grasp_task/src/deep_grasp_demo.cpp:42:
/home/user/ws_grasp/src/moveit_task_constructor/core/include/moveit/task_constructor/marker_tools.h:3:10: fatal error: rviz_marker_tools/marker_creation.h: No such file or directory
 #include <rviz_marker_tools/marker_creation.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/deep_grasp_demo.dir/src/deep_grasp_demo.cpp.o] Error 1
make[1]: *** [CMakeFiles/deep_grasp_demo.dir/all] Error 2
make: *** [all] Error 2
cd /home/user/ws_grasp/build/deep_grasp_task; catkin build --get-env deep_grasp_task | catkin env -si  /usr/bin/make --jobserver-fds=6,7 -j; cd -
..............................................................................................................................................................
Failed     << deep_grasp_task:make                                 [ Exited with code 2 ]                                                                     
Failed    <<< deep_grasp_task                                      [ 1.0 seconds ]                                                                            
[build] Summary: 8 of 9 packages succeeded.                                                                                                                   
[build]   Ignored:   None.                                                                                                                                    
[build]   Warnings:  None.                                                                                                                                    
[build]   Abandoned: None.                                                                                                                                    
[build]   Failed:    1 packages failed.                                                                                                                       
[build] Runtime: 2.1 seconds total.