Python module for GQ-CNN training and deployment with ROS integration.
Home Page: https://berkeleyautomation.github.io/gqcnn
License: Other
The gqcnn Python package is for training and analysis of Grasp Quality Convolutional Neural Networks (GQ-CNNs). It is part of the ongoing Dexterity-Network (Dex-Net) project created and maintained by the AUTOLAB at UC Berkeley.
Please see the docs for installation and usage instructions.
If you use any part of this code in a publication, please cite the appropriate Dex-Net publication.
I try to reproduce the GQ-CNN training model. There is always an thread error about the FIFO queue occurring in the process of my training.
Here is the code relevant to the error:
with tf.Session() as sess:
self.q = tf.FIFOQueue(self.queue_capacity, [tf.float32, tf.float32, train_label_dtype], shapes=[(self.train_batch_size, self.im_height, self.im_width, self.num_tensor_channels), (self.train_batch_size, self.pose_dim), (self.train_batch_size,)])
self.enqueue_op = self.q.enqueue([self.train_data_batch, self.train_poses_batch, self.train_labels_batch])
self.train_labels_node = tf.placeholder(train_label_dtype, (self.train_batch_size,))
self.input_im_node, self.input_pose_node, self.train_labels_node = self.q.dequeue()
self.queue_thread = threading.Thread(target=self._load_and_enqueue)
self.queue_thread.start()
for step in training_range:
self._check_dead_queue()
"""run optimization"""
_, l, lr, predictions, batch_labels, output, train_images, conv1_1W, conv1_1b, pose_node = self.sess.run(
[optimizer, loss, learning_rate, train_predictions, self.train_labels_node, self.train_net_output, self.input_im_node, self.weights.conv1_1W, self.weights.conv1_1b, self.input_pose_node], options=GeneralConstants.timeout_option)
def _load_and_enqueue(self):
""" Loads and Enqueues a batch of images for training """
train_data = np.zeros(
[self.train_batch_size, self.im_height, self.im_width, self.num_tensor_channels]).astype(np.float32)
train_poses = np.zeros([self.train_batch_size, self.pose_dim]).astype(np.float32)
label_data = np.zeros(self.train_batch_size).astype(self.numpy_dtype)
while not self.term_event.is_set():
time.sleep(self.cfg['queue_sleep'])
# Omit the code for geeting data
# send data to queue
if not self.term_event.is_set():
try:
self.sess.run(self.enqueue_op, feed_dict={self.train_data_batch: train_data,
self.train_poses_batch: train_poses,
self.train_labels_batch: label_data})
except:
pass
del train_data
del train_poses
del label_data
self.dead_event.set()
logging.info('Queue Thread Exiting')
self.queue_thread_exited = TrueWhen I train the model for some time, there will be an error:
INFO:root:Step 41817 (epoch 3.35), 0.1 s
INFO:root:Minibatch loss: 1.256, learning rate: 0.000857
INFO:root:Minibatch error: 26.562%
INFO:root:Step 41818 (epoch 3.35), 0.1 s
INFO:root:Minibatch loss: 1.790, learning rate: 0.000857
INFO:root:Minibatch error: 59.375%
INFO:root:Step 41819 (epoch 3.35), 0.1 s
INFO:root:Minibatch loss: 0.868, learning rate: 0.000857
INFO:root:Minibatch error: 3.125%
2017-08-02 07:26:54.769345: W tensorflow/core/kernels/queue_base.cc:302] _0_data_queue/fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2017-08-02 07:26:54.769537: W tensorflow/core/framework/op_kernel.cc:1158] Cancelled: Dequeue operation was cancelled
[[Node: data_queue/fifo_queue_Dequeue = QueueDequeueV2[component_types=[DT_FLOAT, DT_FLOAT, DT_INT64], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](data_queue/fifo_queue)]]
2017-08-02 07:26:54.769608: W tensorflow/core/framework/op_kernel.cc:1158] Cancelled: Dequeue operation was cancelled
[[Node: data_queue/fifo_queue_Dequeue = QueueDequeueV2[component_types=[DT_FLOAT, DT_FLOAT, DT_INT64], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](data_queue/fifo_queue)]]
2017-08-02 07:26:54.769715: W tensorflow/core/framework/op_kernel.cc:1158] Cancelled: Dequeue operation was cancelled
[[Node: data_queue/fifo_queue_Dequeue = QueueDequeueV2[component_types=[DT_FLOAT, DT_FLOAT, DT_INT64], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](data_queue/fifo_queue)]]
2017-08-02 07:26:54.769747: W tensorflow/core/framework/op_kernel.cc:1158] Cancelled: Dequeue operation was cancelled
[[Node: data_queue/fifo_queue_Dequeue = QueueDequeueV2[component_types=[DT_FLOAT, DT_FLOAT, DT_INT64], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](data_queue/fifo_queue)]]
Traceback (most recent call last):
File "training.py", line 269, in <module>
sgdOptimizer.optimize()
File "/home/liuquande/Desktop/SenceTime/gqcnn/test/gqcnn/sgd_optimizer.py", line 254, in optimize
[optimizer, loss, learning_rate, train_predictions, self.train_labels_node, self.train_net_output, self.input_im_node, self.weights.conv1_1W, self.weights.conv1_1b, self.input_pose_node], options=GeneralConstants.timeout_option)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.DeadlineExceededError: Timed out waiting for notification
INFO:root:Queue Thread Exiting
INFO:rospy.core:signal_shutdown [atexit]I can't figure out the reason. Have you ever met the error in your training process?
Thanks!
Hi,
I'm trying to install ROS Installation by following https://berkeleyautomation.github.io/gqcnn/install/install.html#quick-start-guide. However, after cloning and running catkin_make I didn't see "autolab_core" folder in path/to/catkin_ws/devel/lib/python2.7/dist-packages/ . That's why when I run the command python examples/policy.py the error ImportError: No module named autolab_core always appears. I'm using Ubuntu 14.04, Trusty and ROS Indigo. Do you know what's the problem is?
Output of catkin_make:
Base path: /home/tu/catkin_ws
Source space: /home/tu/catkin_ws/src
Build space: /home/tu/catkin_ws/build
Devel space: /home/tu/catkin_ws/devel
Install space: /home/tu/catkin_ws/install
####
#### Running command: "cmake /home/tu/catkin_ws/src -DCATKIN_DEVEL_PREFIX=/home/tu/catkin_ws/devel -DCMAKE_INSTALL_PREFIX=/home/tu/catkin_ws/install -G Unix Makefiles" in "/home/tu/catkin_ws/build"
####
-- Using CATKIN_DEVEL_PREFIX: /home/tu/catkin_ws/devel
-- Using CMAKE_PREFIX_PATH: /home/tu/catkin_ws/devel;/opt/ros/indigo
-- This workspace overlays: /home/tu/catkin_ws/devel;/opt/ros/indigo
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/tu/catkin_ws/build/test_results
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.6.19
-- BUILD_SHARED_LIBS is on
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~ traversing 1 packages in topological order:
-- ~~ - gqcnn
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'gqcnn'
-- ==> add_subdirectory(gqcnn)
-- Using these message generators: gencpp;genlisp;genpy
-- gqcnn: 2 messages, 1 services
-- Configuring done
-- Generating done
-- Build files have been written to: /home/tu/catkin_ws/build
####
#### Running command: "make -j8 -l8" in "/home/tu/catkin_ws/build"
####
Scanning dependencies of target geometry_msgs_generate_messages_cpp
Scanning dependencies of target _gqcnn_generate_messages_check_deps_GQCNNGrasp
Scanning dependencies of target _gqcnn_generate_messages_check_deps_GQCNNGraspPlanner
Scanning dependencies of target _gqcnn_generate_messages_check_deps_BoundingBox
Scanning dependencies of target sensor_msgs_generate_messages_py
Scanning dependencies of target sensor_msgs_generate_messages_lisp
Scanning dependencies of target geometry_msgs_generate_messages_py
[ 0%] Scanning dependencies of target sensor_msgs_generate_messages_cpp
Built target geometry_msgs_generate_messages_cpp
[ 0%] Built target sensor_msgs_generate_messages_py
[ 0%] Scanning dependencies of target geometry_msgs_generate_messages_lisp
Built target geometry_msgs_generate_messages_py
[ 0%] [ 0%] Built target sensor_msgs_generate_messages_cpp
Built target sensor_msgs_generate_messages_lisp
[ 0%] Built target geometry_msgs_generate_messages_lisp
[ 0%] Built target _gqcnn_generate_messages_check_deps_BoundingBox
[ 0%] [ 0%] Built target _gqcnn_generate_messages_check_deps_GQCNNGrasp
Built target _gqcnn_generate_messages_check_deps_GQCNNGraspPlanner
Scanning dependencies of target gqcnn_generate_messages_py
Scanning dependencies of target gqcnn_generate_messages_lisp
Scanning dependencies of target gqcnn_generate_messages_cpp
[ 9%] [ 18%] [ 27%] [ 45%] [ 45%] [ 54%] [ 63%] Generating C++ code from gqcnn/GQCNNGrasp.msg
[ 72%] Generating C++ code from gqcnn/GQCNNGraspPlanner.srv
Generating Python from MSG gqcnn/GQCNNGrasp
Generating C++ code from gqcnn/BoundingBox.msg
Generating Python from MSG gqcnn/BoundingBox
Generating Lisp code from gqcnn/GQCNNGrasp.msg
Generating Lisp code from gqcnn/BoundingBox.msg
Generating Lisp code from gqcnn/GQCNNGraspPlanner.srv
[ 81%] Generating Python code from SRV gqcnn/GQCNNGraspPlanner
[ 81%] Built target gqcnn_generate_messages_lisp
[ 90%] [100%] Generating Python msg __init__.py for gqcnn
Generating Python srv __init__.py for gqcnn
[100%] Built target gqcnn_generate_messages_py
[100%] Built target gqcnn_generate_messages_cpp
Scanning dependencies of target gqcnn_generate_messages
[100%] Built target gqcnn_generate_messages
I am trying to test your GQCNN ROS package on an Emika Franka Panda robot.
Before applying it on the real robot I first am testing out the GQCNN package inside a gazebo simulation. To do this I tried launching the ROS node both in a python2 and python3 environment.
When I try to launch the ROS node using the roslaunch gqcnn grasp_planning_service.launch model_name:=GQCNN-4.0-PJ command inside the pyhton 2.7 enviroment I get the following error:
Traceback (most recent call last):
File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 45, in <module>
from autolab_core import YamlConfig
ImportError: No module named autolab_core
Due to this error I assumed I needed to install the packages present inside the setup.py file. I, therefore, created a python 2.7.15 virtual environment, sourced the environment and executed the following commands:
python -m pip install -r gqcnn/requirements/p2/gpu_requirements.txt
However, if while in the enviroment if I want to build the ROS package again I get the following error:
ImportError: "from catkin_pkg.package import parse_package" failed: No module named catkin_pkg.package
Make sure that you have installed "catkin_pkg", it is up to date and on the PYTHONPATH.
CMake Error at /opt/ros/kinetic/share/catkin/cmake/safe_execute_process.cmake:11 (message):
execute_process(/virtualenvs/test/bin/python
"/opt/ros/kinetic/share/catkin/cmake/parse_package_xml.py"
"/opt/ros/kinetic/share/catkin/cmake/../package.xml"
"/catkin_ws/build/catkin/catkin_generated/version/package.cmake") returned
error code 1
Call Stack (most recent call first):
/opt/ros/kinetic/share/catkin/cmake/catkin_package_xml.cmake:74 (safe_execute_process)
/opt/ros/kinetic/share/catkin/cmake/all.cmake:167 (_catkin_package_xml)
/opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:20 (include)
CMakeLists.txt:56 (find_package)
This is is not strange since the rospackages are installed on the global python 2.7.15 interperter while I'm inside a virtual enviroment. I therefore deactivated the virtual env and build the ROS package again. When I then try to launch the ROS package from within in the virtual enviroment I get the following error:
process[gqcnn/grasp_planner-2]: started with pid [1701]
Traceback (most recent call last):
File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 43, in <module>
import rospy
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/__init__.py", line 49, in <module>
from .client import spin, myargv, init_node, \
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/client.py", line 52, in <module>
import roslib
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslib/__init__.py", line 50, in <module>
from roslib.launcher import load_manifest
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslib/launcher.py", line 42, in <module>
import rospkg
ImportError: No module named rospkg
This again is caused by the rospackages being installed in the global python interperter. This behavoir is explained on this topic. I tried to solve this by running the following command:
pip install rospkg
However, I now got a lot of other errors. So I'm unsure if I'm using the package in the right way?
In conda I got similar errors due to the fact that the ROS packages are installed in the global python 2.7 interpreter. I tried solving some of them using the pip install rospkg command. However, due to the fact that the ROS node is written in python 2.7 (no python3.7 support for ROS kinetic), I get the following error:
Traceback (most recent call last):
File "/opt/ros/kinetic/bin/roslaunch", line 34, in <module>
import roslaunch
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/__init__.py", line 51, in <module>
from . import param_dump as roslaunch_param_dump
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/param_dump.py", line 40, in <module>
import roslaunch.config
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/config.py", line 45, in <module>
import rospkg.distro
File "/opt/conda/envs/tf-gpu/lib/python3.7/site-packages/rospkg/distro.py", line 44, in <module>
import yaml
File "/opt/conda/envs/tf-gpu/lib/python3.7/site-packages/yaml/__init__.py", line 374
class YAMLObject(metaclass=YAMLObjectMetaclass):
^
SyntaxError: invalid syntax
I also tried building all berkleyautomation packages using catkin build and installing all the python requirements of each packages using the setup.py files. In this case I also keep running into dependencies and errors. After running the setup.py files the following packages appeared to be missing:
ipython
skimage==0.14
After solving for all the dependencies I get the following error:
Traceback (most recent call last):
File "/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 46, in <module>
from perception import (CameraIntrinsics, ColorImage, DepthImage, BinaryImage,
File "/catkin_ws/devel/lib/python2.7/dist-packages/perception/__init__.py", line 35, in <module>
exec(__fh.read())
File "<string>", line 8, in <module>
File "/catkin_ws/src/perception/perception/camera_intrinsics.py", line 14, in <module>
from .image import DepthImage, PointCloudImage
File "/catkin_ws/src/perception/perception/image.py", line 27, in <module>
import skimage.morphology as morph
File "/usr/local/lib/python2.7/dist-packages/skimage/__init__.py", line 167, in <module>
from .util.dtype import (img_as_float32,
File "/usr/local/lib/python2.7/dist-packages/skimage/util/__init__.py", line 8, in <module>
from .arraycrop import crop
File "/usr/local/lib/python2.7/dist-packages/skimage/util/arraycrop.py", line 8, in <module>
from numpy.lib.arraypad import _validate_lengths
ImportError: cannot import name _validate_lengths
I have yet to solve this issue but I first wanted to check if I'm not installing the GQCNN package the wrong way.
Can you maybe elaborate on how the ROS package should be used? Do I need to install the python packages in order to use the ROS package? And if so in which environment do I need to install the python packages? And in that case which python version do I need to use? Can I use it with the python 3.7 packages out of the box or do I need to use the multiprocessing library and the socket library in order to achieve this?
Thanks a lot in advance,
System information
Describe the result you are trying to replicate
If you can, provide a link to the section in the documentation.
Provide the exact sequence of commands / steps that you executed to replicate this result
Describe the unexpected behavior
Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
In the paper, Fig2 shows that fc3's kernel shapes is [18, 18], but when I run the code, I got filter_dim for fc3 [17, 17] in fc_network_tf.py--> _build_im_stream(), from input size 96*96--kernel (9, 9)--kernel (5, 5) --kernel (5, 5) --kernel (5, 5)
Could you please confirm which one is correct?
Hi,
after installing all the dependencies I try to get the policy example running, but there's this error:
uburobot@uburobot:~/catkin_ws/src/gqcnn$ python examples/policy.py
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
/usr/lib/python2.7/dist-packages/pkg_resources.py:1031: UserWarning: /home/uburobot/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Consider a more secure location (set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).
warnings.warn(msg, UserWarning)
WARNING:root:Unable to import pylibfreenect2. Python-only Kinect driver may not work properly.
WARNING:root:Unable to import openni2 driver. Python-only Primesense driver may not work properly
WARNING:root:primesense_sensor.py not installed as catkin package. ROS functionality not available.
Traceback (most recent call last):
File "examples/policy.py", line 106, in <module>
from gqcnn import CrossEntropyAntipodalGraspingPolicy, RgbdImageState
ImportError: No module named gqcnn
I downloaded the GQ (from RSS 2017 paper) and changed the policy/gqcnn_model parameter in the policy.yaml config file to the location of the model (also renamed the folder)
# gqcnn params gqcnn_model: ~/Downloads/GQ_Models
I also tried to start the example by adding --config_filename ./cfg/examples/policy.yaml behind the command.
My goal is to try the grasp planning in ROS indigo on the robot Sawyer from RethinkRobotics with a Kinect.
Line 1156 in be55e0d
Seems like this line should definitely be indented to be within the if-statement, otherwise train_image is undefined. It never errors, because gradient_dropout is by default 0 in the YAML.
It also wouldn't error because train_image is defined for a previous distortion above that line, but that's where the behavior gets unknown. It could be setting to the train_image from a previous iteration, or even from the previous distortion.
Hi,
we tried to start the sample visualization of the GQCNN Package as in https://berkeleyautomation.github.io/gqcnn/tutorials/tutorial.html .
Unfortunately we stumble upon the error
No such file or directory: '/home/autolab/Public/data/dex-net/data/grasp_quality_regression/3dnet_kit_04_09_17_minimal'
after starting the last command:
visualizer = GQCNNPredictionVisualizer(visualization_config)
Output:
INFO:root:Setting up for visualization.
INFO:root:Reading filenames
Traceback (most recent call last):
File "", line 1, in
File "gqcnn/gqcnn_prediction_visualizer.py", line 56, in init
self._setup()
File "gqcnn/gqcnn_prediction_visualizer.py", line 198, in _setup
self._setup_data_filenames()
File "gqcnn/gqcnn_prediction_visualizer.py", line 214, in _setup_data_filenames
all_filenames = os.listdir(self.data_dir)
OSError: [Errno 2] No such file or directory: '/home/autolab/Public/data/dex-net/data/grasp_quality_regression/3dnet_kit_04_09_17_minimal'
visualizer.visualize()
Traceback (most recent call last):
File "", line 1, in
NameError: name 'visualizer' is not defined
Why is there no folder "autolab"? We installed the autolab_core and perception packages.
Thanks in advance :-)
Hi,
trying to do a training like in the Documentation in https://berkeleyautomation.github.io/gqcnn/tutorials/tutorial.html
will result in returning an error by using
with gqcnn.get_tf_graph().as_default():
... SGDOptimizer.optimize()
Output:
INFO:root:Saving model to /home/mr/gqcnn/myWeights/model_skfcmqrwkj
Traceback (most recent call last):
File "", line 2, in
File "gqcnn/sgd_optimizer.py", line 154, in optimize
self._setup()
File "gqcnn/sgd_optimizer.py", line 981, in _setup
self._copy_config()
File "gqcnn/sgd_optimizer.py", line 955, in _copy_config
shutil.copyfile(this_filename, out_train_filename)
File "/usr/lib/python2.7/shutil.py", line 96, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: ''
Is there a way to use an already existing pretrained model so we save the time needed by this task?
At the moment we only want display some pointclouds/grasp points as some kind of "demonstration".
Hi
Just to inform that there is a small problem of versions between package.xml (0.1.0) and __version__ in the setup.py (0.0.2) such that catkin_make fails to build the package (on ROS Kinetic).
Change of __version__ = 0.1.0 fixes the problem
HTH
I want to start the serive of gqcnn but failed. The info is like next,
NODES
/
GQCNN_grasp_planner (gqcnn/grasp_planner_node.py)
auto-starting new master
process[master]: started with pid [17722]
ROS_MASTER_URI=http://localhost:11311
setting /run_id to 355ddcf4-4949-11e9-9d57-f4d108abcd6d
process[rosout-1]: started with pid [17735]
started core service [/rosout]
Unable to launch [GQCNN_grasp_planner-2].
If it is a script, you may be missing a '#!' declaration at the top.
The traceback for the exception was written to the log file
[GQCNN_grasp_planner-2] killing on exit
[rosout-1] killing on exit
[master] killing on exit
Anybody can help me? Thanks very much
I created an autonomous grasping solution for the Kinect v2 and the Panda Emika Franka robot using your pre-trained GQ-CNN model. And I have to say that it performs reasonably well considering it was not retrained for my specific setup. The code of this solution can be found here. I just wanted to do a quick check whether I am allowed to publish my grasping solution which uses your pre-trained CNN the way It is currently published. I published my code under an MIT license and added a note that the license of the submodules, including yours should be respected.
Trying the GQ-CNN ROS installation. Coming across an error in mismatch version number upon catkin_make, just after git clone.
CMake Error at /opt/ros/kinetic/share/catkin/cmake/catkin_python_setup.cmake: 79: catkin_python_setup() version in setup.py (0.0.2) differs from version in package.xml (0.1.0)
Currently running ROS Kinetic on Ubuntu 16.04.1
I'll try to play with some version re-numbering in the mean time.
System information
Hi, I found docker installation really useful and effectively, however, I'm wondering if it's possible to have gqcnn with ROS in Docker version.
I'm asking this since I would like to test the solution on a physical robot using a Kinect v2.
Many thanks and have a nice day.
In cfg/examples/policy.yaml, sensor: type: virtual_primesense
but in rgbd_sensors.py
there is no branch with the name of "virtual primesense", only "virtual" or "primesense", which one should put in the policy.yaml? Thank you.
Hi @visatish ,
I am trying to train Gqcnn model with Dexnet 2.0 from scratch .
It starts training but after few moments throwing an error as follows :
Process Process-1:
Process Process-3:
Traceback (most recent call last):
File "/home/dl/miniconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/home/dl/miniconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "build/bdist.linux-x86_64/egg/gqcnn/training/tf/trainer_tf.py", line 1216, in _load_and_enqueue
train_poses[start_i:end_i,:] = train_poses_arr.copy()
ValueError: could not broadcast input array from shape (64,7) into shape (64,1)
Traceback (most recent call last):
File "/home/dl/miniconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/home/dl/miniconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "build/bdist.linux-x86_64/egg/gqcnn/training/tf/trainer_tf.py", line 1216, in _load_and_enqueue
train_poses[start_i:end_i,:] = train_poses_arr.copy()
ValueError: could not broadcast input array from shape (64,7) into shape (64,1)
Process Process-2:
Traceback (most recent call last):
File "/home/dl/miniconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/home/dl/miniconda3/envs/py27/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "build/bdist.linux-x86_64/egg/gqcnn/training/tf/trainer_tf.py", line 1216, in _load_and_enqueue
train_poses[start_i:end_i,:] = train_poses_arr.copy()
ValueError: could not broadcast input array from shape (64,7) into shape (64,1)
Thanks in advance !
Vaibhav
Hello,
I have installed autolab_core, perception, and gqcnn using a Conda environment with Python 2.7.13 on macOS. However, when I try to run the example policy:
$ python2.7 examples/policy.py --config_filename ./cfg/examples/policy.yaml areddy@panzer
WARNING:root:Failed to import geometry msgs in rigid_transformations.py.
WARNING:root:Failed to import ros dependencies in rigid_transforms.py
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
WARNING: AUTOLab Perception Module not installed as Catkin Package. ROS msg conversions will not be available for Perception wrappers
WARNING: AUTOLab Perception module not installed as Catkin Package. ROS msg conversions will not be available for Perception wrappers.
WARNING:root:Unable to import pylibfreenect2. Python-only Kinect driver may not work properly.
WARNING:root:Unable to import openni2 driver. Python-only Primesense driver may not work properly
WARNING:root:Failed to import ROS in primesense_sensor.py. ROS functionality not available
WARNING:root:primesense_sensor.py not installed as catkin package. ROS functionality not available.
DEBUG:PIL.PngImagePlugin:STREAM 'IHDR' 16 13
DEBUG:PIL.PngImagePlugin:STREAM 'IDAT' 41 65536
Traceback (most recent call last):
File "examples/policy.py", line 143, in <module>
policy = CrossEntropyAntipodalGraspingPolicy(policy_config)
File "/Users/areddy/Research/DexNet/gqcnn/gqcnn/policy.py", line 377, in __init__
GraspingPolicy.__init__(self, config)
File "/Users/areddy/Research/DexNet/gqcnn/gqcnn/policy.py", line 104, in __init__
self._gqcnn = GQCNN.load(self._gqcnn_model_dir)
File "/Users/areddy/Research/DexNet/gqcnn/gqcnn/neural_networks.py", line 71, in load
with open(config_file) as data_file:
IOError: [Errno 2] No such file or directory: '/home/user/data/models/GQ-Image-Wise/config.json'
System information
Describe what you are trying to do
I'm trying to get the prediction of suction and parallel jaw using the pre-trained network gathering depth image from real environment.
Describe current behavior
I'm gathering depth image from my Kinect using the method frames of Kinect2Sensor, it collects some noisy points which can be removed using inpaint method but in both cases I can't produce a good segmentation image.
Describe the expected behavior
segmented image
Describe the input images
I cut the image collected in order to have just the table under the camera and nothing else.
Describe the physical camera setup
I have the kinect camera over the table 60cm far.
Other info / logs
This is the original image coming from the camera:
and the output of depth_im.invalid_pixel_mask().inverse() is:
applaying inpaint to the original depth_im we obtain:
and the corresponding depth_im.invalid_pixel_mask().inverse() is
Can you please help me understanding the problem?
Many thanks.
I recently installed GQCNN(python-only) and tested it with policy.py which works perfectly .Now I installed it as ROS package for real robot. but it fails to start service and following is the log on my command line.
from gqcnn.srv import GQCNNGraspPlanner
ImportError: No module named srv
I checked /path/to/catkin_ws/devel/lib/python2.7/dist-packages/gqcnn/srv/init.py exists
And I checked Python sys.path have /path/to/catkin_ws/devel/lib/python2.7/dist-packages.
But Before this path there has '/usr/local/lib/python2.7/dist-packages/gqcnn-0.1.0-py2.7.egg'.
Should I uninstall the gqcnn(python-only) first?
Thanks very much.
System information
Description of the problem
I am using the GQ-CNN to perform robust grasps on a physical robot. I have successfully trained a GQ-CNN model on the Adv-Synth dataset, with a final validation error of 1.1%. When I run the following code from the Prediction heading on the page https://berkeleyautomation.github.io/gqcnn/tutorials/tutorial.html, then an 'Attribute error' shows up that : 'NoneType' object has no attribute 'close'.
Source code
images = np.load('/home/aver2/Documents/extract/arr_0.npy')
poses = np.load('/home/aver2/Documents/extract/arr_02.npy')
poses = poses.reshape(4000,1)
gqcnn = GQCNN.load('/home/aver2/Documents/output_weights_final/model_nzpnizutxr')
output = gqcnn.predict(images, poses[1000:2000])
pred_p_success = output[:,1]
gqcnn.close_session()
Error
AttributeError Traceback (most recent call last)
in ()
7 output = gqcnn.predict(images, poses[1000:2000])
8 pred_p_success = output[:,1]
----> 9 gqcnn.close_session()
/home/aver2/catkin_ws/src/gqcnn/gqcnn/neural_networks.pyc in close_session(self)
545 """ Close tensorflow session """
546 with self._graph.as_default():
--> 547 self._sess.close()
548 self._sess = None
549
AttributeError: 'NoneType' object has no attribute 'close'
Also, if I print the pred_p_success, then the values get printed and error shows up at the last. I have attached a snapshot of the error, with the printed values of pred_p_success just above it. If I remove the line - gqcnn.close_session(), then the values are printed and no error shows up.
Now, when I run the analysis module after predict module, there is a 'Not Found Error' every time. The code and error for analysis module (https://berkeleyautomation.github.io/gqcnn/tutorials/tutorial.html) are as follows:
Code
analysis_config = YamlConfig('/home/aver2/catkin_ws/src/gqcnn/cfg/tools/analyze_gqcnn_performance.yaml') # Sample config: 'cfg/tools/analyze_gqcnn_performance.yaml'
analyzer = GQCNNAnalyzer(analysis_config)
analyzer.analyze()
Error
NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/aver2/Documents/output_weights_final/model_nzpnizutxr/mini_dexnet_all_trans_01_20_17_trained_06_18_17/model.ckpt
Also, there is no model.ckpt file in my system.
Please let me know if I need to give any other details. Thanks in advance for the consideration and help!!!
Hi!
I recently tried to use the Realsense D435 depth sensor to sample some grasps canditates. Previously I used the Asus Xtion Pro Live, which was working just fine as I was able to sample grasps with a high grasp quality. For reference:
Asus Xtion result:
Realsense result:
Here's how I got the Images:
ids = discover_cams()
cfg = {}
cfg['cam_id'] = ids[0]
cfg['filter_depth'] = False
cfg['frame'] = 'realsense_overhead'
self.sensor = RgbdSensorFactory.sensor('realsense', cfg)
self.sensor.start()
self.camera_intr = self.sensor.color_intrinsics
..........
color_im, depth_im, _ = self.sensor.frames()
frame = self.camera_intr.frame
color_im = color_im.inpaint(0.5)
depth_im = depth_im.inpaint(0.5)
# optionally set a bounding box
bounding_box = BoundingBox()
bounding_box.minY = 0
bounding_box.minX = 0
bounding_box.maxY = color_im.height
bounding_box.maxX = color_im.width
segmask = BinaryImage(255 * np.ones(depth_im.shape).astype(np.uint8),
frame=frame)
try:
start_time = time.time()
planned_grasp_data = self.plan_grasp_client(color_im.rosmsg, # T_grasp_camera
depth_im.rosmsg,
self.camera_intr.rosmsg,
bounding_box,
segmask.rosmsg)
...........
The images were taken at roughly the same angle and the same height (about 60 cm).
It seems that the depth is correct, since the pose has a z value of around 60 cm.
What I tried:
strm = self._profile.get_stream(rs.stream.depth) (Line 125 in realsense_sensor.py) --> scaling seems to be wrong nowI read the other issues but I don't think that I have a scaling issue that's why I am opening a new one.
Hopefully I can get some help here, the depth image looks weird but I don't know why.
Thank you!
Describe the result you are trying to replicate
If you can, provide a link to the section in the [documentation]
https://berkeleyautomation.github.io/gqcnn/replication/replication.html
Provide the exact sequence of commands / steps that you executed to replicate this result
$ ./scripts/policies/run_all_dex-net_2.0_examples.sh
Describe the unexpected behavior
i just use the example model and policy
Hello, I have followed the instructions here to install gqcnn as a ros package. However, when I execute
python tools/training.py
it posts these warnings:
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
WARNING: AUTOLab Perception Module not installed as Catkin Package. ROS msg conversions will not be available for Perception wrappers
WARNING:root:Unable to import pylibfreenect2. Python-only Kinect driver may not work properly.
WARNING:root:Unable to import openni2 driver. Python-only Primesense driver may not work properly
WARNING:root:primesense_sensor.py not installed as catkin package. ROS functionality not available.
I have followed the instructions here to install autolab_core as a ros package.
Besides, I find the perception module does not have ros installation. How to install it as a catkin package to fix the second warning?
By the way, could anyone explains the last three warnings? It seems that I should install some more packages?
Thank you!
I download the directory gqcnn>datasets>dexnet_2 dataset and read the README.txt but there are some doubts.
There are 13 file style in one sequence.
1.binary_ims_raw_00000.npz
if depth == 0 then value = 0
else depth =1 ?
2.depth_ims_raw_00000.npz
primary depth image?
3.depth_ims_raw_table_00000.npz
primary depth image pre-rotation-and-translation align the grasp axis and x axis?
4.force_closure_00000.npz
Did not consider friction and interference compute the force_close?
And the value 1 equal that is stable grasp?
5.hand_poses_00000.npz
I can't understand the difference between channels 0,1,2 and 4,5,6.
I drew the 0,1,2 channel, its right?
6.object_labels_00000
the same value represent the same object?
7.pose_labels_00000.npz and image_labels_00000.npz
I don’t know what they represent and what it means
System information
Describe the problem
pip natively downloads python3 modules, specifying python27 versions in install.py file doesn't help because of nested module dependency.
Running on clean install of OS.
Provide the exact sequence of commands / steps that you executed before running into the problem
pip install .
Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
....
Running scikit-image-0.15.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ZV5iUC/scikit-image-0.15.0/egg-dist-tmp-C93hgK
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-5LtMVA/meshrender/setup.py", line 47, in
'sphinx_rtd_theme'
File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 269, in init
self.fetch_build_eggs(attrs['setup_requires'])
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 313, in fetch_build_eggs
replace_conflicting=True,
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 826, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 1092, in best_match
return self.obtain(req, installer)
File "/usr/lib/python2.7/dist-packages/pkg_resources/init.py", line 1104, in obtain
return installer(requirement)
File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 380, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 663, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 693, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 873, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1101, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1087, in run_setup
run_setup(setup_script, args)
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 246, in run_setup
raise
File "/usr/lib/python2.7/contextlib.py", line 35, in exit
self.gen.throw(type, value, traceback)
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python2.7/contextlib.py", line 35, in exit
self.gen.throw(type, value, traceback)
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 166, in save_modules
saved_exc.resume()
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 141, in resume
six.reraise(type, exc, self._tb)
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 154, in save_modules
yield saved
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 195, in setup_context
yield
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 243, in run_setup
DirectorySandbox(setup_dir).run(runner)
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 273, in run
return func()
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 242, in runner
_execfile(setup_script, ns)
File "/usr/lib/python2.7/dist-packages/setuptools/sandbox.py", line 46, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-ZV5iUC/scikit-image-0.15.0/setup.py", line 31, in
classifiers = [
File "/home/user/.local/lib/python2.7/site-packages/numpy/init.py", line 142, in
from . import core
File "/home/user/.local/lib/python2.7/site-packages/numpy/core/init.py", line 40, in
from . import multiarray
File "/home/user/.local/lib/python2.7/site-packages/numpy/core/multiarray.py", line 12, in
from . import overrides
File "/home/user/.local/lib/python2.7/site-packages/numpy/core/overrides.py", line 46, in
""")
RuntimeError: implement_array_function method already has a docstring
in rgbd_sensors.py , s = KinectSensorBridged(quality=cfg['quality'], frame=cfg['frame'])
In cfg/examples/policy.yaml,the sensor: type: bridged_kinect2
and what is the quality put in the policy.yaml?
Thank you.
OS: Fresh Ubutun 16.04 installation
Container: Yes created with singularity 3.2.1
GQCCN installation: Installed with pip
Python version: 3.7
Virtual environment: Both tested in conda and Virtualenv.
I tried installing and running both the GQCNN python2 and python3 policy both in a python virtualenv as well as in a conda environment. The python2 policy would both install and execute correctly without any problems but while installing the python3 policy however I encounter a number of problems. These problems are likely caused by my setup but for documentation, I listed them below.
The errors below were present in both the conda and virtual enviroment.
During the pip installation i get a ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-nks1vqne/meshrender/ . The full error log can be found here.
This was caused by the python3 version of python-dev not being present on my system this can be solved by using the apt install python-dev command.
The when running the setup.py through pip the pyglet package unfortunatelly is not updated to the right version. As a result we get the ERROR: meshrender 0.0.10 has requirement pyglet>=1.4.0b1, but you'll have pyglet 1.3.2 which is incompatible error.
This can be solved by manually installing pyglet (v1.4.0b1) using the pip install pyglet==1.4.0b1 command.
To test the policy I downloaded the trained models using the ./scripts/downloads/models/download_models.sh command. Following I tested the python policy running the following command:
python3 examples/policy.py GQCNN-2.0 --depth_image data/examples/single_object/primesense/depth_0.npy --segmask data/examples/single_object/primesense/segmask_0.png --camera_intr data/calib/phoxi/phoxi.intr
Traceback (most recent call last):
File "/usr/lib/python3.5/tkinter/__init__.py", line 36, in <module>
import _tkinter
ImportError: No module named '_tkinter'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "examples/policy.py", line 46, in <module>
from perception import (BinaryImage, CameraIntrinsics, ColorImage, DepthImage,
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/perception/__init__.py", line 8, in <module>
from .camera_intrinsics import CameraIntrinsics
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/perception/camera_intrinsics.py", line 14, in <module>
from .image import DepthImage, PointCloudImage
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/perception/image.py", line 12, in <module>
import matplotlib.pyplot as plt
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/matplotlib/pyplot.py", line 115, in <module>
_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/matplotlib/backends/__init__.py", line 62, in pylab_setup
[backend_name], 0)
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/matplotlib/backends/backend_tkagg.py", line 4, in <module>
from . import tkagg # Paint image to Tk photo blitter extension.
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/matplotlib/backends/tkagg.py", line 5, in <module>
from six.moves import tkinter as Tk
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/six.py", line 92, in __get__
result = self._resolve()
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/six.py", line 115, in _resolve
return _import_module(self.mod)
File "/root/virtualenviroments/gqcnn/lib/python3.5/site-packages/six.py", line 82, in _import_module
__import__(name)
File "/usr/lib/python3.5/tkinter/__init__.py", line 38, in <module>
raise ImportError(str(msg) + ', please install the python3-tk package')
ImportError: No module named '_tkinter', please install the python3-tk package
Install the python3-tk package using the apt-get install python3-tk command.
First when I try to install the GQCNN package using catkin build I get a lot of dependency errors. These errors were caused by the fact that I did not run the setup.py before building the ROS package. It might be an idea to add this step to the gqcnn documentation.
GQCNN main folder and execute the following command:python3 -m pip install .When I try to install the package I get the following a you are using pip version 8.1.1, however, version 19.1.1 is available warning. This can be solved by updating pip using the pip install --upgrade pip. On some systems, due to an error introduced in pip 10 (see this topic) you need to install pip using the get-pip.py file instead of using the apt repository. This can be done using as explained below:
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && python3 get-pip.py && rm get-pip.py
Error running python tools/training.yaml if val_batch_size and batch_size (in the gqcnn config) aren't set to the same value.
Hello, I download the newest codes, but I find the result does not converge and the validation error has not changed. I doubt that I did not use the dataset properly. I use the dataset adv_synth.zip. I just extract it and set corresponding data_dir in cfg/tools/training.yaml, then I run python tools/training.py. Is it right? And I find that the large dataset is two unformatted files, how can I use them?
Hi,
I have installed all the required dependencies for running gqcnn code following your GQCNN 1.0.0 documentation .I am trying to run examples/policy.py with dexnet 2.0 pretrained model but getting error :
1)When I am running the script using python 2.7 : I get different warnings and error as :
./scripts/policies/run_all_dex-net_2.0_examples.sh
RUNNING EXAMPLE 1
WARNING:root:Failed to import geometry msgs in rigid_transformations.py.
WARNING:root:Failed to import ros dependencies in rigid_transforms.py
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
WARNING:root:autolab_perception is not installed as a catkin package - ROS msg conversions will not be available for image wrappers
WARNING:root:autolab_perception is not installed as a catkin package - ROS msg conversions will not be available for image wrappers
WARNING:root:Unable to import CNN modules! Likely due to missing tensorflow.
WARNING:root:TensorFlow can be installed following the instructions in https://www.tensorflow.org/get_started/os_setup
WARNING:root:Unable to import pylibfreenect2. Python-only Kinect driver may not work properly.
WARNING:root:Unable to import openni2 driver. Python-only Primesense driver may not work properly
WARNING:root:Failed to import ROS in primesense_sensor.py. ROS functionality not available
WARNING:root:primesense_sensor.py not installed as catkin package. ROS functionality not available.
WARNING:root:Failed to import ROS in ensenso_sensor.py. ROS functionality not available
Segmentation fault (core dumped)
2)When I am running the script using python 3.6 : I get the following error :
Traceback (most recent call last):
File "examples/policy.py", line 41, in
from gqcnn.grasping import RobustGraspingPolicy, CrossEntropyRobustGraspingPolicy, RgbdImageState, FullyConvolutionalGraspingPolicyParallelJaw, FullyConvolutionalGraspingPolicySuction
File "/home/dl/virtual_env3/lib/python3.7/site-packages/gqcnn-1.0.0-py3.7.egg/gqcnn/init.py", line 22, in
ModuleNotFoundError: No module named 'model'
But I can see the module is present there !
Thank you in advance !
Hi, i use realsense r300, and i send the ROS message to gqcnn.
However, the result is shown as follows:
I can hardly understand why the grasps are shown like that?
I want to get better segmask image from color image. I attemp to use the method of image.py in perception packages. But I don't get better segmask image, so that I can't to handle object grasp.
figure 1 and 2 is the original color and segmask image which is the example image of data/example/clutter/primesense. Then, the picture 3 that I get by using ColorImage.to_binary() method. Please tell me how to get better segmask image. Thanks.
Accidentally created
Hello,
I recently installed GQCNN ROS Service and tested it with policy.py which works perfectly for me.Now I did gqcnn.launch for launching grasp planner service for giving input of my custom image but it fails to start service and following is the log on my command line.
root@akashp-Latitude-E5570:~/catkin_ws/src/gqcnn/launch# roslaunch gqcnn.launch
... logging to /home/akashp/.ros/log/389615c4-c382-11e7-b384-f0d5bf915032/roslaunch-akashp-Latitude-E5570-26586.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
started roslaunch server http://akashp-Latitude-E5570:34359/
PARAMETERS
NODES
/
GQCNN_grasp_planner (gqcnn/grasp_planner_node.py)
auto-starting new master
process[master]: started with pid [26597]
ROS_MASTER_URI=http://localhost:11311
setting /run_id to 389615c4-c382-11e7-b384-f0d5bf915032
process[rosout-1]: started with pid [26610]
started core service [/rosout]
process[GQCNN_grasp_planner-2]: started with pid [26614]
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
WARNING:root:Unable to import pylibfreenect2. Python-only Kinect driver may not work properly.
WARNING:root:Unable to import openni2 driver. Python-only Primesense driver may not work properly
WARNING:root:primesense_sensor.py not installed as catkin package. ROS functionality not available.
Traceback (most recent call last):
File "/home/akashp/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 19, in
from gqcnn.srv import GQCNNGraspPlanner
ImportError: No module named srv
[GQCNN_grasp_planner-2] process has died [pid 26614, exit code 1, cmd /home/akashp/catkin_ws/src/gqcnn/ros_nodes/grasp_planner_node.py __name:=GQCNN_grasp_planner __log:=/home/akashp/.ros/log/389615c4-c382-11e7-b384-f0d5bf915032/GQCNN_grasp_planner-2.log].
log file: /home/akashp/.ros/log/389615c4-c382-11e7-b384-f0d5bf915032/GQCNN_grasp_planner-2*.log
Kindly provide me further info to solve this problem.Also I installed autolab_core still it shows autolab_core is not installed as catkin package.I did make install in path/to/my/catkin/ws/build/ also with no success.I am using ROS Kinetic on ubuntu 16.04.
Hello Community,
I would like to train the network myself.
I've seen that on the website there is a repository for the npz file dataset. However there is also an hdf5 file which is also a dataset. Besides, using the dex-net API I can only get hdf5 dataset. What's the difference of these two datasets and how can I train the network with the hdf5 dataset.
Thank you in advance!
Hello,
I installed gqcnn ros service on ROS kinetic and used pretrained models for doing some sample tests for Grasp planning on some of my cuboid shaped objects.But Grasp sampler seems to be doing wrong sampling or not be to sample proper antipodal points along edges of my object.In first 2 iterations it seems to have got some antipodal points but still not perfectly aligned to my object geometry.In final iteration all those grasps are also filtered leaving sampler with some on body grasps.Kindly have a look into images attached below.
RGBD Cropped image
Sampled antipodal points
Following is my log output.I did enabled debug logs to get more clearity. Kindly help me solve this issue.Also find grasp_sampler_node.yaml file data following the logs.
root@akashp-Latitude-E5570:~/catkin_ws# roslaunch gqcnn gqcnn.launch
... logging to /home/akashp/.ros/log/af0cabda-c938-11e7-a093-f0d5bf915032/roslaunch-akashp-Latitude-E5570-10112.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
started roslaunch server http://akashp-Latitude-E5570:46762/
PARAMETERS
NODES
/
GQCNN_grasp_planner (gqcnn/grasp_planner_node.py)
ROS_MASTER_URI=http://localhost:11311
core service [/rosout] found
process[GQCNN_grasp_planner-1]: started with pid [10161]
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
WARNING:root:Unable to import pylibfreenect2. Python-only Kinect driver may not work properly.
WARNING:root:Unable to import openni2 driver. Python-only Primesense driver may not work properly
WARNING:root:primesense_sensor.py not installed as catkin package. ROS functionality not available.
[INFO] [1510669240.601043]: Creating Grasp Policy
2017-11-14 19:50:40.799114: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-14 19:50:40.799143: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-14 19:50:40.799157: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-11-14 19:50:40.799166: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-14 19:50:40.799174: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
[INFO] [1510669240.990180]: Grasp Sampler Server Initialized
[INFO] [1510669241.244419]: Planning Grasp
[INFO] [1510669243.259270]: Planning Grasp
2017-11-14 19:50:43,259 - root - DEBUG - Sampling 2d candidates
2017-11-14 19:50:43,262 - root - DEBUG - Depth edge detection took 0.002 sec
2017-11-14 19:50:43,262 - root - DEBUG - Found 3322 edge pixels
2017-11-14 19:50:43,318 - root - DEBUG - Normal computation took 0.056 sec
2017-11-14 19:50:43,601 - root - DEBUG - Normal pruning 0.283 sec
/home/akashp/catkin_ws/src/gqcnn/./gqcnn/image_grasp_sampler.py:40: RuntimeWarning: invalid value encountered in arccos
in_cone_2 = (np.arccos(n2.dot(v)) < alpha)
2017-11-14 19:50:44,108 - root - DEBUG - Sampled 200 grasps from image
2017-11-14 19:50:44,108 - root - DEBUG - Sampling grasps took 0.849 sec
2017-11-14 19:50:44,108 - root - DEBUG - Computing the seed set took 0.849 sec
2017-11-14 19:50:44,153 - root - DEBUG - Tensor conversion took 0.044 sec
2017-11-14 19:50:44,154 - root - DEBUG - CEM iter 0
2017-11-14 19:50:44,154 - root - DEBUG - Predicting file 0
2017-11-14 19:50:44,286 - root - DEBUG - Predicting file 16
2017-11-14 19:50:44,416 - root - DEBUG - Predicting file 32
2017-11-14 19:50:44,545 - root - DEBUG - Predicting file 48
2017-11-14 19:50:44,677 - root - DEBUG - Predicting file 64
2017-11-14 19:50:44,818 - root - DEBUG - Predicting file 80
2017-11-14 19:50:44,953 - root - DEBUG - Predicting file 96
2017-11-14 19:50:45,088 - root - DEBUG - Predicting file 112
2017-11-14 19:50:45,222 - root - DEBUG - Predicting file 128
2017-11-14 19:50:45,356 - root - DEBUG - Predicting file 144
2017-11-14 19:50:45,489 - root - DEBUG - Predicting file 160
2017-11-14 19:50:45,632 - root - DEBUG - Predicting file 176
2017-11-14 19:50:45,772 - root - DEBUG - Predicting file 192
2017-11-14 19:50:45,913 - root - DEBUG - Prediction took 1.759 sec
2017-11-14 19:50:52,938 - root - DEBUG - GMM fitting with 20 components took 0.038 sec
2017-11-14 19:50:52,941 - root - DEBUG - GMM sampling took 0.003 sec
2017-11-14 19:50:52,960 - root - DEBUG - Tensor conversion took 0.017 sec
2017-11-14 19:50:52,960 - root - DEBUG - CEM iter 1
2017-11-14 19:50:52,960 - root - DEBUG - Predicting file 0
2017-11-14 19:50:53,113 - root - DEBUG - Predicting file 16
2017-11-14 19:50:53,249 - root - DEBUG - Predicting file 32
2017-11-14 19:50:53,375 - root - DEBUG - Predicting file 48
2017-11-14 19:50:53,507 - root - DEBUG - Prediction took 0.547 sec
2017-11-14 19:50:56,777 - root - DEBUG - GMM fitting with 6 components took 0.004 sec
2017-11-14 19:50:56,781 - root - DEBUG - GMM sampling took 0.003 sec
2017-11-14 19:50:56,802 - root - DEBUG - Tensor conversion took 0.019 sec
2017-11-14 19:50:56,802 - root - DEBUG - CEM iter 2
2017-11-14 19:50:56,802 - root - DEBUG - Predicting file 0
2017-11-14 19:50:56,981 - root - DEBUG - Predicting file 16
2017-11-14 19:50:57,117 - root - DEBUG - Predicting file 32
2017-11-14 19:50:57,254 - root - DEBUG - Predicting file 48
2017-11-14 19:50:57,391 - root - DEBUG - Prediction took 0.589 sec
2017-11-14 19:51:03,555 - root - DEBUG - GMM fitting with 6 components took 0.022 sec
2017-11-14 19:51:03,556 - root - DEBUG - GMM sampling took 0.001 sec
2017-11-14 19:51:03,576 - root - DEBUG - Tensor conversion took 0.018 sec
2017-11-14 19:51:03,577 - root - DEBUG - Predicting file 0
2017-11-14 19:51:03,742 - root - DEBUG - Predicting file 16
2017-11-14 19:51:03,883 - root - DEBUG - Predicting file 32
2017-11-14 19:51:04,013 - root - DEBUG - Predicting file 48
2017-11-14 19:51:04,151 - root - DEBUG - Final prediction took 0.574 sec
[INFO] [1510669284.796026]: Total grasp planning time: 41.5362010002 secs.
2017-11-14 19:51:24,795 - rosout - INFO - Total grasp planning time: 41.5362010002 secs.
grasp_sampler_node.yaml
vis:
vis_cropped_rgbd_image: 1
vis_uncropped_color_image: 0
vis_uncropped_depth_image: 0
width_pad: 0
height_pad: 0
inpaint_rescale_factor: 0.5 # amount to rescale the images before inpainting (smaller numbers == faster code)
policy:
num_seed_samples: 200 # number of seed samples
num_gmm_samples: 50
num_iters: 3
gmm_refit_p: 0.25
gmm_component_frac: 0.4
gmm_reg_covar: 0.01
gqcnn_model: /home/akashp/Work/GQ-Image-Wise
deterministic: 1
gripper_width: 0.140
crop_height: 96
crop_width: 96
#logging_dir: /home/akashp/Work
sampling:
# type
type: antipodal_depth
# antipodality
friction_coef: 1.0
depth_grad_thresh: 0.0025
depth_grad_gaussian_sigma: 1.0
downsample_rate: 4
max_rejection_samples: 2500
# distance
max_dist_from_center: 10000000
min_dist_from_boundary: 40
min_grasp_dist: 2.5
angle_dist_weight: 5.0
# depth sampling
depth_samples_per_grasp: 1
depth_sample_win_height: 1
depth_sample_win_width: 1
min_depth_offset: 0.015
max_depth_offset: 0.0
vis:
grasp_sampling : 0
tf_images: 0
grasp_candidates: 1
elite_grasps: 0
grasp_ranking: 0
grasp_plan: 1
final_grasp: 1
k: 25
System information
Describe the result you are trying to replicate
If you can, provide a link to the section in the documentation.
Test Dex-Net 4.0 w/ Suction.
Provide the exact sequence of commands / steps that you executed to replicate this result
./scripts/policies/run_all_dex-net_4.0_suction_examples.sh
Describe the unexpected behavior
Errors out.
Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
Traceback (most recent call last): File "examples/policy.py", line 260, in <module> action = policy(state) File "/home/vsatish/Workspace/gqcnn_release/gqcnn/gqcnn/grasping/policy/policy.py", line 228, in __call__ return self.action(state) File "/home/vsatish/Workspace/gqcnn_release/gqcnn/gqcnn/grasping/policy/policy.py", line 405, in action action = self._action(state) File "/home/vsatish/Workspace/gqcnn_release/gqcnn/gqcnn/grasping/policy/policy.py", line 1287, in _action grasps, q_values = self.action_set(state) File "/home/vsatish/Workspace/gqcnn_release/gqcnn/gqcnn/grasping/policy/policy.py", line 1172, in action_set axis=grasp_axis) File "/home/vsatish/Workspace/gqcnn_release/gqcnn/gqcnn/grasping/grasp.py", line 376, in from_feature_vec if v.shape > 2 and axis is None: TypeError: unorderable types: tuple() > int()
System information
Describe the result you are trying to replicate
If you can, provide a link to the section in the documentation.
Provide the exact sequence of commands / steps that you executed to replicate this result
gqcnn/tools/finetune.py
Describe the unexpected behavior
Traceback (most recent call last):
GQCNNTrainerTF INFO Cleaning and preparing to exit optimization...
File "C:/cProjects/picking/fcnn/gqcnn/tools/finetune.py", line 140, in
GQCNNTrainerTF INFO Terminating prefetch queue workers...
trainer.finetune(model_dir)
GQCNNTrainerTF INFO Flushing prefetch queue...
File "C:\cProjects\picking\fcnn\gqcnn\gqcnn\training\tf\trainer_tf.py", line 217, in finetune
GQCNNTF INFO Closing TF Session...
self._finetune(base_model_dir)
File "C:\cProjects\picking\fcnn\gqcnn\gqcnn\training\tf\trainer_tf.py", line 241, in _finetune
self._optimize_weights(finetune=True)
File "C:\cProjects\picking\fcnn\gqcnn\gqcnn\training\tf\trainer_tf.py", line 320, in _optimize_weights
p.start()
File "C:\Users\ecl\AppData\Local\Continuum\miniconda3\envs\py36tf\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\Users\ecl\AppData\Local\Continuum\miniconda3\envs\py36tf\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\ecl\AppData\Local\Continuum\miniconda3\envs\py36tf\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\Users\ecl\AppData\Local\Continuum\miniconda3\envs\py36tf\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "C:\Users\ecl\AppData\Local\Continuum\miniconda3\envs\py36tf\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle _thread.RLock objects
WARNING:root:Failed to import geometry msgs in rigid_transformations.py.
WARNING:root:Failed to import ros dependencies in rigid_transforms.py
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
Process finished with exit code 1
Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
System information
Hi,
I have finish installed the GQCNN package for ROS installation and tested the python policy with pre-trained network and sample image using the following command included in documentation with no issue
$ python examples/policy.py GQCNN-4.0-PJ --depth_image data/examples/clutter/phoxi/dex-net_4.0/depth_0.npy --segmask data/examples/clutter/phoxi/dex-net_4.0/segmask_0.png --camera_intr data/calib/phoxi/phoxi.intr
However, I encountered parameter missing issue when testing the ROS policy.
The commands that I used to test the ROS policy are:
$ roslaunch gqcnn grasp_planning_service.launch model_name:=GQCNN-4.0-PJ
$ python examples/policy_ros.py --depth_image data/examples/clutter/phoxi/dex-net_4.0/depth_0.npy --segmask data/examples/clutter/phoxi/dex-net_4.0/segmask_0.png --camera_intr data/calib/phoxi/phoxi.intr
I modified the config_filename in grasp_planner_node.py line 333 to read the ros_node yaml from 'cfg/policies/gqcnn_pj.yaml' to 'cfg/ros_nodes/grasp_planner_node.yaml'
The error i encountered:
[INFO] [1555922099.759817]: Planning Grasp
[ERROR] [1555922099.800620]: Error processing request: 'rgbd_state'
['Traceback (most recent call last):\n', ' File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 625, in _handle_request\n response = convert_return_to_response(self.handler(request), self.response_class)\n', ' File "/home/junri/catkin_gqcnn/src/gqcnn/ros_nodes/grasp_planner_node.py", line 122, in plan_grasp\n return self._plan_grasp(color_im, depth_im, camera_intr)\n', ' File "/home/junri/catkin_gqcnn/src/gqcnn/ros_nodes/grasp_planner_node.py", line 221, in _plan_grasp\n if self.cfg['vis']['rgbd_state']:\n', "KeyError: 'rgbd_state'\n"]
^C[gqcnn/grasp_planner-2] killing on exit
From the error report, it seems like the process is looking for vis - rgbd_state parameter from yaml but this vis-rgbd_state parameter is missing in the cfg/ros_nodes/grasp_planner_node.yaml
Am I suppose to keep on adding the missing parameters to the yaml?
Hi, I tried to follow the installation instructions for ROS install. My setup is U16.04 with ROS Kinetic.
For making autolab_core and perception I get:
Base path: /home/admin/catkin_ws
Source space: /home/admin/catkin_ws/src
Build space: /home/admin/catkin_ws/build
Devel space: /home/admin/catkin_ws/devel
Install space: /home/admin/catkin_ws/install
####
#### Running command: "make cmake_check_build_system" in "/home/admin/catkin_ws/build"
####
####
#### Running command: "make -j8 -l8" in "/home/admin/catkin_ws/build/autolab_core"
####
[ 0%] Built target std_msgs_generate_messages_lisp
[ 0%] Built target std_msgs_generate_messages_nodejs
[ 0%] Built target std_msgs_generate_messages_py
[ 0%] Built target std_msgs_generate_messages_cpp
[ 0%] Built target std_msgs_generate_messages_eus
[ 0%] Built target _autolab_core_generate_messages_check_deps_RigidTransformListener
[ 0%] Built target _autolab_core_generate_messages_check_deps_RigidTransformPublisher
[ 9%] Built target autolab_core_generate_messages_cpp
[ 36%] Built target autolab_core_generate_messages_py
[ 63%] Built target autolab_core_generate_messages_eus
[ 81%] Built target autolab_core_generate_messages_nodejs
[100%] Built target autolab_core_generate_messages_lisp
[100%] Built target autolab_core_generate_messages
and
Base path: /home/admin/catkin_ws
Source space: /home/admin/catkin_ws/src
Build space: /home/admin/catkin_ws/build
Devel space: /home/admin/catkin_ws/devel
Install space: /home/admin/catkin_ws/install
####
#### Running command: "make cmake_check_build_system" in "/home/admin/catkin_ws/build"
####
####
#### Running command: "make -j8 -l8" in "/home/admin/catkin_ws/build/perception"
####
[ 0%] Built target std_msgs_generate_messages_cpp
[ 0%] Built target std_msgs_generate_messages_lisp
[ 0%] Built target std_msgs_generate_messages_eus
[ 0%] Built target std_msgs_generate_messages_py
[ 0%] Built target std_msgs_generate_messages_nodejs
[ 0%] Built target _perception_generate_messages_check_deps_ImageBuffer
[ 16%] Built target perception_generate_messages_lisp
[ 50%] Built target perception_generate_messages_py
[ 66%] Built target perception_generate_messages_nodejs
[ 83%] Built target perception_generate_messages_eus
[100%] Built target perception_generate_messages_cpp
[100%] Built target perception_generate_messages
Also /path/to/catkin_ws/devel/lib/python2.7/dist-packages/autolab_core and /path/to/catkin_ws/devel/lib/python2.7/dist-packages/autolab_core/srv exist.
When after sourcing ...devel/setup.bash I import autolab_core in python, I get
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable.
Moreover when i import gqcnn, in addition to the warnings i get
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/admin/catkin_ws/devel/lib/python2.7/dist-packages/gqcnn/__init__.py", line 35, in <module>
exec(__fh.read())
File "<string>", line 1, in <module>
File "/home/admin/catkin_ws/src/gqcnn/./gqcnn/neural_networks.py", line 17, in <module>
from gqcnn import InputDataMode, TrainingMode
ImportError: cannot import name InputDataMode
Could anybody give me some hint please?
Some more details:
import cv2 is okHi, I am wondering how to setup realsense SR300. Can I test your code with a set of RGB and depth images from SR300? or should I need to change camera extrinsic/intrinsic?
Hello, i try the suction code.
But i find the calculation for the normals may be not right.
For example：
and the i do the following in suction_policy.py：
print action.grasp.axis
The output for the normal of the red point is [-0. -0. 1.]
But obviously this is not right.
So i wonder whether the calculation of the normal for your code has some limitation.
Hi,
after installing all the dependencies I try to get the suction_policy example running(from dev_jeff branch), but there's this error:
python examples/suction_policy.py
WARNING:root:Failed to import geometry msgs in rigid_transformations.py.
WARNING:root:autolab_core not installed as catkin package, RigidTransform ros methods will be unavailable
/usr/local/lib/python2.7/dist-packages/h5py-2.7.1-py2.7-linux-x86_64.egg/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
WARNING:root:Unable to import pylibfreenect2. Python-only Kinect driver may not work properly.
WARNING:root:Unable to import openni2 driver. Python-only Primesense driver may not work properly
WARNING:root:primesense_sensor.py not installed as catkin package. ROS functionality not available.
WARNING:root:Failed to import ROS in phoxi_sensory.py. PhoXiSensor functionality unavailable.
DEBUG:root:Contour 0 area: 1433.500
DEBUG:root:Contour 1 area: 4260.000
DEBUG:root:Contour 2 area: 562.500
DEBUG:root:Contour 3 area: 1490.000
DEBUG:root:Contour 4 area: 3746.500
DEBUG:root:Contour 5 area: 7460.500
DEBUG:root:Sampling 2d candidates
INFO:root:Filtering took 0.034 sec
/home/hans/catkin_ws2/src/perception/perception/image.py:3150: RuntimeWarning: invalid value encountered in divide
normal_data = pc_grads / np.tile(pc_grad_norms[:, np.newaxis], [1, 3])
/home/hans/catkin_ws2/src/perception/perception/image.py:3224: RuntimeWarning: invalid value encountered in greater
if np.any((np.abs(np.linalg.norm(data, axis=2) - 1.0) > 1e-4)
INFO:root:Normal cloud took 0.150 sec
INFO:root:Loop took 0.053 sec
DEBUG:root:Sampled 200 grasps from image
DEBUG:root:Sampling grasps took 0.247 sec
INFO:root:Computing the seed set took 0.260 sec
INFO:root:CEM iter 0
/usr/local/lib/python2.7/dist-packages/gqcnn-0.1.0-py2.7.egg/gqcnn/grasp_quality_function.py:132: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
INFO:root:Prediction took 0.105 sec
Traceback (most recent call last):
File "examples/suction_policy.py", line 181, in <module>
action = policy(state)
File "build/bdist.linux-x86_64/egg/gqcnn/policy.py", line 161, in __call__
state = RgbdImageState(rgbd_im, camera_intr)
File "build/bdist.linux-x86_64/egg/gqcnn/policy.py", line 555, in action
KeyError: 'vmin'
INFO:rospy.core:signal_shutdown [atexit]
When running example/policy.py every things works great, any clue whats wrong? ``
Hi.
I have installed the suction gqcnn using the pretrained model, but i encounter several issues when running the suction_policy.py example on my own setup. The script is not runned in ROS, only as a python script.
Setup is as follows;
Ubuntu 16.04
Zivid 3D camera, with the objects placed 50-60 cm from the camera. The depth_0.npy file is in float32 format, and the values are in meters. The image is resized to fit the 640x480 requirement, and NAN values are set to the maximum distance.
The main issue seems to be that the network prefers the pick suction locations that does not make sense, such as edges (see image). The network will also favor areas that are only marginally closer to the camera (in the optimal range), even if the area is not suited for suction. The calculated probability of grasp success is also not a logical result. This is illustrated in the logdir files.
This is our suction_policy.yaml file:
`
sensor:
image_dir: /home/minion/BinPick_ws/src/gqcnn_pkg/src/gqcnn-dev_jeff/data/rgbd/multiple_objects
type: virtual_primesense
frame: primesense_overhead
calib_dir: /home/minion/BinPick_ws/src/gqcnn_pkg/src/gqcnn-dev_jeff/data/calib
policy:
num_seed_samples: 700
num_gmm_samples: 300
num_iters: 2
gmm_refit_p: 0.25
gmm_component_frac: 0.4
gmm_reg_covar: 0.01
deterministic: 1
gripper_width: 0.06
crop_height: 100
crop_width: 100
logging_dir: /home/minion/BinPick_ws/src/gqcnn_pkg/src/gqcnn-dev_jeff/log
sampling:
# type
type: suction
# params
max_suction_dir_optical_axis_angle: 20
delta_theta: 1
delta_phi: 1
sigma_depth: 0.00025
min_suction_dist: 1.0
angle_dist_weight: 5.0
depth_gaussian_sigma: 2.0
num_grasp_samples: 1000
max_dist_from_center: 1
max_num_samples: 10000
metric:
type: gaussian_curvature
window_size: 5
sample_rate: 1.0
gqcnn_model: /home/minion/BinPick_ws/src/gqcnn_pkg/src/model_zoo/GQ-Suction
crop_height: 96
crop_width: 96
vis:
grasp_sampling : 0
tf_images: 0
plane: 0
grasp_candidates: 1
elite_grasps: 0
grasp_ranking: 0
grasp_plan: 0
final_grasp: 1
vmin: 0.2
vmax: 0.8
k: 25
inpaint_rescale_factor: 0.75
detection:
type: point_cloud_box
foreground_mask_tolerance: 60
min_pt:
- 0.205
- -0.3
- 0.01
max_pt:
- 0.650
- 0.3
- 0.15
selection_policy: min
focus: 0
min_contour_area: 500.0
max_contour_area: 1000000.0
min_box_area: 500.0
max_box_area: 1000000.0
box_padding_px: 0
rescale_factor: 1.0
interpolation: bilinear
depth_grad_thresh: 1.0
contour_dist_thresh: 2.5
point_cloud_mask_only: 1
image_width: 640
image_height: 480
filter_dim: 1
`
Logdir files are attached. Please let me know if additional files/logs would be helpful for trouble shooting!
Hope someone can help me out.
System information
Describe what you are trying to do
I'm trying to use your Grasp_Sampler_Server for computing my robot grasps.
Describe current behaviour
Currently, I get the following error:
Traceback (most recent call last):
File "/home/ricks/panda_autograsp_ws/src/gqcnn/ros_nodes/grasp_planner_node.py", line 410, in <module>
grasping_policy = CrossEntropyRobustGraspingPolicy(policy_cfg)
File "/home/ricks/panda_autograsp_ws/src/gqcnn/gqcnn/grasping/policy/policy.py", line 726, in __init__
GraspingPolicy.__init__(self, config)
File "/home/ricks/panda_autograsp_ws/src/gqcnn/gqcnn/grasping/policy/policy.py", line 310, in __init__
metric_type, self._metric_config)
File "/home/ricks/panda_autograsp_ws/src/gqcnn/gqcnn/grasping/grasp_quality_function.py", line 1303, in quality_function
return GQCnnQualityFunction(config)
File "/home/ricks/panda_autograsp_ws/src/gqcnn/gqcnn/grasping/grasp_quality_function.py", line 931, in __init__
self._gqcnn = get_gqcnn_model().load(self._gqcnn_model_dir)
File "/home/ricks/panda_autograsp_ws/src/gqcnn/gqcnn/model/tf/network_tf.py", line 218, in load
gqcnn.init_mean_and_std(model_dir)
File "/home/ricks/panda_autograsp_ws/src/gqcnn/gqcnn/model/tf/network_tf.py", line 245, in init_mean_and_std
except FileNotFoundError:
NameError: global name 'FileNotFoundError' is not defined
Describe the expected behaviour
I want to have no errors.
Describe the input images
Images of the sensor_msg.Image create by the iai_kinectv2 toolbox
Solution
This is caused because FileNoteFound was released in python3.3. I created a version for myself in which the error is solved so I can use the toolbox with python 2.7 in ROS Melodic till a ROS version is out that fully supports python3. This Version can be found in pull request #79.
Hi，
I tried to install the "perception" module and it always claimed that "gqcnn 0.1.0 requires tensorflow>=1.0, which is not installed.". However, I am sure that tensorflow-gpu (version1.6) has been installed in my laptop, and I can print the version number successfully by "print(tensorflow.version)". Additionally, if I install the tensorflow with cpu version, everything will be OK, and I can use the GQCNN package to train my data. But it's really time-consuming, what should I do?
Looking forward to your kindly reply.
Sincerely,
Hui.
System information
Describe the problem
When Installing the python package, I get several dependency errors:
ERROR: visualization 0.0.6 has requirement matplotlib<=2.2.0, but you'll have matplotlib 2.2.4 which is incompatible.
ERROR: autolab-core 0.0.11 has requirement matplotlib<=2.2.0, but you'll have matplotlib 2.2.4 which is incompatible.
I now changed your setup.py requirements from:
requirements = [
'autolab-core',
'autolab-perception',
'visualization',
'numpy>=1.14.0',
'scipy',
'matplotlib<3.0.0',
'opencv-python',
'scikit-image<0.15.0',
'scikit-learn',
'psutil',
'gputil'
]
to:
requirements = [
'autolab-core',
'autolab-perception',
'visualization',
'numpy>=1.14.0',
'scipy',
'matplotlib<2.2.0',
'opencv-python',
'scikit-image<0.15.0',
'scikit-learn',
'psutil',
'gputil'
]
This could be due to my specific setup, but I thought you should know.
Provide the exact sequence of commands/steps that you executed before running into the problem
pip install . commandHi～
I am running your wiki's trainnig.py demo. But i found the trainning doesn't converge. It is so sad .
I am new to the tensorflow.
Hello @jeffmahler ,
I pulled latest dev_jeff branch and started getting following error.On your previous commit it wasnt occuring.I am using 'suction_disc_approach_planarity' metric for grasp quality computation.Kindly look into following issue.
[ERROR] [1523441358.434798]: Error processing request: Out of bounds indexing
['Traceback (most recent call last):\n', ' File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 625, in _handle_request\n response = convert_return_to_response(self.handler(request), self.response_class)\n', ' File "/home/akash.p/catkin_ws/src/gqcnn/ros_nodes/suction_grasp_planner_node.py", line 248, in plan_grasp\n return self.execute_policy(state, self.grasping_policy, cropped_camera_intrinsics.frame)\n', ' File "/home/akash.p/catkin_ws/src/gqcnn/ros_nodes/suction_grasp_planner_node.py", line 269, in execute_policy\n action = grasping_policy(rgbd_image_state)\n', ' File "/home/akash.p/catkin_ws/src/gqcnn/./gqcnn/policy.py", line 174, in call\n return self.action(state)\n', ' File "/home/akash.p/catkin_ws/src/gqcnn/./gqcnn/policy.py", line 627, in action\n q_values = self._grasp_quality_fn(state, grasps, params=self._config)\n', ' File "/home/akash.p/catkin_ws/src/gqcnn/./gqcnn/grasp_quality_function.py", line 52, in call\n return self.quality(state, actions, params)\n', ' File "/home/akash.p/catkin_ws/src/gqcnn/./gqcnn/grasp_quality_function.py", line 459, in quality\n points = self._points_in_window(point_cloud_image, action, segmask=state.segmask) # x,y in matrix A and z is vector z.\n', ' File "/home/akash.p/catkin_ws/src/gqcnn/./gqcnn/grasp_quality_function.py", line 418, in _points_in_window\n contact_point = point_cloud_image[int(action.center.y),int(action.center.x)]\n', ' File "/home/akash.p/catkin_ws/src/perception/./perception/image.py", line 474, in getitem\n raise ValueError('Out of bounds indexing')\n', 'ValueError: Out of bounds indexing\n']
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.