ros-perception / perception_open3d Goto Github PK

I would like to address #18, so I need a maintainer access to this repository.

When building the examples via catkin build. open3d_conversions builds fine. Running on ros-melodic "official" docker container

EDIT: same happens when running catkin_make

http://build.ros.org/job/Nbin_uF64__open3d_conversions__ubuntu_focal_amd64__binary/1/console

18:47:37   Could not find a package configuration file provided by "Open3D" with any
18:47:37   of the following names:
18:47:37 
18:47:37     Open3DConfig.cmake
18:47:37     open3d-config.cmake

Error is above, the biggest item is that open3D couldn't be found to build with @nkhedekar

The namespace should be open3d_conversions and not open3d_ros in this readme

There are a few problems with this function in terms of custom fields:

• the output field type in the open3d structure is float32, regardless of the ros field type
• custom fields are entered as triplets into the open3d structure, where the triplet contains 3 copies of the same data entry
• only uint8/int8/float32 ros field types are handled in the iterator

This is a code snippet from the rosToOpen3d function:

        sensor_msgs::PointCloud2ConstIterator<float> ros_pc2_fx(*ros_pc2, ros_pc2->fields[num_fields].name);
        sensor_msgs::PointCloud2ConstIterator<float> ros_pc2_fy(*ros_pc2, ros_pc2->fields[num_fields].name);
        sensor_msgs::PointCloud2ConstIterator<float> ros_pc2_fz(*ros_pc2, ros_pc2->fields[num_fields].name);
        std::vector<Eigen::Vector3d> o3d_TensorList_fields;

        for (size_t i = 0; i < ros_pc2->height * ros_pc2->width; ++i, ++ros_pc2_fx, ++ros_pc2_fy, ++ros_pc2_fz)
        {
          o3d_TensorList_fields.push_back(Eigen::Vector3d(*ros_pc2_fx, *ros_pc2_fy, *ros_pc2_fz));
        }
        open3d::core::Tensor o3d_tpc_fields =
          open3d::core::eigen_converter::EigenVector3dVectorToTensor(o3d_TensorList_fields, dtype_f, device_type);
        o3d_tpc.SetPointAttr(ros_pc2->fields[num_fields].name, o3d_tpc_fields);

Currently the available method is variadic, whereby the user passes arguments like "xyz", "float" and , which (as far as i'm aware) requires knowledge of the fields at compile-time. By replacing the variadic structure with a std::vector<std::pair<std::string,std::string>> or similar, the fields could be determined at run-time. Alternatively, 2 functions could be provided for the open3dToRos function for converting tensor types, since the variadic one is convenient if you know the fields.

The example application where I have run into this issue is very simple:

1. Receive a pointcloud in ROS with fields I don't know about in advance (e.g. intensity, ring, ambience, x, y, z, r, g, b), which is common if dealing with lidars from different sensor drivers
2. Perform some operation on this pointcloud, e.g. filter out all points beyond a certain range, with the assumption that there are x, y, z points
3. Publish the resultant pointcloud with all the fields (this is not possible with open3d::geometry::PointCloud but is with the tensor versoin)

A pure python solution takes above 100ms to convert pointclouds ROS<->Open3d. It would be good to have python bindings using pybind11 for this to be faster.

FYI: isl-org/Open3D#6730

Thank you very much for writing this package and sharing it with the community.

I wanted to point out that the current implementation makes very strong assumptions on the fields of the input point clouds in format sensor_msgs/PointCloud2. While the ROS msgs is quite flexible and compatible with any type of point cloud data, the conversion and Open3D itself are not:

https://github.com/ros-perception/perception_open3d/blob/main/open3d_conversions/src/open3d_conversions.cpp#L81

This means that for example, you automatically discard curvature, reflectivity, ambient noise, and anything that goes beyond RGB or normals.

Would it be possible to implement both in this package and in the Open3D side a more flexible type? Libpointmatcher is a really good example, which is also light in memory (everything is represented as Eigen matrices), and intrinsically handles aspects like memory alignment.

Thank you in advance.

Hi,

The deadline for projects to get into the first cut of Galactic is today! You don't have to do it right now, but it would be good to get this released in the next week or two so that it gets in by the first sync.

For rolling, you should create a new ros2 branch that all ROS2 PRs are against (the "master" for ros2) and then cut distributions off of it. That way if you update rolling, it'll get pulled into the next distribution automatically (assuming no new changes were made requiring a new release)

I am not really sure when the API change happened but lined 159-161 and 252 need to change to accommodate new Open3D API.

Update:

v0.13.0 tag offers GetPoints()
v0.14.1 tag offers GetPointPositions() convenience function for GetPointAttr("positions")

I haven't tested if this change brakes the behavior and results but it compiles successfully.

# 159-161
const open3d::core::Tensor& o3d_TensorList_points = pointcloud.GetPointAttr("positions");
modifier.resize(pointcloud.GetPointAttr("positions").GetShape()[0]);
ros_pc2.data.reserve(pointcloud.GetPointAttr("positions").GetShape()[0]);
----
# 252
o3d_tpc.SetPointAttr("positions",o3d_tpc_points);

Check isl-org/Open3D#4093 (comment)