alpaca-zip / ultralytics_ros Goto Github PK

Can this ROS package be run under ROS1 (ROS melodic)? Since ROS melodic depends on python 2.7, but yolov8 has python >= 3.8, I use $ roslaunch ultralytics_ros tracker.launch debug:=true and get ModuleNotFoundError: No module named 'ultralytics'. Is it possible to run this project under ROS melodic and if so what should I do?

作者你好，我使用gazebo搭建了一个环境，并且在我的车体模型上安装了深度相机插件，我想实现识别结果以视频的形式展示出来，随着摄像头检测到画面的变化而不断变化，但是当我运行了readme里面指定的launch文件后，我只得到了一张识别结果图片，并且rviz显示无响应，当我关闭后控制台报错如图所示

我想知道我应该怎么配置或者编写launch文件才可以实现我想要的效果？还有我该怎么训练自己的模型？

Is it possible to make the model run on GPU instead of CPU?

When I catkin_make. I was prompted with
fatal error: ultralytics_ros/YoloResult.h: No such file or directory. and I didn't find the YoloResult.h file, where is it please?

Hello @Alpaca-zip and thank you for sharing your project.

Could you provide more direction on how someone should get started with your project?
I am not sure if I am doing something wrong on my side, but I cannot get an output image produced.
I am using a webcam.

I started:

1. roslaunch ultralytics_ros tracker.launch debug:=true
2. I started RVIZ and subscribed to image topic.

I do not receive an output image from any of the subscribed topics. RVIZ says "no image received."

Thank you for your help.

Branch

noetic-devel

Description

Thanks to sharing great project.

I've noticed that the tracker_with_cloud node currently only publishes /detection_points. However, it would be highly beneficial to also have it publish excluded_points, which would be the original Point Cloud excluding the /detection_points.

For example, when doing Mapping or Localization, we may need a point cloud excluding people and cars. I think excluded_points would be useful in such cases.

Additional

I've implemented the publisher in my fork of the project. If you're interested, I'd be happy to send over a PR. Just let me know!

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Branch

noetic-devel

Bug

https://github.com/Alpaca-zip/ultralytics_ros/actions/runs/7881844137?pr=58

ERROR: Could not install packages due to an EnvironmentError: [Errno 28] No space left on device: '/tmp/pip-unpacked-wheel-38i3wtow/torch/lib/libtorch_python.so' -> '/usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_python.so'

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Branch

noetic-devel

Description

The two functions that take the most time are projectCloud() and downsampleCloudMsg(), both of which seem to take similar amounts of time. While I can't think of any way to improve the speed of downsampleCloudMsg(), it seems that projectCloud() could benefit from parallel processing with OpenMP.

Additional

Within projectCloud(), the most time consuming processes are processPointsWithBbox() and processPointsWithMask(), with euclideanClusterExtraction() taking about a fifth of the time, which is less than I originally thought.

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Branch

noetic-devel

Bug

To my understanding, the issue occurs because in the tracker_with_cloud_node.cpp the cloud2TransformedCloud function asks for the cam_model_ frame, while it has not been initialized yet.

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Branch

melodic-devel, noetic-devel, humble-devel

Bug

It seems that bag files are not being downloaded correctly from google drive as shown below. The same issue appears in noetic-devel and melodic-devel.

#10 [6/6] RUN cd ~/ &&     wget --load-cookies /tmp/cookies.txt "https://drive.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://drive.google.com/uc?export=download&id=1kN8boJxfMIl7hjytxdgmzANZBhWV7YXU' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1kN8boJxfMIl7hjytxdgmzANZBhWV7YXU" -O kitti_2011_09_26_drive_0106_synced.zip &&     rm -rf /tmp/cookies.txt &&     unzip kitti_2011_09_26_drive_0106_synced.zip &&     rm kitti_2011_09_26_drive_0106_synced.zip
#10 0.449 --2024-01-18 14:26:57--  https://drive.google.com/uc?export=download&confirm=&id=1kN8boJxfMIl7hjytxdgmzANZBhWV7YXU
#10 0.451 Resolving drive.google.com (drive.google.com)... 142.250.189.14, 2607:f8b0:4007:815::200e
#10 0.453 Connecting to drive.google.com (drive.google.com)|142.250.189.14|:443... connected.
#10 0.481 HTTP request sent, awaiting response... 303 See Other
#10 0.556 Location: https://drive.usercontent.google.com/download?id=1kN8boJxfMIl7hjytxdgmzANZBhWV7YXU&export=download [following]
#10 0.556 --2024-01-18 14:26:57--  https://drive.usercontent.google.com/download?id=1kN8boJxfMIl7hjytxdgmzANZBhWV7YXU&export=download
#10 0.556 Resolving drive.usercontent.google.com (drive.usercontent.google.com)... 142.250.72.161, 2607:f8b0:4007:803::2001
#10 0.589 Connecting to drive.usercontent.google.com (drive.usercontent.google.com)|142.250.72.161|:443... connected.
#10 0.617 HTTP request sent, awaiting response... 200 OK
#10 0.815 Length: 2450 (2.4K) [text/html]
#10 0.815 Saving to: 'kitti_2011_09_26_drive_0106_synced.zip'
#10 0.815 
#10 0.815      0K ..                                                    100% 14.9M=0s
#10 0.816 
#10 0.816 2024-01-18 14:26:57 (14.9 MB/s) - 'kitti_2011_09_26_drive_0106_synced.zip' saved [2450/2450]
#10 0.816 
#10 0.820 Archive:  kitti_2011_09_26_drive_0106_synced.zip
#10 0.820   End-of-central-directory signature not found.  Either this file is not
#10 0.820   a zipfile, or it constitutes one disk of a multi-part archive.  In the
#10 0.820   latter case the central directory and zipfile comment will be found on
#10 0.820   the last disk(s) of this archive.
#10 0.820 unzip:  cannot find zipfile directory in one of kitti_2011_09_26_drive_0106_synced.zip or
#10 0.820         kitti_2011_09_26_drive_0106_synced.zip.zip, and cannot find kitti_2011_09_26_drive_0106_synced.zip.ZIP, period.
#10 ERROR: process "/bin/bash -c cd ~/ &&     wget --load-cookies /tmp/cookies.txt \"https://drive.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate 'https://drive.google.com/uc?export=download&id=1kN8boJxfMIl7hjytxdgmzANZBhWV7YXU' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\\1\\n/p')&id=1kN8boJxfMIl7hjytxdgmzANZBhWV7YXU\" -O kitti_2011_09_26_drive_0106_synced.zip &&     rm -rf /tmp/cookies.txt &&     unzip kitti_2011_09_26_drive_0106_synced.zip &&     rm kitti_2011_09_26_drive_0106_synced.zip" did not complete successfully: exit code: 9

Additional

I think it would be easier to maintain the bag files directly using Git LFS.

Tasks

• melodic-devel
• noetic-devel
• humble-devel

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Branch

noetic-devel

Question

there is isue something with message filter , syncCallback does not receive data, when I remove the PointCloud2 from the the system syncCallback working, there is isue with pcl data , I try everything , I increas que size and I try differen sync policies nothing changing, I also want to say my PointCloud2 data came form RGBD camera not lidar , but message type same. when I create seperate calback everything working, is it possioble that related rgbd data resulation high that create problem I don't know. can you give some advice .
I also want to say to you thank you for doing this great package and share with community , you are doing awsome job.
I also ask you when using this package . lidar and camera should located closed or when we add with tf some 1.5 or 2 meter offest it is will be okey ? in code I see algorithm take account ros tf.

Additional

No response

Hello,

This really a great work.
I would like to understand the background of it, especially the projection of the pointcloud to the 2d image space and conversion of the 2d detection pixels to 3d coordinates. Is there any journal or article that you referred to?

hello Alpaca-zip,
Thank you for your program.
I encountered the following error when running tracker_node, it seems that ros_numpy is not compatible.
Looking forward to your response.

Error:
Traceback (most recent call last):
File "/home/gh/v8_0.2/devel/lib/v8/track.py", line 15, in
exec(compile(fh.read(), python_script, 'exec'), context)
File "/home/gh/v8_0.2/src/v8/src/track.py", line 3, in
import ros_numpy
File "/opt/ros/noetic/lib/python3/dist-packages/ros_numpy/init.py", line 7, in
from . import point_cloud2
File "/opt/ros/noetic/lib/python3/dist-packages/ros_numpy/point_cloud2.py", line 224, in
def get_xyz_points(cloud_array, remove_nans=True, dtype=np.float):
File "/home/gh/.local/lib/python3.8/site-packages/numpy/init.py", line 305, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'float'.
np.float was a deprecated alias for the builtin float. To avoid this error in existing code, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

Dear Alpaca,
Thank for you project, I am using your results. However, when I run for tracker_clound ... I can not get the value 3D bounding box of topic /detection3d_topic. Please check about your project this issue. Thank you so much.
"Detected objects(3D bounding box) to detection3d_topic parameter. (vision_msgs/Detection3DArray)"

Branch

melodic-devel

Question

This work is significant, how did you get the coordinate system between the camera and the radar, and how do I proceed if I want to modify the coordinate relationship based on your work? Thank you for your busy reply!

Additional

No response

Branch

noetic-devel

Question

Hello,
Thank you for open sourcing your great work. I attempted to reproduce the object detection results using a point cloud (3D object detection). I downloaded the same kitti dataset (2011_09_26_drive_0106_sync) and run the "kitti_tracker_with_cloud.launch". 2D object detection with and without segmentation is fine, and there is no error. However, I could not see any 3D bounding boxes on the point cloud. This is illustrated in the attached screenshots. Could you please let me know what could be a possible solution to this problem?
Note: I tried all–yolov8 models (yolov8n-seg. pt, yolov8m-seg.pt, etc.) and also increases voxel_leaf_size, as suggested, but nothing works.

Here's my working configuration:

• Operating System: Ubuntu 20.04
• Hardware: Nvidia Agx_orin
• ROS Version: ROS Noetic

Branch

noetic-devel

Question

I want to know track id for use this information but i wasn't take it. Could you give me a trick about that?

Additional

No response

when i ehco the /yolo_result ERROR: Cannot load message class for [ultralytics_ros/YoloResult]. Are your messages built?

Quoted from: #15

The current implementation I've written does not explicitly handle the encoding of the subscribed topics; it simply passes the images to the track function as a numpy array. However, as per the official Ultralytics documentation, images passed to this function are expected to be in BGR encoding.

Tasks

• melodic-devel
• noetic-devel
• humble-devel

In Ubuntu-18.04, there is a problem that lap cannot be installed as follows.

$ python3 -m pip install lap
ERROR: Could not build wheels for lap, which is required to install pyproject.toml-based projects

At least the pip, numpy, and setuptools version changes did not solve these problems. Therefore, modify [melodic-devel] to disable the use of lap.

If anyone is familiar with this issue, please reopen it and offer a solution.

The relevant isuue is:
https://github.com/ultralytics/ultralytics/issues?q=lap

Since the desktop was changed to Ubuntu Mate with the following PR, colcon_ws is not built correctly. Also, ultralytics_ros is not present in ros2 pkg list.
Tiryoh/docker-ros2-desktop-vnc#92

Temporary solution: use tiryoh/ros2-desktop-vnc:humble before June 17, 2023.