monemati / px4-ros2-gazebo-yolov8 Goto Github PK

Aerial Object Detection using a Drone with PX4 Autopilot and ROS 2. PX4 SITL and Gazebo Garden used for Simulation. YOLOv8 used for Object Detection.

Python 100.00%

gazebo object-detection px4 px4-autopilot px4-ros2-gazebo pygame ros ros2 ros2-humble simulation

px4-ros2-gazebo-yolov8's Introduction

PX4-ROS2-Gazebo-YOLOv8

Aerial Object Detection using a Drone with PX4 Autopilot and ROS 2. PX4 SITL and Gazebo Garden used for Simulation. YOLOv8 used for Object Detection.

Demo

px4-ros2-gazebo-yolov8-demo.mp4

Installation

Create a virtual environment

# create
python -m venv ~/px4-venv

# activate
source ~/px4-venv/bin/activate

Clone repository

git clone https://github.com/monemati/PX4-ROS2-Gazebo-YOLOv8.git

Install PX4

cd ~
git clone https://github.com/PX4/PX4-Autopilot.git --recursive
bash ./PX4-Autopilot/Tools/setup/ubuntu.sh
cd PX4-Autopilot/
make px4_sitl

Install ROS 2

cd ~
sudo apt update && sudo apt install locales
sudo locale-gen en_US en_US.UTF-8
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export LANG=en_US.UTF-8
sudo apt install software-properties-common
sudo add-apt-repository universe
sudo apt update && sudo apt install curl -y
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null
sudo apt update && sudo apt upgrade -y
sudo apt install ros-humble-desktop
sudo apt install ros-dev-tools
source /opt/ros/humble/setup.bash && echo "source /opt/ros/humble/setup.bash" >> .bashrc
pip install --user -U empy pyros-genmsg setuptools

Setup Micro XRCE-DDS Agent & Client

cd ~
git clone https://github.com/eProsima/Micro-XRCE-DDS-Agent.git
cd Micro-XRCE-DDS-Agent
mkdir build
cd build
cmake ..
make
sudo make install
sudo ldconfig /usr/local/lib/

Build ROS 2 Workspace

mkdir -p ~/ws_sensor_combined/src/
cd ~/ws_sensor_combined/src/
git clone https://github.com/PX4/px4_msgs.git
git clone https://github.com/PX4/px4_ros_com.git
cd ..
source /opt/ros/humble/setup.bash
colcon build

mkdir -p ~/ws_offboard_control/src/
cd ~/ws_offboard_control/src/
git clone https://github.com/PX4/px4_msgs.git
git clone https://github.com/PX4/px4_ros_com.git
cd ..
source /opt/ros/humble/setup.bash
colcon build

Install MAVSDK

pip install mavsdk
pip install aioconsole
pip install pygame
sudo apt install ros-humble-ros-gzgarden
pip install numpy
pip install opencv-python

Install YOLO

pip install ultralytics

Additional Configs

Put below lines in your bashrc:

source /opt/ros/humble/setup.bash
export GZ_SIM_RESOURCE_PATH=~/.gz/models

Copy the content of models from main repo to ~/.gz/models
Copy default.sdf from worlds folder in the main repo to ~/PX4-Autopilot/Tools/simulation/gz/worlds/
Change the angle of Drone's camera for better visual:

# Go to ~/PX4-Autopilot/Tools/simulation/gz/models/x500_depth/model.sdf then change <pose> tag in line 9 from:
<pose>.12 .03 .242 0 0 0</pose>
to:
<pose>.15 .029 .21 0 0.7854 0</pose>

Run

Fly using Keyboard

You need several terminals.

Terminal #1:
cd ~/Micro-XRCE-DDS-Agent
MicroXRCEAgent udp4 -p 8888

Terminal #2:
cd ~/PX4-Autopilot
PX4_SYS_AUTOSTART=4002 PX4_GZ_MODEL_POSE="268.08,-128.22,3.86,0.00,0,-0.7" PX4_GZ_MODEL=x500_depth ./build/px4_sitl_default/bin/px4

Terminal #3:
ros2 run ros_gz_image image_bridge /camera

Terminal #4:
source ~/px4-venv/bin/activate
cd ~/PX4-ROS2-Gazebo-YOLOv8
python uav_camera_det.py

Terminal #5:
source ~/px4-venv/bin/activate
cd ~/PX4-ROS2-Gazebo-YOLOv8
python keyboard-mavsdk-test.py

When you run the last command a blank window will open for reading inputs from keyboard. focus on that window by clicking on it, then hit "r" on keyboard to arm the drone, and use WASD and Up-Down-Left-Right on the keyboard for flying, and use "l" for landing.

Fly using ROS 2

You need several terminals.

Terminal #1:
cd ~/Micro-XRCE-DDS-Agent
MicroXRCEAgent udp4 -p 8888

Terminal #2:
cd ~/PX4-Autopilot
PX4_SYS_AUTOSTART=4002 PX4_GZ_MODEL_POSE="283.08,-136.22,3.86,0.00,0,-0.7" PX4_GZ_MODEL=x500_depth ./build/px4_sitl_default/bin/px4

Terminal #3:
ros2 run ros_gz_image image_bridge /camera

Terminal #4:
source ~/px4-venv/bin/activate
cd ~/PX4-ROS2-Gazebo-YOLOv8
python uav_camera_det.py

Terminal #5:
cd ~/ws_offboard_control
source /opt/ros/humble/setup.bash
source install/local_setup.bash
ros2 run px4_ros_com offboard_control

Acknowledgement

px4-ros2-gazebo-yolov8's People

Contributors

Stargazers

Watchers

Forkers

eogks8921 kalpa123456 elebaid vdoom rastna12 vibgyor-s ironpower9k

px4-ros2-gazebo-yolov8's Issues

Can't run camera topic

when i use ros2 run ros_gz_image image_bridge /camera to open the camear, but failed, the screen is empty, i inspect the topic list, it's does't have /camera topic. What 's the problem ?

ros2 topic list
/fmu/in/obstacle_distance
/fmu/in/offboard_control_mode
/fmu/in/onboard_computer_status
/fmu/in/sensor_optical_flow
/fmu/in/telemetry_status
/fmu/in/trajectory_setpoint
/fmu/in/vehicle_attitude_setpoint
/fmu/in/vehicle_command
/fmu/in/vehicle_mocap_odometry
/fmu/in/vehicle_rates_setpoint
/fmu/in/vehicle_trajectory_bezier
/fmu/in/vehicle_trajectory_waypoint
/fmu/in/vehicle_visual_odometry
/fmu/out/failsafe_flags
/fmu/out/position_setpoint_triplet
/fmu/out/sensor_combined
/fmu/out/timesync_status
/fmu/out/vehicle_attitude
/fmu/out/vehicle_control_mode
/fmu/out/vehicle_global_position
/fmu/out/vehicle_gps_position
/fmu/out/vehicle_local_position
/fmu/out/vehicle_odometry
/fmu/out/vehicle_status
/parameter_events
/rosout

About keboard control and image capture problem

@monemati Hello, it's me, again! Sorry for bother! Now, I build all environment. I execute the keboard control part. As you mentioned, I open 5 terminal. When I execute the 5th terminal ( execute python keyboard-mavsdk-test.py) and try to control the UAV, The "detected frame" window does not chnage. There is no message in Terminal 3 (execute ros2 run ros_gz_image image_bridge /camera).
When I press key AWSD, there are some value changed in terminal 5, the third value become 0.5 or 1 or -0.5 etc.
Could you give me some suggestion help me to debug it?
Justin Hsu

The messages in terminal 2 are:
ubuntu@DESKTOP-129CA14:~/PX4-Autopilot$ PX4_SYS_AUTOSTART=4002 PX4_GZ_MODEL_POSE="268.08,-128.22,3.86,0.00,0,-0.7" PX4_GZ_MODEL=x500_depth ./build/px4_sitl_default/bin/px4

| ___ \ \ \ / / / |
| |/ / \ V / / /| |
| __/ / \ / /| |
| | / /^\ \ ___ |
_| / / |_/

px4 starting.

libEGL warning: failed to open /dev/dri/renderD128: Permission denied

WARN [vehicle_angular_velocity] no gyro selected, using sensor_gyro:0 1310988
INFO [commander] LED: open /dev/led0 failed (22)
INFO [mavlink] mode: Normal, data rate: 4000000 B/s on udp port 18570 remote port 14550
INFO [tone_alarm] home set
INFO [mavlink] mode: Onboard, data rate: 4000000 B/s on udp port 14580 remote port 14540
INFO [mavlink] mode: Onboard, data rate: 4000 B/s on udp port 14280 remote port 14030
INFO [mavlink] mode: Gimbal, data rate: 400000 B/s on udp port 13030 remote port 13280
INFO [logger] logger started (mode=all)
INFO [logger] Start file log (type: full)
INFO [logger] [logger] ./log/2024-01-31/07_29_11.ulg
INFO [logger] Opened full log file: ./log/2024-01-31/07_29_11.ulg
INFO [mavlink] MAVLink only on localhost (set param MAV_{i}BROADCAST = 1 to enable network)
INFO [mavlink] MAVLink only on localhost (set param MAV{i}_BROADCAST = 1 to enable network)
INFO [px4] Startup script returned successfully
pxh> INFO [commander] Ready for takeoff!
INFO [uxrce_dds_client] synchronized with time offset 1706686151688870us
INFO [uxrce_dds_client] successfully created rt/fmu/out/battery_status data writer, topic id: 19
INFO [uxrce_dds_client] successfully created rt/fmu/out/estimator_status_flags data writer, topic id: 86
INFO [uxrce_dds_client] successfully created rt/fmu/out/failsafe_flags data writer, topic id: 92
INFO [uxrce_dds_client] successfully created rt/fmu/out/manual_control_setpoint data writer, topic id: 138
INFO [uxrce_dds_client] successfully created rt/fmu/out/position_setpoint_triplet data writer, topic id: 169INFO [uxrce_dds_client] successfully created rt/fmu/out/sensor_combined data writer, topic id: 191
INFO [uxrce_dds_client] successfully created rt/fmu/out/timesync_status data writer, topic id: 214
INFO [uxrce_dds_client] successfully created rt/fmu/out/vehicle_attitude data writer, topic id: 230
INFO [uxrce_dds_client] successfully created rt/fmu/out/vehicle_control_mode data writer, topic id: 237
INFO [uxrce_dds_client] successfully created rt/fmu/out/vehicle_global_position data writer, topic id: 238
INFO [uxrce_dds_client] successfully created rt/fmu/out/vehicle_gps_position data writer, topic id: 240
INFO [uxrce_dds_client] successfully created rt/fmu/out/vehicle_local_position data writer, topic id: 244
INFO [uxrce_dds_client] successfully created rt/fmu/out/vehicle_odometry data writer, topic id: 249
INFO [uxrce_dds_client] successfully created rt/fmu/out/vehicle_status data writer, topic id: 254
WARN [timesync] time jump detected. Resetting time synchroniser.
WARN [timesync] time jump detected. Resetting time synchroniser.
INFO [mavlink] partner IP: 127.0.0.1
INFO [uxrce_dds_client] successfully created rt/fmu/out/vehicle_command_ack data writer, topic id: 234
INFO [commander] Armed by external command
INFO [tone_alarm] arming warning
INFO [commander] Takeoff detected
WARN [timesync] time jump detected. Resetting time synchroniser.
WARN [timesync] time jump detected. Resetting time synchroniser.
Stack trace (most recent call last) in thread 1033:
#31 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in
#30 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f208c02484f, in
#29 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f208bf92ac2, in
#28 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f20865f6ca0, in
#27 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f20865f5af1, in QThread::exec()
#26 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f20867e275a, in QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag)
#25 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f208683d0b7, in QEventDispatcherGlib::processEvents(QFlagsQEventLoop::ProcessEventsFlag)
#24 Object "/lib/x86_64-linux-gnu/libglib-2.0.so.0", at 0x7f208430a3e2, in g_main_context_iteration
#23 Object "/lib/x86_64-linux-gnu/libglib-2.0.so.0", at 0x7f2084362257, in
#22 Object "/lib/x86_64-linux-gnu/libglib-2.0.so.0", at 0x7f208430cd3a, in g_main_context_dispatch
#21 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f208683da66, in
#20 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f20867e6f26, in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*)
#19 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f20867e3e39, in QCoreApplication::notifyInternal2(QObject*, QEvent*)
#18 Object "/lib/x86_64-linux-gnu/libQt5Widgets.so.5", at 0x7f2085f3b712, in QApplicationPrivate::notify_helper(QObject*, QEvent*)
#17 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f208681141d, in QObject::event(QEvent*)
#16 Object "/usr/lib/x86_64-linux-gnu/gz-gui-7/plugins/libMinimalScene.so", at 0x7f2063864ee8, in gz::gui::plugins::RenderThread::RenderNext(gz::gui::plugins::RenderSync*)
#15 Object "/usr/lib/x86_64-linux-gnu/gz-gui-7/plugins/libMinimalScene.so", at 0x7f2063874f57, in gz::gui::plugins::RenderThreadRhiOpenGL::RenderNext(gz::gui::plugins::RenderSync*)
#14 Object "/usr/lib/x86_64-linux-gnu/gz-gui-7/plugins/libMinimalScene.so", at 0x7f20638697eb, in gz::gui::plugins::GzRenderer::Render(gz::gui::plugins::RenderSync*)
#13 Object "/usr/lib/x86_64-linux-gnu/gz-rendering-7/engine-plugins/libgz-rendering-ogre2.so", at 0x7f2009497b0d, in
#12 Object "/usr/lib/x86_64-linux-gnu/gz-rendering-7/engine-plugins/libgz-rendering-ogre2.so", at 0x7f2009531bb7, in gz::rendering::v7::Ogre2Scene::FlushGpuCommandsAndStartNewFrame(unsigned char, bool)
#11 Object "/usr/lib/x86_64-linux-gnu/gz-rendering-7/engine-plugins/libgz-rendering-ogre2.so", at 0x7f2009531942, in gz::rendering::v7::Ogre2Scene::FlushGpuCommandsOnly()
#10 Object "/usr/lib/x86_64-linux-gnu/OGRE-2.3/libOgreNextMain.so.2.3.1", at 0x7f2009180602, in Ogre::CompositorManager2::_updateImplementation()
#9 Object "/usr/lib/x86_64-linux-gnu/OGRE-2.3/OGRE/RenderSystem_GL3Plus.so", at 0x7f2008b0ec2b, in Ogre::GL3PlusVaoManager::_update()
#8 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7f206c797dd5, in
#7 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7f206ca532ef, in
#6 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7f206ca4c57a, in
#5 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7f206ca4b611, in
#4 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7f206ca50547, in
#3 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7f206cf5f11a, in
#2 Object "/usr/lib/wsl/lib/libd3d12core.so", at 0x7f2063c193af, in
#1 Object "/usr/lib/wsl/lib/libd3d12core.so", at 0x7f2063c0583a, in
#0 Object "/usr/lib/wsl/lib/libd3d12core.so", at 0x7f2063caba60, in
Segmentation fault (Address not mapped to object [0x180])
Stack trace (most recent call last) in thread 928:
#28 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in
#27 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7fa2e0cc784f, in
#26 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7fa2e0c35ac2, in
#25 Object "/lib/x86_64-linux-gnu/libstdc++.so.6", at 0x7fa2dcf27252, in
#24 Object "/usr/lib/x86_64-linux-gnu/gz-sim-7/plugins/libgz-sim-sensors-system.so", at 0x7fa2adb0ac37, in gz::sim::v7::systems::SensorsPrivate::RenderThread()
#23 Object "/usr/lib/x86_64-linux-gnu/gz-sim-7/plugins/libgz-sim-sensors-system.so", at 0x7fa2adb0a333, in gz::sim::v7::systems::SensorsPrivate::RunOnce()
#22 Object "/lib/x86_64-linux-gnu/libgz-sensors7.so.7", at 0x7fa2adb9a7ce, in gz::sensors::v7::Manager::RunOnce(std::chrono::duration<long, std::ratio<1l, 1000000000l> > const&, bool)
#21 Object "/lib/x86_64-linux-gnu/libgz-sensors7.so.7", at 0x7fa2adb9a765, in gz::sensors::v7::Sensor::Update(std::chrono::duration<long, std::ratio<1l, 1000000000l> > const&, bool)
#20 Object "/lib/x86_64-linux-gnu/libgz-sensors7-camera.so.7", at 0x7fa2ad94b6bf, in gz::sensors::v7::CameraSensor::Update(std::chrono::duration<long, std::ratio<1l, 1000000000l> > const&)
#19 Object "/usr/lib/x86_64-linux-gnu/gz-rendering-7/engine-plugins/libgz-rendering-ogre2.so", at 0x7fa29dc3283b, in
#18 Object "/usr/lib/x86_64-linux-gnu/gz-rendering-7/engine-plugins/libgz-rendering-ogre2.so", at 0x7fa29dcc3a94, in gz::rendering::v7::Ogre2RenderTarget::Copy(gz::rendering::v7::Image&) const
#17 Object "/usr/lib/x86_64-linux-gnu/OGRE-2.3/libOgreNextMain.so.2.3.1", at 0x7fa2acd21c22, in Ogre::Image2::copyContentsToMemory(Ogre::TextureGpu*, Ogre::TextureBox, Ogre::TextureBox, Ogre::PixelFormatGpu, bool)
#16 Object "/usr/lib/x86_64-linux-gnu/OGRE-2.3/libOgreNextMain.so.2.3.1", at 0x7fa2acc76d9c, in Ogre::AsyncTextureTicket::download(Ogre::TextureGpu*, unsigned char, bool, Ogre::TextureBox*, bool)
#15 Object "/usr/lib/x86_64-linux-gnu/OGRE-2.3/OGRE/RenderSystem_GL3Plus.so", at 0x7fa28713f543, in Ogre::GL3PlusAsyncTextureTicket::downloadFromGpu(Ogre::TextureGpu*, unsigned char, bool, Ogre::TextureBox*)
#14 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e522c3d, in
#13 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e52160e, in
#12 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e521449, in
#11 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e2e94f1, in
#10 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e5219e6, in
#9 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e2e6bbb, in
#8 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e30b8ec, in
#7 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e7d0864, in
#6 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e7d057a, in
#5 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e7cf611, in
#4 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28e7d4547, in
#3 Object "/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so", at 0x7fa28ece311a, in
#2 Object "/usr/lib/wsl/lib/libd3d12core.so", at 0x7fa29ca203af, in
#1 Object "/usr/lib/wsl/lib/libd3d12core.so", at 0x7fa29ca0c83a, in
#0 Object "/usr/lib/wsl/lib/libd3d12core.so", at 0x7fa29cab2a60, in
Segmentation fault (Address not mapped to object [0x180])

The messages in terminal 4 are:

ubuntu@DESKTOP-129CA14:~/PX4-ROS2-Gazebo-YOLOv8$ python3 uav_camera_det.py
/home/ubuntu/.local/lib/python3.10/site-packages/matplotlib/projections/init.py:63: UserWarning: Unable to import Axes3D. This may be due to multiple versions of Matplotlib being installed (e.g. as a system package and as a pip package). As a result, the 3D projection is not available.
warnings.warn("Unable to import Axes3D. This may be due to multiple versions of "
[INFO] [1706686307.530698747] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 234.6ms
Speed: 6.4ms preprocess, 234.6ms inference, 2.7ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686309.811796528] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 230.5ms
Speed: 2.2ms preprocess, 230.5ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686310.059574347] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 195.7ms
Speed: 1.6ms preprocess, 195.7ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686310.267478064] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 196.3ms
Speed: 1.5ms preprocess, 196.3ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686310.476035180] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 197.4ms
Speed: 1.5ms preprocess, 197.4ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686310.805740106] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 196.8ms
Speed: 1.7ms preprocess, 196.8ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686311.440424157] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 199.4ms
Speed: 2.1ms preprocess, 199.4ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686312.094040908] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 201.5ms
Speed: 1.8ms preprocess, 201.5ms inference, 0.7ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686312.737367859] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 199.8ms
Speed: 1.5ms preprocess, 199.8ms inference, 0.6ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686313.417425313] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 225.6ms
Speed: 2.9ms preprocess, 225.6ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686314.182934448] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 202.2ms
Speed: 1.5ms preprocess, 202.2ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686314.887464659] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 224.2ms
Speed: 2.3ms preprocess, 224.2ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686315.614201869] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 204.1ms
Speed: 2.0ms preprocess, 204.1ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686316.351212380] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 219.6ms
Speed: 2.6ms preprocess, 219.6ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686317.037596090] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 200.0ms
Speed: 1.5ms preprocess, 200.0ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686317.695627400] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 233.0ms
Speed: 3.4ms preprocess, 233.0ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686318.388757910] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 195.5ms
Speed: 1.6ms preprocess, 195.5ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686319.004148019] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 194.2ms
Speed: 1.8ms preprocess, 194.2ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686319.641245528] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 197.1ms
Speed: 2.1ms preprocess, 197.1ms inference, 0.4ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686320.269485438] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 204.3ms
Speed: 1.6ms preprocess, 204.3ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686320.920355147] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 205.8ms
Speed: 1.7ms preprocess, 205.8ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686321.635106158] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 237.2ms
Speed: 2.5ms preprocess, 237.2ms inference, 0.7ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686322.397014672] [image_subscriber]: Receiving video frame

0: 384x640 (no detections), 199.2ms
Speed: 2.1ms preprocess, 199.2ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
[INFO] [1706686323.064647686] [image_subscriber]: Receiving video frame

The messages in terminal 5 are:
ubuntu@DESKTOP-129CA14:~/PX4-ROS2-Gazebo-YOLOv8$ python3 keyboard-mavsdk-test.py
pygame 2.5.2 (SDL 2.28.2, Python 3.10.12)
Hello from the pygame community. https://www.pygame.org/contribute.html
Waiting for drone to connect...
-- Connected to drone!
-- Connected to drone!
Task exception was never retrieved
future: <Task finished name='Task-3' coro=<getKeyboardInput() done, defined at /home/ubuntu/PX4-ROS2-Gazebo-YOLOv8/keyboard-mavsdk-test.py:12> exception=ActionError(<mavsdk.action.ActionResult object at 0x7fc3fc9dd420>, 'arm()')>
Traceback (most recent call last):
File "/home/ubuntu/PX4-ROS2-Gazebo-YOLOv8/keyboard-mavsdk-test.py", line 36, in getKeyboardInput
await my_drone.action.arm()
File "/home/ubuntu/.local/lib/python3.10/site-packages/mavsdk/action.py", line 320, in arm
raise ActionError(result, "arm()")
mavsdk.action.ActionError: COMMAND_DENIED: 'Command Denied'; origin: arm(); params: ()
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 1 0
0 0 1 0
0 0 1 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0
0 0 0.5 0

About execution issue

Hello! I try to follow your project. Your projrct is good for my research topic. My environment is Ubuntu 22.04(jammy). However, when I try to install PX4, there is an error for protoc version. I check the protoc version and found that the version is 3.12. Therefore, I change the environment to Ubuntu 20.04 (Focal). The PX4 complier is OK and the protoc version is 3.6.1. But in Ubuntu 20.04, ROS2 Humble could not be installed! Could you give me a suggestion how should I do?
Thanks much!!
Justin Hsu

The problem of running simulation

Dear sir:
I have the problem of running the gazebo:
~/PX4-Autopilot$ PX4_SYS_AUTOSTART=4002 PX4_GZ_MODEL_POSE="268.08,-128.22,3.86,0.00,0,-0.7" PX4_GZ_MODEL=x500_depth ./build/px4_sitl_default/bin/px4

| ___ \ \ \ / / / |
| |/ / \ V / / /| |
| __/ / \ / /| |
| | / /^\ \ ___ |
_| / / |_/

px4 starting.

INFO [px4] startup script: /bin/sh etc/init.d-posix/rcS 0
env SYS_AUTOSTART: 4002
INFO [param] selected parameter default file parameters.bson
INFO [param] importing from 'parameters.bson'
INFO [parameters] BSON document size 310 bytes, decoded 310 bytes (INT32:13, FLOAT:3)
INFO [param] selected parameter backup file parameters_backup.bson
INFO [dataman] data manager file './dataman' size is 7872608 bytes
INFO [init] Gazebo simulator
INFO [init] starting gazebo with world: /home/tang/PX4-Autopilot/Tools/simulation/gz/worlds/default.sdf
INFO [init] PX4_GZ_MODEL_POSE set, spawning at: 268.08,-128.22,3.86,0.00,0,-0.7
INFO [gz_bridge] world: default, model name: x500_depth_0, simulation model: x500_depth
INFO [gz_bridge] Requested Model Position: 268.08,-128.22,3.86,0.00,0,-0.7
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
[Err] [SDFFeatures.cc:843] The geometry element of collision [collision] couldn't be created
[Err] [SDFFeatures.cc:843] The geometry element of collision [collision] couldn't be created
[Err] [SDFFeatures.cc:843] The geometry element of collision [collision] couldn't be created
[Err] [SDFFeatures.cc:843] The geometry element of collision [collision] couldn't be created
ERROR [gz_bridge] timed out waiting for clock message
ERROR [gz_bridge] Task start failed (-1)
ERROR [init] gz_bridge failed to start and spawn model
ERROR [px4] Startup script returned with return value: 256
tang@tang-GL502VSK:~/PX4-Autopilot$ [Err] [FuelClient.cc:691] Failed to download model.
Server: https://fuel.ignitionrobotics.org
Route: openrobotics/models/sonoma raceway/2/sonoma raceway.zip
REST response code: 504
[Err] [SystemPaths.cc:432] URI [https://fuel.ignitionrobotics.org/1.0/openrobotics/models/sonoma raceway/2/files/meshes/Raceway.obj] resolved to path [/meshes/Raceway.obj] but the path does not exist
[Err] [SystemPaths.cc:525] Could not resolve file [https://fuel.ignitionrobotics.org/1.0/openrobotics/models/sonoma raceway/2/files/meshes/Raceway.obj]
[Err] [MeshManager.cc:193] Unable to find file[https://fuel.ignitionrobotics.org/1.0/openrobotics/models/sonoma raceway/2/files/meshes/Raceway.obj]

And after that the GAZEBO could show the drone and car but can't show the world and it also can't connect to the MicroDDS and QgroundControl.

And also I can't running ros2 run ros_gz_image image_bridge /camera it show like:
$ ros2 run ros_gz_image image_bridge /camera
Package 'ros_gz_image' not found

Thanks for your model it looks very good, i really wish to running it.

PX4 Arming Problem

The installation was successful.
And all components except PX4 working well !

PX4 erros:
Compass Sensor 0 missing
Accel #0 fail: STALE!
Gyro #0 fail: STALE!

Error Gazebo - PX4-ROS2-Gazebo-YOLOv8

Hi, I hope you are well. I have problem to see Gazebo scene after installing [PX4-ROS2-Gazebo-YOLOv8], Gazebo page appear but completely black. I have ROS2 Humble on my pc. I installed ROS2 before, so I did not just follow that part related to install Ros2, but I did all process step by step.

When I run this code in Terminal, Gazebo page appear in black scene.
(Terminal #2:
cd ~/PX4-Autopilot
PX4_SYS_AUTOSTART=4002 PX4_GZ_MODEL_POSE="268.08,-128.22,3.86,0.00,0,-0.7" PX4_GZ_MODEL=x500_depth ./build/px4_sitl_default/bin/px4)

Would you please guide me to solve my problem? Thanks in advance

This is my Error:
desktop:~/PX4-Autopilot$ cd /PX4-Autopilot
desktop:/PX4-Autopilot$ PX4_SYS_AUTOSTART=4002 PX4_GZ_MODEL_POSE="268.08,-128.22,3.86,0.00,0,-0.7" PX4_GZ_MODEL=x500_depth ./build/px4_sitl_default/bin/px4

| ___ \ \ \ / / / |
| |/ / \ V / / /| |
| __/ / \ / /| |
| | / /^\ \ ___ |
_| / / |_/

px4 starting.

INFO [px4] startup script: /bin/sh etc/init.d-posix/rcS 0
env SYS_AUTOSTART: 4002
INFO [param] selected parameter default file parameters.bson
INFO [param] selected parameter backup file parameters_backup.bson
SYS_AUTOCONFIG: curr: 0 -> new: 1
SYS_AUTOSTART: curr: 0 -> new: 4002
CAL_ACC0_ID: curr: 0 -> new: 1310988
CAL_GYRO0_ID: curr: 0 -> new: 1310988
CAL_ACC1_ID: curr: 0 -> new: 1310996
CAL_GYRO1_ID: curr: 0 -> new: 1310996
CAL_ACC2_ID: curr: 0 -> new: 1311004
CAL_GYRO2_ID: curr: 0 -> new: 1311004
CAL_MAG0_ID: curr: 0 -> new: 197388
CAL_MAG0_PRIO: curr: -1 -> new: 50
CAL_MAG1_ID: curr: 0 -> new: 197644
CAL_MAG1_PRIO: curr: -1 -> new: 50
SENS_BOARD_X_OFF: curr: 0.0000 -> new: 0.0000
SENS_DPRES_OFF: curr: 0.0000 -> new: 0.0010
ERROR [param] Parameter SIM_GZ_EN not found.
ERROR [param] Parameter SIM_GZ_EC_FUNC1 not found.
ERROR [param] Parameter SIM_GZ_EC_FUNC2 not found.
ERROR [param] Parameter SIM_GZ_EC_FUNC3 not found.
ERROR [param] Parameter SIM_GZ_EC_FUNC4 not found.
ERROR [param] Parameter SIM_GZ_EC_MIN1 not found.
ERROR [param] Parameter SIM_GZ_EC_MIN2 not found.
ERROR [param] Parameter SIM_GZ_EC_MIN3 not found.
ERROR [param] Parameter SIM_GZ_EC_MIN4 not found.
ERROR [param] Parameter SIM_GZ_EC_MAX1 not found.
ERROR [param] Parameter SIM_GZ_EC_MAX2 not found.
ERROR [param] Parameter SIM_GZ_EC_MAX3 not found.
ERROR [param] Parameter SIM_GZ_EC_MAX4 not found.
INFO [dataman] data manager file './dataman' size is 7872608 bytes
INFO [init] Gazebo simulator
INFO [init] starting gazebo with world: /default.sdf
INFO [init] PX4_GZ_MODEL_POSE set, spawning at: 268.08,-128.22,3.86,0.00,0,-0.7
etc/init.d-posix/rcS: 117: /home/Desktop/PX4-Autopilot/build/px4_sitl_default/rootfs/etc/init.d-posix/px4-rc.simulator: gz_bridge: not found
ERROR [init] gz_bridge failed to start and spawn model
ERROR [px4] Startup script returned with return value: 256
desktop:~/PX4-Autopilot$ [Err] [SystemPaths.cc:534] File [/default.sdf] resolved to path [/default.sdf] but the path does not exist
[Err] [Server.cc:109] Failed to find world [/default.sdf]

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble