pvela2017 / ksam-2022-robotic-competition

DigiAg Team code - Second place KSAM Robotic Competition

KSAM-2022-Robotic-Competition

Introduction

로봇 미션: 제작한 로봇을 이용하여 과수원 모사 경기장 내의 출발점부터 도착점까지 주행하면서 과수 모형에 달린 과일을 검출, 분류, 계수함(로봇 임무 제한시간은 5분 이내)

주행: 과수 열을 인식하여 열 사이로 충돌없이 주행해야 함
  - 각 팀별로 주어지는 예비 시험에서 맵작성 가능
  - 주행시간은 로봇의 출발점부터 주행하면서 각 과수의 과일 검출을 끝내고 도착점까지 이동한 시간으로 측정함

회피: 주행 동안 주어진 장애물 모형을 충돌 없이 회피하여야 함
  - 장애물은 본선대회 당일 임의 재배치 됨

검출: 과수 모형에 달린 과일을 정상과와 질병과로 분류하고 계수해야 함

Requirements

sudo apt-get install ros-noetic-slam-karto
sudo apt-get install ros-noetic-teb-local-planner

# Turtlebot
sudo apt install ros-noetic-turtlebot3-msgs
sudo apt install ros-noetic-turtlebot3
sudo apt install ros-noetic-dynamixel-sdk

Instalation

To install this repository on your home folder:

cd ~
git clone [email protected]:pvela2017/KSAM-2022-Robotic-Competition.git
cd KSAM-2022-Robotic-Competition/ros
catkin_make

Before running the repository the models path needs to be setup:

echo 'export GAZEBO_MODEL_PATH=~/KSAM-2022-Robotic-Competition/ros/src/robot_gazebo/models:${GAZEBO_MODEL_PATH}' >> ~/.bashrc
source ~/.bashrc

Run Simulation

Gazebo

source ./devel/setup.bash
roslaunch robot_gazebo scenario_1_world.launch

Simultaneous Localization And Mapping

In another terminal:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash

Karto SLAM:

roslaunch robot_slam robot_slam_simulation.launch slam_method:=karto

Gmapping SLAM:

roslaunch robot_slam robot_slam_simulation.launch slam_method:=gmapping

Navigation

In another terminal:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash

DWA planner:

roslaunch turtlebot3_navigation turtlebot3_navigation_simulation.launch planner:=dwa

TEB planner:

roslaunch turtlebot3_navigation turtlebot3_navigation_simulation.launch planner:=teb

Run Real Robot

Bring Up

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash
roslaunch turtlebot_bringup turtlebot_robot

Simultaneous Localization And Mapping

In another terminal:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash

Karto SLAM:

roslaunch robot_slam robot_slam_real.launch slam_method:=karto

Gmapping SLAM:

roslaunch robot_slam robot_slam_real.launch slam_method:=gmapping

Navigation

In another terminal:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash

DWA planner:

roslaunch turtlebot3_navigation turtlebot3_navigation_real.launch planner:=dwa

TEB planner:

roslaunch turtlebot3_navigation turtlebot3_navigation_real.launch planner:=teb

Other Scripts

Teleoperation

In another terminal:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash
roslaunch turtlebot3_teleop turtlebot3_teleop_key.launch

To initiate the waypoints:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash
rosrun turtlebot3_navigation goals.py

To reset the robot position in the simulation:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash
rosrun turtlebot3_navigation reset.py

Launch obstacle LED and camera servo:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash
rosrun robot_competition leds_servos.py

Launch tree identification using lidar:

cd ~/KSAM-2022-Robotic-Competition/ros
source ./devel/setup.bash
rosrun robot_competition tree_labels.py

YOLOX - Apple Counting

YOLOX is a single-stage object detector that makes several modifications to YOLOv3 with a DarkNet53 backbone. We chose YOLOX because it is a lightweight model able to run in a Jetson Nano. Also because our familiarity working with other YOLO neural network.

Requirements

To install the Yolo requirements:

cd ./YOLOX
conda create -n yolox python=3.7
conda activate yolox
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge
pip3 install -r requirements.txt
pip3 install -v -e .

Running

Pc Test:

python3 tools/demo.py video -f ./nano.py -c ./best_ckpt.pth --path ./1_2.mp4 --conf 0.25 --nms 0.45 --tsize 320 --save_result --device [cpu/gpu]

For counting the apples:

python3 digiag_1st.py capture -f ./nano.py -c ./best_ckpt.pth --conf 0.25 --nms 0.45 --tsize 320 --device gpu --save_result

Then to save the results to the USB:

sudo fdisk -l
sudo mount ~/media/myusb2

Sometimes there are some errors with the camera drivers, if that is the case this is the workaround:

rm ~/.cache/gstreamer-1.0/registry.aarch64.bin
export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1
sudo service nvargus-daemon restart

Training

To train the model, please refer to the guidelines provided by YOLOX.

ksam-2022-robotic-competition's Issues

Index out of range

Description
The index of the array was out of range during the competition.
Cause: Unknown

High Priority

https://github.com/mefisto2017/KSAM-2022-Robotic-Competition/blob/a81e5617a9d78d891208bf637130694c682b05a3/ros/src/robot_competition/src/tree_labels.py#L31

Fix Gazebo model according to our real robot

Description
Adapt our simulation model to our actual robot

Tasks

• Change Lidar range
• Set Lidar position as the one we will use in the real robot
• Set cameras position as the one we will use in the real robot
• Set cameras parameters ( FOV, etc)
• Modify robot structure (optional)
• Test in Gazebo

Implement TEB planner

When running TEB following error is displayed on GAZEBO:
[ERROR] [1660560429.465558481, 17.960000000]: Client [/move_base] wants topic /scan to have datatype/md5sum [sensor_msgs/PointCloud/d8e9c3f5afbdd8a130fd1d2763945fca], but our version has [sensor_msgs/LaserScan/90c7ef2dc6895d81024acba2ac42f369]. Dropping connection.

DWA

Description

Add DWA to our ROS package.

Tasks

• Test TEB in real world

• Tune TEB in real world

• Create more scenarios in Gazebo

• #8

• Tune TEB

• Test TEB in Gazebo

• Add DWA

• Test it on Gazebo

• Test it with real robot

Real robot

Description

Tasks to complete our real robot. (2 robots)

Tasks

• Install ubuntu 20.04
• Install ros-noetic
• Install graphical env.
• Install VNC
• Setup VNC
• Check all the ros package are installed by running our repo.
• Design piece to mount cameras.
• Test IMU.
• Test LIDAR
• Make real obstacles
• Decrease height of the robot
• Design piece to mount Jetson Nano on the back
• Mount Jetson Nano
• Implement LEDs.
• Implement launch button for the competition.
• Fix Lidar position and give me the x,y,z coordinates.
• Define our test cases for the real robot.
  Doheon
• Mount cameras.
• Test cameras.
• Print 2 board bridge
• Back on NX2
• Connect 2 board
• Connect fan
• 아크릴
• Standing tree without bottom tree

roslaunch cant pass 2 arguments?

In file: turtlebot3_navigation.launch

<launch>
  <!-- Arguments -->
  <arg name="model" default="burger" doc="model type [burger, waffle, waffle_pi]"/>
  <arg name="sim_real" default="" doc="Run simulation or real robot [simulation, real]"/>
  <arg name="planner" default="" doc="model type [dwa, teb]"/>
  <arg name="map_file" default="$(find turtlebot3_navigation)/maps/map_005.yaml"/>
  <arg name="open_rviz" default="true"/>
  <arg name="move_forward_only" default="false"/>

Parameters sim_real and planner cannot work together

SLAM

Description

Karto and Gmapping SLAM related problems.

High Priority

• Simulation: Gmapping map size is 2240x2240 meters.

   How to run:  `roslaunch robot_slam robot_slam_simulation.launch slam_method:=gmapping`
  

• Real: Karto lidar reading