starceees / robot-perception-

robot-perception

System Overview:

The cornerstone of our system is the KeyboardPlayerPyGame class, which facilitates both manual and autonomous navigation. By utilizing libraries such as PyGame for interaction and OpenCV for computer vision, the class enables the robot to explore, detect targets, and record navigational commands for future use.

Exploration Phase:

During the manual exploration phase, the operator directs the robot using keyboard inputs. The system captures and records each command, along with the corresponding visual data from the robot's first-person view (FPV). This phase lays the groundwork for autonomous navigation by building a repository of navigational data.

Autonomous Navigation Phase:

Leveraging the recorded data, the robot can autonomously retrace its steps, using visual cues to navigate through previously explored areas. The ORB detector and FLANN matcher work in tandem to enable real-time target detection and path plotting.

Feature Matching and Path Plotting:

Our approach uses the ORB algorithm to detect keypoints and compute descriptors. The FLANN matcher then identifies similar features in sequential images, allowing the robot to understand its movement and detect dead-ends.

Command Recording:

Each action taken by the robot is meticulously recorded, capturing the command type, duration, and associated FPV image. This historical log is crucial for the robot's learning and provides a rich dataset for navigation optimization.

Target Detection and Positioning:

Through continuous feature matching, the system detects when a target enters the robot's FPV. It then records the robot's position, updating the map with the target's location.

Navigation Commands and Path Visualization:

The robot executes recorded commands to navigate, with the path and targets visualized in a 2D space. This visualization helps in monitoring the robot's progress and analyzing its navigational efficiency.