This project implements a Handover or Handoff (HO) operation using reinforcement learning (RL) techniques. It focuses on training an agent to perform HO operations effectively in a simulated environment that is satellite communication environment. This project includes two parts designed to perform Handover: inter-satellite HO (Satellite HO) and intra-satellite HO (Spotbeam HO). The reinforcement learning algorithm employed here enables the agent to learn and adapt its actions to achieve optimal performance.
List the software and libraries required to run your project :
- Python 3.x
- OpenAI Gym
- stable-baselines3
- pygame
- tensorboard
You can also include any hardware requirements if necessary.
-
Clone the repository to your local machine:
git clone https://github.com/RezaEs79/Bachelor-Project.git
-
To install dependencies (It works on both Linux and Windows) :
pip install gym conda install stable-baselines3 conda install multipledispatch conda install pygame pip install Shimmy conda install -c conda-forge tensorboard
-
Also, for any possible errors, these may be useful:
pip install stable-baselines3[extra] pip install tensorflow --upgrade --force-reinstall pip install stable-baselines3[extra] --upgrade --force-reinstall pip3 install torch torchvision torchaudio
To use this project, follow these steps:
-
Chose your handover scenario in satellite communication: Proj1 is inter satellite HO and Proj2 is intra satellite HO. You can try following command to test whether it works or not:
# in Proj1 python Test.py
or :
# in Proj2 python mytest.py
-
Train the RL Agent: Train the reinforcement learning agent to optimize handover operations.
-
Test the Agent: Evaluate the agent's performance and visualize the results.
To train the reinforcement learning agent, run the following command:
python Model_builder.py
and then type ppo or a2c or dqn for your specific algorithm.
To test the trained agent and evaluate its performance, use the following command:
python Model_runner.py
and then type ppo or a2c or dqn for your specific algorithm.
For watching results obtained from your project, such as performance metrics, graphs, and visualizations. use codes in MATLAB_Codes folder and results that writed in .txt file.