rl-robot's Introduction

RL-ROBOT

Ángel Martínez-Tenor - 2016

This repository provides a Reinforcement Learning framework in Python from the Machine Perception and Intelligent Robotics research group (MAPIR).

Reference: Towards a common implementation of reinforcement learning for multiple robotics tasks. Arxiv preprint ScienceDirect

Getting Started

Setup

Create a python environment and install the requirements. e.g. using conda:

conda create -n rlrobot python=3.10
conda activate rlrobot
pip install -r requirements.txt
# tkinter: sudo apt install python-tk

Run

Execute python run_custom_exp.py (content below)

import exp
import rlrobot

exp.ENVIRONMENT_TYPE = "MODEL"   # "VREP" for V-REP simulation
exp.TASK_ID = "wander_1k"
exp.FILE_MODEL = exp.TASK_ID + "_model"

exp.ALGORITHM = "TOSL"
exp.ACTION_STRATEGY = "QBIASSR"
 
exp.N_REPETITIONS = 1
exp.N_EPISODES = 1
exp.N_STEPS = 60 * 60

exp.DISPLAY_STEP = 500

rlrobot.run()

Full set of parameters available in exp.py
Tested on Ubuntu 14,16 ,18, 20 (64 bits)

V-REP settings:

Tested: V-REP PRO EDU V3.3.2 / V3.5.0

Use default values of remoteApiConnections.txt

portIndex1_port 		= 19997
portIndex1_debug 		= false
portIndex1_syncSimTrigger 	= true

Activate threaded rendering (recommended): system/usrset.txt -> threadedRenderingDuringSimulation = 1

Recommended simulation settings for V-REP scenes:

Simulation step time: 50 ms (default)
Real-Time Simulation: Enabled
Multiplication factor: 3.00 (required CPU >= i3 3110m)

Execute V-REP (./vrep.sh on linux). File -> Open Scene -> <RL-ROBOT path>/vrep_scenes

rl-robot's People

Contributors

Stargazers

Watchers

rl-robot's Issues

MemoryError

When I run the code rlrobot.run() , there is something wrong with the following information

Task Wander_12Ks_25a 12500 states 25 actions
Connected to Robot
wander_12k_TOSL_QBIASSR
Traceback (most recent call last):
File "", line 1, in
File "/home/maroon/RL-ROBOT/rlrobot.py", line 109, in run
lp.setup() # Learning process setup
File "/home/maroon/RL-ROBOT/lp.py", line 65, in setup
agent.setup()
File "/home/maroon/RL-ROBOT/agent.py", line 109, in setup
action_selection.setup()
File "/home/maroon/RL-ROBOT/action_selection.py", line 32, in setup
action_qbiassr.setup()
File "/home/maroon/RL-ROBOT/action_qbiassr.py", line 53, in setup
index = np.full(([n_states, n_inputs, n_states]), -1, dtype=np.int)
File "/home/maroon/.local/lib/python3.5/site-packages/numpy/core/numeric.py", line 298, in full
a = empty(shape, dtype, order)
MemoryError

Is this a common problem and what should I do?

hello,i am using vrep for reinforcement learning ,but my simulation speed is very slow ,i want to know your simulation speed ,in your paper ,only show computational cost of Sample-modeled wander-1K,without any computational cost of vrep simulation .
thank you very much,and look forward to you replay!