Can I use reinforcement learning to control the drone navigate to the point goal ? about gym-pybullet-drones

[gmmhhh]

Thank you!

[JacopoPan]

Have a look at class RLTakeoffAviary (used in scriptlearn.py)

The current _computeReward is a step-wise function of the drone's z in the world frame

You can hard-code a goal position by making _computeReward return a (dense or sparse) reward computed as a function of the distance from the desired point

Note that _computeDone terminates episodes outside the cube with edge=1, on top of the world frame origin (0,0,0)

To fly outside this space, you'd need to re-implement both _computeDone and _clipAndNormalizeState

[chris-aeviator]

In my RL problem I'm introducing a design that has tilting motors, where I just mind about the thrust vectoring, not about collision of the propellers (though every visual has a corresponding collision box). I extended the actionspace for two additional values (n=6), made sure to set the upper and lower limits of the joints and I can confirm I can control them by setting a POSITION inside BaseAviary and see correct rotations in fly.py).

0 degree, 0 degree

+30 degree (jointUpperLimit), -30 degree (jointLowerLimit)

+30 degree ,+30 degree

I now wonder how _clipAndNormalizeState plays a role here. I cannot find any call to this function, not in this repo nor in openai/gym or all github :) .

Do I append my two link positions (both can travel -30 to +30 degrees, set in RAD) that I want the singleAgent to optimize?

Would I modify norm_and_clipped inside HoverAviary (

gym-pybullet-drones/gym_pybullet_drones/envs/single_agent_rl/HoverAviary.py

Line 158 in f4d3186

norm_and_clipped = np.hstack([normalized_pos_xy,

) and computeControl

gym-pybullet-drones/gym_pybullet_drones/control/SimplePIDControl.py

Line 71 in f4d3186

def computeControl(self,

in SimplePIDController (which I'm using in baseAviary) to include the two angles of the motor joints?

[JacopoPan]

@chris-aeviator

I only briefly read this message (I'm not sure why it's in this issue?).
_clipAndNormalizeState is called in _computeObs of BaseSingleAgentAviary.py (or the multi agent class)

obs = self._clipAndNormalizeState(self._getDroneStateVector(0))

It's just an extra function to clip and normalized the state vector IF a certain RL problem requires it (or can benefit from it).
It can be an identity function if you want to, you probably don't want to clip and normalize if you are using one of the PID controllers.

If you change the state vector, you would want to use computeControl instead of computeControlFromState in BaseControl (one is just a wrapper for the other to use fewer arguments).