Question about ch7 about hands-on-intelligent-agents-with-openai-gym

[praveen-palanisamy]

Hi Ali,

The error is because in your code snipped, action is a 3 dimensional array instead of an integer.
As documented here , when "discrete_actions" is set to true (the default), the action space of the Carla-v0 environment is Discrete(9).

Therefore, you can simplify your pick action function something like below:

def pick_random_discrete_action(t, current_action):
#     a = env.action_space.sample()
#     return a
    if t < 60:
        return np.array(3)  # Go forward
    
    if t % 5 > 0:
        return current_action

    rn = random.randint(0,9)
   return rn

To see what each discrete action means, please refer to the table here.

If you want to use the continuous action space, you can set the "discrete_actions": False here:

Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch7/carla-gym/carla_gym/envs/carla_env.py

Lines 70 to 72 in df9ab39

	# Default environment configuration
	ENV_CONFIG = {
	"discrete_actions": True,

and then provide a 2 dimensional action (example: np.array([0, 1]) ) to the env.step(...) function. The definitions for the continuous action space is listed here.

Hope that helps.

[AliBaheri]

Thanks Praveen,

In fact, In my dataset, I have 3 dimensional control action in the form of [steer, gas, break]. One quick hack may be change this part in carla-env.py:

def step_env(self, action):
        if self.config["discrete_actions"]:
            action = DISCRETE_ACTIONS[int(action)]
        assert len(action) == 3, "Invalid action {}".format(action)
        throttle = float(np.clip(action[0], 0, 1))
        brake = float(np.abs(np.clip(action[0], -1, 0)))
        steer = float(np.clip(action[1], -1, 1))
        reverse = False
        hand_brake = False

where I changed assert len(action) == 2 to assert len(action) == 3. It solves the problem, but it turns out this is not good for driving policy as it is pointless to accelerate while braking. In fact, as we already discussed, it is good idea to concatenate gas and brake into a single [-1 , 1], which indicates [-1 , 0) for brake and (0, 1] for gas. However, I do not know how can I modify the code or dataset to take into account this? Is there any way to resolve the issue without modifying or
re-collecting the dataset? In other words, is there any way to keep the size of the control action as 3?

[praveen-palanisamy]

@AliBaheri : Sorry for the delay. I was just able to get to this.

You could use a 3 dimensional control action in this form [steer, gas, brake] with the Carla Gym environment implementation.
In that case, you can use a modified step function in place of the the original step function in carla_env.py or for quick experiments, you can replace the following lines:

Hands-On-Intelligent-Agents-with-OpenAI-Gym/ch7/carla-gym/carla_gym/envs/carla_env.py

Lines 324 to 330 in df9ab39

	def step_env(self, action):
	if self.config["discrete_actions"]:
	action = DISCRETE_ACTIONS[int(action)]
	assert len(action) == 2, "Invalid action {}".format(action)
	throttle = float(np.clip(action[0], 0, 1))
	brake = float(np.abs(np.clip(action[0], -1, 0)))
	steer = float(np.clip(action[1], -1, 1))

with

def step_env(self, action):  # action: [ steer, gas, brake]
    steer = float(np.clip(action[0], -1, 1))
    throttle = float(np.clip(action[1], 0, 1))
    brake = float(np.clip[action[2], 0, 1))

[praveen-palanisamy]

@AliBaheri: Is your question answered?

[githubce]

Author provided the solution. Reopen if you want to continue discussion.