Issue when progress at 100% about deepracer-core

[crr0004]

3df4b46 should fix it

[crr0004]

No you haven't done anything wrong. It's a bug I haven't fixed yet. It's because the method is trying to call an AWS API to cancel the simulation job. It needs to be wrapped in a check for the LOCAL environment variable and bypassed if so.

…

On Sun., 30 Jun. 2019, 3:59 am bhannebi, ***@***.***> wrote: Hello I have the following error when progress reach 100 *An error occurred (UnrecognizedClientException) when calling the CancelSimulationJob operation: The security token included in the request is invalid.* Is it an issue or am I doing something wrong? File "/usr/local/lib/python3.5/dist-packages/rl_coach/environments/gym_environment.py", line 448, in _take_action self.state, self.reward, self.done, self.info = self.env.step(action) File "/usr/local/lib/python3.5/dist-packages/gym/wrappers/time_limit.py", line 31, in step observation, reward, done, info = self.env.step(action) File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 567, in step return super().step([self.steering_angle, self.speed]) File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 271, in step self.infer_reward_state(self.steering_angle, self.speed) File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 436, in infer_reward_state self.finish_episode(current_progress) File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 472, in finish_episode self.cancel_simulation_job() File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 519, in cancel_simulation_job job=self.simulation_job_arn File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 661, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the CancelSimulationJob operation: The security token included in the request is invalid. ***@***.***:/app/robomaker-deepracer/simulation_ws# [agent-9] killing on exit — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#26>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABPNBIOI4M2MTURMNMFSXJTP46PHJANCNFSM4H4K4PCQ> .

[crr0004]

I'm just going to close this, if you still have issues, comment and we can reopen this.

[bhannebipro]

It seems to work thanks.

[kimwooglae]

When I run local training, it is failed after 12 hours with same error message.
In the model folder in minio server, I found exact 200 model files.
The same error occurred twice.
Any advice?

SIM_TRACE_LOG:19999,26,2.7384,-1.1359,-2.0313,0.14,8.00,4,0.0014,False,False,13.7719,15,22.92,1565108136.847502

SIM_TRACE_LOG:19999,27,2.6546,-1.2924,-2.0545,0.00,8.00,3,0.0000,True,False,14.4170,16,22.92,1565108136.9079044

reward: 21.731499599643186
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/rollout_worker.py", line 303, in <module>
    main()
  File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/rollout_worker.py", line 298, in main
    memory_backend_params = memory_backend_params
  File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/rollout_worker.py", line 169, in rollout_worker
    graph_manager.act(EnvironmentEpisodes(num_steps=act_steps))
  File "/usr/local/lib/python3.5/dist-packages/rl_coach/graph_managers/graph_manager.py", line 443, in act
    result = self.top_level_manager.step(None)
  File "/usr/local/lib/python3.5/dist-packages/rl_coach/level_manager.py", line 230, in step
    env_response = self.environment.step(action_info.action)
  File "/usr/local/lib/python3.5/dist-packages/rl_coach/environments/environment.py", line 299, in step
    self._take_action(action)
  File "/usr/local/lib/python3.5/dist-packages/rl_coach/environments/gym_environment.py", line 448, in _take_action
    self.state, self.reward, self.done, self.info = self.env.step(action)
  File "/usr/local/lib/python3.5/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 566, in step
    return super().step([self.steering_angle, self.speed])
  File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 271, in step
    self.infer_reward_state(self.steering_angle, self.speed)
  File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 437, in infer_reward_state
    self.finish_episode(current_progress)
  File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 473, in finish_episode
    self.cancel_simulation_job()
  File "/app/robomaker-deepracer/simulation_ws/install/sagemaker_rl_agent/lib/python3.5/site-packages/markov/environments/deepracer_racetrack_env.py", line 518, in cancel_simulation_job
    job=self.simulation_job_arn
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the CancelSimulationJob operation: The security token included in the request is invalid.

[kimwooglae]

I fix the error changing NUMBER_OF_EPISODES value in robomaker.env