training not use GPU full perfermance , is just spikes about lightzero

[puyuan1996]

Hello,

Thank you for your diligent efforts in eliminating warnings within your local environment. In respect to the issue of low GPU utilization and occasional spikes encountered while running cartpole_sampled_efficientzero.py, potential reasons are as follows.

Within LightZero, the MCTS+RL (Monte Carlo Tree Search + Reinforcement Learning) algorithm predominantly encompasses three stages: collect, train, and eval:

• During the collect and eval stages, the sampled_efficientzero algorithm is utilized where the network inference is conducted mainly at the leaf nodes of the MCTS. This operation is executed on the GPU, while the MCTS search itself is primarily conducted on the CPU. Consequently, throughout the search process, GPU utilization may remain relatively low. Spikes are likely to manifest during the inference at the leaf nodes.

• Throughout the train stage, data is sampled from the replay buffer, model inference is executed, the corresponding loss is calculated, and subsequently, backpropagation is performed to update the model. During this stage, GPU utilization is anticipated to be higher. Nevertheless, as the model applied in RL algorithms and the batch size for training are not exceptionally large, the GPU utilization might not reach its maximum capacity of 100%.

In pursuit of enhancing the efficiency of MCTS search and improving GPU utilization, we will delve further into these areas in our forthcoming research. We deeply appreciate your valuable suggestions.

Best Wishes.

[amineoui]

i remove warnings but issu still there , GPU is used + take some gpu ram but still processor just spikes and

warning removed by :
pip3 install numba
pip3 install numpy==1.23.0

on cartpole_lightzero_env.py

line:32 replace self._env = gym.make('CartPole-v1') by self._env = gym.make('CartPole-v1', new_step_api=True)
line:45 Comment # self._env.seed(self._seed + np_seed)
line:49 and # self._env.seed(self._seed)
line:48 replace obs = self._env.reset() by obs = self._env.reset(seed=self._seed)
line:74 replace obs, rew, done, info = self._env.step(action) by obs, rew, done, _ , info = self._env.step(action)

this remove all my warning , but still GPU not go full perfermence only some gpu ram and sometimes spike on gpu processor

Please help me to solve this