BUG about iqn-and-extensions

[BY571]

hey,

you mean the memory usage is steadily increasing?
how big is your replay buffer? remember that each tuple in the buffer consists of two states so two (4, 84, 84).

haven't really tested the algorithm for several hours yet, but I can try later.

[THSWind]

@BY571 yes, it increasing until killed by system.
the size of replay buffer is 1,000,000.

I tested it of course, several games. They were all killed in several hours(about 3 ~ 5 hours), no exception.

[BY571]

@THSWind How much memory do you have on your GPU or in general? I know from my PC that it cant handle 1mio samples in the replay buffer.
however you could try to save the sates as dtype=np.uint8 this should reduce memory

[THSWind]

@BY571 This machine's memory is 64G. (2080Ti GPU).
Have you really tested the code implementation? Run 20000000 steps or see if it can complete the training task properly?
I will try it as you say to save the sates as dtype=np.uint8 , but I don't think it's the root of the problem.
buddy, I'm following up on your work. It is important for me~

[BY571]

I trained on the Pong environment several times and had no problems even though with a 10.000 replay buffer since my system does not have that much memory.

[BY571]

@THSWind if you want to discuss the problem you can join this discord channel: https://discord.gg/kHrBHC