Comments (2)
Hi @yangysc , thanks for testing the RNN. The shared network from the spec ppo_rnn_shared_cartpole
works slightly better because there are less hyperparameters to run. It yields slightly better results:
[2019-07-14 22:53:07,321 PID:73583 INFO __init__.py log_summary] Trial 0 session 0 ppo_rnn_shared_cartpole_t0_s0 [train_df] epi: 169 t: 200 wall_t: 360 opt_step: 234560 frame: 23465 fps: 65.1806 total_reward: 200 total_reward_ma: 173.03 loss: 0.0292752 lr: 4.55652e-17 explore_var: nan entropy_coef: 0.001 entropy: 0.112986 grad_norm: nan
[2019-07-14 22:53:10,775 PID:73583 INFO __init__.py log_summary] Trial 0 session 0 ppo_rnn_shared_cartpole_t0_s0 [train_df] epi: 170 t: 185 wall_t: 363 opt_step: 236480 frame: 23650 fps: 65.1515 total_reward: 185 total_reward_ma: 173.2 loss: 0.679745 lr: 4.55652e-17 explore_var: nan entropy_coef: 0.001 entropy: 0.228988 grad_norm: nan
[2019-07-14 22:53:14,093 PID:73583 INFO __init__.py log_summary] Trial 0 session 0 ppo_rnn_shared_cartpole_t0_s0 [train_df] epi: 171 t: 200 wall_t: 367 opt_step: 238400 frame: 23850 fps: 64.9864 total_reward: 200 total_reward_ma: 173.35 loss: 0.624804 lr: 4.55652e-17 explore_var: nan entropy_coef: 0.001 entropy: 0.315934 grad_norm: nan
We have not thoroughly tested RNNs yet, but your observation is true and the RecurrentNet
class is limited in that sense. The hidden state is discarded and not used as input in the next forward pass. We can implement this by storing the hidden state alongside the state in agent Memory, and retrieve it during memory.sample()
.
This will take some time to implement, but we're currently busy with benchmarking tasks. I'm making this issue as a feature request so we can get on it as soon as we have time.
from slm-lab.
In the meantime, you could try increasing the sequence length (seq_len
) in the net
component of the spec
file. This will persist the hidden state for more steps.
from slm-lab.
Related Issues (20)
- Are there any benchmarks for SIL? HOT 2
- Empty Multi trial Graph in 'search' mode HOT 1
- Crashing with seccomp-bpf failure in syscall 2030 HOT 2
- RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED on Pong HOT 1
- Cannot open shared object file: No such file or directory when running demo HOT 2
- Book branch problem, need your help HOT 2
- Installation on MacOs 11.6.1 fails HOT 2
- IndexError: list index out of range
- ModuleNotFoundError: No module named 'cv2' HOT 4
- subprocess.CalledProcessError: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128. HOT 4
- How to add a non-gym envrinment? HOT 6
- I have encounter one problem, maybe a bug. HOT 1
- How to run the code on multiple GPUs? HOT 1
- how to improve the convergence performance of training loss?
- python setup.py test
- `optimizer.step()` before `lr_scheduler.step()` Warning Occurred
- Docker build fails on environment.yml installation HOT 1
- How to add a custom gym environment in json spec file.
- Exception: pyglet 2.0.0 requires Python 3.8 or newer HOT 1
- Docker build failed HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from slm-lab.