joshvarty / alphazerocpp Goto Github PK

Currently we create our tensor on the CPU and then manually move it to our CUDA device. We would prefer to create it directly on the device.

See:

• https://stackoverflow.com/questions/58631466/create-a-torchtensor-from-c-c-array-without-using-from-blob/58631791#58631791
• https://discuss.pytorch.org/t/create-tensor-on-cuda-device-in-c/113232

It might be more natural to have Backpropagate() in Node. I think we could get rid of this "PlayerId" thing and just operate on _toPlay directly the same way we're doing it in Expand()

Right now during training we're using a fixed temperature of 0. We should consider implementing this properly and decaying temperature.

We would like our Monte Carlo Tree Search to operate on base class Model objects instead of our Connect2Model as it currently does. This will make it easier to write tests and to support other games going forward.

My understanding is that Adam is not great near the beginning of training (when it has seen few examples) but is good once it has been exposed to enough of your dataset.

In our project we're creating a new Adam optimizer every time we try to train our network. This feels like it might be putting us in a bad place. Should we consider other optimizers? Has anyone written about this?

Right now we have to create a mock model for each test method. It's probably easier if we refactor the test code to provide a mock model.

I think most of these methods can be made const, but it will take some time to see if those consts propagate anywhere else.

Currently we're using Connect2Game, but we should be using a more generic game class so we can support other games easily.

See: https://google.github.io/styleguide/cppguide.html#Variable_Names

I think private fields like columns in game.h should be columns_.

We should probably have Connect2Game inherit from Game and then have other classes (Trainer, MCTS etc.) operate on Game objects.

This will allow us to add new games easily.

I think the condition is backwards.

AlphaZeroSimple: https://github.com/JoshVarty/AlphaZeroSimple/blob/b68171a5cb9367b407017e07f5de3f65b10e888e/monte_carlo_tree_search.py#L126-L136

AlphaZeroCpp: https://github.com/JoshVarty/AlphaZeroCpp/blob/main/src/monte_carlo_tree_search.cpp#L151-L166

There are some weird things about saving models in C++ PyTorch so I've currently just commented out the implementation. At some point we should come back and save the model correctly during training.