Alpha Zero algorithm applied to the game Othello (aka Reversi).
MCTS framework from: https://jeffbradberry.com/posts/2015/09/intro-to-monte-carlo-tree-search/
Bitboard algorithm to find possible moves from: http://eprints.qut.edu.au/85005/1/__staffhome.qut.edu.au_staffgroupm$_meaton_Desktop_bits-7.pdf
$_ Bitboard algorithm to reflect over diagonals from: https://www.chessprogramming.org/index.php?title=Flipping_Mirroring_and_Rotating&mobileaction=toggle_view_mobile#FlipabouttheDiagonal
Neural network architecture adapted from: https://web.stanford.edu/~surag/posts/alphazero.html
- Basic implementation of mcts for othello.
- Basic implementation of mcts for ttt (in /ttt).
- Links to sources used.
- Punctuation to this log.
- Moved ttt mcts out of /ttt.
- /ttt.
- Bitboard implementation of Othello 1 (show_poss.py, tested with AI Grader).
- 64-char string to bitboard conversion methods.
- Display methods for bitboards.
- Bitboard implementation of Othello 3 (make_moves.py, tested with AI Grader).
- Bitboard to str conversion methods.
- Bitboard implementation of get_score.
- 0.2.0 README section wording.
- Streamlined get_poss to match new place method (in make_moves.py).
- Turned start() and s_board_to_bitboard() into one-liners.
- Debug print/displays in show_poss and make_moves.
- Bitboard implementation of Othello 6 (terminal_alphabeta.py, tested with AI Grader, 93.5).
- State recursion for get_poss.
- To-dos.
- Made bit_moves_to_s_moves() return a set rather than a list.
- get_score() now returns score as a single +/- int from perspective of token rather than a pair of ints.
- bit_moves_to_s_moves renamed as bit_poss_to_moves.
- bit_poss_to_moves now integrated into get_poss.
- To-dos (impractical).
- Speed tested bitboard get_poss vs string get_poss (bitwise is faster by a factor of 5, results in speed_test.py).
- Bitboards to check corners and center 4 squares added.
- Bitboard implementation of MCTS (doubled the amount of positions searched in 5 s).
- Tweaked cutoff val from 0.01 to 0.1 in both MCTS searches.
- Fixed small bug in make_moves.py.
- Bitboard reflections across diagonals implemented.
- Reflections added to state recursion in get_poss, unsure of speed upgrade.
- To-dos.
- Bitboard reflections removed from state recursion dicts (was slower).
- Organized files to start implementing neural net.
- Fixed UCB-selection in bit_mcts.py.
- New folder (/nnet) for neural net implementation.
- Modified MCTS to use policy/eval funcs.
- Separated game and display functions.
- (ONLY IN /NNET)
- Game methods take a state arg instead of separate board, token args
- Get_poss returns list
- (ONLY IN /NNET)
- Shortened bit_poss_to_moves.
- Added stochastic play option for early game mcts.
- Changed is_terminal to return -1, 0, or +1.
- A working neural net for policy/eval funcs (neural_net.py)–currently assesses positions in conjunction with MCTS, albeit randomly.
- To-do.
- Methods for saving/loading weights of nnet.
- Self_play.py, which handles the self_play for training the nnet.
- Methods for saving/loading training_examples from self play.
- Verbose/training flags for play_game (maybe this method is overloaded...).
- Made get_probs return a list of length 65 for probs.
- Renamed mcts to player (more accurate).
- Passes now indicated by 65 rather than -1 (to match prob vector).
- Combined get_probs with get_best_move for clarity.
- Merged display into game (no reason for sep file).
- Reflect funcs now take a state arg rather than a single bitboard arg.
- To-dos.
- To-dos.
- Reflecting boards/pis for training data.
- run_adversarial_episode() returns win_pct and win/loss/tie record
- latest_weights.h5 added with git LFS.
- After 100 games of self_play (~6000 states for training), nnet player beat completely random player in 18/25 games and beat random MCTS based player in 17/25 games.
- Full training/testing loop established in self_play.py (for testing on K80s).
- Saving of nnet pre-training iteration for testing purposes.
- Hyperparameter tuning begun.
- Rand_MCTS given same parameters as normal self_player.
- Debug statements.
- Saves training examples after full eps of self-play.
- Takes training examples from the last episodes' worth of games rather than from the last episode.
- Further testing (around 10 full training blocks) with similar settings as in 1.0.4 yield up-and-down performance both against random and previous selves. Need to implement a win_pct check.
- Win percentage check against previous selves (to prevent decrease in performance).
- More hyperparameters.
- Adv episodes take games rather than pairs of games (will be rounded down for odd numbers).
- NNet instantiation not verbose anymore.
- Last_nnet in self_play.py is a NeuralNet object now rather than a Keras model.
- Hyperparameters.
- Old debug comments.
- NeuralNet class removed because instantiating Keras models in a loop is memory-intensive.
- Player class and self_play.py changed to reflect this class removal.
- Most instances of "nnet" replaced by "model" for naming accuracy.
- Games played in self_play loop are now saved in saved_games.
- Helper script
see_old_games.pyadded to see saved games (but doesn't work).
- Rework state recursion
- Add folder for saved self_play games (as lists of moves)
- Figure out systematic way to load/save/clear training examples
- Since planning to use terminal_alphabeta competitively, enable for self-play and remove those training examples that alphabeta covers
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent