maxhumber / carlo

Monte Carlo Tree Search Library

About

A Monte Carlo Tree Search (MCTS) library for turn-based games built with Swift

Import

Import Carlo by adding the following line to any .swift file:

import Carlo

Implement

Implement Carlo by designing player, move, and game structs that conform to the CarloGamePlayer, CarloGameMove, and CarloGame protocols.

While the first two protocols don't explicitly require anything, "conforming" to them might look like this:

enum ConnectThreePlayer: Int, CarloGamePlayer, CustomStringConvertible {
    case one = 1
    case two = 2
    
    var opposite: Self {
        switch self {
        case .one: return .two
        case .two: return .one
        }
    }
    
    var description: String {
        "\(rawValue)"
    }
}

typealias ConnectThreeMove = Int
extension ConnectThreeMove: CarloGameMove {}

Conforming to CarloGame requires the following:

public protocol CarloGame: Equatable {
    associatedtype Player: CarloGamePlayer
    associatedtype Move: CarloGameMove
  
    var currentPlayer: Player { get }
    func availableMoves() -> [Move]
    func update(_ move: Move) -> Self
    func evaluate(for player: Player) -> Evaluation
}

Properly implemented it might look like this:

struct ConnectThreeGame: CarloGame, CustomStringConvertible, Equatable {
    typealias Player = ConnectThreePlayer
    typealias Move = ConnectThreeMove

    var array: Array<Int>
  
    // REQUIRED
    var currentPlayer: Player
    
    init(length: Int = 10, currentPlayer: Player = .one) {
        self.array = Array.init(repeating: 0, count: length)
        self.currentPlayer = currentPlayer
    }

    // REQUIRED
    func availableMoves() -> [Move] {
        array
            .enumerated()
            .compactMap { $0.element == 0 ? Move($0.offset) : nil}
    }
    
    // REQUIRED
    func update(_ move: Move) -> Self {
        var copy = self
        copy.array[move] = currentPlayer.rawValue
        copy.currentPlayer = currentPlayer.opposite
        return copy
    }
    
    // REQUIRED
    func evaluate(for player: Player) -> Evaluation {
        let player3 = three(for: player)
        let oppo3 = three(for: player.opposite)
        let remaining0 = array.contains(0)
        switch (player3, oppo3, remaining0) {
        case (true, true, _): return .draw
        case (true, false, _): return .win
        case (false, true, _): return .loss
        case (false, false, false): return .draw
        default: return .ongoing(0.5)
        }
    }
    
    private func three(for player: Player) -> Bool {
        var count = 0
        for slot in array {
            if slot == player.rawValue {
                count += 1
            } else {
                count = 0
            }
            if count == 3 {
                return true
            }
        }
        return false
    }
    
    var description: String {
        return array.reduce(into: "") { result, i in
            result += String(i)
        }
    }
}

Use

Use Carlo by scaffolding a CarloTactician on a CarloGame:

typealias Computer = CarloTactician<ConnectThreeGame>

Instantiate with arguments for CarloGamePlayer and a limit for the number turns to rollout in a single search iteration:

let computer = Computer(for: .two, maxRolloutDepth: 5)

Call the .iterate() method to perform one search iteration in the tree, .bestMove to get the best move (so far) as found by search algorithm, and .uproot(to:) to recycle the tree and update the internal game state:

var game = ConnectThreeGame(length: 10, currentPlayer: .one)
/// 0000000000
game = game.update(4)
/// 0000100000
game = game.update(0)
/// 2000100000
game = game.update(7)
/// 2000100000
game = game.update(2)
/// 2020100000
game = game.update(9)
/// 2020100001 ... player 2 can win if move => 1

computer.uproot(to: game)
for _ in 0..<50 {
    computer.iterate()
}

let move = computer.bestMove!
game = game.update(move)
/// 2220100001 ... game over

Install

If you use Swift Package Manager adding Carlo as a dependency is as easy as adding it to the dependencies of your Package.swift:

dependencies: [
    .package(url: "https://github.com/maxhumber/Carlo.git", from: "1.0.2")
]

carlo's Issues

Backpropagation assigns same value to both players

Backpropagation should assign "1" as value to a node in case of win and "0" in case of loss. The current implementation assigns after a rollout the same value to all nodes. It should however assign different values, depending on who is the currentPlayer. Right now for one player not the best moves are picked during the Selection phase, but the worst moves. Because of this the algorithm picks rather random moves.

I have a fix I'm going to commit.