GithubHelp home page GithubHelp logo

victorialena / mcts.jl Goto Github PK

View Code? Open in Web Editor NEW

This project forked from juliapomdp/mcts.jl

0.0 0.0 0.0 67.75 MB

Monte Carlo Tree Search for Markov decision processes using the POMDPs.jl framework

License: Other

Julia 100.00%

mcts.jl's Introduction

MCTS

Documentation Build Status Coverage Status

MCTS Tree for Grid World, visualized

This package implements the Monte-Carlo Tree Search algorithm in Julia for solving Markov decision processes (MDPs). The user should define the problem according to the generative interface in POMDPs.jl. Examples of problem definitions can be found in POMDPModels.jl.

There is also a BeliefMCTSSolver that solves a POMDP by converting it to an MDP in the belief space.

Special thanks to Jon Cox for writing the original version of this code.

For reference, see the UCT algorithm in this paper: Kocsis, Levente, and Csaba Szepesvári. "Bandit Based Monte-Carlo planning." European Conference on Machine Learning. Springer, Berlin, Heidelberg, 2006.

Installation

In Julia, type, ]add MCTS

Documentation

Documentation can be found on the following site: juliapomdp.github.io/MCTS.jl/latest/

Usage

If mdp is an MDP defined with the POMDPs.jl interface, the MCTS solver can be used to find an optimized action, a, for the MDP in state s as follows:

using POMDPs
using POMDPModels # for the SimpleGridWorld problem
using MCTS
using StaticArrays
mdp = SimpleGridWorld()
solver = MCTSSolver(n_iterations=50, depth=20, exploration_constant=5.0)
planner = solve(solver, mdp)
a = action(planner, SA[1,2])

See this notebook for an example of how to visualize the search tree.

See this notebook for examples of customizing solver behavior, specifically the Rollouts section for using heuristic rollout policies.

mcts.jl's People

Contributors

bozenkhaa avatar etotheipluspi avatar fredcallaway avatar juliatagbot avatar lassepe avatar maximebouton avatar mossr avatar pslade2 avatar rcnlee avatar rejuvyesh avatar shushman avatar zsunberg avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.