agausmann / perftree Goto Github PK

View Code? Open in Web Editor NEW

48.0 3.0 3.0 33 KB

Perft debugger. Compare your chess engine to Stockfish and quickly find discrepancies in move generation.

License: MIT License

Rust 100.00%

chess testing debugging perft rust cli

perftree's Introduction

perftree

A perft debugger. Compare your chess engine to Stockfish and quickly find discrepancies in move generation.

How it works

When debugging a chess engine, it is common to compare its move generation to a known-good engine using the results of the perft function, which counts all of the nodes at some given depth from some starting position. Using these results, one can quickly isolate the problematic subtrees and figure out where the move generation differs between the two engines.

Instead of comparing the results and walking the tree manually, I use and maintain perftree, a semi-automatic debugger that does that hard work for you. It can keep track of where you are in the game tree, evaluate the perft function at the current position, and compare the results automatically, highlighting the differences so they are easy to pick out.

Install

perftree uses Stockfish, a well-known engine used widely throughout the chess community, as a trusted source of perft results. Download and install Stockfish if you haven't already, and make sure you can run it from the command line with the command stockfish.

Install the perftree CLI application from the crates.io repository with cargo:

cargo install perftree-cli

Usage

Your perft script

perftree requires some way to invoke the perft function on your chess engine. Currently, it expects the user to provide a script, which will be invoked like this:

./your-perft.sh "$depth" "$fen" "$moves"

where

$depth is the maximum depth of the evaluation,
$fen is the Forsyth-Edwards Notation string of some base position,
$moves is an optional space-separated list of moves from the base position to the position to be evaluated, where each move is formatted as $source$target$promotion, e.g. e2e4 or a7b8Q.

The script is expected to output the results of the perft function to standard output, with the following format:

For each move available at the current position, print the move and the number of nodes at the given depth which are an ancestor of that move, separated by whitespace.
After the list of moves, print a blank line.
Finally, print the total node count on its own line.

For example, this is what the depth-3 perft of the starting position should look like:

$ ./your-perft.sh 3 "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"
a2a3 380
b2b3 420
c2c3 420
d2d3 539
e2e3 599
f2f3 380
g2g3 420
h2h3 380
a2a4 420
b2b4 421
c2c4 441
d2d4 560
e2e4 600
f2f4 401
g2g4 421
h2h4 420
b1c3 440
g1h3 400
b1a3 400
g1f3 440

8902

Running perftree

Run perftree from the commandline, and pass the script path as the first argument:

perftree ./your-script.sh

perftree understands the following commands:

fen [new_fen] - Set the FEN string of the root node, and clears the move list. When new_fen is not provided, the current FEN string will be printed instead.
moves [new_moves ...] - Set the move list. When new_moves is not provided, the current move list will be printed instead.
depth [new_depth] - Set the max depth for perft. When new_depth is not provided, the current depth will be printed instead.
root - Clears the move list, effectively changing to the root node of the game tree.
child|move <move> - Pushes the given move onto the move list, effectively changing to the child node identified by the given move in the current state.
parent|unmove - Pops a move from the move list, effectively changing to the parent node of the current state.
diff - Calculates and outputs a diff of the perft results for the current node. Your results will be on the left, and Stockfish will be on the right. A missing number means that the move did not exist in the output from the corresponding engine.
exit|quit - Exits the program.

Example

perftree's People

Contributors

Stargazers

Watchers

Forkers

evenstrom greatgodoffire justlinuxuser

perftree's Issues

Chess 960 support

Hi! Really useful project! I used it to debug my move generator a while back.

Are there any plans on supporting Chess960 (Fisher Random Chess) in the future?

Graphical UI

Likely going to use Qt to build it, as it is a relatively mature framework with existing support for tree views. It's also what I have experience with (albeit mainly with PyQt) and there appears to be decent bindings for Rust.

Concept: Display the game tree using a tree model.

The full tree doesn't have to be present in memory, it can be built on demand, similar to the way that it is already (manually) done using the CLI.

If you `move` more than ones Stockfish gets confused the turn of the player

Imagine this case:

$ perfttree my_movegen
fen rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
depth 5
move e2e4
move e7e5
move d2d4
diff

In the above case, Stockfish's output should consider that it is black's turn, not white, but what I get is possible moves of white! Not sure if this is a Stockfish problem (I doubt it), or perfttree's

cannot compute diff: unexpected end of line

Hello, thank you for developing this tool. However, I am struggling to make it work.

My perft program, e.g, my_perft.out, receives two args (depth and FEN) and outputs the nodes in the expected format according to the README.md file.

./my_perft.out 3 "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"
a2a3 380
b2b3 420
c2c3 420
d2d3 539
e2e3 599
f2f3 380
g2g3 420
h2h3 380
a2a4 420
b2b4 421
c2c4 441
d2d4 560
e2e4 600
f2f4 401
g2g4 421
h2h4 420
b1a3 400
b1c3 440
g1f3 440
g1h3 400

8902

When running perftree my_perft.out:

> fen
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
> depth 3
> diff
cannot compute diff: unexpected end of line
>

I tried removing the \n character after the total node count but the error persists. What can I do? Thank you in advance.

Interactive prompt

A prompt should be displayed whenever the program expects input from an interactive user, to make it clear when commands finish running (especially in the case of unexpected output / errors displayed to the user)

Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }

Hey, I am really excited to try this to get the final few bugs out of my engine. However, whatever I pass in as the <script> it throws the error:
Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }.
I have tried to pass .sh files, as well as a plethora of other files and file types to no success.

For example, I run
perftree ./script.sh
and get
Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }

but, running just ./script works fine and gives the correct output.
Thanks

Hangs with infinite newlines (maybe because of checkmate?)

Hey, I have been using this for my chess engine and it has been really helpful.

However, in certain situations it seems to get hung in a loop where it outputs newlines to stdout.

I can reproduce with the following input:

fen 5k2/8/5K1R/8/8/8/8/8 w - - 0 1
depth 2
diff

My program outputs a correct (or at least diffable output) for this fen:

./target/debug/stubot-perftree 2 "5k2/8/5K1R/8/8/8/8/8 w - - 0 1"
f6e5 5
f6e6 3
f6f5 5
f6g5 5
f6g6 3
h6g6 1
h6h1 2
h6h2 2
h6h3 2
h6h4 2
h6h5 2
h6h7 2
h6h8 1

35

My instinct is this relates to checkmate (which is what I am debugging right now). There is a mate in 1, so not all leaf nodes are actually of depth 2.

For each move available at the current position, print the move and the number of nodes at the given depth which are an ancestor of that move, separated by whitespace.

Does "the current position" mean the same thing as what you call "the base position" earlier in this section? If not, what does "the current position" mean?

Also, you meant descendant instead of ancestor, right?