Lichess sends invalid move about botli

[Torom]

I have created a new issue because although the result is similar, the cause is completely different.

It looks like the local board does not match with Lichess' board or Lichess sent an invalid move for some other reason.

@IbaiBuR Can you please post the link to the game.

[IbaiBuR]

Thanks for moving it, you a right, the result is similar but the cause is completely different.

Link to the game: https://lichess.org/eN0MpXmc (my bot is black).

Full logs of the game:

ID: eN0MpXmc     BOT ArasanX (2814)   -   BOT ReinforcementTrial (2903)     TC: 5+0     Rated     Variant: Standard

‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾

ArasanX (player): Hi! I'm ArasanX. Good luck! Type !help for a list of commands I can respond to.

ArasanX (spectator): Hi! I'm ArasanX. Type !help for a list of commands I can respond to.

Book:    1...   e5        100 %     BlackHole

Book:    2...   Nf6       100 %     BlackHole

Book:    3...   Nc6       100 %     BlackHole

Book:    4...   Bb4       100 %     BlackHole

Book:    5...   Bc5       100 %     BlackHole

Book:    6...   d6        100 %     BlackHole
Book:    7...   O-O       100 %     BlackHole

Book:    8...   Re8       100 %     BlackHole

Book:    9...   Bg4       100 %     BlackHole

Book:    10...  e4        100 %     BlackHole

Book:    11...  exf3      100 %     BlackHole

Book:    12...  fxg2      100 %     BlackHole

Book:    13...  dxc5      100 %     BlackHole

Book:    14...  Ne5       100 %     BlackHole

Engine:  15...  Nxc4      +0.05     29/48      Nodes:   5.1 M     NPS:   1.2 M     MT: 00:04.1     Hash:  10.8 %

Engine:  16...  Nxd5      +0.07     32/56      Nodes:  10.8 M     NPS:   1.1 M     MT: 00:09.8     Hash:  26.3 %     TB:     1

Engine:  17...  Nb4        0.00     36/62      Nodes:  11.5 M     NPS:   1.2 M     MT: 00:09.9     Hash:  26.4 %

Engine:  18...  Nd3        0.00     42/54      Nodes:  15.5 M     NPS:   1.2 M     MT: 00:13.3     Hash:  39.4 %     TB:     5

Engine:  19...  Qd4        0.00     43/51      Nodes:  13.2 M     NPS:   1.1 M     MT: 00:11.6     Hash:  36.1 %     TB:    50

Engine:  20...  Qxc3      +0.08     30/50      Nodes:   8.0 M     NPS:   1.1 M     MT: 00:07.1     Hash:  16.9 %     TB:     5

Engine:  21...  Rad8      +0.10     31/49      Nodes:   8.7 M     NPS:   1.1 M     MT: 00:07.9     Hash:  21.4 %     TB:    36
Engine:  22...  Rd7       +0.07     38/46      Nodes:   6.9 M     NPS:   1.2 M     MT: 00:05.8     Hash:  16.4 %     TB:    21

Engine:  23...  Re6       +0.09     28/45      Nodes:   4.3 M     NPS:   1.1 M     MT: 00:03.8     Hash:   9.4 %

Engine:  24...  b6        +0.06     30/53      Nodes:  10.7 M     NPS:   1.1 M     MT: 00:10.0     Hash:  27.3 %     TB:    33

Engine:  25...  Rc6       +0.19     32/40      Nodes:   5.8 M     NPS:   1.1 M     MT: 00:05.4     Hash:  13.5 %     TB:     2

Engine:  26...  Nc5        0.00     48/48      Nodes:     1       NPS:     1                                         TB:     1

Engine:  27...  Ne6       +0.08     32/50      Nodes:   4.3 M     NPS:   1.1 M     MT: 00:03.8     Hash:   9.2 %     TB:    79

Engine:  28...  Nxg5      +0.16     32/46      Nodes:   4.3 M     NPS:   1.2 M     MT: 00:03.7     Hash:   9.8 %     TB:   100

Engine:  29...  Ne6       +0.08     39/46      Nodes:  25.3 M     NPS:   1.1 M     MT: 00:23.2     Hash:  56.3 %     TB:   993

Engine:  30...  Rd8       +0.12     39/47      Nodes:   5.1 M     NPS:   1.1 M     MT: 00:04.7     Hash:  11.7 %     TB:   113

Engine:  31...  Nxd8      +0.09     35/52      Nodes:   8.7 M     NPS:   1.1 M     MT: 00:08.0     Hash:  22.1 %     TB:   259
________________________________________________________________________________________________________________________________

ID: nD43xgWa     Challenger: BOT mayhem23111 (2934)     TC: 5+5     Rated     Color: Random     Variant: Standard

Challenge added to queue.

‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾

Engine:  32...  Rc4       +0.48     36/34      Nodes:   3.2 M     NPS:   1.1 M     MT: 00:02.9     Hash:   6.4 %     TB:     2

Engine:  33...  Rxa4      +1.36     23/40      Nodes:   3.4 M     NPS:   1.1 M     MT: 00:03.1     Hash:   8.0 %     TB:   108

Exception in thread Thread-913:

Traceback (most recent call last):

  File "/usr/local/lib/python3.12/threading.py", line 1052, in _bootstrap_inner

    self.run()

  File "/game.py", line 71, in run

    updated = self.lichess_game.update(event)

              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/lichess_game.py", line 88, in update

    self.board.push(chess.Move.from_uci(moves[-1]))

  File "/usr/local/lib/python3.12/site-packages/chess/__init__.py", line 2259, in push

    assert piece_type is not None, f"push() expects move to be pseudo-legal, but got {move} in {self.board_fen()}"

AssertionError: push() expects move to be pseudo-legal, but got a1a4 in 3n2k1/2p2ppp/1p6/8/r2BP3/2P2PP1/5K2/5R2

________________________________________________________________________________________________________________________________ArasanX will not be challenged to a new game pair before 2023-11-22 08:33:16.

I also analyzed the position locally to check if it was related to the engine or something, I got a completely different move tho:

position fen 3n2k1/2p2ppp/1p6/8/P1rBP3/2P2PP1/5K2/R7 b - - 0 33
go depth 25
info string NNUE evaluation using nn-0000000000a0.nnue
info depth 1 seldepth 0 multipv 1 score cp 0 nodes 0 nps 0 hashfull 0 tbhits 0 time 2 pv b6b5
info depth 2 seldepth 3 multipv 1 score cp 6 nodes 46 nps 23000 hashfull 0 tbhits 0 time 2 pv d8c6
info depth 3 seldepth 2 multipv 1 score cp 6 nodes 66 nps 33000 hashfull 0 tbhits 0 time 2 pv d8c6
info depth 4 seldepth 4 multipv 1 score cp 9 nodes 91 nps 45500 hashfull 0 tbhits 0 time 2 pv d8c6
info depth 5 seldepth 4 multipv 1 score cp 14 nodes 136 nps 68000 hashfull 0 tbhits 0 time 2 pv d8c6
info depth 6 seldepth 6 multipv 1 score cp -14 nodes 1045 nps 522500 hashfull 0 tbhits 0 time 2 pv g8f8 a4a5 b6a5 a1a5
info depth 7 seldepth 7 multipv 1 score cp -17 nodes 2113 nps 704333 hashfull 0 tbhits 0 time 3 pv h7h6 f2e3
info depth 8 seldepth 7 multipv 1 score cp -18 nodes 2438 nps 609500 hashfull 0 tbhits 0 time 4 pv d8c6 f2e3 c6a5 e3d3
info depth 9 seldepth 9 multipv 1 score cp -15 nodes 4005 nps 801000 hashfull 0 tbhits 0 time 5 pv d8c6 f2e3 c6a5 g3g4 c7c5 d4e5 f7f6
info depth 10 seldepth 9 multipv 1 score cp -14 nodes 5161 nps 1032200 hashfull 1 tbhits 0 time 5 pv d8c6 f2e3 h7h5 e3d3 c6a5 g3g4 h5g4
info depth 11 seldepth 13 multipv 1 score cp -13 nodes 7543 nps 1077571 hashfull 2 tbhits 0 time 7 pv d8b7 f2e3 g8f8 e3d3
info depth 12 seldepth 14 multipv 1 score cp -13 nodes 13457 nps 1223363 hashfull 4 tbhits 0 time 11 pv d8c6 f2e3 c6a5 g3g4 h7h6 e3d3 f7f6
info depth 13 seldepth 16 multipv 1 score cp -6 nodes 18656 nps 1243733 hashfull 7 tbhits 0 time 15 pv d8c6 f2e3 c6a5 e3d3 h7h6 e4e5 g8f8 g3g4 f8e8 f3f4 e8d7
info depth 14 seldepth 16 multipv 1 score cp -1 nodes 31607 nps 1374217 hashfull 12 tbhits 0 time 23 pv d8b7 f2e3 g8f8 e3d3 c4c6 g3g4 f7f6 a1h1 h7h6
info depth 15 seldepth 19 multipv 1 score cp -3 nodes 53572 nps 1409789 hashfull 22 tbhits 0 time 38 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 f3f4 f7f6 f4f5 f8g8
info depth 16 seldepth 24 multipv 1 score cp 0 nodes 68959 nps 1407326 hashfull 28 tbhits 0 time 49 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 f3f4 f7f6 f4f5 f8g8 e4e5 f6e5 d4e5
info depth 17 seldepth 28 multipv 1 score cp 0 nodes 129600 nps 1408695 hashfull 51 tbhits 0 time 92 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 a1a2 f7f6 e4e5 f6e5 d4e5 c4c6 a2h2 f8f7 f3f4 a5b3 d3e4
info depth 18 seldepth 25 multipv 1 score cp -1 nodes 143668 nps 1422455 hashfull 57 tbhits 0 time 101 pv d8c6 f2e3 c6a5 e3d3 g8f8 f3f4 h7h6 e4e5 f8e7 f4f5 e7e8 a1a2 h6h5 a2a1
info depth 19 seldepth 20 multipv 1 score cp 0 nodes 160177 nps 1430151 hashfull 62 tbhits 0 time 112 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 a1a2 f7f6 e4e5 f6e5 d4e5 c4c6 f3f4 f8f7 e5d4 c6g6 d4e5
info depth 20 seldepth 30 multipv 1 score cp 0 nodes 187858 nps 1434030 hashfull 75 tbhits 0 time 131 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 a1a2 f7f6 e4e5 f6e5 d4e5 c4c6 f3f4 f8f7 e5d4 c6e6 f4f5 e6e1
info depth 21 seldepth 29 multipv 1 score cp 0 nodes 212386 nps 1444802 hashfull 83 tbhits 0 time 147 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 a1a2 f7f6 e4e5 f6e5 d4e5 c4c6 f3f4 f8f7 a2g2 a5b3 d3e4 c6c4 e4d5 c4c5 d5e4
info depth 22 seldepth 31 multipv 1 score cp 0 nodes 318541 nps 1422058 hashfull 122 tbhits 0 time 224 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 a1a2 f7f6 e4e5 f6e5 d4e5 c4c6 a2b2 a5c4 b2b5 c4e5 b5e5 c6d6 d3e3 d6c6 e3d3
info depth 23 seldepth 31 multipv 1 score cp 0 nodes 415428 nps 1417843 hashfull 158 tbhits 0 time 293 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 a1a2 f7f6 e4e5 f6e5 d4e5 c4c6 a2b2 a5c4 b2b5 c4e5 b5e5 c6d6 d3e3 d6c6
info depth 24 seldepth 29 multipv 1 score cp 0 nodes 452983 nps 1424474 hashfull 171 tbhits 0 time 318 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 a1a2 f7f6 e4e5 f6e5 d4e5 c4c6 a2b2 a5c4 b2b5 c4e5 b5e5 c6d6 d3e3 d6c6
info depth 25 seldepth 35 multipv 1 score cp 0 nodes 530285 nps 1433202 hashfull 200 tbhits 0 time 370 pv d8c6 f2e3 c6a5 e3d3 g8f8 g3g4 h7h6 a1a2 f7f6 e4e5 f6e5 d4e5 c4c6 a2b2 a5c4 b2b5 c4e5 b5e5 c6d6 d3e3 g7g6 a4a5 b6a5 e5a5 d6e6 e3d3 e6f6 d3e3 f6c6 e3d3
bestmove d8c6 ponder f2e3

[Torom]

To be honest, I would just blame it on the Lichess reboot that was done at the time. ToromBot also got stuck, which normally doesn't happen very often anymore. RaspFish continues to run normally.

I would be interested to know what the gameState event looked like that could cause this error. The push to the local board is only executed if more moves are sent than those we already know (because our own move is also sent to us again, but we already push it to the local board ourselves). So Lichess must have sent more moves than we knew and yet the newest move in this list is the one we just played ourselves a1a4. Which in turn triggers the error, because another a1a4 is of course not possible in the position after a1a4.

[Torom]

Ah no, we're black, then that's not true.

[IbaiBuR]

Yes my bot is black, I think the Lichess reboot has probably something to do with it too

[Torom]

That makes it even weirder :D

Our local board is in this position:
3n2k1/2p2ppp/1p6/8/r2BP3/2P2PP1/5K2/5R2

I have no idea how we get there, it's only possible if after our 33... Rxa4 white played 34. Rf1 (which is also just a bad move). Which didn't happen in the game, BotLi doesn't make up moves, so it must have gotten it from the Lichess API.

[Torom]

So it looks like we got a gameState event with 34. Rf1 (which neither makes sense nor is really played in the game) and then a gameState event with 34. Rxa4 which then of course cannot be executed on our local board.

Edit: On the other hand, we would have had to make a move after the imaginary 34. Rf1, which didn't happen. So I don't know what happened.

Very interesting but probably caused by the Lichess reboot, as long as it doesn't happen again I'd say it's not a problem.

[IbaiBuR]

Yes, quite weird so far, I was watching the game live and I was shocked when I saw my engine giving a rook for free somehow, which made absolutely no sense.

I know BotLi does not make up the moves, thats why I was surprised.

So yeah, I think this is solely because of the lichess update because if not I cant find the reason.

[IbaiBuR]

Edit: On the other hand, we would have had to make a move after the imaginary 34. Rf1, which didn't happen. So I don't know what happened.

After the error BotLi ignored the game and started matchmaking again, so I guess that is why there were 2 instances of the engine running.

That is why my bot lost on time.

See:

Exception in thread Thread-913:

Traceback (most recent call last):

  File "/usr/local/lib/python3.12/threading.py", line 1052, in _bootstrap_inner

    self.run()

  File "/game.py", line 71, in run

    updated = self.lichess_game.update(event)

              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/lichess_game.py", line 88, in update

    self.board.push(chess.Move.from_uci(moves[-1]))

  File "/usr/local/lib/python3.12/site-packages/chess/__init__.py", line 2259, in push

    assert piece_type is not None, f"push() expects move to be pseudo-legal, but got {move} in {self.board_fen()}"

AssertionError: push() expects move to be pseudo-legal, but got a1a4 in 3n2k1/2p2ppp/1p6/8/r2BP3/2P2PP1/5K2/5R2

________________________________________________________________________________________________________________________________ArasanX will not be challenged to a new game pair before 2023-11-22 08:33:16.

ID: nD43xgWa     Challenger: BOT mayhem23111 (2934)     TC: 5+5     Rated     Color: Random     Variant: Standard

Challenge has been canceled.
Skipping FantacticEman (-3) as white ...

Skipping mayhem23111 (-5) as white ...

Skipping KQRBNPkqrbnp (-14) as white ...

Removing ResoluteBot from online bots because it is offline ...

Challenging LeelaMultiPoss (+22) as white to rapid_3 ...

Challenge against LeelaMultiPoss has timed out.

LeelaMultiPoss will not be challenged to a new game pair before 2023-11-25 03:39:40.

Skipping FantacticEman (-3) as white ...

Skipping mayhem23111 (-5) as white ...

Skipping KQRBNPkqrbnp (-14) as white ...

Challenging EmanBot (-30) as white to rapid_3 ...

Challenge against EmanBot has timed out.

EmanBot will not be challenged to a new game pair before 2023-11-22 16:10:14.

Skipping FantacticEman (-3) as white ...

Skipping mayhem23111 (-5) as white ...

Skipping KQRBNPkqrbnp (-14) as white ...

Skipping caissa-ai (-46) as black ...

Challenging ArasanX (-53) as white to rapid_3 ...

[Torom]

Well, for 33... Rxa4 I don't really see any other reason than the engine just wanted to play it. What's normally your NPS? That's usually the best way to recognize a second engine, as the NPS is then roughly halved.

[IbaiBuR]

In the game the NPS were normal (1.2M which is what I normally get with 2 threads), the second instance was launched after the error.

[Torom]

That's what I thought. BotLi does not terminate the engine correctly when an error occurs, I would probably have to solve that better.

[IbaiBuR]

I see :)

At least we were able to spot where the problem was, still very weird though