Losing on time A LOT about zahak

[CMCanavessi]

I will gladly test it, but later or tomorrow. Will let you know what I find.

[CMCanavessi]

Ok, reporting back. That last version works ok. Not many games played yet, but the old one was losing on time on almost every game. Now i've already seen several with no time losses, and always with ponder ON. Looks good.

[amanjpro]

Ah, thanks for testing Zahak out. Any idea if it was losing when in time trouble or at some other situation? Currently I preserve 5ms as a buffer, I believe this is too low, maybe 10-15ms is a better range

[CMCanavessi]

I really don't know because I left it playing overnight. Arena is quite crappy and latency between gui and engine is huge, maybe it's like you say, the buffer is too small.

[amanjpro]

This PR should bump the buffer time to 100ms,
#42

I'll make an alpha release and let you test it. Thanks for using Zahak :)

[amanjpro]

Can you please test this, and see if the issue has disappeared: https://github.com/amanjpro/zahak/releases/tag/1.0.0-alpha-1

[CMCanavessi]

Great, I will test it and let you know. Just give me time.

[CMCanavessi]

Ok, that was faster than I thought. I started a new tournament with the new binary (ignore that Arena says Zahak 0.2.1, it's actually 1.0.0-alpha-1 playing). 2 games, both games lost by time:

Pairing 1/61: Zahak 0.2.1 x64  -  Loki 1.2.0 x64
Round 1 of 4: Zahak 0.2.1 x64  -  Loki 1.2.0 x64
Game end: 0-1 {0-1 White forfeits on time}
Last game won by Loki 1.2.0 x64 with Black in 46 moves
Zahak 0.2.1 x64  -  Loki 1.2.0 x64: 
   1 B 2    1,0:0,0        0,0:1,0       11:36:06     0.00    46 8,023 :8,205  0-1-0
Saving file F:\Chess\Arena Windows 2\Tournaments\261 - Loki 1.2.0 Gauntlet.pgn Round:1 2021.03.24 11:32:27

Pairing 2/61: Zahak 0.2.1 x64  -  Sissa 2.0.0.0 x32
Round 1 of 4: Zahak 0.2.1 x64  -  Sissa 2.0.0.0 x32
Game end: 0-1 {0-1 White forfeits on time}
Last game won by Sissa 2.0.0.0 x32 with Black in 51 moves
Zahak 0.2.1 x64  -  Sissa 2.0.0.0 x32: 
   1 B 2    1,0:0,0        0,0:1,0       11:40:08     0.00    51 8,818 :8,545  0-1-0
Saving file F:\Chess\Arena Windows 2\Tournaments\261 - Loki 1.2.0 Gauntlet.pgn Round:1 2021.03.24 11:36:12

Pairing 3/61: Zahak 0.2.1 x64  -  MatMoi VII 7.15.0-cct x64
Round 1 of 4: Zahak 0.2.1 x64  -  MatMoi VII 7.15.0-cct x64
Game end: * {}
Last game finished without result in 3 moves
Zahak 0.2.1 x64  -  MatMoi VII 7.15.0-cct x64: 
   1 * *     0,0:0,0        0,0:0,0       11:40:17    +0.74     3 9,000 :0 0-0-0
Saving file F:\Chess\Arena Windows 2\Tournaments\261 - Loki 1.2.0 Gauntlet.pgn Round:1 2021.03.24 11:40:12
Tournament aborted

So I'm going to give you a little more data and maybe you can check yourself recreating my settings:
Using Arena for Windows, engine configuration is 4096 MB of hash (i know it's excessive, but it still should work), ponder ON, time control is 1 minute + 1 second, 1 thread for each engine. CPU is Ryzen 7 1700, 32 gigs of ram, Win 10.

Here is the engine configuration directly from the ArenaENG.cfg file (remember that 0.2.1 is pointing to the new binary, I just didn't change the name):

[Zahak 0.2.1 x64]
Ponder=true
Hash=4096
Book=false

[Zahak 0.3.0 x64]
Ponder=true
Hash=4096
Book=false

Zahak 0.2.1 x64=%ArenaDrive%\Chess\Engines Windows\Zahak\0.2.1\zahak-windows-amd64-1.0.0-alpha-1.exe||721|Amanj|1||||100|0|1|0|||||0|0||

[amanjpro]

Ah! Pondering is on! it might be a bug in my pondering implementation. As I don't have access to a Windows box, would you be able to send me the logs to one of the games? Or can you play Zahak and deliberately avoid the moves that Zahak ponders on, and see how it handles it (and send me the logs for the game?)?

Mmm, looking into the logs, the forfeit happens at moves like 40-50 (which eliminates the pondering hypothesis). I will look into Loki (or Sissa) and see how do they handle the time buffer

[amanjpro]

Loki uses 50 milliseconds!

[CMCanavessi]

Just let me set everything up again, and i'll turn logging to on and wait for a forfeit to happen, then I'll post it here.

[CMCanavessi]

Ok, again, it happened on the 1st try lol

Here is the Arena log: https://file.io/CzBGgSOmfIt8
Here is the PGN, if you want to check the game: https://pastebin.com/DcreMVij

Here's what I'll do now, I'll disable pondering and see what happens. If it works ok, then you know where to look for bugs. I'll report back.

[CMCanavessi]

Ok, no. I disabled pondering and the 1st game was already a loss on time. I let it continue for 1 more game, and again, loss on time.

Here is the Arena log for those 2 games: https://file.io/wVmX4vUHKAew
Here is the PGN: https://pastebin.com/c1RFiacQ

I just watched the games live, and nothing really abnormal is happening... it's just that when Zahak is low on time it will not move fast enough.

[amanjpro]

Ah, the second sample is not losing on time, but:

2021-03-24 15:58:03,546<--1:info depth 7 seldepth 6 tbhits 9 hashfull 1000 nodes 462930 score cp -148 time 996 pv g4g5 b5b4 g5e5 b8c7 f3f4 b6b5
2021-03-24 15:58:03,577<--1:bestmove g4g5 ponder b5b4
2021-03-24 15:58:03,582*1*---------> Arena:Illegal move!: "bestmove g4g5 ponder b5b4" ()
2021-03-24 15:58:03,594****Tour**Game end: 0-1 {0-1 White forfeits on time}

That is intersting! looking at the PGN, it is not actually illegal!

[amanjpro]

Same thing applies for the first one (pondering on), Zahak replies with b5a4 which is not illegal, but marked as illegal by the Arena!!

[amanjpro]

I noticed, only when I was running a parallel tournament, sometimes CuteChess was claiming that Zahak is playing an illegal move, but I could never generate the illegal move... something strange is happening, either in Zahak side, or GUI side, or maybe my understanding to UCI

[amanjpro]

OK, looked further up in the logs and I see this:

2021-03-24 15:58:02,754-->1:stop

Probably Zahak ran out of time earlier, and received stop, and as per the protocol it sent the best move, but as it is already out of time, it is considered illegal. So yeah, it is time management issue

I need to debug this, and see how much time is allocated for search:

position startpos moves b2b4 d7d5 c1b2 d8d6 a2a3 c8f5 e2e3 b8d7 g2g4 f5g6 g1f3 e8c8 b1c3 c8b8 c3b5 d6e6 b2d4 b7b6 h1g1 e6c6 a1c1 h7h5 f3e5 c6e6 e5g6 e6g6 f1d3 g6g5 d3f5 e7e6 f5d3 c7c6 d1f3 c6b5 f3f7 d7e5 f7e6 e5f3 e1e2 f3g1 c1g1 g5e7 d4e5 b8b7 e6e7 g8e7 g4h5 h8h5 e5g7 a7a6 g7f8 d8f8 g1g2 h5h6 e3e4 b7b8 e2e3 h6h3 e3e2 d5e4 d3e4 h3a3 g2g3 a3a2 g3g7 f8e8 e2d3 a2a1 h2h4 a1e1 f2f3 e1h1 g7h7 h1b1 h4h5 b1b4 h5h6 b4b1 h7g7 b1h1 h6h7 h1h4 g7g4 h4h2
2021-03-24 15:58:01,258-->1:go wtime 1 btime 21881 winc 1000 binc 1000

[amanjpro]

Oh! white has 1 millisecond left on its clock! I doubt I handle this :D

[amanjpro]

Zahak, thinks that the game is going to stretch for 30 moves, and plans the time for that many moves. That is why it consumes more time than most engines in the beginning. This usually pays off, as it outplays other engines early off. But when faced against a stronger engine, it runs out of time at the end (as even with more thinking, it can't actually harm them :D)

I probably need to work on timemanagement and trying to make it adaptive (based on eval and other factors), than adding more buffer

Oh, and it also thinks that the increment is part of the remaining time, this I can easily trim

[amanjpro]

Can you please try this, and see if the issue still persists:

https://file.io/6M1V0tFbLm6j

I basically remove the increment from the "available time"

Amanjs-MBP:zahak amanj$ git diff
diff --git a/search/timemanager.go b/search/timemanager.go
index 09eb6d5..c10e391 100644
--- a/search/timemanager.go
+++ b/search/timemanager.go
@@ -4,22 +4,22 @@ import (
        . "github.com/amanjpro/zahak/engine"
 )
 
-const COMMUNICATION_TIME_BUFFER = 100
+const COMMUNICATION_TIME_BUFFER = 50
 
 func (e *Engine) InitiateTimer(game *Game, availableTimeInMillis int, isPerMove bool,
        increment int, movesToTimeControl int) int64 {
        maximumTimeToThink := 0
        if isPerMove {
-               maximumTimeToThink = availableTimeInMillis - COMMUNICATION_TIME_BUFFER
+               maximumTimeToThink = availableTimeInMillis
        } else {
                movestogo := 30
                if movesToTimeControl != 0 {
                        movestogo = movesToTimeControl
                }
                availableTimeInMillis /= movestogo
-               maximumTimeToThink = availableTimeInMillis - COMMUNICATION_TIME_BUFFER
+               maximumTimeToThink = availableTimeInMillis
        }
-       return int64(maximumTimeToThink + increment)
+       return int64(maximumTimeToThink - COMMUNICATION_TIME_BUFFER)
 }

[amanjpro]

I'll wait until the match finishes before merging the PR... Thanks a lot for your interest in Zahak

[CMCanavessi]

Well I let it play several matches and aborted the tournament because I had other one to run (which had to pause). Didn't get a single time forfeit. Maybe eventually one would appear, but so far, this is way better than before.

[amanjpro]

Cool... I merged the PR... will be included in the next alpha release