How much disk-space for training about maia-chess

[reidmcy]

The files we used for training are from https://database.lichess.org and we host the testing sets in the CSV format as that's easier to analyze but not the right format for training.

The amount of disk space required is dependant on the number of games and if you delete the intermediate files as those (uncompressed PGN) are larger than the final data files (gzipped protobuffer). The rating of the players also affects things as higher rated players tend to play longer games, e.g. 1900 has about a 30% increase in file size over 1100.

I don't have the processed files anymore as they're too large to keep in our archive and the rebuilding can be done deterministically, but I believe it was under 100GB per model. I also wouldn't suggest you just do a single month as there likely aren't enough games for players in a narrow rating bucket and I've never been able to get models with a wide bucket to perform well.