Game list building improvements about mamesaver

[nullpainter]

Changing the //machine XPath to mame/machine reduces the elapsed time to a bit over three minutes. I've only done a single run on both so this isn't hugely scientific, however a more specific XPath query should theoretically be more performant.

If we ignore the separate button suggestion for the time-being, I'm happy to do a proper performance test and identify the bottleneck.

[nullpainter]

Have reduced the full rebuild time to around five seconds 600ms (for 20 roms, to be fair) by also running -listxml on each verified game rather than retrieving the full output. Which I've just realised is what you initially suggested and I promptly forgot about. This is included in the PR.

Think I might leave this now 😄

Edit: Updated both calls to Mame to pass in chunked list of roms to speed things up even further. Definitely finished with this one now.

[mika76]

Fixed in #8

[mika76]

Hey thanks for the interest @nullpainter!

Yeah the build list functionality is a pain. Options I thought of...

1. Looking at http://docs.mamedev.org/commandline/commandline-all.html I could probably pull the list using -listfull which comes back within a second if piped, but there is no year included only the game name.

2. One idea I had was to actually get the rom folders by using the -showconfig param, get the files list there, and then get the -listxml info for only the roms in the folders. This would probably have a bug effect for most people since I doubt most people have 1000's of roms.

3. One other issue is the fact that if I pipe the -listxml for all games it's like 430MB! I didn't really want such a huge cached file permanently saved. Of course there is the option to just pull out the info needed (after the list has been pulled - so still quite a while) and then cache only that info (rom name, game name and year really). If doing this it would probably be good to store the hash of the mame.exe and compare so we know if it was changed/updated. This would save time on subsequent tries but not on the first one.

Either opt 1 or 2 sound the best to me. 1 if showing the year is dropped (seems silly to do all this to just show year), or 2 if year is important.

If you don't have the time or inclination to do this work, I may be able to submit a pull request.

I'm very happy to receive contributions 😊 Please just try keep them small and focused one thing at a time - last one I got changed everything including formatting and had the binaries included!

[nullpainter]

Good point about the size of the full XML file; I had forgotten that! Given that piping directly to a file from mame.exe takes under a minute, I may start by rewriting the parsing using XMLReader instead of XPath to see how much this alleviates the problem.

Even though it pains me, I promise to keep the project using .NET Framework 2.0 ;-)

[mika76]

Even though it pains me, I promise to keep the project using .NET Framework 2.0 ;-)

Hey I'm quite happy to update to a newer version of .net - update away 😄

[nullpainter]

Good news. By reading directly from the StandardOutput stream instead of first reading the contents into a string, and using a combination of XmlReader traversal with LINQ to XML to simplify the parsing, I've reduced the -listxml parsing from 4.5 minutes and 3Gb of RAM (!) down to 26 seconds and 33Mb of RAM. This is about the amount of time that the -listxml command itself takes.

Unfortunately the -verifyroms step still takes just under three minutes - again because the MAME command takes that long. 😢

So, this is a good first step that I think should be done regardless of any future smarts.

I've upgraded to .NET 4.6 in order to use the LINQ to XML package and am tempted to shift all of this to a separate class. It would also be nice to be able maintain previous game selections after scanning, so there could potentially be quite a bit of related game preparation code. I've never been a huge fan of #region, however I'm happy to follow your preferred style.

I'll tidy this up and hope to create a PR later this weekend.

[nullpainter]

I've found a solution for the verify roms step. Instead of running -verifyroms with no further arguments, you can enumerate through the contents of the roms directory and invoke it per rom. I've tested with a missing rom, and a rom with missing files, and all checks out.

I only have 15 roms as I'm running this screensaver at work with a curated set of vector games. Running -verifyroms appears to take a constant time, regardless of the number of roms. I imagine you would need a significant amount of roms before checking them individually will result in worse performance.

Implementing this approach, the full rebuild time takes just under 30 seconds for me, with the verification step down to 10 seconds. I'll create a PR shortly which you can take if you like.

[mika76]

Definitely finished with this one now

🤣