Should we merge the `open-all` branch? about methylmallet

[pegeler]

Update: 3303a78 raises exception if too many files are given. An alternative could be falling back to the open-files-as-needed approach.

[bnovotny]

I would wait to merge until running more trials to find out if the increase in speed is worth the issue with larger datasets. If it turns out to be worth it, maybe with too-large datasets the program continues with the open-files-as-needed approach? Not sure if that's helpful input at all but that's my current take on it.

[nullderp]

So, there's some tomfoolery one could use to increase the maximum open files, but it is a hazard given that it takes configuration, and could be catastrophic if the power went off or something. I would not advise using this approach, usability and reliability are more important here.

[pegeler]

You're right. The way to get this approach to work is to write out a bunch of temp files and then gather them. I'll write some code against that tonight.